adverse selection and integrity

[Mark Wegener]

Hi there,

 

I'd like to spin up a discussion about the integrity of the people reviewing new editor applications.

 

My main problem with my application is the responds I've got from it. I applied to the category of Arts/Music/Charts containing 91 links. Now charts is a subject which is either an archive or highly up to date (weekly max monthly). There are actually dead links in this category, links which haven't been updated for more than a year, and perhaps more surprisingly links to promote party DJ's.

So I thought I apply for this directory. I personally believe I have good credentials, it's been my hobby for 5 years and been doing data mining related to charts for a few as well. Which I now publish, and as I may say myself is pretty damn informative (and according to my stats, most of my visitors back me up on this). Now as one who's hobby it is, I do like to participate and actively relate to the parts I believe are top notch (backed up by my upwards sloping experience curve). Some of these weren't in the directory so I supplied these.

 

The responds to my application was about the following: "Thanks but no thanks. We already have more then enough editors. Start small and work your way up.", with additional comments saying: "The application specifically asks for 2 or 3 suggested URLs that fit the category. Ideally at least 2 of these should not be affiliated with you or any of your clients." I honestly do not believe the person who said that, actually clicked the links and saw what they were about, or checked the awful state the category is in right now. And on the matter of the additional comment, bend the rules where necessarily (it's like your on the highway cruising at the limit, 5 cars approach at double the limit, there is no way out except breaking the limit, and you still won't do it).

 

My conclusion:

someone who administered my application has a larger then normal personal interest in this category, as the application was reviewed within 4 days (probably to make sure a honest objective editor wouldn't accept my application). And second it says there are no editors for this category.

Honestly I hope this is the problem, because if it isn't, it seems the directory has some uninterested people volunteering for them. This wouldn't be a problem if they were lazy, but they don't seems to be both, and I see this as a lethal combination for a volunteer-driven site.

 

My solution:

Tell me who reviewed my application, and let me defend it.

I believe the key to 99% of the potential success is transparency, unfortunately no more then let's say 5% actually use it. Just to name an example of the past few years: wikipedia (upupup), dmoz (downdowndown). Therefore I suggest to rebrand dmoz to: The closed Directory, because face it, that's what it is.

 

I'm curious about your opinion.

 

Cheers,

Mark

 

ps. what is the long term vision and goals of dmoz??

[chaos127]

I haven't looked at your application, but from what you've written I would assume the following:

 

1/ The reviewed felt that the category was too large for you to take on straight away as new editor. (Regrettably there is a standard letter that we need to use, and its content is often misinterpreted. The size of the category is almost always the reason it's used.)

 

2/ In the sites you suggested you included more that one that you were affiliated with in some way. (We don't want to accept editors who are only going to add their own sites to the category.)

 

Both of these are reasonable grounds for rejecting an application, and both of them can easily be solved by the applicant should they wish to apply again. (And there's nothing to stop you doing so.) As for your arguments about the current state of the category or the content of the example sites you suggested, these are both irrelevant when it comes to deciding if either of the above grounds applies.

 

Editor applications are only reviewed by meta editors and catmods, who are experienced editors with a proven track record of impartial editing. I would very much doubt that there is any conflict of interest over the category you applied to.

 

(BTW: The 'open' part of the name refers to the fact that the data produced is made available for others to use without charge, not that the whole editing process is open. Though you might have noticed all our policies and procedures are publicly available, including the parts that explain how meta editors evaluate applications.)

[jimnoble]

What chaos127 said...and more.

 

I have looked at your application.

 

We ask for example websites so that we can see how well the candidate can find good ones for the category. By proposing two that you're associated with, you prevented us from doing that. This meant that we couldn't properly evaluate your application. I'd have declined it on those grounds also.

 

You want to know the name of the person who evaluated it. Metas have the option to be anonymous when declining an application and most of us have leaned the hard way that to do so is a good idea. It would seem that this one did too and I'm not about to breach that wish.

 

We understand your disappointment but the email you received is an honest one and pretty routine under your circumstances. By all means try again in the light that it and the further explanations in this thread should provide.

 

Finally, the only folks who can post in this thread are people who don't have access to the data and meta editors who do. That's not really a good foundation for sensible debate.

[Mark Wegener]

2/ In the sites you suggested you included more that one that you were affiliated with in some way. (We don't want to accept editors who are only going to add their own sites to the category.)

 

If for example a manager of UNICEF would apply for the editorship of let's say Child Welfare. And in it's application he would suggest UNICEF and some other welfare organizations they work with, because he know they do good work. Don't you think he's the man for the job? The part I find weird is the one where an editor follows the guidelines while the editor supplies links which already should have been in the category and with these links shows he knows the category. I believe this simple calls on your common sense.

 

As for your arguments about the current state of the category or the content of the example sites you suggested, these are both irrelevant when it comes to deciding if either of the above grounds applies.

 

I didn't say that would be a valid ground of acceptances, but then again, I didn't agree with your second argument.

 

(BTW: The 'open' part of the name refers to the fact that the data produced is made available for others to use without charge, not that the whole editing process is open. Though you might have noticed all our policies and procedures are publicly available, including the parts that explain how meta editors evaluate applications.)

 

Yes, that's exactly why I suggested the name change. Corporations also have public also policies and procedures but that is not what makes people find them to be open or closed.

 

[...] Finally, the only folks who can post in this thread are people who don't have access to the data and meta editors who do. That's not really a good foundation for sensible debate.

 

I agree on the first part about proper evaluation (and therefore the rejection of my application). Though I find it somewhat weird to anonymously reject an application WITH comments, but then again my definition of open is somewhat different from dmoz's.

 

Finally, the only folks who can post in this thread are people who don't have access to the data and meta editors who do. That's not really a good foundation for sensible debate.

 

Agreed on as my example has it's far share in my post, but if you take it out, there is enough to have an open debate on.

 

Also still wondering about the dmoz vision/goal. And what makes editor experience important? is it to show your capabilities, or because there is a policy that an editor can't start with category which has subcategories (or whatever the exact policy is). I'm asking because in this particular category, each subcategory is a way different league in terms of taste and origin.

[jimnoble]

Rather than try and answer your questions myself, here is some reading material.

About DMOZ

FAQ and General Advice About Becoming an Editor

[shadow575]

...as the application was reviewed within 4 days (probably to make sure a honest objective editor wouldn't accept my application).

I often go through the applications for the branches I have the most experience in, and process the easier ones first. Often that means that incomplete applications or application with obvious major issues can be rejected within a matter of a day or two of submission. An application waiting for 4 days has had more than enough time for many Meta's to have looked at it. Whether the application was looked at within hours of its submission or 2 months later wouldn't have changed the outcome of the review.

 

Staff has launched an official blog, its a good time as any to remind everyone interested to check it out.

[chaos127]

If anyone (even the head of UNICEF) applied to Society/Philanthropy/Organizations/Child_Welfare/ their application would likely be rejected since the category is too large for a new editor. And even if it wasn't, including http://www.unicef.org/ as an example website isn't going to help since it's already listed there

 

More seriously, if the head of UNICEF gave only UNICEF sites as his examples, then I think I would probably reject the application, and ask him/her to apply again and show that he/she is able to find some other suitable sites. (The exception might be if it was to a category that only held UNICEF websites so there wasn't anything else to choose from.)

 

Note that while it's useful for an editor to have some experience in the topic their editing it's certainly not essential. Far more important are other qualities such as being able to work as part of a team, understanding the editing guidelines and community practices, and being able to take advice and constructive criticism from your fellow editors.

[Papa Spot]

Regrettably there is a standard letter that we need to use, and its content is often misinterpreted.

 

Please don't take offense at this but, if its content is "often misinterpreted" then maybe its content needs to be edited for better clarity. I don't know who's responsible for that but if people frequently misinterpret what I say, I have to take responsibility for my failure to be clear.

[shadow575]

The letter that chaos127 mentions as often (not usually, but more often than anyone would like) misinterpreted is the small section that reads:

 

Although we would like you to join us as a volunteer editor, you have

chosen a category that is already well represented, or is broader than

we typically assign to a new editor.

This particular letter appears when a reviewer recommends that the applicant narrow their focus to a more specific category. In most cases this is because the category originally chosen is to big for a beginner learning the ropes and/or the suggestions waiting review are spammy or in large quantity. What is typically misinterpreted is that applicants only read the first line and stop at "well represented,".

 

This being said, some reviewers (not all) may use a different letter (with a complete list of the most common reasons for rejection) and then include a short reviewer comment recommending a smaller category/narrower focus. Which alleviates most of the misinterpretations.

 

While it is regrettable when someone misinterprets the wording, the way the letter is worded and how a Meta chooses to use it (including additional comments, etc) can also make for a very clear and effective way of expressing the problems with the rejected application.

[ftw_420]

Another point

 

Part of the process, from what I understand is the ability to actually edit. That would require proper use of grammatical tenses. In your post you use the word "responds" instead of "response." If I were an editor and saw that on an application, I would reject it too because it's not showing attention to detail, among other things.

[Guest Erick880]

The letter that chaos127 mentions as often (not usually, but more often than anyone would like) misinterpreted is the small section that reads:

 

 

This particular letter appears when a reviewer recommends that the applicant narrow their focus to a more specific category. In most cases this is because the category originally chosen is to big for a beginner learning the ropes and/or the suggestions waiting review are spammy or in large quantity. What is typically misinterpreted is that applicants only read the first line and stop at "well represented,".

 

This being said, some reviewers (not all) may use a different letter (with a complete list of the most common reasons for rejection) and then include a short reviewer comment recommending a smaller category/narrower focus. Which alleviates most of the misinterpretations.

 

While it is regrettable when someone misinterprets the wording, the way the letter is worded and how a Meta chooses to use it (including additional comments, etc) can also make for a very clear and effective way of expressing the problems with the rejected application.

 

I agree with you, as a editor hopeful who has had a few rejections, I can tell the difference between a well written rejection and the stock letter. The most recent one I received had the stock letter, and then below had a point by point reason why I was rejected, utilising specific examples. This level of detail does reflect well on the person rejecting you. There was no ambiguity / room for misinterpretation. He cited the standard responses, and then said, for example, your titles do not follow our guidelines, and linked to the guidelines.