Should ODP learn from Zeal?

Hello Everyone

I'm a big fan of ODP but despair of the editor selection and site submission process. In my humble opinion the directory is becoming stale due to the lack of editors. And I know I’m not alone in the belief that it’s very tricky to be accepted as an editor, I myself have been rejected 3 times.

In the last couple of months I’ve become an editor of zeal and I have been very impressed with many aspects of their setup. Is anybody else familiar with zeal?

For the benefit of those that aren't, then to become an editor you have to pass a test which allows you to accept submissions. However every submission needs to be approved by a senior editor until you have proved yourself. Every submission receives a rating from your editor. To move up you need to submit several sites, have a submission rating average above 7 and have completed an advanced editor test.

The whole process is excellent and helps to ensure quality and editor engagement. I would really like ODP to adopt a similar setup, what does everyone else think?

Been there, didn't like it. I think the system Zeal has is stiff, complicated and bureaucratic.

One has to pass a test before s/he even can submit a link suggestion. This means, that Zeal gets much less submissions than ODP.

Then there is the points one has to earn before getting his/her "own" category. I tried to update an outdated link in another category, but because it was already listed somewhere else I couldn't. Therefore I submitted it as a new link with a comment, and was punished with a minus point because of that. I didn't find it very encouraging, but there's no accounting for tastes. If somebody thinks, that the system is better in Zeal, perhaps Zeal is the best choice for him/her.

Personally I think the system ODP has is much more open and functional. And the results speak for themselves.

Thanks for the feedback!

However I wasn't suggesting ODP adopt everything zeal do but look at them to see how to improve. I also didn't like the way you had to take a test to submit a site, but I think ODP would be better for asking any new editor take a test. Any site reviews by new editors could then be monitored and rated in a similar manner.

Does anyone else think something needs to be done to improve ODP? I can't see it remaining as influential as it currently is unless it improves. I've submitted several quality sites in the past that have taken up-to 9 months to be listed.

I’m not trying to slag off ODP, I think the directory is great and very useful. Nevertheless I really believe constructive criticism is needed if ODP is going to remain a quality resource.

Gavin, I've been an editor in the ODP for nearly two years, now, and people were still saying these sorts of things about the ODP back then - that it had a limited future, that it "just wasn't working" and that, if it didn't "improve" it would lose its status. None of this has happened.

Are there things which could be improved? Yes, of course. I believe that everyone is doing the best they can with the resources available. We suffer from a shortage of editors, and there have been threads on this forum and others which have discussed lowering the standards for admission. After long discussion, the cons outweigh the pros. The system is being maintained by two paid staff - everything else is on a volunteer basis. Given the way the ODP has grown, it is amazing to me how well staff do to keep this going - and the recent major inlux of capital to upgrade the servers is a nice vote of confidence in the future support of the directory.

A test for editors to get in? Yes there is one - the application form. As you'll note if you read around a bit, quite a few people have complained that because of the requirements on that form (and the metas that review them) there is DEFINITELY a test. Most, if not all, metas will informally "watch over" a new editor to make sure that they aren't too far off in their editing. There is also a peer review process where editors will look at another editor's maintenance of a category and critique them. We also have a program whereby editors can be given the ability to edit in a category, but additions are only after the review of another editor.

I know it's difficult to see this from the "outside, looking in", if you will - all suggestions are taken seriously, and those that are feasible are taken on. Once you get into the inside of the ODP, you realise why some of the suggestions made just aren't possible or desirable. There is a whole discussion forum in the editor's area which talks about how we can make the ODP better - many ideas brought up on this board are discussed there, plus a LOT more ideas that editors come up with.

Are we making the right decisions? Only time will tell. If Zeal grow to become more reputable than the ODP, then maybe the decisions weren't good. If the ODP continues to have the respectability that it does today, then maybe we did the best job we could, on balance. Yes, we can learn from others, but that doesn't mean that we should *become* others.

For another, similar, discussion on this board, (just so we don't repeat ourselves) please see "Could ODP learn from Wikipedia"

I want to re-iterate what alucard said. The application ~is~ the test. It is how a potential editor demonstrates:

- That they can spell, punctuate, and capitalize correctly in the appropriate language.
- That they have the ability to describe a website without hype or keyword stuffing.
- That they can find new sites that are appropriate to the scope of the category.

The ODP needs editors that are self motivated to learn and are good thinkers. That is why our "test" is an essay and not multiple choice.

-Lissa

Thanks for the excellent response! Like you say I am only an outsider looking in and I don't assume I could ever begin to solve these problems.

However burying your heads in sand won’t solve anything. If the shortage of editors isn't addressed then the ODP will become weaker and poorer quality. The current status you enjoy will soon disappear if Google stop using you, and without improvement I can’t see they’ll have an alternative.

I'd like to hear what ODP plans to do too increase the amount of editors. People want to help contribute, people that are more than capable. But the current process rejects these applications. Surely there must be a better way of approving editors that wouldn’t lower standards.

It seems as if too many responses hide behind a concrete and patronising belief that the current editor selection is fine because it ensures only quality editors get in.

However burying your heads in sand won?t solve anything.

Click to expand...

I don't feel that anyone is burying their heads in the sand. We recognise that we could always use more good editors to help make the directory better - we have ALWAYS wanted that. We can *never* have "enough", and we can never be perfect. We are always coming up with ideas on how to recruit good editors. Read a few posts on this board and you will find very few opportunities that are passed to encourage people to apply to become an editor.

If the shortage of editors isn't addressed then the ODP will become weaker and poorer quality. The current status you enjoy will soon disappear if Google stop using you, and without improvement I can't see they'll have an alternative.

Click to expand...

You are definitely entitled to your opinion. Mine differs from yours. The only way to tell which one is correct is after the fact. We are doing the best job we can given the tools and resources available, and given what we are trying to achieve. History will show whether we succeed or not.

I'd like to hear what ODP plans to do too increase the amount of editors. People want to help contribute, people that are more than capable. But the current process rejects these applications. Surely there must be a better way of approving editors that wouldn?t lower standards.

Click to expand...

Lissa has stated what is required - that is what we mean by "capable" and what I refer to above as "good". If they can't pass that test they they are not allowed to edit, you are correct. I am not a meta, and I have no knowledge of approvals directly, but I see the result of it in the editors that come on board and contribute. Personally, I WANT metas to be picky about who they let in - if we lower the bar, then all we're going to be doing is mopping up after editors who make messes. That's not exactly gratifying work. We are always emphasising quality over quantity in the ODP - editors are encouraged to list GOOD sites, not ALL sites. To me, the editor selection process reflects this too.

And applying sucessfully isn't rocket-science. Many do it, are successful, and the external message forums never hear from them, because they are busy editing. You are reading about all the cases of people saying "I got rejected and have no idea why".

It seems as if too many responses hide behind a concrete and patronising belief that the current editor selection is fine because it ensures only quality editors get in.

Click to expand...

I don't think anyone will say that the process is perfect. It doesn't catch all bad editors and it may reject some good ones. We weigh up the pros and cons in loooong discussions between editors. It gets raised about every month on one of the fora, believe me. Noone is sitting back on their laurels and pretending that more can't be done.

The doomsayers that have been foretelling the death of the ODP have been around as long as the ODP. I believe that right now it is very much alive and kicking and it will keep getting better - but everyone is entitled to their own opinion, right?

>>However burying your heads in sand won't solve anything. <<

Just because we aren't jumping up and down with joy over your suggestion doesn't mean we aren't doing anything.

>>People want to help contribute, people that are more than capable. But the current process rejects these applications.<<

If someone is rejected by the application process, then they failed to demonstrate that they are capable. The reviewing Meta has access to no information except what is put on the application. Applicants should treat it as seriously as a real-world job application.

Additionally, it doesn't matter how "expert" someone is in a topic, they have to meet the same qualifications as everyone else. Someone may be a physics genius, but if they can't spell (or at least can't figure out to use a word processor to check their spelling) then thanks but no thanks. Someone may be a complete whiz and knows everything about every car, but they'll have to start out editing a smallish category just like everyone else.

>>I'd like to hear what ODP plans to do too increase the amount of editors.<<

It is not just getting editors to apply and be accepted, it's about keeping the ones we get. ODP has a lot of ongoing internal discussions and efforts to increase editor participation and retention. Weekly, monthly, and yearly the community evolves and internal communcation improves. It will be really nice to have the servers and forums back to normal after 6+ months of problems so we can focus more strongly on some of those efforts again.

-Lissa

In general the ODP is blazing a different trail, and single features from one project can't be simply dive-bombed into the other one. (No feature stands alone; the project is a combination of features chosen in an attempt to achieve a particular balance.)

In this context it should perhaps be mentioned that the ODP's "greenbuster" feature may owe some of its ideas to the Zeal scheme; it has not yet achieved its potential (in scope or activity), even so, it may already deal with more sites than Zeal's dual review system does.

Also, it should be mentioned that there are high-level Zealots editing on the ODP, and high-activity ODP editors at Zeal. There is continual cross-pollination. They can see the various past discussions and implementations of things like "editing exams." Basically, although some editors saw value in the mockups, the editing community as a whole wasn't in favor of imposing them on everyone.

You shouldn't assume that all ODP improvement discussions will ever be centered on any one ill-conceived and undeveloped idea. A good many of the differences between the ODP of today and the 2000 ODP are a result of editor initiative (and editor initiatives.) With the hardware upgrade, staff even provided a server for editor-created tools. And editors are already working to fill up its hard drive. If the editing community cares enough about more testing to create the tests, tests will be created. If they are then found to be of almost universal value, they could be made mandatory at that time.

I found the Zeal test to be helpful and educational. I wished that the ODP had something similar when I applied to be an editor.

I would be supportive of making a similar test available to applicants for the ODP at some time in the future. Although I don't necessarily think that passing such a test should be a requirement of editorship. I also do not believe that passing such a test should result in automatic acceptance to editorship. (Even as a greenbuster.) Editing is about much more than being able to craft a satisfactory site description.

>It is not just getting editors to apply and be accepted, it's about keeping the ones we get.

Lissa makes a good point. There are tens of thousands of editors registered, but only a small fraction are very active. What should be remembered is that the ODP is a volunteer effort; if we get anything out of this, it won't be in this lifetime But people are different and there are different reasons to join. Some do it as a hobby,
others do it as a public service. And then there are the self-promoting slickers who are in it to promote their own business, and ideological cranks who join to use the ODP to push their agenda. The self-interested types usually don't stick around too long before they are shown the logout door for good.

The thing about Zeal is that from what I hear it is an 'open book test' - no real knoweledge is needed - you can take their test 1000 times over and over until you pass.
Plus it is tough for some volunteer there to suggest changes
that are taken seriously. There are some serious ontological problems with their organzation in some areas, but the biggest difference is that it's a money making venture for getting businesses listed. Cough up the dough and maybe we'll accept you.

Frankly, I think that Zeal has more to learn from the ODP than vice-versa.

In defense of Zeal, it is really intended to be a non-commercial directory (Look$mart has the commercial sites) and they have "rules" in place to protect that ideal.

The problem, with Zeal as with the ODP, is finding people to take responsibility for enforcing the guidelines. Zeal tends toward more rigid guidelines (and paid submittals for commercial sites, which eliminates the complaints about long review delays, but limits their comprehensiveness in other ways.) The ODP has explicitly repudiated both of those approaches. In this respect, rather than trying to force everyone into the same mold, we are better off supporting multiple approaches, each with their own advantages in their own communities.

If the shortage of editors isn't addressed then the ODP will become weaker and poorer quality.

Click to expand...

Who said that we had a shortage of editors? What do you base this on?

The current status you enjoy will soon disappear if Google stop using you, and without improvement I can’t see they’ll have an alternative.

Click to expand...

I think that you may be missing the point about the purpose of the ODP. Our goal is not to be the directory of choice for search engines Take a read through the About DMOZ page to see what we're all about.

People want to help contribute, people that are more than capable. But the current process rejects these applications.

Click to expand...

What are you basing this on? People that are capable are becoming editors. Nothing is stopping them. The current process rejects those who are not capable.

It seems as if too many responses hide behind a concrete and patronising belief that the current editor selection is fine because it ensures only quality editors get in.

Click to expand...

I don't see this. The current system actually works quite well for us. The current system is not carved in stone, and is modified as needed to streamline and improve the process. I don't think that we've ever claimed that only "quality editors" are accepted. Obviously we have many different types of editors. Sometimes we discover, after the fact, that an editor may not be the best that they can be, but we have a process for dealing with this.

At the risk of being hounded (or bannishment as one editor has threatend me with), but here goes, has DMOZ employed any new suggestions from outsiders?

What are your reasons for pursuing this topic? Are you truly interested in making the ODP a better place, or are you just interested in trying to find fault? I'm just curious

I think it would be really hard to determine whether an idea has been used from an outside source. Usually those ideas get posted internally and they are discussed and modified and added to by many people until a workable solution can be implemented. Not something that happens overnight, and not something that is going to have a blurb attached that says "Thanks to the participation of the following people for making this solution possible"

"Our goal is not to be the directory of choice for search engines" was not intended as a mission statement. For that you would have to read About DMOZ. You will not find anything in there about us suppling data to downstream users. Somehow people have twisted the mission of the ODP into a directory that is for web site owners instead of a directory for web surfers.

I'm sure that this will not be the last word here

"Our goal is not to be the directory of choice for search engines"

Click to expand...

It's not that our primary goal is to *not* to be the directory of choice for search engines. Rather, it's that being the directory of choice for search engines is not our goal.

Our goal is to make the best directory possible. It just so happens that enough people think we are doing a good job of it, that it has become the directory of choice for important search engines.

If that ever changes, I'm sure some of our editors may leave the project. That happens sometimes anyway. Others will stay.

I joined without even realizing how connected dmoz was to so many other directories, never mind search engines. So I guess I'd stay.

This thread was supposed to be about learning from Zeal, but it's drifted far from that, indicating that the topic has run its course.