OK, this will get long.
What would be the problem with requiring all editors to first clean up the submissions needing review and staying on top of them going forward?
This is an important question, actually. Many (I'd say most) editors who have been around for a while do think that it would be directly harmful to the directory to do that, but I can see how it won't be obvious to somebody without the insider knowledge.
Suppose you are an editor, with editing rights in the category Sports/Waterfowl_Wrestling, its five bird-specific subcategories and their respective subcategories. Suppose you review the home page of the Australian Duck Wrestling Federation, which has been suggested for review by the site's webmaster. On it you find links to 40 local sites for Australian duck wrestling clubs. Some of these are obviously abandoned sites with information last updated in 1998, but most of them are good, and listing them would really make the category Waterfowl_Wrestling/Ducks/Clubs/Australia a much better resource.
However, because it has been decided that you must first review all suggested sites before going on, you can't build this category you are itching to create. You have to review 210 other sites first, which have been suggested by the public. 102 of these sites have nothing whatsoever to do with waterfowl wrestling; they were suggested to the category by a lazy SEO who didn't bother to look up the best category to suggest the sites to. If you are a conscientious editor (which we will assume that you are) you'll do your best to find the right category to forward these sites to -- we're assuming that you don't have editing rights anywhere outside S/W_W. Many editors make an attempt to re-write the description of sites they send to the unreviewed pile of other categories as well, to make the reviewing job a little quicker for the next editor to handle the site. This takes at least 5-10 minutes per site (determining what the site is about, finding the best category, moving it... and possibly writing a new description). 25 of the mis-submitted sites are in languages you don't recognise. Finding out the most likely language is fun (for many of us at least -- I love that process myself) but can also be time-consuming, especially when it comes to navigating the category structure of an unknown language to find the best category.
Interspersed with these are sites that have to do with the subject area; these are probably more fun for you to review since they are about the thing you are there to deal with. Again, review times vary from site to site, say between 7 and 25 minutes. Some of the sites are really bad, such as the fan sites for the players in the Gull Wrestling World Cup; they are deserving of as much of a review as any other site though, and as already mentioned you are a conscientious editor.
Let's also assume that you have a life outside the ODP (I have heard rumours that some editors do, though I don't hold truck with that stuff myself). You devote 3-4 hours to editing, twice a week -- I have no idea of how much time the mythical average editor puts into editing, but there are many who don't have time for that much, and many who edit much, much more than that, so let's say 8 hours a week all in all. I'm not going to do the math to find out how long it will take before you can get around to the Australian local clubs; of course, new site suggestions trickle in all the time while you are working on the other sites. Sooner or later you will probably have cleared all the unreviewed sites, though -- and only then can you do what you were so enthusiastic about a couple of months ago. Except if you're like me at all, you may have forgotten about the Australian clubs, because you have also found dozens of listable links on other sites, not to mention in the Albatross Wrestler magazine you subscribe to.
So, in the end you have done the waterfowl wrestling community a disservice by not listing the Australian club sites whose webmasters have no idea of what the DMOZ is, and no intention of paying a SEO person to promote their site (probably they, like most people, have no contept of what SEO is. Lucky them.) You also haven't had the most fun you can have as an editor: creating a new category from scratch. (And an editor who is having fun is an editor who will stay around, and may tomorrow be cleaning spam from the Web Designers category). Instead you have moved a number of undoubtedly very worthy sites around, without feeling any interest in them at all, simply because you had to do so. Without the pressure, you would eventually have found homes for the mis-suggested sites, but you'd have been able to get some proper editing in as well. As it is, you feel that a lot of your time has been wasted. If I were you I'm not sure I'd stay around. After all, you joined because you are passionate about the subject and wanted to create a great resource.
Am I making stupid examples that have no foundation in reality? Not at all. (Well, the category names are made up, I'll confess to that.) The number 102 is real -- it's the number of sites a lazy SEO in the space of 2-3 days pushed into a category I edited. (A fairly high-level category to be sure, but on the other hand that meant that fewer editors had access to it). When I started taking a special interest in the Benin category last year (the category then had fewer than 40 listed sites), I found over 50 sites about Ireland that had been suggested there (and there are still the occasional suggestion of Ireland-related sites to Regional/Africa/Benin -- no other European country, only Ireland. It's one of the great mysteries of our time why this should be). That sort of thing happens all the time, on a larger or smaller scale. Many people think Recreation/Models has something to do with beautiful people showing clothes, and suggest their sites wrong in good faith. And let's not even start on the language thing. Which, again, is something I personally enjoy, but I know it's a pain to a lot of people.
All right then. How about a solution where all the sites which are suggested to the right category get reviewed immediately? That's one of these things that sounds obvious, but is actually impossible. Because in many, many cases, it won't be all that easy to see that it is mis-sent. A toy shop in Waterfowl_Wrestling -- sure, that's an easy one. But a site about computer consultants doing a particular kind of programming in a category devoted to another programming language, that's not all that simple. Oh, and this is assuming that the description of the site actually says something about the content of the site, and in our experience that's usually not the case. So there's more time wasted trying to ascertain whether a site should be reviewed or not -- a pre-review process. I don't need to tell you that adding another step to the process is not the way to make it more efficient!
So the example is one that could very well be taken straight from reality. And this is in a category with a named editor who is editing out of a love of the subject. Does this help you see why implementing a "clear unrevieweds first" rule would be directly harmful to the directory?
