Submission review times

Just a question here - it might come off "combatative" or argumentative, but it's not meant to be, I'm simply trying to understand the way the system for DMOZ submissions works. I read the following quote earlier today from a moderator, and had to swallow once hard before I really realized what it meant:

Due to the voluntary nature of the dmoz.org directory it is impossible to guess how long a submission will take to get a review. Wait times are between 2 hours and 2 years or more.

Click to expand...

So basically, you could get reviewed and in the same day you submitted, or... reviewed and not listed 1.5 years after your site closes down (if it's only up for 6 months). Doesn't that length of duration for a review basically go against what the DMOZ is trying to accomplish? When someone goes to a directory to find a website related to their needs, they want something both 1) relevant, and 2) timely.

Take a game site for example: Say "Diablo 8" comes out in August of 2010, and a fan creates a site for it's players sometime in February, a full 6 months ahead of when the game is launched. They submit their site to DMOZ at the same time, 6 months before the game comes out. Now, say the DMOZ submission takes 1.5 years (18 months) - the game's already been on shelves for a full year, and this fan who so diligently planned ahead with their site, gets a listing after the game's fans are already burned out, and the game is essentially dead.

What kind of information is DMOZ giving someone browsing their directory? Outdated, irrelevant info. The site might be dead, or winding down - the information is over a year past being useful.

I guess my point is - what point is it having a directory that has submission times of up to 2 years? Art might be forever, information is not. Websites don't always last for a decade, and many are frequently *very* timely in the nature of their content. Perhaps certain categories should be moved up in importance, those dealing with period-sensitive material. Perhaps DMOZ needs more editors/reviewers than it has now, as things are obviously not getting done in a timely nature.

Take this all with a grain of salt, it's speculation and thoughts only. I think the concept of DMOZ is good, but the execution is slightly lagging - things like huge submission delays, some sites getting in while others don't make it (and they both offer similar services/info). Perhaps if an editor/moderator has differing views, they'd care to reply with them. I'd be interested to hear it from someone on the inside.

Your first problem is your bizarre obsession with submittals.

Submittals do not assume the central position in the cosmological scheme that you suppose: they are merely gingerbread on the fringe of the universe. If all submittals vanished instantly, the ODP would still proceed with its activities leading to its goals. Forget about submittals. The simple fact is, they have no relevance whatsoever in any conceivable measure of relevance or currency.

Now if we really wanted to measure that (as we evidently don't), then what we would measure is the delay between the time a site is first published, and the time it's first reviewed by an editor. Whether or not it's been submitted isn't relevant. If it is there, then it is a fit subject for our review.

Now, if we really wanted currency (as we obviously don't), we'd be very concerned about the sites that are up for six months and then go down. But -- that would be a colossal waste of our time. Remember, what we publish is delayed by further months until it's actually hosted by our licensees. There's no way we're going to get that site into AOL before it's gone, and we just have to take it out again. And in order to achieve that failure, we'd have had to review 4 million sites a year -- to have a directory of one million sites. You may waste your time that way if you want to, but we shall not ask our editors to waste theirs.

So for sites that do not show indications of continuity, everyone is much better off if we deliberately wait 13 months after the site is published, to make sure its sponsor shows some evidence of commitment to provide a continuing service. It has been a proverb for years: "Directories favor large, stable sites." Volunteerism can't change that aspect of the fundamendal laws of information.

Now, how about currency?

It should have been obvious that NO directory can do that, and the Open Directory even less than most (with its built-in lag between creation and general publication). You're confusing the Daily Tattler with the Encyclopedia Galactica just because they're both words printed on recycled vegetable matter. We aim to be more like the encyclopedia, and (like the encyclopedia) we don't take advertisements for puppy giveaways, By the time the next edition is distributed, the puppies will be grown.

Just to put things in perspective: Even a search engine can't NATURALLY be current: it takes a special kind of spidering. Hence the difference between Google News and Google Web. And there are numerous other ways of slicing the internet that the Open Directory can't do.

Intellectual tools aren't in that way different from carpenters' tools. You're whining because your screwdriver does a lousy job of setting rivets. Think of us as just one of Sears Craftsman's suppliers, not as Ronco. And for the tools you really use, buy the one that does the best job at what it does, not the one that when sufficiently overpowered can be made to do a half-baked splintered job of a half-dozen tasks.

Unfortunate, but it is what it is. My own hopes were for a centralized directory of up to date and good info, apparently not to be found here after all. Your analogy of an Encyclopedia is a fairly good one I'd say, except that websites are living breathing entities, with lifespans all their own. Two years is, imho, far too long to let a submission go. I would think at the absolute outside, 9-12 months tops, and even that long on irrelevant or shady prospects - much less for good or timely sites.

As for submissions not being the bread and butter of the directory.. how could you possibly accumulate unbiased information based on a set number of people pursuing their interests on the web, adding in sites they thought were at least partially unique or informative? You would nowhere near cover even 1/10th of what is available, and perhaps miss out on hundreds or thousands of great sites. I suppose that is why you have the "submissions" after all. Too bad they aren't timely, or they'd definitely count for something.

But thank you for replying, even if it was a somewhat shaded reply (meaning you *did* take my original post as argumentative, it was not meant to be - nor is this one). It's good to get the scoop from the inside of the ballpark, so you really know the score.

Just to chime in on what you are saying - if we continue to liken the ODP with an Encyclopedia, your argument would be that how could anyone build an unbiased encyclopedia based on a set number of people whose job it is to compile it, based on the resources that they have at their disposal?

The Encyclopedia Britannica seems to have been doing that quite well for a number of years, so I don't really see that the model falls down when it is applied to an online medium like the ODP.

Does the fact that sites sit and wait for 2 years bother us? yes, of course it does - we would like to have every site that contains good information listed in the ODP just as soon as the site is available. We'd like to change the descriptions as the sites change, so that people can always know what is on the site; we'd like to delist the site the moment the site goes out of business, or changes to be a spammy p0rn redirect.

...and that's where reality comes in. We have a team of trusted editors, and governance process in place to make sure they earn their trust who are doing the best job they can given the situation we work in.

yes it has flaws, but in as far as it goes, it works, too.

I think (on occasion!) that if you look at ODP as a whole, and look at the 2-year comment, you can come to some really false conclusions.

Many, many parts of the directory are very close to being current -- meaning that the editors are active and the wait times are relatively short (a few months, versus a couple of years).

Regional is a good example. It is all about where rather than what, and the Regional editors (who edit in one of the largest parts of the directory) work reall hard at keeping the community listing reasonably up-to-date.

Let's look at the opposite end of the spectrum. Shopping is often an area where the wait times are the greatest. It is also where a lot of the spam is the highest. There is a correlation in there, and many editors simply won't even apply for permissions to edit in shopping. My personal opinion is that this is no great loss in terms of directory quality. So it takes 2 years for yet another merchandise hawker to get listed? Hey, in the same time I might be able to research and build out several new sections of the directory in Society or Recreation.

Think about this: when I go home tonight, rather than fighting submissions and spam, I'm going to add about two dozen summer camps in Arizona that are not listed (I found a really nifty resource while I was at work today). Now that really lights my fire, and I'll feel good when the night is done, and if some Shopping or Computers or Business site sits in the muck for an additional 48 hours it is a good tradeoff. And tomorrow when I go home? When, there is this idea I have for Florida (that I stole from Arizona) and I'm going to work on that and hope that when I have the concept fleshed out my peers will like it.

You see, the day I don't like working on ODP stuff is the day I stop editing. If I want to deal with submissions and spam, well, my wife has a nice honeydoo list for me, and if I get some of that stuff done, I get a really neat reward. If I wade through the spam and submissions, there is only more there tomorrow.

That's what floats my boat.

>As for submissions not being the bread and butter of the directory..
They aren't, I assure you. I build the directory, and I know perfectly well which side of the butter the bread is on, and which side the dust is on.

You still miss the point. In all matters of relevance, submittals are conceptually irrelevant. The unsubmitted and unlisted sites are just as detrimental to our coverage as the submitted but unreviewed sites!

>how could you possibly accumulate unbiased information based on a set number of people pursuing their interests on the web, adding in sites they thought were at least partially unique or informative?

The pragmatic answer is: just watch us. And see if you can find a comparable resource. Just try. The proof is in the eating, and we have a royal feast.

The theoretical answer is: you obviously have never taken a statistics course. How can Harris and Gallup predict the course of an election by only polling a few hundred people? And yet, they generally do a fairly good job, with no more bias than, say, a major metropolitan newspaper shows, and far less than a TV mouthpiece. And the ODP has THOUSANDS of sampled interests!

I don't care much for commercial contempop entertainment; the next editor probably doesn't care for German high baroque counterpoint. I'm not very interested in Hindu mythology; the next editor may not care about Lutheran chorales. I've no interest in casinos; the next editor may not care for national parks. It averages out.

There are institutional biases, of course: We're volunteers, and we're likely to be more interested in volunteerish stuff. That's fine; so does Encyclopedia Britannica. Overture has much the opposite bias; so does USA Today. As a user (surfer), you can be a "customer" of whichever one you want; as a producer of content, you can offer your product to whichever one you think is most likely to be interested in "buying" (listing) it.

And we're more interested in palpably unique sites, which often means building up new categories from nothing. That way we know we're adding obvious value (breadth) to the directory. This is not a bias we need apologize for. A library with books in 10000 DDS categories is MUCH more useful than a comprehensive collection of Harlequin romances. And for nearly all the world, it's even MUCH more useful than, say, the Lowell Mason collection of nineteenth-century hymnals, valuable as that might be for hymnologists.

And topical biases that we have are self-correcting. If someone feels strongly enough that we're missing out on the rise in African Tribal Religions or Recreational Tractor usage -- feels really strongly -- feels strongly enough to be willing to edit such a category ... then they can apply to be an editor. How do you think people get involved in such a project?

A couple of other side points:

As for the ODP shouldering the responsibility for the failure of e-businesses without a marketing clue:

Shrug. Just shrug. I can walk off tonight and never edit another site, and I have abundantly fulfilled my responsibility to the project. And every other editor can say the same thing. So whose are the shoulders that responsibility is on? Go blame it on the universe that owes you a living. And sue it. Good luck serving the papers.

As for game and movie categories being out of date: We have active editors that like to build up new game and movie categories as fast as the industry releases them. In such categories, there are usually NO submittals, buttered or not, and the editor has to rely on Googling, personal knowledge, and gaming community connections to find the sites. And it is in such categories that the ODP typically shines brightest. You can drop your bread and butter in the muck (half of it seems to come directly from there anyway): we let our surfers eat cake!