How long a wait for review is reasonable?

In the FAQ I found the statement "eventually review every suggestion" and wonder whether it is really true in pratical terms.

By in pratical terms I mean while the original suggestion still has value.

Eventually every site will cease to be. The law of thermodynamics tells us that. Things change. People move on etc.

For reviews to be useful they need to happen within a reasonable time.

I suspect everyone would agree with the last statement however I also suspect that ones view on what is a reasonable time will differ from person to person.

I also suspect that the biggest divide on this question will be whether one is an overworked editor or a suggestion submitter.

I would like to see some stats about how long a suggestions takes for review in terms of average time and standard deviation from the mean.

I would also like to conduct a staw poll as to what is considered a reasonable time.

If you are reading this post then you presumably have an opinion. Please add a reply stating it and also state whether you are an editor or a site suggestor or any other factors that might be considered to be biasising your view.

The assumption behind your question seems to be that the volunteers here are providing some kind of service to website owners. They are not.

the subtext is that the volunteers aren't volunteering hard enough to suit your purposes. That's OK too because their purposes aren't yours.

I don't think that a straw poll based upon false premises is particularly useful. Perhaps you should undertake it somewhere else.

Providing a service?

jimnoble said:

The assumption behind your question seems to be that the volunteers here are providing some kind of service to website owners. They are not.

the subtext is that the volunteers aren't volunteering hard enough to suit your purposes. That's OK too because their purposes aren't yours.

I don't think that a straw poll based upon false premises is particularly useful. Perhaps you should undertake it somewhere else.

Click to expand...

It all depends on your definition. ...

The volunteers are doing something that website owners applaude and wish to encourage. As such it's a service. However, if one defines a service as something paid for then it's not a service.

However what one calls it is irrelevent. The fact is that the volunteers are doing something at the suggestion of those who suggest URLs.

As such it seems reasonable that these volunteers would have aims in this regard.

It has been said that a man who aims at nothing will achieve what he aims for.

Click to expand...

So I assume as an editor you have aims. It just unclear to me what they are? I would certainly be interested in understanding them better if you are prepared to enter into a dialogue. This is particularly true because I am considering being an editor myself and it would be helpful to that consideration to have a better understanding of the ethos of this directory.

If I were an editor my ethos would be to make a part of this directory a better and fuller representation of the real world than it currently is. Would that seems to you to be a reasonable ethos?

The fact is that the volunteers are doing something at the suggestion of those who suggest URLs.

Click to expand...

Not a fact at all. In very many categories, editors go out and find websites for themselves.

In some categories, there are very few listing suggestions and so we have to go out and find appropriate websites for ourselves. Yes I do write down URLs from the back of trucks.

In others, there is so much spam in the suggestion pool (99% in some competitive areas) that it's far more efficient to ignore them and, once again, find ones own.

Most editors take great satisfaction in building something good and great pleasure in working within such a supportive team. At the end of the day though, editing is just an absorbing hobby.

The volunteers are doing something that website owners applaude and wish to encourage. As such it's a service. However, if one defines a service as something paid for then it's not a service.

Click to expand...

"A service" implies an obligation on behalf of the provider to do something for the user, something that does not exist between the ODP and the people suggesting sites.

If I were an editor my ethos would be to make a part of this directory a better and fuller representation of the real world than it currently is. Would that seems to you to be a reasonable ethos?

Click to expand...

Yes, but you can do that without ever dipping your toes into the pool of suggested sites.

Totally agree

motsa said:

"A service" implies an obligation on behalf of the provider to do something for the user

Click to expand...

If thats the definition then I totally agree.

I thought I'd better look on dictionary.com just in case my understanding is non standard or another example of the difference between UK and US english.

The first definition amongst others is

an act of helpful activity; help; aid: to do someone a service

Click to expand...

Using that definition and without any extra implications such as obligations I would say that what you editors are doing is helpful and therefore a service.

However I totally agree that there is no obligation involved. I believe that is made clear in the use of the word voluntary.

We're providing a service to our users, to people looking to find sites and sites using our data. Site owners can help us in that task by suggesting their sites for inclusion but that doesn't in turn mean that we are in way providing a service to those site owners. It's an important distinction. Any help provided to site owners is a side-effect of what we're actually doing.

A common interest?

I think motsa makes an important point --- of course it is true that editors don't owe any elevated duty of care to site owners or those who make submissions. But is it not also safe to say that those who create content intended to be useful and dmoz editors have a common interest; that is, ensuring that those who would find the content useful have a means of finding it, free from commercial influence?

While I certainly can try to imagine the frustration that 99% spam in submissions might cause, is it not also true that common interests are best served by working together, to whatever practical extent interests are common?

To that end, I offer the following suggestions and questions for discussion.

1) Obviously submitters should follow posted guidelines to the letter --- a point that always bears repeating.

2) I believe that a source of anxiousness on the part of submitters is not so much the indeterminate length of the review process (an inherent aspect of the volunteer effort), but rather, not knowing if the submission is awaiting review at all (after all, glitches do happen --- submissions can get lost: http://www.resource-zone.com/forum/index.php?showtopic=50192). If there were a way to check on the status of a particular submission (perhaps accessible even from the dmoz homepage) then this might serve to assuage some concern, and maybe even cut down on the incessant re-submissions and status inquiries. Specifically I envision providing every submitter a unique serial number associated to the submission. To check on the status of a submission later, a submitter could then submit this serial number, and then all information associated with the original submission would then be returned to the user (aside from the originally submitted email address, for privacy). This would serve as a confirmation that the submission has not been lost, and could be accompanied by useful information appearing in some of the resource-zone forums (http://www.resource-zone.com/forum/faq.php?faq=how_long for example). In my opinion this would reduce status inquiries as well as the need for repetitious information in the resource-zone forums.

3) Given the increasingly important role of dmoz in search engines, to what extent are dmoz listings becoming a "choke point" in a site's ability to be found? In turn, is it possible that widespread failure to obtain a dmoz listing for otherwise listable sites within a certain timeframe (thus resulting in failure to bring the site's useful content to its intended users while it is still useful) might dampen the fervor for creating useful content in the first place --- especially time sensitive content? Does this second order effect alter the optimal balance of approaches for editing the directory (use of submissions versus other methods of finding sites)?

4) Since anyone can submit a site or request a change to a listing, it is conceivable that a site's competitor could attempt to do so for deleterious effect. While editors are of course clearly capable of dealing with this possibility, is it not of concern that a legitimate submission could be overwritten by an illegitimate one? ("Please note that a new suggestion to the same category overwrites the previous one." at http://www.resource-zone.com/forum/faq.php?faq=faq_site_questions#faq_no_editor) Moreover, in deference to the overwhelming editorial capability of dmoz volunteers, who better to initially suggest a title/description for a useful site then those who labored to make it useful in the first place. For both of these considerations, would it not make sense to at least record which submissions come from site owners (by asking those who claim site ownership to confirm it by posting a resource with a dmoz-generated random string on their site, which the dmoz server would then validate)? Editors could still exercise their discretion in lending credence to site ownership but at least the information would be known, and an owner's submission could not be overwritten by a nefarious one.

I truly appreciate those dmoz editors taking the time to participate in discussions such as these. Ultimately those of us committed to bringing useful information to users (in whatever role) all want the same thing --- a neutral internet, free from commercial bias, but also free from exploitation by opportunists (i.e. spammers) and institutional idiosyncrasies (which can lead to random and/or unintended results).

It would be my pleasure to help with the implementation of any of these suggestions in any capacity. Apologies for the length of the post.

Experience shows that your number 4 isn't really a problem at all. Unfortunately a lot of site owners seem incapable of following the site suggestion guidelines, and even fewer of them manage to write acceptable (by our standards) titles and descriptions for their websites. In tens of thousands of edits, I think I've only found one or two public suggestions that didn't need editing in some way before being published.

It would be my pleasure to help with the implementation of any of these suggestions in any capacity. Apologies for the length of the post.

Click to expand...

The best way you could help the project is by applying to become an editor. From the inside you'll get a much better idea of what the real problems we face are, and perhaps be able to help address some of them. I'm afraid that we're unable to accept any help other than suggesting sites and pointing out bugs and quality control issues from non-editors.

thus resulting in failure to bring the site's useful content to its intended users while it is still useful

Click to expand...

Most worthwhile websites that could become stale if left to their own devices are updated regularly in order to avoid doing so .

We see a fair few abandoned ones of course. I was looking at a prospective listing this morning (part of a bulk suggestion from an editor who'd been data mining for choirs some months ago) who's programme page was headed What's coming up in 2005. I didn't think that our directory would be well served by listing it. AFAICT it was never suggested by the website owner BTW.

communities are built on common interests, regardless of perspective

Full disclosure: I too am a little fustrated at my pending submission (though just 4 months, which seems better than others I read about) and the lack of transparency as to its status.

That out of the way, I have to respectfully disagree with a couple of positions represented in this thread by Mr. Noble....

1) Your quick dismissal of DavidMK's original posting...

DavidMK said:

In the FAQ I found the statement "eventually review every suggestion" and wonder whether it is really true in pratical terms.

Click to expand...

It is a valid question, raised repeatedly in some form or another in these forums. And within the "... > General Discussion > Suggesting a Site" forum, it seems very appropriate and relevant.

Though I cannot speak for DavidMK, I don't believe that any assumption or subtext was implied and an open dialogue would serve to improve the overall value and service provided by the ODP and more specifically, the volunteers which make it up, than to dismiss it so rashly.

I understand the large burden on the editors, but I find that all members of any community, whether they be merchants or consumers, are dependent on each other. So to discount feedback, or be exessively punitive, just because it represents the merchant's perspective, seems to me to miss the true point of all communities, which are built on mutually benefitial relationships and interests, regardless of perspective.

2)

jimnoble said:

In others, there is so much spam in the suggestion pool (99% in some competitive areas) that it's far more efficient to ignore them and, once again, find ones own.

Click to expand...

To quote three key aspects of the ODP (including DavidMKs opening line)...

"...largest, most comprehensive human-edited directory..."

"The web continues to grow... commercial directory sites can't keep up with submissions.... Link rot is setting in..."

"...eventually review every suggestion..."

In particular, your comment goes directly against the later.

It seems to me that for an editor to seek and add useful sites is a great value add which differentiates the ODP and has made it the leading directory. However, a wholesale avoidance of submissions based on quantity is just as likely to hurt ODP moving forward, as it likely misses relevant sites along with spam; especially in unedited categories, of which there are many. This is a disservice to surfers as it represents exactly the clutter they are trying to cut through.

This is further accentuated by the relevance that ODP data plays in search engines, which does not seem to be lost by its members; that is, "The Internet Brain".

Lastly, some of the other suggesstions posted in this thread seem very reasonable and would undeniably cut down on the "spam", so I don't understand why the constructive feedback would be dismissed so easily.

However, a wholesale avoidance of submissions based on quantity is just as likely to hurt ODP moving forward, as it likely misses relevant sites along with spam....

Click to expand...

It's only a small proportion of our categories that are spam magnets. I hope I didn't give the impression that most category editors ignore suggested websites because I don't believe that's true.

Let's for the moment look at things from a surfer's perspective. If I want to buy a blue widget, go to the Blue_Widget category and find 20 sellers, my needs have been met. There might well be many other sellers that aren't listed but that doesn't particularly matter to me.

Now look at things from the editor's perspective. First and foremost, editing is a hobby - something done for fun/pleasure/satisfaction. There's little of that to be had digging through a pile of spam hoping to find a pearl - particularly if the category already has a goodly number of listings. However, i notice that the nearby Green_Gadget category only has a couple of listings and there aren't any awaiting listing suggestions. Finding some more and making the category a better resource is very satisfying. It also better serves the surfer.

We have finite resources here. If you were a surfer, or an editor for that matter, would you really prefer us to give priority to the Blue_Wigets over the Green_gadgets?

...especially in unedited categories, of which there are many

Click to expand...

I'd dare to say that there are none - it's just that they aren't edited as often as you or I might prefer. That's not recalcitrance; it's a matter of resources .

Unusually, I've spent the last week focused on working at the editorface in an English county which doesn't seem to have attracted much effort in the last year. Over 500 listing suggestions and queries thrown up by our QA tools have been examined and most processed to completion. The remainder are now in their correct categories (They were suggested to the wrong ones) and just need final processing. During the exercise, the structural integrity of the county has been checked and fixed where needed. I've also noted where we seem to need some new locality categories. Later, I'll create them and populate them by finding suitable websites for them. So far, it's been a very satisfying exercise. When I'm done some time next week, the surfer should have a better experience there too.

I hope that the above gives you some idea as to where I'm coming from .

Did you notice that DavidMK is now an editor?

Becoming an editor

jimnoble said:

Did you notice that DavidMK is now an editor?

Click to expand...

I hadn't, thank you for pointing that out. Also, I appreciate your point and applaud your effort.

I too considered to become an editor, however, while I would never try to adversely affect a competitor's submission, or even avoid approving it, I can't in good consious say that I would go out and find my best competitor's websites and add them to the directory.

So you see the true point of my personal fustration and likely that of other site submitters... the categories aren't necessarily in bad shape, just missing a few tweaks, of which we believe our site is one. So it hurts to have all of your competitors there and you're on the outside looking in, especially when you're the new kid on the block. Hence my earlier point that site owners are a valid constituency too, spammers aside.

I too considered to become an editor, however, while I would never try to adversely affect a competitor's submission, or even avoid approving it, I can't in good consious say that I would go out and find my best competitor's websites and add them to the directory.

Click to expand...

Thank you for being honest enough to admit that. If you were to consider becoming an editor, you could always pick a category where you wouldn't have that problem, for example, the category for a favourite hobby or the town you grew up in.