Feature request: Bring back the automated status checker.

I've just been reading through archive material about the discontinuation of the automated status checking service. As a relative newcomer to DMOZ I have not had the chance to experience this service. However, I guess that applicants filled in an html form, indicating which site submission they were inquiring about. In response to submitting the form, some simple automated routine checked a database of submitted URLs and returned one of a few possible responses such as:

- no record of it ever being submitted.
- a record of it being submitted (with details) but status is: currently not reviewed, please check back later.
- a record of it being submitted, being reviewed and the outcome of that review (inclusion/non-inclusion in the directory).

It sounds like the kind of system which would eliminate the seemingly large volume of forum-based requests to DMOZ volunteers, asking repetitive questions about whether sites were submitted and if so, what their submission status is. It shouldn't be hard to design, operate and maintain such a system (a simple query routine plus a trivially small database containing nothing but submit URL strings, current status flags and possibly authentication details for site submitters).

The arguments for not doing this just seem very flimsy.

"...not enough resources". With a large, free, volunteer workforce and most of the data required already being held within the DMOZ server systems, this argument doesn't stand up to scrutiny. Most newcomer web developers could write the HTML and server-side code to authenticate the service users, filter out automated requestors, and then query the database and produce a response. Technical resources should not be a genuine limitation.

"...it might help spammers". How? It's difficult to see what leverage a spammer could gain from knowing that their submission is still in an unreviewed state or not. For genuine webmasters, the service would offer them peace of mind and acknowledgement. For a spammer? To them I would presume they only care whether a site is listed or not. If its not, they'll keep on spamming. If it is... ...they'll probably keep on spamming.


With DMOZ operating largely on good will, the reasons for putting this system in place are pretty obvious. Most potential category editors’ first contact with DMOZ will be as people submitting URLs for review. Making them unnecessarily frustrated with DMOZ in their first points of contact is hardly a great start to encouraging them to volunteer their time to help. By putting the feedback service back in place, this frustration and discontent is eliminated. DMOZ reputation has one less thing to sully it and encourages new volunteers, plus: webmasters get the reassurance and acknowledgement they seek. Everyone wins.

So can this service be brought back please?

There never has been an automated status checker, so we can't bring it back.

You seem to have seen the FAQ, but other readers of this thread might not. So please see also FAQ: Why don't you provide an automatic status reporting system for URL listing suggestions?

"...not enough resources". With a large, free, volunteer workforce and most of the data required already being held within the DMOZ server systems, this argument doesn't stand up to scrutiny. ... Technical resources should not be a genuine limitation.

Click to expand...

Unfortunately, those with access to the core systems are rather small in number, and rather busy with other more important things. I suppose tht any editor could in theory produce a tool to achieve this. Don't you think it says something that no-one (to my knowledge) has attempted to do so.

"...it might help spammers". How?

Click to expand...

Trust me it would. Unfortunately going in to the details would probably help some of the less intelligent spammers too. So you'll have to either take my word for it, or think a bit harder.

Yes it would be a nice thing to have for legitimate submitters, but would it really provide any truely useful / actionable information for them? You can already tell if your site is listable by consulting our editing guidelines. If you've suggested a listable site, and it's not yet listed, you can assume it's waiting to be reviewed and get on with your life. If the site's not listable, then you should never have suggested it in the first place.

I'm afriad that from an editors point of view, the negatives out-weight the positives, so I don't think what you're suggesting is likely to happen any time soo.

If you really want to help the project, why don't you volunteer to become an editor. Then you might get a feel for what the real problems we're facing are...

Hi Chaos127,

I think I can relate to what you're saying, though I don't pretend to grasp the finer detail of the spammer-related implications of an automated status checker. I don't buy the "Don't you think it says something that no-one has..." argument though. If you start out thinking along those lines, you might as well give up thinking altogether and assume someone else has probably already done the thinking for you.

And yes - I do plan to do what you suggest: as soon as the 'apply to be an editor' scripts are back online, I'm planning to do just what you suggest.

I kind of have interests in DMOZ from both sides of the fence: On one side, I'd like to see a commercial site of mine be reviewed and listed in a suitable and relevant category. On the other side, I want to to do my bit to support sport in my area and hopefully help local clubs get their sites listed if they're relevant and maybe encourage a few more people to take up a sport.

I'm currently experiencing 'DMOZ frustration' in the form of inoperable editor application links (currently just giving 404 page not found errors) but I'll stick at it. This all too common mantra of "be patient, it'll work in time" is one I'm coming to understand in the DMOZ arena! Just hope that my editor application is reviewed while I still have an interest in the subject matter. Otherwise it'll add to the timewasting pile along with all those webmasters periodic checks to DMOZ to see if there's been any progress on their listings. Maybe some kind of automated... (only kidding!)


Apologies to pvgool - I did find some reference to an "automated status checker" in the archives but cannot trace my path to it now. Midway through someone's post, someone added a short note to the effect that there had once been an automated status checker with forms that submitters filled in to check status - but that this had been pulled (In 2005, I believe). If only I had a search tool to find it again... (!) If I do find the link again, I'll post it here.

Midway through someone's post, someone added a short note to the effect that there had once been an automated status checker with forms that submitters filled in to check status - but that this had been pulled (In 2005, I believe). If only I had a search tool to find it again... (!)

Click to expand...

No, there has never been anything like that. The only ODP element that has a automatic status check is the public abuse reporting system. We *did* stop doing manual status checks for people here in this forum in May of 2005 so maybe that's what you read.

I don't buy the "Don't you think it says something that no-one has..." argument though. If you start out thinking along those lines, you might as well give up thinking altogether and assume someone else has probably already done the thinking for you.

Click to expand...

I think you're missing the point of what I was saying: in this case it is perfectly reaonsble to assume that others (with more information than you) have already thought about this, and deduce that there must be a good reason (even if you can't see it at the moment) why it hasn't been done.

As you can see by searching this form, the idea of status checks (automated or otherwise) is hardly new. You just said how easy you thought it would be for someone to code such a solution. It's public knowledge how many editors there have been, and a reasonable assumption that more than a handfull of them are techncially competant enough per your assertion.

Hi chaos127,

I've no doubt that many of the editors working on the DMOZ are technically very competent and I genuinely have a great respect for their abilities. In my own commercial organisation many of the staff are highly competent too but the combination of numerous staff and highly competent staff does not neccesarily mean that the best practice is always performed. Its a badly flawed assumption to think otherwise.

The fact that there are so many people working on behalf of DMOZ means that it is extremely easy to assume that someone else has thought a recurring problem through, when the reality is that everyone may simply be following the wisdom of the first person to express an opinion on the matter.

Presumably, core site development work for DMOZ is not carried out or decided upon by the general volunteer force. The actual number of people involved in performing or deciding upon such an upgrade is relatively small. (Correct me if there is a wider panel of decision makers for core development work.) The present ongoing downtime of the cgi scripts on the site, now at least 3 days old, suggests the availability of competent developers is a lot more restrictive than many outsiders would believe.

(By the way, the emphasis there was on the 'availability' of DMOZ's core developers, not on their competency. In text, it's hard to put emphasis in the right places in sentances.)

If the number of developers with direct programming access to core systems is limited, and the amount of time they can commit to debating/decision making, then the whole argument about '...others (with more information than you) have already thought about this...' is a bit lame.

Courtesy E-mails
Going slightly off-topic but taking on board many of the valuable comments people have made in this one, I don't understand the aversion to releasing an automated courtesty confirmation e-mail to applicants. Whilst I can acknowledge (but am unconvinced of) the potential for spammers to make use of data about site status, the production of a simple confirmation e-mail for applicants would be a good feature for the people involved in using this system. The best argument against it I've heard in other threads so far goes something like "We don't care about webmasters" suggesting that they're not part of the wider community which DMOZ serves. Or, (worse), "they / their interests are irrelevant" (followed by the sound of power-mad maniacal laughter!).

So.

Any fors / againsts an e-mail confirmation feature?

I'm open minded on this - if there's a deluge of people responding with excellent reasons not to do it, I'll eat humble pie and accept that its a poor idea.

Otherwise, who are the movers and shakers who can actually get it actioned?

monkeyhanger said:

The best argument against it I've heard in other threads so far goes something like "We don't care about webmasters" suggesting that they're not part of the wider community which DMOZ serves.

Click to expand...

That is correct. In their role as webmaster / websiteowner they are not part of the community which DMOZ serves. In their role as a person searching for information they ofcourse are part of that community.

So.

Any fors / againsts an e-mail confirmation feature?

I'm open minded on this - if there's a deluge of people responding with excellent reasons not to do it, I'll eat humble pie and accept that its a poor idea.

Otherwise, who are the movers and shakers who can actually get it actioned?

Click to expand...

There's really nothing for you to be open (or closed) minded about. You've made your suggestion and we thank you for that. But that's all you can do. Debate here about the merits for or against any given suggestion really isn't going to help because, ultimately, the decision to implement or not implement a suggestion lies with the editorial community and ODP staff. Pretending otherwise will likely just lead to unreal expectations. I say that not to be rude but to be realistic.

motsa said:

Debate here about the merits for or against any given suggestion really isn't going to help because, ultimately, the decision to implement or not implement a suggestion lies with the editorial community and ODP staff.

Click to expand...

Ah, perhaps I misunderstood the purpose of this forum. I read the 'bugs and features title' and took it to mean this was a forum for users to report bugs or request features. If I understand you correctly, the forum is actually for people with administrative priveleges to advise that there are bugs or have made available new features.

Does anyone know of a forum for actually requesting and discussing new features or is that avenue simply not open to non-priveledged DMOZ users?


By the way, I thought the community which DMOZ serves included pretty much everyone without prejudice. You would have thought that webmasters, who are the people which are 100% responsible for the content which DMOZ deals with every day might be included in this. You never know, you might even find a few webmasters working within the category editor community - it's amazing how many editor's profiles have links to their own personal or commercial sites...

I stand corrected!

monkeyhanger said:

Does anyone know of a forum for actually requesting and discussing new features or is that avenue simply not open to non-priveledged DMOZ users?

Click to expand...

As motsa wrote "the decision to implement or not implement a suggestion lies with the editorial community and ODP staff". That means the discussion (which has happened already many times) is only open for editors.

monkeyhanger said:

By the way, I thought the community which DMOZ serves included pretty much everyone without prejudice. You would have thought that webmasters, who are the people which are 100% responsible for the content which DMOZ deals with every day might be included in this. You never know, you might even find a few webmasters working within the category editor community - it's amazing how many editor's profiles have links to their own personal or commercial sites...

Click to expand...

We don't have anything against webmasters. We just don't offer any service to them. People can have many roles in their live, in their role as a DMOZ editor they have to forget anything they would be doing in their role as a webmaster. If people can't make this separation they wouldn't make good editors. Luckely the meta editors are very good in filtering the people who only want to become an editor to list their own sites.

Ah, perhaps I misunderstood the purpose of this forum. I read the 'bugs and features title' and took it to mean this was a forum for users to report bugs or request features. If I understand you correctly, the forum is actually for people with administrative priveleges to advise that there are bugs or have made available new features.

Click to expand...

You've misread my post. Requesting features is not the same thing as discussing or debating requested features.

Does anyone know of a forum for actually requesting and discussing new features or is that avenue simply not open to non-priveledged DMOZ users?

Click to expand...

We have no control over what people discuss in other forums, rightly so. But discussions of this nature generally take place inside the editorial community. Keep in mind that you are not suggesting something new, something that has never been discussed before, here. We've been here before.

Hi motsa,

Thanks for your comments. I can see how this point has probably been raised again and again and how you're probably all getting a bit tired with being asked for the same features.

If nothing else, I hope my posts and those with similar requests before me brings it home to you guys that there remain features which there is an ongoing demand for, which DMOZ has failed to provide. For every person like me who makes the effort to raise the point, there's a thousand more who are quietly getting pissed off about it without joining a forum to say so. As a result, there are many people who's first (and as a result, only) experience of DMOZ leaves them feeling somewhat dissapointed in the lack of what is really only relatively basic courtesy feedback.

For an organisation that already has less volunteers working for it than would be ideal for the size of task it faces, maybe DMOZ's public reputation is something that it can no longer afford to dismiss so lightly.

Anyway, I've made these service requests. If the current wisdom is that they are not worth pursuing, sad though that is, so be it. I think you've given me some insight into the prevailing DMOZ mentality and hope that the wider distribution of DMOZ volunteers are as poistive and open-minded, if not moreso.


My best regards to all who've contributed to this thread. Keep up the good work and don't be afraid to re-evaluate and change the status quo (no, not the group!) every so often.

We haven't just been in DISCUSSIONS before (although that is altogether too true.) We've been in the activity of offering site suggestion status before. One of the purposes of our activity was to determine (1) what could be done, (2) how it should be done, (3) what were the advantages, and (4) what were the disadvantages. We got our answers out of the crucible of experience, so your speculation about what those answers might have been, are really of no interest or value. We Know What The Real Answers Were.

That's what the decision was based on.

Demand isn't a concern. Need is a concern. And basically, no honest person needs what you demand.

...features which there is an ongoing demand for, which DMOZ has failed to provide.

Click to expand...

There may well be a demand for such a feature, but it's not coming from a group that the ODP is trying to serve. We are here to produce a useful directory for surfers and other downstream data users. We're not here to meet the needs / wishes / expectations or webmasters. Apart from perhaps a slight PR benefit, implementing such a system wouldn't really help further the aims of the project.

Yes, we might get a bit less bad press, but we'd also get more people screaming when they knew we'd definitely rejected their affiliate-spam-unlistable-rubbish-whatever. We may get a few more editors as a result, but are most of the people moaning they can't get their sites in at the moment, really the non-self-interested people we want as editors? I'm sorry but the benefits to the project are minimal, then risk of helping spammers is real, and given the ODP data structures implementing an automated system would probably not be trivial. We've already tried a manual version, and you can see the results for yourself in the forum archive.