Responding to Submitters

It was a staff decision.

Click to expand...

I'm curious... has the staff discussion and decision been posted anywhere to be viewed?

from the ODP Social Contract
3. We Don't Hide Our Official Editorial Policies

We will keep all official ODP editorial guidelines and policies open for public view at all times.

Click to expand...

I realize that this may not be considered an official guideline or policy, but making the information public would fall under the spirit of the contract.

Again, I must express my surprise at the opposition... and please note that I, personally, have absolutely nothing to gain from this. I simply find it odd enough to comment on.

Why would it be posted? This doesn't fall under the guise of editorial policies or guidelines. The only things that need to be posted anywhere are things that affect a person's ability to use the ODP, be that suggesting a site, becoming or being an editor, using the dmoz.org site, or using the RDF data. There is absolutely no reason for staff to issue an official edict explaining why they might decide not to not pursue a suggestion offered by a member of the public.

>I have been surprised by the strong opposition to a suggestion that would almost certainly improve the directory.

If you had followed the analysis given, you would have seen exactly how this would have certainly harmed the directory. And that is why editors are opposed to it. Not because they don't like quizzes -- quite the contrary! (Everytime the idea comes up, some editors go off and enthusiastically create them. I've done it myself.) But because, at the root, we don't want to do anything to cut off the casual submitter who is willing to help a little, and who in our experience is not really hurting us at all.

Professional submitters often have the impression that they are the center of the earth to us: but the fact is, it is the tidbits of unique, precious knowledge possessed by those casual submitters that are the primary justification for the public editor interface.

no need to quote here, just refer back a couple postings...

The lack of any type of filtering process on URL submissions does indeed affect a person's ability to suggest a site (ok, not the ability to suggest it, but the timeliness and probabilty of it being reviewed).

As I had said, maybe not an official guideline or policy, but certainly within the spirit of the "social contract," both in its title and its content.

Absolutely no reason? How about good will; good public relations; openness; common courtesy... just to name a few.

I have followed the analysis given, and none of it has given me the impression the directory would be harmed, let alone defined exactly how.

The idea that Joe Surfer would be unable or unwilling to verify that he read a couple paragraphs of information about the submittal process just doesn't ring true to me. In fact, the "casual submitter's" desire to suggest a site to this directory may indicate that he or she already has some appreciation for the ODP, and would likely welcome any step that would aid in getting the site, which he or she must feel to be of value, listed in the directory.

To state the potential benefits once again, a simple verification process, as I have suggested, could:

• Inhibit automated submissions

• Deter the "Professional submitters"

• Reduce editor time and effort needed to slog through garbage submissions

• Reduce the frustration on the part of the editors in the review process

• Reduce the frustration on the part of the editors in responding to requests for status of submissions

• Reduce the delay between submission and inclusion in the directory

• Reduce the frustration on the part of the (non-spam and non-professional) submitters

Or, perhaps I have been mislead by some of the postings from various editors in this and other forums... Is it actually the case that the vast majority of suggested URLs are from casual submitters, and the frequently long delay before approval and listing is simply due to the incredible volume of high-quality sites being suggested?

but the fact is, it is the tidbits of unique, precious knowledge possessed by those casual submitters that are the primary justification for the public editor interface.

Click to expand...

And IMO, the proof of this is in the number of quirky little Geocities-type sites we list.

Joe and Sally surfer can create some wonderfully intersting websites using free hosting services, that they create out of love for a given topic. They never think to submit them, so we love to go find them.

I am completely baffled as to how a professional submitter proposes a solution for his own benefit while simultaneously asserting that all OTHER professional submitters would be deterred.

As for the "garbage" submittals are not the ones from the public--the public is kind and helpful to us. We prize their input beyond jewels! Even their excrement grows prize-winning roses. The true toxic waste comes exclusively professional affiliate spammers -- the kind of people that submit every little affiliate banner farm every month -- the people that memorize the answers to the questions after the first hundred submittals -- the kind that had their Russian-mafia cracker friends write automated submittal programs that got past Altavista's security systems -- the kind that we really really want to be frustrated by our every word and action.

As for a quiz "reducing the level of frustration" -- sir, I perceive that you are no user interface engineer!

As for "reducing the frustration of editors in responding to status requests" -- Not a problem. we get frustrated, we do something else.

The issue has been discussed in the internal forums, at length, multiple times. Based on the editors' experience with reality, it is not a viable idea in our context. Even in the Zeal context, it has contributed to a product that IMO is demonstrably not superior to the ODP in editor or surfer satisfaction. But your mileage may differ, and we will not be offended if you feel Zeal's approach is better, or even if you give them your patronage, based on that opinion.

In fact, there are thousands of other sites out there with whom you can share goals and methods; exchange money, admiration, and promotions; and constructively share criticism based on experential and theoretical knowledge. And on the other hand, there are thousands of people who share the ODP goals, appreciate its approach, and work for its improvement. Where there is competition, the consumer has a choice -- and that's good. Where there is contradiction, then anyone who wants to have a voice can start a website; anyone who wants to have the background experience that makes that voice worth hearing can start a project with other people with similar goals and ideas.

Absolutely no reason? How about good will; good public relations; openness; common courtesy... just to name a few.

Click to expand...

So, if you write to Yahoo saying that you think they should make their colour scheme purple and yellow, do you really think they should issue an official proclamation stating why they won't do that, thankyouverymuch? I don't understand why you would think any organization should feel an obligation to announce, let alone explain, why they are declining to use a suggestion. You've made your suggestion; it's hardly appropriate to attempt to dictate how we respond to said suggestion, officially or otherwise.

>Is it actually the case that the vast majority of suggested URLs are from casual submitters, and the frequently long delay before approval and listing is simply due to the incredible volume of high-quality sites being suggested?

No, the slight majority of submittals are pure toxic waste rancid spam from professional submitters. The direct (*1) and indirect (*2) costs of that are the main (and indeed, almost the sole) cause of the delay in reviewing sites, as many editors have said in many different ways.

Of the other sites, I don't know that there's a preponderance of any one kind. There are lots of small but businesses (legitimate businesses with people that are busy providing goods and services, I mean, not fright masks for affiliate drop-ship programs or even trollers for financial predators!) and lots of sites of personal, artistic, community, or academic interest -- really, the full gamut of aspects of being human. (The people who see the web as only or primarily an economic phenomenon are dangerously socially maladjusted!) And the ODP represents all those aspects.

Footnote 1: the direct effect is that more than half our time is wasted reviewing the affiliate/lead generator/dropship spammers.

Footnote 2: The indirect effects are more complex. Editors avoid "spam magnets" because they can review 50 sites in a row without finding a single one that would contribute a quarter-byte of information to the category: and we feel like we're wasting our time searching for haute cuisine in a toxic waste dump. And even when they review such sites, they have to do it from this viewpoint: "I know 99.9% of these sites are spam. And I know the spammers are malicious, deceptive, devious jerks with no productive activity to engage in -- and in fact nothing to do with their time but to conceal their deceptions. So how deeply do I have to investigate this site before deciding it is legitimate, as opposed to merely more cleverly disguised spam?" Even the legitimate sites suffer from this: and even the most experienced editors are often afraid to actually list a site: after twenty minutes of careful investigation, all that can be safely said is, "it doesn't seem to be one of the normal scams." So legitimate sites take much longer to review, wait longer for an editor willing to review them, and sometimes don't get listed even after a careful review -- because of editor's concerns.

That's the problem we really face. And we know exactly what these "professional spammers" are willing to do, and what the really malicious ones can do automatically. That's how we can be so confident that the proposal you make will not address any problem we have. (And, since you had not yet gotten that message, and had an unrealistic idea of the actual problems, it's easy to see why you'd go haring after a will o'the wisp rather than proposing anything that would actually help.

That's also why editors spend so much time trying to explain how things actually work now. Once you've comprehended this note, you can go back over the forum thread, and see that there's nothing in this post that hasn't been said three times before -- but you were too busy evaluating emotions and counting votes and attributing motives, and overlooked the simple facts.

motsa: Are you equating an actively discussed suggestion on improving the site submission process for (and thus the overall quality of) the ODP to a color-scheme suggestion?

As I am sure that is not the case, I will address your comments.

Suppose there was a similar discussion about improving some aspect of Yahoo, along with thousands of comments from users (I base this on the fact that there are 10,000+ threads in the “Site Submission Status” forum) related to that aspect. Suppose further that the Yahoo equivalent of an ODP editor had stated that the staff had discussed the issue and made a decision against the suggestion. Were that the case, then yes, I would expect a public comment from the staff at Yahoo, even though Yahoo does not tout a “Social Contract with the Web Community.”

it's hardly appropriate to attempt to dictate how we respond to said suggestion

Click to expand...

Well, I did not realize that is what I was doing. Another editor had stated that a decision had been made by the staff, and it had been made clear that quizzes will not be attached to ODP submissions. That editor either made up those statements, or he had knowledge that there was in fact a discussion (hopefully a thoughtful, rational one) that resulted in a decision. Choosing to believe the latter, I asked, quite non-confrontationally, if that discussion and decision had been posted for public view. Not exactly what I would call an attempt to “dictate” the format of a response. In your reply to my question posed to the other editor you asked why it would be posted and stated that there is “absolutely no reason” for a public comment from the staff, to which I responded with several reasons which I considered valid, as to why I thought it appropriate. I see no grounds for a claim that I am attempting to dictate how the ODP should respond.

You quoted part of my message, so I will assume that you read it, and evidently you do not feel that my reasoning was valid. Fair enough. Our interpretations of the spirit of the ODP’s Social Contract, specifically article 3, obviously differ. I do feel, however, that the sardonic nature of your reply was out of line.

along with thousands of comments from users (I base this on the fact that there are 10,000+ threads in the “Site Submission Status ...

emphasis added

Click to expand...

That illustrates an all too common misconception based, I strongly suspect, on wishful thinking. People who submit suggestions to promote sites are not our users, they are our suppliers. We may pick and choose among their products according to our conception of what our users want. Our users are web surfers.

I think I'd prefer one of those "type the six letters in this blurry photograph into the box" quizzes to a trivia quiz about the ODP. It wouldn't take long for a spammer to memorize the six correct answers to the quiz and quickly input them every time, but it *would* be a real pain for them to have to look at the letters and enter them each and every time.

But yeah, though it's frustrating that so many people don't understand basic things about the ODP like "We're not Google" and "Our customers are searchers, not website owners" and "We cannot predict the time it will take until any given site is reviewed by us," it doesn't seem fair to force someone to memorize and repeat that information before we'll let them help us by suggesting a site. Boy, people think we're fascist elitist ingrates already. That sure wouldn't help matters. (-:

Posted by: hutcheson
we know exactly what these "professional spammers" are willing to do, and what the really malicious ones can do automatically. That's how we can be so confident that the proposal you make will not address any problem we have.

Click to expand...

This type of response, perhaps including “we have tried various methods and/or researched similar attempts by other sites, and have concluded that…” would have been excellent. Much more informative and productive than the “not on smarts or test-taking ability of the submitters” and “preposterous in the extreme” type of comments, along with sarcastic and sardonic “example questions” that initially surfaced (from other editors, not from yourself).

As to your specific comment:

I am completely baffled as to how a professional submitter proposes a solution for his own benefit while simultaneously asserting that all OTHER professional submitters would be deterred.

Click to expand...

It appears that you believe, along with other posters here, that I am a “professional submitter” trying to further my own agenda. This could not be further from the truth. While it is likely that nothing I say could convince you otherwise, I will say it anyway.

When I first found the ODP, roughly three months ago, I was, in fact, “Joe Surfer.” I submitted (suggested) a site, and checked back to see the subsequent listing. After noting that the category had been updated several times, yet the site I suggested was not yet listed, I re-read the guidelines and FAQs and found the Site Status forum. That is where I began to get the impression that the ODP is deluged with a backlog of submissions, to the point where it can take months or even years to get a site reviewed. Heading over to the Becoming an Editor forum, I noticed a great deal of frustration on the part of some individuals whose editor applications were being denied.

I went back and again re-read the submission guidelines, and even read through many of the editor guidelines and resource pages. Everything in those documents (most specifically the description of GreenBusters) indicated that the ODP is built as a process of: URL submission (whether by promoters, webmasters, business owners or Joe Surfer), review by an editor, and eventual acceptance or rejection of the suggested site. So it struck me as odd that some people were having such difficulty being accepted as editors while the backlog of submitted sites continued to grow.

This prompted me to post in these forums. In my first message, I stated upfront that I had submitted a site. I emphasized the fact that it was a non-commercial site, not because I thought it should receive priority over a commercial site, but rather to indicate that I was not a typical submitter whining about how I needed my URL listed to drive customers my way.

I quickly learned from the responses to my post that the suggested URLs are only one source used by the editors, and are not even a priority. This cleared up the misconception I had about the process, based on my reading of the submission and editor guideline pages. What followed was, I thought, a reasonable suggestion to be upfront with this information during the URL submission process.

I reiterated this thought earlier in this thread, along with a suggestion, not of an intelligence or test-taking ability measuring quiz, but rather of a few simple questions to verify that the submitter had read the information. Seemed like a good idea to me at the time. The responses have now indicated that while some editors agree it is not necessarily an absurd idea in principal, it has been dismissed and is unlikely to be reconsidered. I do hope, though, that this applies only to the verification questions, and not also to the first part of my suggestion. I would definitely be disappointed to think that the staff is content with the inaccurate (and, quite frankly, misleading, in my opinion) information currently being provided.

Again, I do not expect you to believe me when I say that I am not a “professional submitter” and that I have nothing to gain here. However, if you look at what I have posted on this thread, as well as my original discussion here: http://resource-zone.com/forum/index.php?showtopic=17622 perhaps you (and the other editors who have responded to my posts) might view my comments in a slightly different light.

Thanks, Don

Deleted post because this wasn't the right place for it.

I would definitely be disappointed to think that the staff is content with the inaccurate (and, quite frankly, misleading, in my opinion) information currently being provided.

Click to expand...

I would hope, since you have a pretty good investment in time to this point, that you would be kind enough to tell us (please be explicit) where you feel the guidelines and other ODP-published information is inaccurate or misleading. Sometimes we can read things over and over again, and not really see what we are reading.

In doing this you could be performing a real service to your fellow surfers. I'm not saying we will agree with your all of observations and run right out to make changes, but I can guarantee a fair reading, and if experience counts for anything, some/all of the items you point out would likely lead to internal discussions (we are very consensus driven). When others have pointed out such inconsistencies in our published materials, those observations have received very serious attention.

Now to a personal matter.

Since the "humorous" quiz suggestions may have touched a bit of a nerve, and I was one of the posters, let me apologize. Strange, sometimes sarcastic humor sometimes breaks out here, and what is hilarous on this side of the keyboard can sometimes prove offensive or demeaning on the other side of the keyboard. Please accept my word that no offense to you was intended.

Don, I don't think you are a professional submitter but the rest of hutcheson's post was excellent and should have given you a better idea of the real problem the ODP has in regards to the bulging pool of unreviewed site suggestions. Implementing a quiz will not help that particular problem. Personally, I would like to see something like you purpose, just because it might help joe-user do a better job. I think it's very little to ask that a submitter actually read the guidelines. A small quiz would force them to do the very thing the ODP asks them to do before submitting. How well it would work or how much it might detour joe-user is unknown. Apparently the staff felt it wasn't worth exploring further or the negatives outweighed the positive.

This part of hutcheson's post is sooooo true!

Footnote 2: The indirect effects are more complex. Editors avoid "spam magnets" because they can review 50 sites in a row without finding a single one that would contribute a quarter-byte of information to the category: and we feel like we're wasting our time searching for haute cuisine in a toxic waste dump. And even when they review such sites, they have to do it from this viewpoint: "I know 99.9% of these sites are spam. And I know the spammers are malicious, deceptive, devious jerks with no productive activity to engage in -- and in fact nothing to do with their time but to conceal their deceptions. So how deeply do I have to investigate this site before deciding it is legitimate, as opposed to merely more cleverly disguised spam?" Even the legitimate sites suffer from this: and even the most experienced editors are often afraid to actually list a site: after twenty minutes of careful investigation, all that can be safely said is, "it doesn't seem to be one of the normal scams." So legitimate sites take much longer to review, wait longer for an editor willing to review them, and sometimes don't get listed even after a careful review -- because of editor's concerns.

Click to expand...

99% of all the suggestions I get in one of the cats I edit is spam and 90% of those are repeats. I have a dozen or so just setting there because I'm unsure of their legitimacy, even after reviewing them more than one time. Some or all may be legit but I will not add them until I'm sure and I may never be sure.

As far as all the rejection of those applying to be an editor is concerned, its really not that hard to become an editor. There are new editors added every day. Those that were rejected did not understand the guidelines and/or applied to a cat that's too large for new editors, just to name a few possible errors. They might think they understood and did everything right, but they didn't. That becomes evident when one finally gets in and realizes the mistakes they made originally. And you have to understand the majority of those that are approved do not come in here and announce it.

I'm sure you are only trying to help but the best way to do that is to apply to be an editor yourself. After you have some experience, then you can make helpful suggestions based on that experience. The internal flora is buzzing most of the time and you might be surprised at what actually happens in editor land and the tools the editors have at their disposal.

As the one who suggested the quiz, let me say that I am sorry that I did and at the same time, happy that I did. I’m sorry in that it was not really a solution to the problem. That’s because I did not really understand the problem. I had seen all submitters as one big group.

I’m happy in that the true issue seems to have surfaced. That is, spammers. I won’t put all professional submitters in that group because I assume that there are at least some "professional" submitters. And, I don’t think that professional versus non-professional should be the criteria anyway. If Joe Surfer spams the ODP it’s just as harmful as a professional spamming it.

Understanding the situation better now, I retract my quiz suggestion (with a degree of embarrassment). But I agree that it would be great on bobrat’s dmoz for dummies site.

Preface 1: Any practical method of reducing the amount of spam and otherwise abusive submissions would be welcomed by the staff of the ODP, and likely implemented forthwith.

Preface 2: All currently suggested methods of accomplishing this, including but not limited to any type of quiz being attached to the submission process, have been thoroughly discussed and investigated by the staff, and have been deemed impractical, unreliable and / or too much of a hindrance to Joe Surfer’s ability to suggest a URL.

Preface 3: Statements posted within this thread supporting prefaces 1 and 2 are taken at face value and accepted as being true, with no further documentation necessary.

All who agree with the above, and feel that no additional discussion on this topic within this thread is needed, say “Aye.” Well, please don’t submit a bunch of posts saying “Aye.” Shouting “Aye” at your monitor will be more than sufficient. On the other hand, as I am the one writing this, I, and only I, am allowed to post it here: Aye!

Further discussion is warranted, however, on spectregunner’s request for additional information regarding my statement:

I would definitely be disappointed to think that the staff is content with the inaccurate (and, quite frankly, misleading, in my opinion) information currently being provided.

Click to expand...

The word “misleading” is, itself, a bit misleading, but I could not come up with a better term. The American Heritage Dictionary defines it as:

1. To lead in the wrong direction.
2. To lead into error of thought or action, especially by intentionally deceiving.

Unfortunately, the common inference includes the intent to deceive, which I absolutely do not believe is the case here. Rather, I believe the ODP documents to be outdated and inaccurate, thus providing a misleading explanation of how the ODP directory is built.

For the purpose of discussion, my use of the term user will apply to any individual who has retrieved a page from the dmoz.org servers. This includes Joe Editor, Joe Webmaster, Joe Businessowner, Joe Sitepromoter and specifically Joe Surfer.

In general, without having read multiple postings in these forums, the average ODP user is under the impression that the directory is built on a process of:

• User URL submission
• Timely editor review of that submission
• Acceptance or rejection of the site

Specifically, many documents include phrases which foster that assumption.

The URL submission form itself:

• Submitting a site is easy
• helps expedite our review of your site
• ODP's consideration of the site I am submitting

“How to suggest a site to the Open Directory” – http://dmoz.org/add.html

• all submissions are subject to editor evaluation
• We don't accept all sites
• policies for submitting sites for our consideration
• An ODP editor will review your submission
• it may take several weeks or more before your submission is reviewed
• If a site you submitted has not been listed after a month, you may check its status

“Submitting Your Site” – http://dmoz.org/help/submit.html

• it may take up to 2 weeks or more for your site to be reviewed
• help expedite review of your submission
• thoughtful submission has a greater chance of getting reviewed sooner
• The editors have the ability to move your site

“Our Social Contract with the Web Community” – http://dmoz.org/socialcontract.html

• We will make every effort to evaluate all sites submitted to the directory

The statements which I find particularly inaccurate and / or misleading are “it may take several weeks or more,” “it may take up to 2 weeks or more” and “has not been listed after a month.” Yes, a literal interpretation of “or more” includes months or even years, which may be the actual timeframe. However, the clear implication is that one could expect a review within a one-month period, at which point it becomes prudent to check on the status of the submission.

If a user goes so far as to follow the suggestion on the “Submitting Your Site” page and reads the “Open Directory Editorial Guidelines” – http://dmoz.org/guidelines/describing.html, he will still see phrases such as:

• Verify a URL is correct and working
• remove superfluous information from the end of the URL
• derive a concise title from the site's contents if the title is ambiguous
• If "aaa Website" is the submitted title

All of which imply that the editor is reviewing submitted URLs.

Prior to reading these forums, nothing in the documents sited above (which are the “front-line” of information) would indicate otherwise.

Only after coming to these forums, and reading various threads, does the user find information indicating that the ODP editors (in general, of course there are exceptions) consider submissions as only one resource for new site listing, and spend their time adding sites they have found on their own, reviewing and updating currently listed sites and finally (although frequently not as a priority) reviewing user-suggested URLs.

Granted, it is the rare user indeed who actually reads what is placed in front of him, and rarer still clicks on a link for additional information.

But, for those few who might read a couple of sentences, I feel it would be absolutely appropriate to include text similar to my original suggestion from item #34 of this thread (excluding, as we have agreed, a “quiz” of any sort) right on the URL submission form. Additionally, it might not be a bad idea to revisit the other documents and include some language more accurately describing the process of building this directory.

By the way, I am not intentionally trying to win the award for the longest single message posted in these forums.

Thanks for reading, Don

P.S. spectregunner: I am, as should anyone who has posted a comment to any internet message board, well aware of how easily the intent of a message can be misinterpreted. No offense was taken.

bobrat said:

[Ok D. is sarcasm, since I just deleted a site that was submitted twice in 15 minutes.

Click to expand...

This bothers me. Why was this done? Maybe the submitter had an issue with his or her browser? Were they told that it was deleted?

Granted if additional submittions were made after the first two, then its a spam issue but how do we not know that this was just a mistake?

-drmike

spectregunner said:

I like the concept of the above, but really struggle with the idea of a quiz. No where in the social contract does it say anything about only considering suggestions from those smart enough to pass a quiz. We consider web sites based on the content of the site, not on smarts or test-taking ability of the submitters, and it is my personal opinion, that putting in any kind of pre-submission test or quiz contravenes the spirit of the social contract.

Click to expand...

It makes sure the submitter understand the material (the rules posted above) presented to them. It doesn't test them on their intellegence but no weither or not they actually read the page they are looking at.

I'd go ahead and add the resubmit after 6 months stuff as well.

-drmike