Site was added to ODP, then removed

prokopton72

Member
Joined
Jun 28, 2006
Messages
8
Our site <url removed> was added to the ODP 2 week ago under the category:

Health: Mental Health: Disorders: Impulse Control: Pathological Gambling: Support Groups

Then suddenly it was dropped from the ODP without any explanation.

:eek:
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
What was the explanation given when it was added? Maybe that explanation turned out to be erroneous? (Actually, all deletions have explanations that are available to all interested parties--parties interested in the ODP, that is: i.e. editors).

And ... "suddenly" is the only way we CAN drop a listing. A slow fade just doesn't make any sense at all, and would be a royal pain to implement (so it hasn't been implemented and won't be either.)
 

prokopton72

Member
Joined
Jun 28, 2006
Messages
8
We had a site, www.abc.com (as an example), that was listed on the ODP for over 2 years. Earlier this month we moved the site to another domain, www.123.com (as an example). It is basically the same site but under a different domain name.

The editor changed the name and the link to reflect the new URL over two weeks ago. Now, for some reason, the site is no longer on the ODP.

:eek:
 

prokopton72

Member
Joined
Jun 28, 2006
Messages
8
A note:

The old domain was <url removed> The new domain is <url removed>. The old domain was listed on DMOZ for over two years.

Thank-you.
 

motsa

Curlie Admin
Joined
Sep 18, 2002
Messages
13,294
I removed your URL from your original post. That didn't mean go ahead and put it in another post.

Unfortunately, we do not give site status checks any more and that includes queries of "why isn't my site listed any more?" If your site is listable, then its absence from the public directory is likely to be temporary. Some reasons why a listed (and listable) site might be temporarily removed from the public directory include site was moved to another category for rereview, site was unavailable for a period of time, or listing was unreviewed for QA reasons.
 

Isometric

Member
Joined
Jun 14, 2006
Messages
28
Unexplained removal

Hi there,

To me it seems that there is a very thick fog around what dmoz is doing with the editing of it's database.

Now I know that it is a free listing and is run by volunteers and so on, but with the amount of effect that the removal of a listing can have on a site which in turn can have a very dramatic effect on the individuals and businesses involved with that site. It seems to me there should be a lot more of an open and available public view of the status of listings, at least a public list of recent removals and reasons should be available, because the lack of such a system leaves enormous room for people on the other side of the fog to make up reasons that could be self serving or could just be misinformed.

Please don't tell me that dmoz doesn't have that great of an effect on a site, as I have seen a number of sites all but disappear from SE listings immediately after DMOZ has removed the same listing, for no obviously apparent reason in most cases that I have seen.

I am sure that the secrecy has been implemented with the best of intentions but to me it seems you are becoming the very opposite of an open directory.

I am sure you can see yourself that the lack of listing simple information such recent removals with reasons has caused numerous unnecessary posts in here. If you had a listing of recent removals and reasons you could have a much more simplified process for dealing with people that disagree with a listings removal.

Hope you consider this as it would make DMOZ much more user friendly and as more people become aware of the ODP there will be more and more pressure on the editors and and admins to use more user friendly methods.

Don't get me wrong I think that the ODP is a great thing and has achieved some awesome stuff, however in my dealings with other web developers and net savvy people there is a massive stigma around DMOZ, it is seen as the most difficult organization to deal with even though it is seen as one of the most important also.

Anyway, I hope that you take this as a friendly plea and also possible a wake up call in regard to all the stigma that surrounds DMOZ. Although I am sure you have heard much the same thing before, which only highlights its importance in my eyes.
 

pvgool

kEditall/kCatmv
Curlie Meta
Joined
Oct 8, 2002
Messages
10,093
Reasons why a site will be removed (or will not be listed) can be found on the DMOZ site. See http://dmoz.org/add.html and http://dmoz.org/guidelines/include.html

Hope you consider this as it would make DMOZ much more user friendly and as more people become aware of the ODP there will be more and more pressure on the editors and and admins to use more user friendly methods.
I can´t see how this would make us more friendly to our users.
O wait, you think webmasters and siteowners are our users. Well they are not. Never have been and never will be.

Although I am sure you have heard much the same thing before, which only highlights its importance in my eyes.
Ofcourse we have read many complaints form webmasters. But they all complain about something DMOZ does not provide. How can we change something we don´t and do not want to provide.
There are only two kind of proposed changes we are interested in
1) how to make the directory better for our users
2) how to make improvements for the editors
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
Isometric, we've heard it before. We've tried it before, and we KNOW, from EXPERIENCE, EXACTLY what happened.

What happened was not good for ANY honest submitters, it was not good for editors, it was not good for the directory. We didn't like it.

We learned better, and we won't make that mistake again. You too can learn by listening to the voice of experience, and replacing your uninformed speculation with genuine (secondhand) knowledge.

Or ... even firsthand knowledge, if you prefer that, and are willing to work to get the experience. Our experience is archived, publicly, in these forums. Read a few thousand posts, and see how many people could act in any way on the information received. Look at the rejected sites, see how many of them bear any evidence of the webmaster being able or willing to generate unique content. Look at the sites waiting, see how many of them were significant losses to our users (who are, it is always worth repeating, not webmasters but surfers).

The facts are there, in plain sight. Speculation is unnecessary, and almost invariably contrafactual. Don't inflict your speculation on people who know better.
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
The stigma among "webmasters" is not a problem. It is the ODP's greatest achievement. There are millions of sites that have been suggested to the ODP, and that will wait forever for acceptance, because they've been rejected for cause. All those webmasters are frustrated: the web was, after all, their route to endless riches, the ODP was the paymaster, and THE PAYOUT WINDOW WAS CLOSED, and the ODP guards dragged their carcase away when they played the constructive-nuisance card in front of the gates.

They don't like the ODP. And that is, in a nutshell, what the ODP is all about -- not being liked by spamming webmasters.

How about the others? Well, the spamming webmasters do their dead level best to make sure we don't FIND the other sites. We find lots of them anyway.

Have we not found your site yet? Don't blame us. Blame the spammers, who have filled every online information conduit with crud. And look at the good sites we HAVE been able to find, in spite of the spammers.
 

Isometric

Member
Joined
Jun 14, 2006
Messages
28
My 2c

Hi to both PVGOOL and Hutcheson,

Firstly PVGOOL,

Don't you think it is a little arrogant to say that webmasters and site owners are not your users? I know this is the vastly overwhelming attitude within the ODP and it is to me a little bit affronting to tell the truth.

You see, I am a web developer, a site owner, a webmaster, an ODP user, a legitimate ODP submitter and with any luck a soon to be ODP editor. I would assume that you are aware that the ODP is also a website, and was developed by a web developer, has it's own (although I hate the term...) webmasters and is actually more dependent on web developers, webmasters, and site owners than any other type of site?

For someone affiliated with such an organization as the ODP I find it hard to believe you have such contempt for the actual owners and operators of websites.

Anyway, I think there is nothing more I can say if the mere mention of a site with status information spurs such a torrent of webmongering. I never suggested that you go back to your old system by the way, and I never suggested that you take peoples complaints about the status of their sites any more seriously than you do now. I was thinking you should implement a new system that is just a monitor and allows legitimate web developers to get information instead of having to guess what is going on on the other side of the ODP Iron curtain. I am a firm believer in the fundamentals that the web was founded on you see. In my eyes that is freedom of information and expression on an even platform where an enterprising individual can make himself heard or can even run a business without having to bow down to the massive multi-nationals to get anywhere. In effect the ODP is sounding more and more like an enemy of that notion. I hope I am wrong.

This turned in to more of a rant than anything else. Sorry, it just really upsets me that people who should be working together end up at each others throats. I want to stop spam just as much as you do. I just don't think that this total secrecy is the answer. It causes more problems than it solves I think.

Hutcheson,

Sorry I think I used up everything I had on PVGOOL.

Just one thing though, you say:

Hutcheson said:
The stigma among "webmasters" is not a problem

You two keep saying webmasters. I hate that term, I never mentioned that in my post, I said web developers.

Hutcheson said:
All those webmasters are frustrated: the web was, after all, their route to endless riches, the ODP was the paymaster, and THE PAYOUT WINDOW WAS CLOSED, and the ODP guards dragged their carcase away when they played the constructive-nuisance card in front of the gates.

It is not only spammers and such that are frustrated, it is everyone I have ever spoken to about dmoz who has had to submit a site to you.

Hutcheson said:
Have we not found your site yet? Don't blame us. Blame the spammers, who have filled every online information conduit with crud.

This is the best point, I agree, damn their dirty spamming hides to hell.

One thing I would like to say though, is this, I don't blame you and was not trying to. I was just trying to voice a desire for some sort of tracking for those of us who are legitimate web developers/owners/submitters and such.

Hutcheson said:
And look at the good sites we HAVE been able to find, in spite of the spammers.

This is why I still love DMOZ and reading this made me feel a whole lot better than after all of that anti web developer stuff.
 

giz

Member
Joined
May 26, 2002
Messages
3,112
>> I have seen a number of sites all but disappear from SE listings immediately after DMOZ has removed the same listing <<

I have seen this happen too. We didn't think the site was listable, and so, it seems neither did they.

But to say that they were delisted simply because we dropped the site is a very long stretch .
 

chaos127

Curlie Admin
Joined
Nov 13, 2003
Messages
1,344
Don't you think it is a little arrogant to say that webmasters and site owners are not your users? I know this is the vastly overwhelming attitude within the ODP and it is to me a little bit affronting
It is the truth though. Web site owners can help us by submitting sites, and in a sense we rely on them for there to be any sites at all for us to list. However, our customers, the people who we make the directory for, are the people who use it to find websites -- it is these people that the project was set up to benefit, and this is the group we continue to target our procedures to help.

Think of it like a restaurant. We have suppliers of produce (site owners) and customers (surfers). There are an abundance of suppliers, and whilst we should be polite to them and provide sufficient ways for them to supply us, it's not in our mission to provide any sort of service to them -- we're here to serve the customers. Potential supliers are free to drop off leaflets through the door with detail of their products, but why should we be forced into providing any individual feedback on whether we want them to supply us or not?

I was just trying to voice a desire for some sort of tracking for those of us who are legitimate web developers/owners/submitters and such.
It's been tried and discussed here before. The conclusion amongst editors is that it takes resources from other areas to do well, doesn't help our customers (the surfers), and would provide useful information to spammers. Yes, it would be nice for legitamate website owners, but that's outweighed by the other effects, and the information would really offer any material benefit to them anyway.
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
Oh, first of all, an apology: I don't much care for the term "webmaster" either, possibly on similar grounds.

>I was just trying to voice a desire for some sort of tracking for those of us who are legitimate web developers/owners/submitters and such.

It's a nice idea. But how can you do that and exclude the spammers -- who provide 90% or so of suggested URLs, and a much higher percentage of the queries about them?

I can't think of a way, without actually reviewing the site first.

Think about the demographics. Most people are single-site owners: they have their business or their hobby, and a website about it, and that's all. There will be no track record for them, and no way of getting one, and no use for it when we've got it. We review their site, we list it or not. That's the end of it.

Then there are the professional web developers or SEO firms: creating or promoting websites for other customers (who are not only genuine businesses and one-off site owners themselves -- surely they advertise their own services online -- but they also can provide a stream of other useful websites from their clientele. They are PROFESSIONALS. Knowing how the ODP works is a required skill for them. Knowing the ODP guidelines and criteria and processes, COLD, is their JOB. They can have a reputation, and it makes sense to keep track of it.

So, should the ODP make some sort of special status for these people? We've discussed it internally, and the analysis is simple. "People who want to help the ODP, and that we trust to help, are called editors. What other category could conceivably be useful?"

It sounds like, practically speaking, you're thinking along similar lines. The best way to get to know a process is to experience it. And web developers or SERP perps aren't forbidden to edit: like everyone else, they MUST declare affiliated sites. like everyone else, they MUST NOT give edit to give advantage to their affiliated sites.

Perhaps most important, although difficult to measure, is the attitude: we're here for the surfers, who are our only users. And once that has been grokked, a lot of decisions become much easier. For instance, could an editor just review suggestions, and not look for sites any other way? In the final analysis, that isn't even serving webmasters in general: it represents an absolute refusal to serve anybody but SEO types! It is clearly abusive editing. An honest editor best serves webmasters who haven't had ANY SEO work done by or for them. And again, should it matter whether a website owner is making (1) only a little money, (2) lots of money, (3) less money than if they had an ODP listing, (4) less money than if they didn't have an ODP listing, or (5) none of the above, ever? Again, with the ODP attitude firmly in place, we can focus on the question: "Does THIS website provide unique information to the surfer?" If it does not, it doesn't matter how many dollars of revenue are being "lost" (that is, are going to someone else instead) because there is no ODP listing. If the site does provide information, then it doesn't matter how much good, or harm, it will to to the site owner for it to be listed.

Now, for some people these conflicts of interest are just too much to handle. Anyone has to be careful. One blatant "lapse", a pattern of subtle preferential edits, and one falls into the category of "people whom, we know, shouldn't be trusted".

Which is, of course, the final category. People who've earned a reputation for suggesting sites that were obviously unlistable (or worse, sites that were subtlely unlistable.) Now, you don't GET a reputation like that without working for it: that is, suggesting (or listing or promoting) AdSense doorways or fraternal mirrors or similar noninformative promotional sites.

Which comes back to conflict of interest. In your day job, you may develop or promote sites that would not be listable in the ODP. (Even I have done that, and professionally speaking I am not in the web promotion or website development field.) But an honest promoter has to tell the client what is listable, and what is not. (How do you know? It's his JOB to know these things, remember? If he HAS to ask the editors, he's not being honest with your client -- he's not telling them he doesn't have the skills he was being paid for. And, once again, it comes back to attitude. There is a lot of information about the ODP available. But when it comes down to it, anyone can be a good editor without EVER spending a millisecond trying to train anyone to be ANYTHING except an editor. The ODP is not and ought not to be a training organization! If none of us ever did any of that, it would be OK. The professionals can learn on their own, or go back to some easier job: either option is (and ought to be) OK.

I've worked for small, medium, and large companies, and been associated with other organizations. An organization that survives will always be making decisions about what it will try to do, and what it won't -- what it will stop doing in order to focus scarce resources on what it needs to do -- in short, what best serves its mission. People who don't want to serve that mission, or want to serve another mission -- can just go to another organization. There's nothing of arrogance about that: it's far more a sign of realistic humility. "We CAN'T do everything, let's focus on doing one thing as best we can. Because if we don't do that one thing better than anyone else, we have no reason to exist." What the ODP does is serve surfers. Period. And ... there is a lot more that should be done even to accomplish THAT mission. Everything else ... can be left for some OTHER organization, designed and equipped for other purposes, which can almost surely do whatever it does better than the ODP. Training? Site promotion? Advertising? Disease control? Weather prediction? E-text generation? All, perhaps, worthy goals. All better done by someone else. Possibly someone else with help from the few ODP editors who are interested; possibly someone else without any help from ODP editors. But certainly, someone else.

And people who are interested in the ODP mission are welcome to help -- even people who contribute to some OTHER mission, somewhere else.
 

charlesleo

Member
Joined
May 3, 2006
Messages
152
But how can you do that and exclude the spammers -- who provide 90% or so of suggested URLs, and a much higher percentage of the queries about them?
Here's some ideas to possibly cut down on spam, Again, I don't fully understand DMOZ/ODPs internal protocol so these are just some thoughts:

1) Automated word searches for inappropriate content.
2) IP tracking/reverse lookup and flagging of multiple IPs originating from same address comparison.
3) Duplicate username tracking comparison.
4) Duplicate site name submission tacking comparison.
5) Duplicate/similar description tracking comparison.
6) Duplicate email tracking comparison.
7) A method of tracking an unusual amount of posts to the same category within a period of time.
8) A DMOZ spider which searchs suggested sites for 404s and inapproriate content.
9) Blacklisting email addresses and IPs for a period of time.
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
charles, we'd really rather not discuss what we do in public. If you'll think about it, every one of the things you propose can be easily gotten around, if you know about it and are trying to get around that thing only.

In reality, I suspect we do catch a lot of people who were planning to avoid getting caught by platonic forms, when what we were really using was occam's razor....if you get my drift.

But one unique thing about the ODP suggestion system is, so much of the spam is one-off. Maybe we can spot the Wallace Spamfords of the world, who submit their own site five hundred times a day. But it causes just as much trouble, and is far harder to catch generically, when ten thousand hotel affiliate spammers EACH suggest their ONE hotelnow doorway once a month. And when you remember that each of those spammers probably also has a FTD doorway, and a world-of-gifts doorway, and a diet plan doorway, and a real estate doorway and .... you get the picture.

Don't overlook the importance of the indeterminate (and, ok, indeterminable) delays between site suggestion and suggestion review! (That's not a problem? It's really a feature?)

Well, I've been told by professional doorway spammers, that if a suggestion isn't reviewed within, say, 6 weeks, it's no use at all to them. See, they have to PLAN their campaigns. A quick rampup of links, followed by dropping the content when Google catches on, then on to the next site.

Real businesses that have their reputation invested in one site for the long haul, aren't nearly so affected. (And, since they have real promotions planned, dependable professional promotions, they are much less likely to place so much emphasis on Google as opposed to .... all the other options, online and off, open to an honest businessman willing to pay for services rendered. (Less likely, I say: I've also heard anecdotal evidence that an ODP listing made a difference to a real business. But that real business could wait for three years.)
 

Isometric

Member
Joined
Jun 14, 2006
Messages
28
Okay, I do see your point. It is very hard to provide information to legitmate developers while not giving spammers something to use to their benefit.

Originally I was thinking along the lines that, even if you do give them basic information about the status it would reduce the spam anyway because they wouldn't spam if the site was still awaiting review, and if it is rejected then there is not much point resubmitting it unless they change the site somehow. Thinking more about it though I suppose that the spammers that are a problem are not really the ones that submit the same URL but the ones that submit duplicate or overly simillar pages on different URLs and try to trick you in that fasion, and for them I suppose the status listing would be quite valuable. I guess.

Okay, well thank you and thanks to everyone else for the willingness to go over this topic yet again.

I still think there is possibly room to do something about it without giving spammers anything, but maybe it is just too hard.

Thanks again.
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
>Originally I was thinking along the lines that, even if you do give them basic information about the status it would reduce the spam anyway because they wouldn't spam if the site was still awaiting review, and if it is rejected then there is not much point resubmitting it unless they change the site somehow.

Well, that would be logical. But a spammer doesn't have to be logical. He just has to be industrious. The spamming approach is not to deduce the right target, but to shotgun everything. How much does an ODP listing matter? Does it matter? NO! Suggest the site anyway, to ten thousand search engines! Maybe only three of them are any good, but ... which three? Who knows, suggest to them all! Spammers are not snipers, they are machine gunners laying down suppressal fire. (Killing innocent bystanders is not their main purpose, it's just a fortituous side effect.)

>Thinking more about it though I suppose that the spammers that are a problem are not really the ones that submit the same URL but the ones that submit duplicate or overly simillar pages on different URLs and try to trick you in that fasion, and for them I suppose the status listing would be quite valuable. I guess.

BINGO!

>I still think there is possibly room to do something about it without giving spammers anything, but maybe it is just too hard.

Yeah, I thought that too, for a long time. But ... you're doing well so far: chew on this one.

What is the owner of a listable site going to do with status information anyway?

Nearly always, the site will be either "in review" or (finally) simply "listed". We're pretty good at NOT losing good URLs. And, unfortunately, we are no good at all at telling when a site WILL be reviewed (which might be some consolation.)

But, NONE of those three responses ("not reviewed", "listed", "to be reviewed in XX weeks") require any action to be taken.

The other conceivable (but rare) response would be "the site was accidentally deleted due to editor error." But how could we know that, except by reviewing the site again (by another editor)? When we get to the point of "site deleted", then we assume you're not helping us, trust is lost, and it's time to cut communications because they wouldn't be productive.

Is trust sometimes lost without a reason? Yes, online as well as off. How do you restore it?

There isn't (and can't be) an "I'm not going to lie to you any more; start trusting me again!" button. All we can do is hope that, IF the site really was worthwhile, it'll be picked up in some way OTHER than outside site suggestions. Any editor can review a site, even one rejected before. And if the former rejection was erroneous, there is no bar to listing the site -- of course, documenting carefully the reason WHY the former rejection was wrong.
 

spectregunner

Member
Joined
Jan 23, 2003
Messages
8,768
For the non editors out there, let me give you two veryreal examples of things that we run across as editors. I won't tell you how we handle these, just tell you what we see.

Within regional, in one fairly large city within one state there is a web development company. About every six weeks or so, they submit half a dozen to a dozen sites they ahve developed to the directory. Nothing wrong with that. However, they are lazy. They will submit all the sites to whatevere subcategory the first one fits in. so we get an art gallery, restaurant, used car dealer, florist, real estate agent and mobile pet groomer all dumped into the single locality's /Restaurants subcategory. The company has been politely asked to submit each site to the single best category and not just dump them into a single subcat.

Pretend you are the editor, you have just opened up a subcategory and come across 9 sites from this developer -- only one of which is even vaguely related to the subcategory. What would you do?

Scenario number two.

Again within Regional.

There is a very competitive industry and a number of companies make websites for the local business that are part of that industry.

ABC Company has a website about their local business.
JKL Company also has a website about their local business.

Both websites are listed, both have sufficient unique content to justify their continued listing.

A company that also builds websites for that industry builds sites for both ABC and JKL. They submit them. There is nothing that indicates that the existing websies are going away. There is no way that either company is going to get two websites listed. What would you do? Keep the existing listings or throw them out to add the new listings?

Are the submitters spammers? What do you think?

And as you think about this, recognize there are about a zillion websites waiting to be reviewed, and that this is the type of quandry that editors run into on a daily basis -- not just in Regional.
 

Isometric

Member
Joined
Jun 14, 2006
Messages
28
Hutcheson said:
But, NONE of those three responses ("not reviewed", "listed", "to be reviewed in XX weeks") require any action to be taken.

True but that would at least let the people that have submitted it know that there is no action to be taken, this was my initial point.

Hutcheson said:
Is trust sometimes lost without a reason? Yes, online as well as off. How do you restore it?

There isn't (and can't be) an "I'm not going to lie to you any more; start trusting me again!" button. All we can do is hope that, IF the site really was worthwhile, it'll be picked up in some way OTHER than outside site suggestions. Any editor can review a site, even one rejected before. And if the former rejection was erroneous, there is no bar to listing the site -- of course, documenting carefully the reason WHY the former rejection was wrong.

I agree that you can't have a restore trust button, but it should not be lost too easily either, and there should be some way for people to at least give an eplanation for any assumed wrong doing. No one is infallible and there should be some kind of method for people to be made aware of what they have been accused of, which may have been done by mistake or maybe even someone else who had some contact with the site has done.

-edit- 'Accused of' is not really the right word, I mean they should have some reason for their site being branded spam. This is purely an ethical perspective.
 

spectregunner

Member
Joined
Jan 23, 2003
Messages
8,768
True but that would at least let the people that have submitted it know that there is no action to be taken, this was my initial point.

Works well in theory but not in practice -- or it would work well in practice if we could get all the spammers to self-identify and opt out of the notifications.

Where the concept of notifications breaks down is in the area of motives.

Sure, we could tell you that your site (hypothetically) was declined because it lacks unique content. You may or may not be mature enough to handle that information and our previous attempts at communication of this sort indicated that most people did not handle that information very well.

But what if you are one of the bad guys? When we tell you we have decline your site because it is a fraternal mirror of XYZ.com, all you are going to do is tweak the site, give it a different URL and submit it another dozen or so times.

What is a polite way to tell a spammer that we have figured out his latest batch of tricks and his last 355 submission have been for naught?

What good does it do to tell someone that their site is nothing but an affiliate/clone of xyz.com. They know what they have. They know they are ineligible for a listings. They were just hoping to catch an editor in the midst of a brain fart, and when that doesn't happen they will be the first one over at webmasterworld or digitalpoint screaming about how corrupt we are.

However, never say never, and if you can figure out a way to implement a notification system that does more good than harm, I'm sure we are all ears.
 
This site has been archived and is no longer accepting new content.
Top