Search Engines Censoring Sites Using ODP Data

There is a report on the net “Search Engine Censoring of Open Directory Data” that says all three major search engines (especially Google) routinely block access by their users to sites that use any ODP data as part of the site. The sites that are banned are sites that use ODP data as part of the site, not sites that are listed in ODP although being listed in the “sites using ODP data” page on ODP may increase the chance that a site will be banned.

This appears to mean that “Open” Directory is not really open anymore since only major sites like Google can use ODP data without major risk of being banned by one or more search engines.

It also seems to mean that ODP editors are mainly functioning as unpaid employees of Google and other major companies as opposed to providing data that can actually be used by many web sites.

The purpose of the ODP is to provide data that can be used by surfers.

This seems to be a search engine issue not an ODP issue. I am not an expert on Google but I believe there have always been issues on duplicate content which this seems to be related to.

regards

My economic analysis is completely different.

I don't see what gain AOL or Google gets from my work. And even if I saw, it wouldn't matter to me.

What I see is people who are looking for information, unsuccessfully, until I show them how to use the Open Directory. THOSE are the people who are being benefited, and THOSE are the people I'm working for.

So far as AOL and Google are concerned, they are giving ME a service -- providing information to users like me, my family, my friends ... and doing it at no financial cost, and with absolutely no irritating, bandwidth-hogging, space-wasting advertisements/promotions (on either directory.google.com or dmoz.org!) How much would you have to pay to get that kind of service elsewhere (if you could find it at all)?

And that's not all. I'm not just a surfer, I'm also a content provider. And for my web link content, AOL is hosting it (for free), providing first-class tools to maintain it (for free), and, not least of all, building a fascinating community of likeminded people to work with.

I'd be doing more-or-less what I'm doing today, for the same pay, with or without the support provided by businesses, schools, or churches. But -- the more people who can use the results, the better. Google and AOL both do an excellent job of making it possible for anyone in the world to use the results.

Oh, but you're thinking the only conceivable "use" of something is "make money". How ... sad.

Because that's not really an effective use of the ODP at all. Sure, the ODP data is available free of cost, and free of significant obligation. You can do almost anything you want with it. But ... it's pretty stupid to think anybody on earth should pay you for just posting a copy of it on your website, isn't it? Because they can get the same information, faster, freer, fresher, from AOL or Google!

So I'm altogether in favor of Google using mass copying of the ODP (or wikipedia, or gutenberg, or vstore, the CIA Factbook, or hotelnow.com, or ... use your imagination), without added value, as a signal of a infinitesimally-small-value website. The better Google does this -- the better Google weeds out people who try to profit from our work WITHOUT paying back to the community like our patron does -- the better Google serves editors, surfers, and all genuine content creators. And who else is on the web? Nobody that is going to use MY material in any way that matters to me. Nobody that is going to create anything that I would use. In other words, nobody that matters.

Upon further consideration, I think the other part of your attitudinal problem is that you are not thinking of the net as a peer-to-peer network.

And that's wrong. We're all peers on the net. The only thing that makes google more influential than, say, MSN, is that most of us OTHER peers think Google gives more reliable (well, less unreliable) information. Google, at least, doesn't have the power to tell anyone where they can't go. And they don't imagine they ought to have that power. From the beginning they have realized how much they depend on what their peers (you and me and all the other voices) say about each other.

So, it's stark raving froth-blowing spittle-drooling nonsense to talk about Google "censoring" ANYONE. Any URL on the internet can be your home page, or mine, without Google or MSN ever knowing.

You are Google's peer, and Google is my peer.

Now that we're all peers, we can decide what we'll do. We can stalk around the internet badmouthing anyone with more influence than we have -- which, if it isn't yet everyone, soon will be.

We can try to interpose ourself between one peer and another, impoverishing them to the full extent of our power (usually, that is, not much) to enrich ourselves.

Or we can freely give something in return for what we've already freely received. I review a hundred thousand sites or so -- but I've already received information concerning tens of millions of site reviews. I scan or proofread or format a few tens of thousands of pages -- but someone else has already gifted me with tens of thousands of completely scanned and proofread and formatted BOOKS. And, for that matter, Google has without my help given me information not beyond my wildest dreams (because I'm insatiably curious) but certainly beyond my ability to collect. Who cares whether my name is on all those actions? Why do I care who has done all the other work? We have all enriched (not impoverished) the whole world to the extent of our ability (usually not very much), we peers cooperate to avoid unproductive duplicate effort, and we have all been enriched -- to the extent of the total of our efforts. And THAT is a VERY great deal all by itself -- even more when we cooperate so that the intersection of our efforts is minimized (and the union maximised). But nobody has to cooperate. It is just that -- nobody has to give any respect at all, either, to the person who (as a result of not cooperating) doesn't add anything to the total.)