The ODP effect on search engines

[dermotz]

in between the lines i always read:

 

"we have nothing to do with it, we are just a bunch of editors reviewing website. it is not our fault search engines work the way they work"

 

it slightly sounds like "we are just producing guns.....it is not our fault if people use them to kill people"..

you could either ask people not to kill each other or avoid guns being produced.

 

the same here..you could either blame the search engines and tell them not to rely to much on dmoz data or to tell dmoz to be more fair with adding websites.

 

ok slightly out of topic..but you know what i mean ;-)

 

in general search engines should never count links of dmoz related directory websites and alexa should not use dmoz data either.

[motsa]

it slightly sounds like "we are just producing guns.....it is not our fault if people use them to kill people"..

you could either ask people not to kill each other or avoid guns being produced.

You might as well say "people die in car accidents so we shouldn't make cars. It's a silly statement to make. We assume no control (nor should we) over what people do with our data. To do otherwise would be wrong.

[jeanmanco]

The problem is, that most search engines work like that:

 

- Someone is looking for a specific website

- The search engines looks for the relevant DMOZ categories

- The search engines shows the busiest/most popular websites from that category using Alexa data which only knows website that do have DMOZ categories.

- Then the remaining results are determined using own search engine algorithms

 

No search engine operates in this way, to the best of my knowledge. You may possibly be thinking of Yahoo! in the old days, before it had its own search engine. As I recall its search would serve up results from its directory first and then further results came from whichever search engine it was in partnership with at the time. (Am I right? - My memory is not my strong point.)

 

You may possibly have assumed that because Google uses a branded copy of the Open Directory as its directory, that it followed the same pattern. But no. Google has never, ever served up ODP-listed sites first for every query.

 

You will quite often see ODP-listed sites coming up on the first page of results for queries. That's not too surprising. We list about 5 million sites. And we try to list the best. But there are plenty of sites not listed in the ODP that do well in Google and other search engines. Examples have already been given in this thread.

 

As for Google using Alexa statistics - where's the proof? I personally don't believe that Google would do so. If Google wants to factor site usage into its algo, it can generate its own statistics for it. But I see no consensus from search engine experts that Google is actually doing that.

[hutcheson]

Alexa has three different and distinct modes.

 

(1) Information on site popularity. It displays the most popular 500 sites, whether or not they are listed in the ODP. It seems to use ODP descriptions, so you can EASILY find MANY non-ODP-listed sites in that list -- just look for the sites without description.

 

(2) Directory. This is a copy of the ODP, and it lists sites (whatever their Alexa rank is). It does allow you to sort a category by Alexa rank, if that's what you want to do.

 

(3) Search engine. Search results for PAGES are shown with no apparent regard for whether the SITE has a listing -- and again, you can look for yourself: Alexa search does return pages from "popular" sites that aren't in the ODP (I picked three sites off the top-500 at random to check this.)

 

This cannot be emphasized too much. All of this is Alexa's choice and the sole responsibility of the ODP. If Alexa wants to use ODP data, they can: that Alexa chooses to do so, is an indication that the ODP priorities are set correctly, and that ODP editors are succeeding at providing a value to surfers. In other words, that ODP editors are fulfilling the responsibility they took on.

 

It is not, not EVER, a sign that ODP editors should be looking for someone else to tell them what responsibilities they should have taken on, instead of doing what has been proven to work.

[charlesleo]

I should add that the folks over at Alexa are extremely helpful and kind. It took a couple weeks but my site was added. Also, they took the time to manually add my site to their directory independent of ODP and were clear to point out a listing in their directory would not carry over to ODP.

 

My site hit #6 today in a popular art/business category today - and it is still not listed in ODP. I have no doubt that it helps to be listed here, but there are many alternative ways to go about search results.

[jeanmanco]

I'm so glad that you have found out the truth of what I've been telling you.

[Callimachus]

Unless they have changed, Alexa's rankings are based on results from the Alexa toolbar (which afaik only works under IE). This means that only those surfers, a small minority though larger proportionally in the past, using the toolbar are contributing those figures.

[jeanmanco]

That was my understanding Callimachus and the reason why I don't believe that Google would want to factor Alexa's data into its algorithm.

[hutcheson]

Well, Alexa is like Google. They get the best information they can (in their judgment, of course.)

 

The ODP provides the best information WE can. And that is, of course, based on OUR judgment.

 

If someone like Alexa or Google decides to use the ODP data, what would that be?

 

Would it be:

(1) A confirmation that someone else -- someone, in fact, with professionals looking at website quality -- thinks what we are doing is good.

 

(2) Evidence that we should stop what we are doing, ignore not only our own experienced judgment but also the recommendation of the expert, and take on a responsibility imposed by some third party? A third party with neither our experience nor Google's reputation as site critic?

 

Folks, this is just flat NOT a difficult question.

[WRMineo]

Alexa rankings and stats are crap - there, I said it.

 

As stated, the traffic stats are soley based on patterns of those who've chosen to download the Amazon dataminer .... er, toolbar.

 

I have a site with listings in DMOZ and it gets nearly 20,000 visitors a day - most of which are from Yahoo and MSN; so much for Google giving such high weight / trust to DMOZ - they're a link like any other.

 

The fact that other online entities choose to use the ODP data is the added benefit, not weight soley because of its origin.

 

My site that gets thousands of visitors a day - its Alexa ranking hovers around 80,000; sites with a fraction of the traffic that have Alexa traffic rankings in below 10,000 ...

[hutcheson]

Yes, well, we none of us are exactly eradicating spam.

 

I think spam is one of those things like poverty and hemorrhagic fevers and mortal sins -- the evils we resist not because we think we can conquer them, but because resistance is a moral or categorical imperative.

[charlesleo]

I'm so glad that you have found out the truth of what I've been telling you.

Well. You are right in many ways and I appreciated everyone's advice. But I put an abnormal amount of work into getting there. I've got nothing to complain about, but I still believe a listing would have (past tense) helped alot - at least in the sense of having reducing the work involved.

 

the reason why I don't believe that Google would want to factor Alexa's data into its algorithm

Google also pulls from the Yahoo Directory and that's a competitor. Google also pulls up alot of Alexa pages in the top 50 of most categories (in mine top 20) so it's not like it's ignoring Alexa altogether. It also assigns PageRank to Alexa pages.

[nuthin]

only one part of there algorithm(s) my friend.

conquer one part, then onto the next.. until you get it sorted and can rest for a bit.

until a search engine like Google pulls a "mass change" to there algo... but heh, if done right you have no worries.

only improving there index against spammers, so once Google works itself out you should all be celebrating like me.

[jeanmanco]

Google also pulls from the Yahoo Directory and that's a competitor. Google also pulls up alot of Alexa pages ...

Google aims to include in its index pretty well everything its bots can reach, except for stuff it considers duplicate content and/or spam and/or sites trying to fool its algorithm. Pages with high PageRank in the Yahoo directory or Alexa have a good chance of appearing in Google's SERPs for relevant queries, just like high PR pages from other sites.

 

But dermotz put forward the theory that Google actually uses in its algo (as a separate factor from PR) Alexa's site popularity data i.e. how often the people using the Alexa toolbar click on a particular site. There is no evidence that Google does this and I would be amazed to learn that they do. They have their own data for how often people click on the sites that they present in their SERPs. And vastly more people use Google than use the Alexa toolbar. So if Google wanted to include click-popularity in its algorithm (and we don't know that they do), they would scarcely need to turn to Alexa for it.

[charlesleo]

Ah. I see what you are saying. Good point.