ODP and secrecy

This is my personal perspective as an average internet user and otherwise search engine/directory/search optimization illiterate. (I know you know this, I just want to show you how I'm thinking.)

I have used Google and only Google for the last 3 years because it was the only thing that ever seemed to get me what I wanted without spending hours sifting junk. The way I figure it, Google finds everything through lots of directories and following associated links and will do its magic spidering pages and sorting the information to present relevant results. But if a site isn't linked somewhere, Google won't find it.

ODP is a directory of sites, not a search engine of pages. The organization is important to assist editors in managing sites, to provide "meta" information about the content of a category, and to assist a user actually using the directory to find information. However, I really think ODP's primary benefit is as a source of sites for Google to search. I also believe that Google strongly considers placement in ODP as part of its relevancy algorithm because they know that a real person looked at the site, that sites are being included impartially, and that every attempt is made to filter out the mirrors and no-actual content sites. To the best of my knowledge, no other major directory does this now. Also (and even more importantly) no other directory is actively ferreting out the content filled sites that aren't being actively promoted.

My perspective as an editor is that ideally, yes I want to list every site relevant to my category and be complete. But given the backlog, what I REALLY want to list are the sites that no one else is listing. At the lower levels I have the luxury of being able to keep up with submissions too. As I move to edit broader areas, my focus will still be primarily on the under-promoted sites. I will look across categories and see what is missing (Hmmm, surely that locality has a police department) and then go look for stuff to fill it. I'm sure I will not be able to keep up with submissions, but I'm not going to worry about it, because in most topics a submission has also probably been submitted to other directories and search-engine optimized and thus will be found by Google. Those sites don't need my help to stand out as relevant.

Yesterday I listed two local pizza shops. Searching Google for pizza and the locality turned up only chains and directory references to those same chains. One store was on the third page of results, the other wasn't even in the top 100. I'm pretty sure that when Google catches that change both those stores will pop to the first page of results. End effect, a person looking for pizza will be presented with 4 actual stores and not just 2. THAT is why I'm here and I feel I can make a difference.

I really don't believe that ODP can actually list all websites, at least not in the next few years, unless there is a broad grass-roots change in the philosophies of web-users to come help. There are a whole lot more people making websites than editing in ODP. I don't think any manually edited directory can be complete. So, when I say I am focussed on the user, I am really thinking about the user of Google, which provides a super-set of directory contents (and more!) Other editors may have different views, but I believe that the one fundamental thing in common is that they feel they make a difference. And I'm sure we do!

Hi Crowbar and thanks for th response - I feel this is a very worthwhile discussion:

>>As far as quality content goes, yes, we like sites with quality content, but, our instructions are to list sites with "useful content", meaning useful to the user, and not, necessarily, built by a professional such as yourself.<<

No one, and especially not me, has suggested that it is necessary for a site to be bult by a professional in order to have good content. Neither should it be penalized if it is built by a professional nd has good content.

IMO the name of the game it to get information into the hands of the User and in the best way possible.

I do not want to be drawn into the "be an editor" scene as this is not the proper thread for it and there is another thread underway talking about that aspect so you'll excuse me if I don't respond to the rest of the post.

Hi Lissa;

Another great post, and one that contributes to my understanding of the inner workings.

>>The way I figure it, Google finds everything through lots of directories and following associated links and will do its magic spidering pages and sorting the information to present relevant results. But if a site isn't linked somewhere, Google won't find it. <<

Thats somewhat of an understatement, and I won't go into all the machinations involved in the Google Algorithm, but Google finds most of its site by spidering directly on the web, there are hundreds of thousands of sites indexed by Google that are not in directories, but you are correct that Google will not list a site if someone somwhere does not link to it from an external site.

>>I also believe that Google strongly considers placement in ODP as part of its relevancy algorithm<<

There is some ancedotal evidence to that effect but there is no proof, and as far as can be determined having or not having an ODP listing will not raise you from fifth to first place in Googles rankings. Google is really very clever in the way it ranks sites, using perhaps as many as ten or twelve different factors in evaluating a site, but this is not a human decision, it is done by software, and I can almost assure you that there is not a line in that code that says if this site is in ODP rank it higher by x points.

>>Yesterday I listed two local pizza shops. Searching Google for pizza and the locality turned up only chains and directory references to those same chains. One store was on the third page of results, the other wasn't even in the top 100. I'm pretty sure that when Google catches that change both those stores will pop to the first page of results.<<

This is realy exciting Lissa, since it could give us a chance to see what effect an ODP listing will have on a real world example when no one is in the background optimizing and tweaking the sites. If you would like to PM the URls to me I would be happy to monitor those sites rankings and report the results back. This is the kind of information that is golden to me.

>>So, when I say I am focussed on the user, I am really thinking about the user of Google, which provides a super-set of directory contents (and more!) Other editors may have different views, but I believe that the one fundamental thing in common is that they feel they make a difference. And I'm sure we do!<<

Well put Lissa, and I tend to agree with you. I am not here to poke holes in ODP but to find out more about it, and this forum is the best source for that I have found.

>>as far as can be determined having or not having an ODP listing will not raise you from fifth to first place in Googles rankings<<

I didn't mean to imply this strong of a relationship. I was mostly thinking of being pulled out of obscurity into the top 10. I'll forward the URLs to you - I listed them several days apart, so an effect in Google on one might come before the other.

Mel,

Thank you for starting this thread and keeping it alive. IMHO this thread alone justifies all efforts to set up this forum.

I do believe that if ODP believe they are something other than another information resource (and to the SEARCHERS they are just another search engine) they are going to have problems.

Please remember that we have to deal with both volunteers and users. Most volunteers want to solve their own need and to do what they love: to list sites. Personally I had a long personal bookmarks list in my browser before I joined ODP and I still consider ODP a huge extention to it. Those like myself would be building and improving their common bookmarks list even without any external users, just for ourselves.

Now let see at ODP from user's (searcher's in your language) point of view. Due to my personal curiosity and my job I do tens of searches a day using both automatic search engines and directories. I do know what I expect from both and I use them differently. On directories I use internal search engine only to find relevant categories. I do expect to find most relevant content listed in appropriate category and related ones. Yes, I actually browse directories.

What I found interesting: during my searches sites are getting more hits through search engines (it equals google in my case) than through directories BUT from my point of view pages found through directories are much more relevant. Let me rephrase it: according to statistics search engine is a winner BUT my impression remains that the best content I was able to find through directories.

Why I wrote it all? Because if the only important use of directory would be through search engine the whole idea of directory would be ridiculus! Why to bother about categories? Just build a huge flat list of sites with keyword lists reviewed by human editor!

Once again: to feed a search engines with relevant data you need only a qualified human editor editing the descriptions and keyword lists and comparing them with actual content of site.

We believe there is around a significant group of users who use directories as such. Are they a minority among general internet public? Personally, I don't care. Our goal is to built most comprehensive and relevant directory. If our data can be used in other ways it is OK but it does not change my goal.

Hi Wladek:

Thanks for another great post, and one that goes a long ways towards convincing me that ODP is superior in some ways.

>>What I found interesting: during my searches sites are getting more hits through search engines (it equals google in my case) than through directories BUT from my point of view pages found through directories are much more relevant. Let me rephrase it: according to statistics search engine is a winner BUT my impression remains that the best content I was able to find through directories.
<<

This is a very interesting point, and really quite logical when you think about it. A human review is much more flexible (read intelligent) than a software review, and so it is logical that there should be nuggets in a well edited directory that are not so well ranked in search engines which use a fixed algorithm to evaluate all sites. This has been your experience and as a "professional" searcher it is a valuable one.

But the statistical approach (and common sense) says that the search engines have listings which the directories do not since the better engines include all the directory listings as well as other listings that they find on their own. Search engines too have the advantage that they index each page independently, while the directory must categorize each entire site in one listing, so a careful searcher might well find additional nuggets in that mine. Busy searchers too will find things faster in the search engines than by browsing in Directories.

>>Because if the only important use of directory would be through search engine the whole idea of directory would be ridiculus! Why to bother about categories? Just build a huge flat list of sites with keyword lists reviewed by human editor! <<

My thoughts exactly, since that is how I tend to use directories, but I can seee that there are other considerations, such as administration and the division of the work among editors, since without the categoires and subcategories putting bounds around each editors workspace, there could well be a great duplication of effort.

It may not be perfect, but it seems to me that discussion such as this will go a long ways towards bridging the perceived gap between the users and the editors.

Mel,

>But the statistical approach (and common sense) says that the search engines have listings which the directories do not since the better engines include all the directory listings as well as other listings that they find on their own. Search engines too have the advantage that they index each page independently, while the directory must categorize each entire site in one listing, so a careful searcher might well find additional nuggets in that mine. Busy searchers too will find things faster in the search engines than by browsing in Directories.<

This is exactly why Google is so effective, it uses both a directory and a meta spider generated contact list. Remember that before the one reads the list the relevant directory category is displayed. This may be because it is a convenient place to put it. What I see is that the user can click the directory for a small group of extremely relevant sites or continue the search through the list.
I am an archaeologist and historian by bent and training, but found the normal search engines are a hindrance to internet scholarship. People spend a great deal of time sorting through 10,000 returns, when all the wanted to know was what year the Wizard of Oz was made. The directory is superior for research but the directory must be staffed with competent and dedicated people. It is like a library; the organization of data in books is the first and most important step to scholarship. One could learn the same thing by searching through the books in a big pile in a football arena, but think of the time and effort. With this the sites must be read, described, and filed only once; then everyone can take advantage of the work of a few.

Godanov
Archaeology and Arts editor

Hi Godanov:

Have I mentioned what a pleasure it is to discuss ODP with intelligent, dedicated individuals who know first hand whereof they speak? Compared to the rants of the "professional ODP complainers" that are common on many forums, this is a gift like a spring day in the middle of winter.

>>The directory is superior for research but the directory must be staffed with competent and dedicated people. It is like a library; the organization of data in books is the first and most important step to scholarship. One could learn the same thing by searching through the books in a big pile in a football arena, but think of the time and effort. With this the sites must be read, described, and filed only once; then everyone can take advantage of the work of a few.<<

I can understand your point, but I guess depending on the necessary depth of your research, IMO you can find that "which has floated to the top" so to speak more easily with the a good search engine, and in addition you will have access to a greater repository of information say on Google than on ODP, simply because Google includes the ODP listings, plus those it has found on its own.

But the quality of the information, and the ranking(or indexing) of it (it has to be ranked/indexed because no one is going to look through 300,000 listings)is a bit of a problem. When an expert in his field puts his specialized knowledge on the web for all to see and make use of, he may not be so expert in presenting it clearly, and generally adopting those tactics which make it easy for his offering to rank well in the eyes of a search engine is likely the last thing on his mind.

ODP address these shortcomings by having a large group of editors who are expert in certain fields do an analysis on behalf of the user and present him with a brief summary neatly categorized and searchable.

Search engines (and the sites author) have to rely on the services of the much maligned SEOs to make the content more understandable to the reader and search engine alike. (Many may be shocked to learn that one tenent of good SEO is good copywriting). The Problem here is that most SEO is directed towards "commercial" sites since they generally are the only ones willing to pay for it.

Bottom line, both ODP editors and good SEO practitioners are in the essentially the same business, making information more easily available to users, IMO.

When an expert in his field puts his specialized knowledge on the web for all to see and make use of, he may not be so expert in presenting it clearly

All too true, and that's why the ODP is essentially a peer review process. There are editors that are experts in a specific subject matter, and there are editors that have experience in presenting and categorizing information in an understandble way. In the ideal case, those will work together, and the result will be much better than what each of them could have acheived alone. (There's also the occasional conflict when a highly specialized expert happens to have difficulties to look beyond his own teapot, but that just shows we're only human...)

and generally adopting those tactics which make it easy for his offering to rank well in the eyes of a search engine is likely the last thing on his mind.

Well, maybe it shouldn't explicitly be the very last thing on his mind, but definitively not the first either. The first goal is to make the description descriptive for someone who browses the directory. Of course, if there's room left after that, then trying to include another keyword or two may not hurt. The nice thing is, however, that any downstream engine using those listings has the possibility to take a hybrid approach, and fetch additional search keywords by actually spidering the linked page. Some engines indeed use such combined approaches, most prominently Google and Alltheweb (the latter in a much less obvious way).