Does DMOZ spider itself?

[shawnjohn]

I've been wondering about this for a while.

 

My site is listed in this section:

 

http://www.dmoz.org/Regional/North_America/Canada/Ontario/Localities/O/Oshawa/Business_and_Economy/

 

But if you do a normal search in DMOZ for my business name or domain name, neither show up.

 

Why is that? Does this mean that the search function DMOZ uses is not yet finished spidering itself?

Is the category too deep?

Is the category set as noindex nofollow in some robots.txt or htaccess file?

Is the search function offered by some 3rd party, like Google? I can see G hasn't cached that page since Dec 27th.

 

Very strange. Can someone shed some light here?

 

Merci,

[photofox]

To see if a URL is listed in the directory you need to search for it using "example.com" rather than "http://www.example.com".

 

The search database is based on the RDF file we generate, so it's not updated in real-time, it can take up to a week for a site to appear in search (so long as the RDF generation process is working correctly).

[hutcheson]

The search function is internal--not Google or any other search engine. It's periodically rebuilt from internal databases; it doesn't use what you'd call a "spider" procedure, so doesn't have any spidering issues.

 

But dmoz search IS frequently out of date, as you notice. We volunteers don't control (or even know ahead of time) when the database will be re-indexed.

[jimnoble]

I wouldn't worry about it. If you can see your website listed in the category, our search system will catch up eventually.