Some files indexed regularly, linked files are not

V

vallini

Hallo

I am a new member so please be indulgent with me: I read the guidelines and at least apparently my question might belong this forum.

I have the following question: I found this happening repeatedly over months and I'd like to know how it works and if I can do sth:
I noticed that webbbots (this applies to DMOZ too anyway) visit regularly at least two files on my website (a noframe list of links, and a framed main page named list.html) and upadte them on at least a monthly base.

The strange fact is: whereas those two mentioned files get visited monthly, the links (instance: in noframe.html) that lead to the newest files are not visited.
This can generate the following strange situation: a file named math5.html whose main keyword can be considered "Aseroth" appears in a search "unitedscripters Aseroth" listed as present in noframe.html and list.html, but is not indexed on its own as math5.html.

I got the habit to add the newest Url to the google add url form (for instance), for there there seem to be no specific prejudice against sending this kind of "briefings": anyway I did it only 5 or 6 times.
Any way to get a new file crawled together with the index files?

I have a long file that I uploaded over 2 months ago and whereas it appears as heving been crawled inside the noframe and list.html files, still it has not bee indexed yet on its own.

I am jot sure whether Urls can be printed so I won't print my url (I bet they can be printed, but I'm unsure and I don't wont to make two mistakes, in case I find out I choose the wrong forum)

ciao
Alberto
 

Alucard

Member
Joined
Mar 25, 2002
Messages
5,920
Alberto,

The only bot that the ODP has is something we nickname "Robozilla". Its only purpose is to look at the URLs listed in the directory and work out if they exist or not. It does nothing more than that - it doesn't look at content or keywords. The ODP doesn't list anything based on keywords or revision date, so the fact that you may have updated pages linked to the page listed in the ODP is of no interest to the bot.

I hope this answers your question.
 
V

vallini

Hallo

actually they are not updated files, they are brand new files, the last two I added are both over 100Kb of text, but after over 2 months they're not indexed yet, so my concern is: how can I make new files with wide contents (as such which can be of some, at least relative, relevance) be included as timely as they include the updates (in this latter case updates indeed) to the *index* files?

thank you in advance
Alberto
 

dfy

Member
Joined
Aug 2, 2002
Messages
2,044
Re: Some files indexed regularly, linked files are

The ODP doesn't have any form of spider, so we will not be crawling any of your site looking for new content. Once your main page is listed, that's it as far as we are concerned.

If you feel that your description needs updating, use the Update_URL link in the top right corner of the category your site is listed in.

If you're talking about Google results, we can't help you. We're not Google, and we have little knowledge of how they work.
 

hutcheson

Curlie Meta
Joined
Mar 23, 2002
Messages
19,136
Re: Some files indexed regularly, linked files are

vallini, you need to take questions about search engines and their spiders to a forum that talks about such things. This one doesn't. I visit webmasterworld.com and searchengineforums.com but there are other forums that will be prepared for such questions. This is not such a forum, we only deal with the Open DIRECTORY, which (as has been said) doesn't spider ANYTHING.
 
This site has been archived and is no longer accepting new content.
Top