Hello;
I'm thinking about designing a search engine. I somehow need to get a list of domain names or URLs for the search engine to crawl and add to the index.
Does anyone know how I can get a listing of just the raw URLs in the dmoz database?
Thanks.
I'm thinking about designing a search engine. I somehow need to get a list of domain names or URLs for the search engine to crawl and add to the index.
Does anyone know how I can get a listing of just the raw URLs in the dmoz database?
Thanks.