Live data users completely stuffed for weeks

Currently (and for at least a week now), live users of the data, including my site www.localpin.com are completely stuffed because DMOZ for one reason or another is not returning pages before a timeout occurs.

This is not just my site, but for example, all users of the DWodp live program (http://www.dominion-web.com/products/dwodp_live/) such as http://www.lightmysite.it/eng/search_engine.html which frequently give the error message:

Unable to connect: Timeout of 20 reached on port 80 to domain dmoz.org

Click to expand...

This is extremely frustrating, and because Netscape holds a de-facto monopoly on DMOZ level of service, there is nothing, absolutely nothing, that live data users like me can do about it.

I think this shows a low regard for the following part of the social contract http://dmoz.org/socialcontract.html

6. Our Priorities are Our Data Users and the Community

We will be guided by the needs of our data users and the ODP editorial community. We will place their interests first in our priorities.

Click to expand...

For my site, which I am trying to make a professional user of the data, it is a major crisis - when my site cannot pull the data from DMOZ, it might as well not exist.

Some questions.

So, what about the priority of sites that actually use the DMOZ data live like mine? I don't think much of this social contract if I have a week or more without being able to use DMOZ properly.

And, as a more general question, what is the real commitment of Netscape to providing the server capacity or technical support or whatever else it is needed to really make DMOZ reliable?

I have been stuck waiting months and months before for the RDF dump. Now I am stuck waiting weeks for live data.

Does Netscape have some conflicts of interest if DMOZ actually had efficient (i.e. fast) turnaround on sites submitted, and was reliable in terms of live performance and RDF dump frequency?

What is the incentive for Netscape to invest in the DMOZ project (something I've never quite understood)?

Wouldn't DMOZ be better off spun off into some sort of Open Data consortium, rather like Open Source development?

Don't get me wrong: I am a bigger fan than most of the readers of this post of the DMOZ principle, since I base my entire website on it, but when it hasn't been working properly for a week or two, and I really need it to be working, then I am extremely frustrated.

Well, there are a few thoughts for y'all ;-).

Had you considered getting your feed from directory.google.com?

Can't you use an ODP clone that is updated regularly, or set up your own clone using the RDF data and update it regularly yourself?

robots.txt said:

# If you need to examine many dmoz pages, please download the rdf file from
# http://rdf.dmoz.org/ instead of crawling us.

Click to expand...

Shouldn't you be using the rdf instead of screen scraping?

So, what about the priority of sites that actually use the DMOZ data live like mine? I don't think much of this social contract if I have a week or more without being able to use DMOZ properly.

Click to expand...

javasprout shouldn't you be complying with the license as I highlighted in the other thread?

Or is that not an important factor to you?

For my site, which I am trying to make a professional user of the data, it is a major crisis - when my site cannot pull the data from DMOZ, it might as well not exist.

Click to expand...

It shouldn't exist without the license being adhered to.