Guest fdwilk1 Posted November 30, 2002 Posted November 30, 2002 Is there a projection on when the rdf dump problem(s) will be resolved and processed? Thanks for any info you can provide.
Meta windharp Posted November 30, 2002 Meta Posted November 30, 2002 No, but staff is working oin the problem first priority. As I posted some times now, a single RDF generation needs about a week and is (for speed reasons) not very good to debug. So doing bugfixing is quite timeconsuming - please dont hold your breath waiting <img src="/images/icons/frown.gif" alt="" /> (We are still hoping for a christmas present - things seem to have become a bit better from last to this week...) Curlie Meta/kMeta Editor windharp
Guest swisstony Posted December 1, 2002 Posted December 1, 2002 Christmas? OMG, doesn't this mean that all versions of the ODP are going to be a minimum of 3 months out of date by the time of the next update? That's a serious issue. I suppose it shows just how dependent upon the ODP editors/staff many sites are. Do you think that this will damage the ODP in the long term, as providers look for a more "reliable" alternative?
giz Posted December 1, 2002 Posted December 1, 2002 There is no-one else with a directory of over 3 million web sites to go to. The problems will be fixed, but are complex and it takes nearly a week to see if each fix has cured all the problems. Staff have said that the RDF is the highest priority, so rest assured that it is being worked on. Information about the RDF can be found at: http://dmoz.org/rdf.html and http://dmoz.org/rdf/Changes.html and http://dmoz.org/rdf/tags.html and http://dmoz.org/help/getdata.html and http://dmoz.org/rdf/ .
Guest swisstony Posted December 22, 2002 Posted December 22, 2002 Any progress yet? Christmas is close and I expect that the staff have finished now for the holiday period...
apeuro Posted December 22, 2002 Posted December 22, 2002 Once the RDF starts to compile it's outside the hands of staff. The current compilation (which was started on the 10) is still chugging through - it should be done sometime this week (earlier rather than later). It looks like this is the one.
Guest swisstony Posted December 22, 2002 Posted December 22, 2002 Thanks. That is a 12 day compilation so far... is it meant to take that long? Last time I heard a time frame it was a week for a build.
beebware Posted December 22, 2002 Posted December 22, 2002 Staff are eager to get an working RDF dump out as soon as possible and it's my thoughts (i.e. not necessarily fact) that staff may have tryed changing the RDF generation system to try and automatically eliminate the problems that have been plaguing the system for the last few weeks: each time it failed, it required human intervention to manually correct database errors. Hopefully, we'll know shortly - I've heard that the current RDF has just gone into to "integrity checking" phase: if it passes, yipee! If it doesn't, well, looks like we didn't get out Christmas present after all...
giz Posted December 22, 2002 Posted December 22, 2002 This RDF has been a few days extra compared to 'normal'. Maybe it reflects that there are a lot more sites now in the directory? Maybe the RDF generation has been slowed down a little by the many other processes going on at the same time? Maybe the way it works has been changed to make it more robust? Whatever the reason, here's hoping that it works OK this time.
Guest swisstony Posted December 23, 2002 Posted December 23, 2002 Yup, I bet there are a whole bunch of people who will be very happy to see the dump working again. Looking forward to it <img src="/images/icons/smile.gif" alt="" />
Guest swisstony Posted January 7, 2003 Posted January 7, 2003 Well, Christmas and the new year have come and gone; do we have any news on the progress of the RDF dump? Some cheery news would be very welcome <img src="/images/icons/smirk.gif" alt="" />
Meta windharp Posted January 8, 2003 Meta Posted January 8, 2003 No good ones, sorry. Staff is still fighting the bugs. Curlie Meta/kMeta Editor windharp
totalxsive Posted January 8, 2003 Posted January 8, 2003 Latest I've heard is that a new dump is being generated now and should work this time. But then, you've probably heard that loads of times so you probably have a right to be sceptical, I'm afraid. Certainly the last failed dump provided some 'new' errors which had been missed before, which have been fixed.
Guest swisstony Posted January 9, 2003 Posted January 9, 2003 Thanks, I will pass the info on <img src="/images/icons/smile.gif" alt="" />
Guest swisstony Posted January 15, 2003 Posted January 15, 2003 The current compilation should be due around now if it started on the 6th - has anyone heard about the progress? Thanks
uzs980 Posted January 15, 2003 Posted January 15, 2003 The process failed, probably due to bad software which has been exchanged in the mean time. The RDF process has been restarted now.
Guest Posted January 15, 2003 Posted January 15, 2003 that'd be 'bad hardware' <img src="/images/icons/smile.gif" alt="" />
uzs980 Posted January 15, 2003 Posted January 15, 2003 Errr ... Sorry about that. You're right, of course. Must have been due to my bad English - I'm German, you know. <img src="/images/icons/wink.gif" alt="" />
Guest swisstony Posted January 16, 2003 Posted January 16, 2003 So are we looking at a 7 day wait, or has it increased to 12 days (like the last one)? Also, is it not possible to have a second set up also creating the RDF at the same time, just in case there are such problems? Thanks
Meta windharp Posted January 16, 2003 Meta Posted January 16, 2003 Such matters are discussed inside the ODP. Servers are set up by AOL/Netscape, so in some way the ODP depends on how many server capacity they provide. Curlie Meta/kMeta Editor windharp
Meta hutcheson Posted January 16, 2003 Meta Posted January 16, 2003 It is unlikely that the staff programmer would seriously consider that approach; it is almost equally unlikely that they have the extra hardware available to do it; and I for one am not going to give up the Quake server until orlady and her evil minions are driven into the Sea of Oblivion (which might be as early as tomorrow morning, but could drag on into the weekend.)
giz Posted January 16, 2003 Posted January 16, 2003 >> So are we looking at a 7 day wait, or has it increased to 12 days (like the last one)? << I believe that it was only 12 days as it "died" after a few days and had to be restarted. An RDF run that fully completes (with or without errors) should take about 5 or 6 days I do believe; though this is subject to other server load as well.
Guest swisstony Posted January 16, 2003 Posted January 16, 2003 OK, thanks. So it looks like we can hope to see something around the 20th/21st. That's 4 months after the last one - a heck of a long time to wait <img src="/images/icons/smirk.gif" alt="" /> Thanks Again
totalxsive Posted January 17, 2003 Posted January 17, 2003 Yeah, we know. Remember not having a good RDF dump affects us as editors too.
totalxsive Posted January 20, 2003 Posted January 20, 2003 Just to let you know, the RDF failed to generate properly again yesterday.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now