ODP structure.rdf.u8.xml is incorrect

Hello,

I try to parse the structure.rdf.u8.xml file and i discover lot of mistake in it :
For exemple all the category below are not declared in the structure.rdf.u8.xml as <Topic r:id="xxx"> but they are use as child of some category declared in the structure.rdf.u8.xml file (<altlang r:resource="Afrikaans:xxx"/> for exemple)

Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/Adult_Check
Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/Adult_Check/Gold
Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/FreeNetPass
Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/SexKey
Top/Arts/Music/Instruments/Stringed/Guitar/Luthiers
Top/Arts/Music/Styles/Country/Bands_and_Artists/W
Top/Arts/Music/Styles/Dance
Top/Arts/Music/Styles/Dance/Disco/Producers_and_Remixers
Top/Arts/Music/Styles/Folk
Top/Arts/Music/Styles/Hip_Hop
Top/Arts/Music/Styles/Hip_Hop/Reviews
Top/Arts/Music/Styles/Jazz/Bands_and_Artists
Top/Arts/Music/Styles/Regional_and_Ethnic/Cajun_and_Zydeco
Top/Arts/Music/Styles/Regional_and_Ethnic/Latin/Bands_and_Artists
Top/Arts/Music/Styles/Rhythm_and_Blues
Top/Arts/Music/Styles/Rock/Heavy_Metal/Bands_and_Artists/W
Top/Arts/Photography/Resources/Directories
Top/Arts/Television/Programs/Science_Fiction_and_Fantasy/S/Stargate_Series/Stargate_SG-1/Fan_Fiction
Top/Arts/Writers_Resources/Conferences
Top/Business/Construction_and_Maintenance/Tools_and_Equipment/Operator_Training_School
Top/Business/Consumer_Goods_and_Services/Toys_and_Games/Models
Top/Business/Consumer_Goods_and_Services/Toys_and_Games/Models/Remote_Control
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/L
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/M
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/N
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/W
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/X
Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/Z
Top/Games/Puzzles/Mazes/A
Top/Games/Puzzles/Mazes/B
Top/Games/Puzzles/Mazes/C
Top/Games/Puzzles/Mazes/D
Top/Games/Puzzles/Mazes/E
Top/Games/Puzzles/Mazes/F
Top/Games/Puzzles/Mazes/G
Top/Games/Video_Games/Developers_and_Publishers/U/Ubi_Soft_Entertainment
Top/Games/Video_Games/Developers_and_Publishers/U/Ubi_Soft_Entertainment/Titles
Top/Games/Video_Games/Shooter/Q/Quake_Series/Quake/Q-Zone
Top/Health/Fitness/Pilates_Method/United_States
Top/Health/Public_Health_and_Safety/Patient_Safety/Q
Top/Health/Public_Health_and_Safety/Patient_Safety/T
Top/Health/Public_Health_and_Safety/Patient_Safety/U
Top/Health/Public_Health_and_Safety/Patient_Safety/V
Top/Home/Cooking/Soups_and_Stews/Techniques
Top/Kids_and_Teens/Games/Computer_and_Video/Final_Fantasy
Top/Kids_and_Teens/International/Español/Artes/Teatro_y_Drama
Top/Kids_and_Teens/International/Español/Juegos/Videojuegos/Noticias_y_CrÃ*ticas
Top/Kids_and_Teens/International/Español/Juegos/Videojuegos/Trucos_y_Consejos
Top/Kids_and_Teens/International/Russian/Школа/Наука
Top/Kids_and_Teens/International/Russian/Школа/Наука/Биология
Top/Kids_and_Teens/People_and_Society/Psychology/Bullying
Top/Kids_and_Teens/Teen_Life/Health/Diseases_and_Disorders
Top/Regional/Asia/Hong_Kong/Travel_and_Tourism/Lodging/Hotels
Top/Regional/Asia/India/Arts_and_Entertainment/Movies/Hindi/Titles/X
Top/Regional/Asia/India/Localities/X
Top/Regional/Europe/United_Kingdom/England/Essex/Colchester/Business_and_Economy/Shopping/Furniture
Top/Regional/Europe/United_Kingdom/England/Essex/Colchester/Government
Top/Regional/Europe/United_Kingdom/England/Society_and_Culture/Gay,_Lesbian_and_Bisexual/Nightlife
Top/Regional/Europe/United_Kingdom/Recreation_and_Sports/Paintball
Top/Regional/Europe/United_Kingdom/Society_and_Culture/Gay,_Lesbian_and_Bisexual/Nightlife
Top/Regional/North_America/Canada/Quebec/Localities/L/Le_Gardeur
Top/Regional/North_America/United_States/Colorado/Localities/W/Winter_Park/Recreation_and_Sports/Cycling
Top/Regional/North_America/United_States/Florida/Localities/P/Pahokee/s
Top/Regional/North_America/United_States/Massachusetts/Travel_and_Tourism/By_Locality/Z
Top/Regional/North_America/United_States/Minnesota/Health/Fitness
Top/Regional/North_America/United_States/Minnesota/Health/Fitness/Personal_Training
Top/Regional/North_America/United_States/Nevada/Localities/I/Incline_Village/Recreation_and_Sports/Gambling/Casino_Resorts
Top/Regional/North_America/United_States/Vermont/Localities/A/Arlington/Recreation_and_Sports
Top/Regional/North_America/United_States/Vermont/Localities/B/Barton/Travel_and_Tourism/Parks
Top/Regional/North_America/United_States/Vermont/Localities/B/Brighton/Weather
Top/Regional/North_America/United_States/Vermont/Localities/H/Hartland/Travel_and_Tourism
Top/Regional/North_America/United_States/Vermont/Localities/H/Huntington/Travel_and_Tourism
Top/Regional/North_America/United_States/Vermont/Localities/L/Londonderry/Business_and_Economy/Shopping
Top/Regional/North_America/United_States/Vermont/Localities/L/Londonderry/Business_and_Economy/Shopping/Books
Top/Regional/North_America/United_States/Vermont/Localities/M/Montpelier/Travel_and_Tourism/Lodging/Hotels_and_Motels
Top/Regional/North_America/United_States/Vermont/Localities/P/Perkinsville/Travel_and_Tourism
Top/Regional/North_America/United_States/Vermont/Localities/P/Pittsford/Business_and_Economy/Shopping
Top/Regional/North_America/United_States/Vermont/Localities/P/Proctor/Travel_and_Tourism
Top/Regional/North_America/United_States/Vermont/Localities/S/Swanton/Travel_and_Tourism
Top/Regional/North_America/United_States/Vermont/Localities/W/Weston/Travel_and_Tourism/Lodging/Hotels_and_Motels
Top/Regional/North_America/United_States/Vermont/Localities/W/White_River_Junction/Travel_and_Tourism/Lodging/Hotels_and_Motels
Top/Regional/North_America/United_States/Vermont/Localities/W/Wilmington/Travel_and_Tourism/Lodging/Hotels_and_Motels
Top/Regional/North_America/United_States/Vermont/Localities/W/Woodstock/Travel_and_Tourism/Lodging/Hotels_and_Motels
Top/Regional/North_America/United_States/Vermont/Regions/Central_Vermont/Travel_and_Tourism/Lodging/Vacation_Rentals
Top/Regional/North_America/United_States/Vermont/Travel_and_Tourism/Lodging/Resorts
Top/Regional/Oceania/Australia/New_South_Wales/Health/Fitness/By_Municipality_
Top/Regional/Oceania/Australia/New_South_Wales/Municipalities/Newcastle_City/Business_and_Economy/Organisations
Top/Regional/Oceania/Australia/New_South_Wales/Municipalities/Sydney_City_Council/Business_and_Economy/Organisations
Top/Regional/Oceania/Australia/New_South_Wales/Regions/Greater_Sydney/Business_and_Economy/Organisations
Top/Regional/Oceania/Australia/New_South_Wales/Regions/Hunter_Valley/Business_and_Economy/Organisations
Top/Regional/Oceania/Australia/New_South_Wales/Regions/Orana/Business_and_Economy/Organisations
Top/Science/Biology/Neurobiology/People/P
Top/Science/Biology/Neurobiology/People/Q
Top/Society/Gay,_Lesbian,_and_Bisexual/Travel/Lodging/Europe/United_Kingdom/England
Top/Society/Law/Organizations/Bar_Associations/V
Top/Society/People/Seniors/Organizations/British_Pensioners_in_Australia
Top/Society/Religion_and_Spirituality/Christianity/Churches/Europe/United_Kingdom/England/G
Top/Society/Religion_and_Spirituality/Christianity/Churches/Europe/United_Kingdom/England/H
Top/Society/Religion_and_Spirituality/Christianity/Denominations/Methodist/United_Methodist/Local_Churches/United_States/North_Carolina/U
Top/Society/Transgendered/Crossdressing/Personal_Pages/N
Top/Society/Transgendered/Crossdressing/Personal_Pages/O
Top/Society/Transgendered/Crossdressing/Personal_Pages/Q
Top/Society/Transgendered/Crossdressing/Personal_Pages/U
Top/Society/Transgendered/Crossdressing/Personal_Pages/W
Top/Society/Transgendered/Crossdressing/Personal_Pages/X
Top/Society/Transgendered/Crossdressing/Personal_Pages/Z
Top/Sports/Baseball/College_and_University/Z
Top/World/Arabic/مجتمـع/صفØ*ات_شخصية
Top/World/CatalÃ*/InformÃ*tica/Internet/Cerca/Directoris/Open_Directory_Project/Llocs_que_usen_les_pÃ*gines_de_l'ODP
Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació
Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació/Disseny_i_desenvolupament_web
Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació/PHP
Top/World/CatalÃ*/Medicina_i_salut/Productes_i_seveis
Top/World/CatalÃ*/Regional/Europa/ItÃ*lia/Regions/Véneto
Top/World/Chinese_Simplified/商业/制é€*业/化工/农药化工
Top/World/Chinese_Simplified/商业/制é€*业/化工/æ—*机原料
Top/World/Chinese_Simplified/商业/制é€*业/化工/日用化工
Top/World/Chinese_Simplified/商业/制é€*业/化工/有机原料
..... and more !

What do you thing about it ??


Regards
stéphane

http://www.arkadia.com/usa/
http://www.arkadia.com/usa/real-estate/
http://www.arkadia.com/usa/commercial-real-estate/
http://www.arkadia.com/usa/vacation-rentals/
http://www.arkadia.com/usa/loan/

http://www.arkadia.com/fra/
http://www.arkadia.com/fra/immobilier/
http://www.arkadia.com/fra/immobilier-entreprise/
http://www.arkadia.com/fra/location-vacances/
http://www.arkadia.com/fra/credit/

What you see simply means that those categories don't exist anymore, but there are still links to them from some places. Which is something our data users should take into account when importing ODP data.

Basically there are two different reasons for this issue:

1) The category has been removed. In this case the link can't be corrected and you could either remove it or leave it (and have it point to an invalid place).

2) The category has been moved. On http://rdf.dmoz.org/ there is a link to a CatMv history that you could use to correct those links while parsing the RDF if you like to. Or you could simply choose to live with those bad links.

We have been thinking about solutions for this problem, but currently our software is not capable of detecting such situations and handling them correctly. If you take into account that the ODP is built on a so called "flat file database" (i.e. lots and lots of files all over a huge directory structure equivalent to the category structure) you can imagine that this task is no easy one. So I guess there won't be any solution soon, sorry to say.

Hmmm,

If ODP are interresting, i build a software than correct all the bad link in the structure.rdf.u8.xml. (It take only few minutes for my software to correct (remove) all the bad entry)

Also, their is some other problemes : Some ressources in topic are duplicated !! For exemple one topic have "related" resource to "xxx" but also have a "narrow" resource to the same "xxx" ! and sometime more craizy, some topy have 2 sames resource "narow" where just the title change !!


Regards
stéphane

http://www.arkadia.com/usa/
http://www.arkadia.com/usa/real-estate/
http://www.arkadia.com/usa/commercial-real-estate/
http://www.arkadia.com/usa/vacation-rentals/
http://www.arkadia.com/usa/loan/

http://www.arkadia.com/fra/
http://www.arkadia.com/fra/immobilier/
http://www.arkadia.com/fra/immobilier-entreprise/
http://www.arkadia.com/fra/location-vacances/
http://www.arkadia.com/fra/credit/

Correcting the links should be done on the directory, not on the dump. Otherwise our directory itself has invalid links remaining. Otherwise, writing a script to do so isn't that much of a problem, so I am sure our technical staff will manage this if it is decided to do so

I talked to one of our technical staff about the duplicate links, and he assumed you mean that a category contains multiple links to the same target category with different names. That is perfectly allowed, to have different titles for the same thing. Especially in long lists of subcategories this can help users find what they want. So that's no bug, but a feature.

Btw: We have a "no signature fof noneditors" policy in this forum to prevent linkspam. That's why the "signatues" feature is turned off. I strongly suggest that you edit your previous postings and remove the links from the footer, thank you.