ODP structure.rdf.u8.xml is incorrect

[arkadia]

Hello,

 

I try to parse the structure.rdf.u8.xml file and i discover lot of mistake in it :

For exemple all the category below are not declared in the structure.rdf.u8.xml as <Topic r:id="xxx"> but they are use as child of some category declared in the structure.rdf.u8.xml file (<altlang r:resource="Afrikaans:xxx"/> for exemple)

 

Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/Adult_Check

Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/Adult_Check/Gold

Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/FreeNetPass

Top/Adult/Image_Galleries/Ethnic/Ebony/Softcore/AVS/SexKey

Top/Arts/Music/Instruments/Stringed/Guitar/Luthiers

Top/Arts/Music/Styles/Country/Bands_and_Artists/W

Top/Arts/Music/Styles/Dance

Top/Arts/Music/Styles/Dance/Disco/Producers_and_Remixers

Top/Arts/Music/Styles/Folk

Top/Arts/Music/Styles/Hip_Hop

Top/Arts/Music/Styles/Hip_Hop/Reviews

Top/Arts/Music/Styles/Jazz/Bands_and_Artists

Top/Arts/Music/Styles/Regional_and_Ethnic/Cajun_and_Zydeco

Top/Arts/Music/Styles/Regional_and_Ethnic/Latin/Bands_and_Artists

Top/Arts/Music/Styles/Rhythm_and_Blues

Top/Arts/Music/Styles/Rock/Heavy_Metal/Bands_and_Artists/W

Top/Arts/Photography/Resources/Directories

Top/Arts/Television/Programs/Science_Fiction_and_Fantasy/S/Stargate_Series/Stargate_SG-1/Fan_Fiction

Top/Arts/Writers_Resources/Conferences

Top/Business/Construction_and_Maintenance/Tools_and_Equipment/Operator_Training_School

Top/Business/Consumer_Goods_and_Services/Toys_and_Games/Models

Top/Business/Consumer_Goods_and_Services/Toys_and_Games/Models/Remote_Control

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/L

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/M

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/N

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/W

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/X

Top/Business/Publishing_and_Printing/Publishing/Literary_Agents/Z

Top/Games/Puzzles/Mazes/A

Top/Games/Puzzles/Mazes/B

Top/Games/Puzzles/Mazes/C

Top/Games/Puzzles/Mazes/D

Top/Games/Puzzles/Mazes/E

Top/Games/Puzzles/Mazes/F

Top/Games/Puzzles/Mazes/G

Top/Games/Video_Games/Developers_and_Publishers/U/Ubi_Soft_Entertainment

Top/Games/Video_Games/Developers_and_Publishers/U/Ubi_Soft_Entertainment/Titles

Top/Games/Video_Games/Shooter/Q/Quake_Series/Quake/Q-Zone

Top/Health/Fitness/Pilates_Method/United_States

Top/Health/Public_Health_and_Safety/Patient_Safety/Q

Top/Health/Public_Health_and_Safety/Patient_Safety/T

Top/Health/Public_Health_and_Safety/Patient_Safety/U

Top/Health/Public_Health_and_Safety/Patient_Safety/V

Top/Home/Cooking/Soups_and_Stews/Techniques

Top/Kids_and_Teens/Games/Computer_and_Video/Final_Fantasy

Top/Kids_and_Teens/International/Español/Artes/Teatro_y_Drama

Top/Kids_and_Teens/International/Español/Juegos/Videojuegos/Noticias_y_CrÃ*ticas

Top/Kids_and_Teens/International/Español/Juegos/Videojuegos/Trucos_y_Consejos

Top/Kids_and_Teens/International/Russian/Школа/�аука

Top/Kids_and_Teens/International/Russian/Школа/�аука/Биологи�

Top/Kids_and_Teens/People_and_Society/Psychology/Bullying

Top/Kids_and_Teens/Teen_Life/Health/Diseases_and_Disorders

Top/Regional/Asia/Hong_Kong/Travel_and_Tourism/Lodging/Hotels

Top/Regional/Asia/India/Arts_and_Entertainment/Movies/Hindi/Titles/X

Top/Regional/Asia/India/Localities/X

Top/Regional/Europe/United_Kingdom/England/Essex/Colchester/Business_and_Economy/Shopping/Furniture

Top/Regional/Europe/United_Kingdom/England/Essex/Colchester/Government

Top/Regional/Europe/United_Kingdom/England/Society_and_Culture/Gay,_Lesbian_and_Bisexual/Nightlife

Top/Regional/Europe/United_Kingdom/Recreation_and_Sports/Paintball

Top/Regional/Europe/United_Kingdom/Society_and_Culture/Gay,_Lesbian_and_Bisexual/Nightlife

Top/Regional/North_America/Canada/Quebec/Localities/L/Le_Gardeur

Top/Regional/North_America/United_States/Colorado/Localities/W/Winter_Park/Recreation_and_Sports/Cycling

Top/Regional/North_America/United_States/Florida/Localities/P/Pahokee/s

Top/Regional/North_America/United_States/Massachusetts/Travel_and_Tourism/By_Locality/Z

Top/Regional/North_America/United_States/Minnesota/Health/Fitness

Top/Regional/North_America/United_States/Minnesota/Health/Fitness/Personal_Training

Top/Regional/North_America/United_States/Nevada/Localities/I/Incline_Village/Recreation_and_Sports/Gambling/Casino_Resorts

Top/Regional/North_America/United_States/Vermont/Localities/A/Arlington/Recreation_and_Sports

Top/Regional/North_America/United_States/Vermont/Localities/B/Barton/Travel_and_Tourism/Parks

Top/Regional/North_America/United_States/Vermont/Localities/B/Brighton/Weather

Top/Regional/North_America/United_States/Vermont/Localities/H/Hartland/Travel_and_Tourism

Top/Regional/North_America/United_States/Vermont/Localities/H/Huntington/Travel_and_Tourism

Top/Regional/North_America/United_States/Vermont/Localities/L/Londonderry/Business_and_Economy/Shopping

Top/Regional/North_America/United_States/Vermont/Localities/L/Londonderry/Business_and_Economy/Shopping/Books

Top/Regional/North_America/United_States/Vermont/Localities/M/Montpelier/Travel_and_Tourism/Lodging/Hotels_and_Motels

Top/Regional/North_America/United_States/Vermont/Localities/P/Perkinsville/Travel_and_Tourism

Top/Regional/North_America/United_States/Vermont/Localities/P/Pittsford/Business_and_Economy/Shopping

Top/Regional/North_America/United_States/Vermont/Localities/P/Proctor/Travel_and_Tourism

Top/Regional/North_America/United_States/Vermont/Localities/S/Swanton/Travel_and_Tourism

Top/Regional/North_America/United_States/Vermont/Localities/W/Weston/Travel_and_Tourism/Lodging/Hotels_and_Motels

Top/Regional/North_America/United_States/Vermont/Localities/W/White_River_Junction/Travel_and_Tourism/Lodging/Hotels_and_Motels

Top/Regional/North_America/United_States/Vermont/Localities/W/Wilmington/Travel_and_Tourism/Lodging/Hotels_and_Motels

Top/Regional/North_America/United_States/Vermont/Localities/W/Woodstock/Travel_and_Tourism/Lodging/Hotels_and_Motels

Top/Regional/North_America/United_States/Vermont/Regions/Central_Vermont/Travel_and_Tourism/Lodging/Vacation_Rentals

Top/Regional/North_America/United_States/Vermont/Travel_and_Tourism/Lodging/Resorts

Top/Regional/Oceania/Australia/New_South_Wales/Health/Fitness/By_Municipality_

Top/Regional/Oceania/Australia/New_South_Wales/Municipalities/Newcastle_City/Business_and_Economy/Organisations

Top/Regional/Oceania/Australia/New_South_Wales/Municipalities/Sydney_City_Council/Business_and_Economy/Organisations

Top/Regional/Oceania/Australia/New_South_Wales/Regions/Greater_Sydney/Business_and_Economy/Organisations

Top/Regional/Oceania/Australia/New_South_Wales/Regions/Hunter_Valley/Business_and_Economy/Organisations

Top/Regional/Oceania/Australia/New_South_Wales/Regions/Orana/Business_and_Economy/Organisations

Top/Science/Biology/Neurobiology/People/P

Top/Science/Biology/Neurobiology/People/Q

Top/Society/Gay,_Lesbian,_and_Bisexual/Travel/Lodging/Europe/United_Kingdom/England

Top/Society/Law/Organizations/Bar_Associations/V

Top/Society/People/Seniors/Organizations/British_Pensioners_in_Australia

Top/Society/Religion_and_Spirituality/Christianity/Churches/Europe/United_Kingdom/England/G

Top/Society/Religion_and_Spirituality/Christianity/Churches/Europe/United_Kingdom/England/H

Top/Society/Religion_and_Spirituality/Christianity/Denominations/Methodist/United_Methodist/Local_Churches/United_States/North_Carolina/U

Top/Society/Transgendered/Crossdressing/Personal_Pages/N

Top/Society/Transgendered/Crossdressing/Personal_Pages/O

Top/Society/Transgendered/Crossdressing/Personal_Pages/Q

Top/Society/Transgendered/Crossdressing/Personal_Pages/U

Top/Society/Transgendered/Crossdressing/Personal_Pages/W

Top/Society/Transgendered/Crossdressing/Personal_Pages/X

Top/Society/Transgendered/Crossdressing/Personal_Pages/Z

Top/Sports/Baseball/College_and_University/Z

Top/World/Arabic/مجتمـع/صÙ�Ø*ات_شخصية

Top/World/CatalÃ*/InformÃ*tica/Internet/Cerca/Directoris/Open_Directory_Project/Llocs_que_usen_les_pÃ*gines_de_l'ODP

Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació

Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació/Disseny_i_desenvolupament_web

Top/World/CatalÃ*/InformÃ*tica/Programari/Llenguatges_de_programació/PHP

Top/World/CatalÃ*/Medicina_i_salut/Productes_i_seveis

Top/World/CatalÃ*/Regional/Europa/ItÃ*lia/Regions/Véneto

Top/World/Chinese_Simplified/商业/制é€*业/化工/农è�¯åŒ–å·¥

Top/World/Chinese_Simplified/商业/制é€*业/化工/æ—*机原料

Top/World/Chinese_Simplified/商业/制é€*业/化工/日用化工

Top/World/Chinese_Simplified/商业/制é€*业/化工/有机原料

..... and more !

 

What do you thing about it ??

 

 

Regards

stéphane

 

http://www.arkadia.com/usa/

http://www.arkadia.com/usa/real-estate/

http://www.arkadia.com/usa/commercial-real-estate/

http://www.arkadia.com/usa/vacation-rentals/

http://www.arkadia.com/usa/loan/

 

http://www.arkadia.com/fra/

http://www.arkadia.com/fra/immobilier/

http://www.arkadia.com/fra/immobilier-entreprise/

http://www.arkadia.com/fra/location-vacances/

http://www.arkadia.com/fra/credit/

[windharp]

What you see simply means that those categories don't exist anymore, but there are still links to them from some places. Which is something our data users should take into account when importing ODP data.

 

Basically there are two different reasons for this issue:

 

1) The category has been removed. In this case the link can't be corrected and you could either remove it or leave it (and have it point to an invalid place).

 

2) The category has been moved. On http://rdf.dmoz.org/ there is a link to a CatMv history that you could use to correct those links while parsing the RDF if you like to. Or you could simply choose to live with those bad links.

 

We have been thinking about solutions for this problem, but currently our software is not capable of detecting such situations and handling them correctly. If you take into account that the ODP is built on a so called "flat file database" (i.e. lots and lots of files all over a huge directory structure equivalent to the category structure) you can imagine that this task is no easy one. So I guess there won't be any solution soon, sorry to say.

[arkadia]

Hmmm,

 

If ODP are interresting, i build a software than correct all the bad link in the structure.rdf.u8.xml. (It take only few minutes for my software to correct (remove) all the bad entry)

 

Also, their is some other problemes : Some ressources in topic are duplicated !! For exemple one topic have "related" resource to "xxx" but also have a "narrow" resource to the same "xxx" ! and sometime more craizy, some topy have 2 sames resource "narow" where just the title change !!

 

 

Regards

stéphane

 

http://www.arkadia.com/usa/

http://www.arkadia.com/usa/real-estate/

http://www.arkadia.com/usa/commercial-real-estate/

http://www.arkadia.com/usa/vacation-rentals/

http://www.arkadia.com/usa/loan/

 

http://www.arkadia.com/fra/

http://www.arkadia.com/fra/immobilier/

http://www.arkadia.com/fra/immobilier-entreprise/

http://www.arkadia.com/fra/location-vacances/

http://www.arkadia.com/fra/credit/

[windharp]

Correcting the links should be done on the directory, not on the dump. Otherwise our directory itself has invalid links remaining. Otherwise, writing a script to do so isn't that much of a problem, so I am sure our technical staff will manage this if it is decided to do so :-)

 

I talked to one of our technical staff about the duplicate links, and he assumed you mean that a category contains multiple links to the same target category with different names. That is perfectly allowed, to have different titles for the same thing. Especially in long lists of subcategories this can help users find what they want. So that's no bug, but a feature.

 

Btw: We have a "no signature fof noneditors" policy in this forum to prevent linkspam. That's why the "signatues" feature is turned off. I strongly suggest that you edit your previous postings and remove the links from the footer, thank you.