I am curious as how it can take some sites years to get in but on a couple of certain sites one has 4,850 entries And then another has over 38,000 entries? Some entries are quite new for both.
There are different kinds of "sites". Wikipedia is a "site", which has had thousands of contributors over the years. Like many fellow-editors, I generally check Wikipedia when I'm starting a new category or building up a small category.
Project Gutenberg is another special site, which has significant (book-length!) contributions from tens of thousands of published authors. Clearly, every author represented there should be listed separately.
Somewhere between those two sites lies the Catholic Encyclopedia, with thousands of articles, written by acknowledged experts, and deemed well worth publishing in print.
Geocities and Youtube are different kinds of "contributed content" sites: not every contributor deserves a listing, but many of them do. From here, the level of content goes down: imdb, rolling stone, and other large sites have earned a reputation as being reliable (for some values of "reliable") and worth considering, at least, for deeplinking.
It depends on whether you're thinking about "fair to the surfer" or "fair to the content creator" or "fair to the webmaster." "Fair to the surfer" means everything. "Fair to the content creator" is a minor issue, but not completely negligable. "Fair to the webmaster" is irrelevant, and ought to be. (Without a content creator and an interested surfer, a webmaster is like a bridge for nobody to nowhere.)
Obviously these sites have people on the inside.
Sometimes, sometimes not. "People on the inside" can't keep a site from being un-deeplinked, if the editors don't agree it's useful to deeplink it: Rolling stone lost thousands of links, even with people on the inside, because the editing community judged their content over-shallow and over-listed.
Furthermore it seems sub-categories are created just for these sites.
It may seem so. In the early days it actually was actually so. The Open Directory Project made agreements with some sites to base the category structure on the site's structure, and seed the categories with that site's deeplinks. After the first couple of years, that practice stopped. And clearly, it's no longer needed now that the ODP is one of the four or five largest taxonomies ever devised.
But usually it happens the other way around. An editor wants to build a new category for a not-so-famous person, a movie, a small town, and the best pages he can find are subpages of the old reliable sites.
An editor would have to work full time for any one of these sites. There are a several more like this and they are all Branded sites that heavily utilize Adwords and I am sure spend huge advertisement dollars with Google.
Ah, here's another place where a small change in perspective makes all the difference. We're looking at the whitespace, you're looking at the colors. See, all that matters to us is -- everything EXCEPT the ads. All that matters to you is the ads. We're not looking at the same website, even though it's the same domain name. We're not looking at the same page, even though it's the same URL. And that's why you're so confused about what we're doing -- you're not seeing anything that matters to us, and we're not looking at anything that matters to you.
That's fine. The web is a big place, and there's something for everyone to look at. If you ever want to look for sites that have something besides ads (or something in addition to ads), we'll have a large list of them for you. And if not -- that's OK too.
