Is Google dying?

Daniel Brandt of Google-Watch says Google is dying. Or, more accurately, its abaility to index pages on the Web is severely broken:

On sites with more than a few thousand pages, Google is not indexing anywhere from ten percent to seventy percent of the pages it knows about. These pages show up in Google’s main index as a listing of the URL, which means that the Googlebot is aware of the page. But they do not show up as an indexed page. When the page is listed but not indexed, the only way to find it in a search is if your search terms hit on words in the URL itself. Even if they do hit, these listed pages rank so poorly compared to indexed pages, that they are almost invisible. This is true even though the listed pages still retain their usual PageRank.

I have been complaining about this since April 2003, and it has become more visible in 2004. There is no method to Google’s madness, which is another way of saying that this phenomenon is not characteristic of any particular type of site. It is happening across the entire landscape of large sites. […]

I first noticed [this problem] in April 2003. That was the month when Google underwent a massive upheaval, which I describe in my Google is broken essay. When that essay was written two months after the upheaval, it would have been speculative to claim that the listed URL phenomenon was a symptom of the 4-byte docID problem described in the essay. It was too soon. But sixteen months later, the URL listings are beginning to look very widespread and very suspicious. It’s a major fault in Google’s index, it is getting worse, and it is much more than a mere temporary glitch.

Basically, Brandt claims that Google’s index is limited to approximately 4 billion pages by the numbering structure it uses. Now that its index of listed pages has reached trhat limit, the only way for a new page to get added is for an old, obsolete one to fall off.

If this is true (and the argument appears to be quite plausible and well reasoned), then this is sobering indeed.

Glad I didn’t buy into the IPO.