I used to be managing editor of PC World, so I think I have some right to say that a story now running on the Today @ PC World blog, Win XP SP2 Halts 15% of Systems, Survey Says is irresponsible nonsense. I don’t know where the author went to school, but he needs to go back and learn some of the fundamentals of journalism. Here’s a quote:
System administrators who have been installing Windows XP Service Pack 2 on their own PCs and on test systems are reporting the results of their practice runs to the SANS Institute Web site–and the failure rate seems to be pretty high.
While the vast majority of the 752 people who published feedback (as of this blog entry) had no problems, 15 percent of these competent technology professionals complain that severe problems prevent the PC from being used after they installed the service pack. About half of those say they had to rebuild the entire system from scratch after they ran SP2.
Let’s go through a few things here:
First, this survey is not statistically weighted, so generalizing with the bold headline “SP2 Halts 15% of Systems” is unsupported by any evidence. On the contrary, the survey’s respondents are, by definition, going to be weighted disproportionately in the direction of those with problems. Sysadmins and computer users who have trouble-free experiences with software do not go out on the Internet looking for information about that software. The group in a self-selecting survey such as this one consists disproportionately of people who had problems, went looking for answers, and found this survey.
Second, see there in the second graf where the author says “15 percent of these competent technology professionals complain…”? Go and read the survey for yourself. Do you see anything that required a test of technical competence? This survey is open to the public, and characterizing the respondents with any degree of accuracy is impossible. Even the site’s sponsor, the SANS Institute, acknowledges that “we will not verify submissions for accuracy (we just can’t).” A reporter interested in accuracy might say that the site has a reputation for attracting sysadmins with more technical knowledge of computers and networks than a general-interest site such as PC World’s, but that’s about it.
In fact, that whole “technically sophisticated” thing cuts two ways. In my experience, power users are more likely to experience problems with any upgrade using any operating system. Why? Because they’re willing to use bargain hardware, install unsigned drivers, and experiment with all sorts of software.
I’d be willing to bet that many (not all, but many) of the specific problems reported by the respondents to the SANS survey involve blue-screen errors. Those errors are almost certainly caused by faulty drivers (hardware or system-level software, such as AV or CD-burning software). A less-likely scenario is that the blue screen is caused by a faulty hardware component such as a bad RAM chip, but my money is on a bad driver. Anyway, installing SP2 didn’t cause the problems they experienced; it simply exposed those problems. The bad driver was a ticking time bomb just waiting to go off. What they need to do is figure out where the driver is that’s causing their problem and replace it with a proper version, after which they can install SP2.
But anyone reading this story, especially if they focus on the headline and don’t think about the details, will assume that there’s a 15% chance that installing SP2 will hose their system. That’s nonsense, and PC World shouldn’t be spreading that message around.
Ed Bott 