I’ve got a new post up at ZDNet: Five things every Windows beta tester should know.
I tried really hard not to take any personal shots at anyone, including Paul Thurrott, who was highly vocal last week in a pair of posts on the subject. But Paul decided to call me out in the headline for his reply this morning:
Ed Bott: The end justifies the means
That’s pretty inflammatory, and it’s based on some interesting selective quoting. Here’s the chunk of my post that Paul quoted,. Notice what he chooses to emphasize in bold:
Frankly, I’m having a hard time working up any level of sympathy for those doing the complaining, partly because I heartily approve of the way Windows 7 development is going right now and partly because I have seen the feedback process up close and personal. Microsoft is getting a bad rap from a group of people who are mourning the reality that they’re no longer being treated as privileged elites.
It’s almost like he didn’t notice the second part of that sentence, the part he chose not to boldface. Here, let me do it: “partly because I have seen the feedback process up close and personal.” Funny how that just slipped past. No, I don’t think the end justifies the means. I disagree with Paul’s thesis that Microsoft isn’t listening. On the contrary, I think they’re doing a better job of incorporating feedback today than they did in the Vista era. Smarter, too.
The reality is a lot of people, including Paul and me, have been talking with Microsoft developers about Windows 7 for a long time. A little over a year ago, I had an NDA meeting with the Media Center team where they showed me some of the stuff they were working on and asked for my opinion on it. Some of that feedback has made it into the product, and some hasn’t. I has a chance to sit down with usability professionals that week as well, and saw firsthand how they were working closely with users to flesh out the goals and design of how Media Center would work in the next release. We also talked a lot about blogs and forums like The Green Button and went into some of the detailed feedback from those sources, much of it solicited directly from forum members by Microsoft developers.
Now, that’s just one example, but it’s a good illustration of something that Sinofsky wrote about in his E7 blog post:
Feedback about Windows 7 of course starts before we’ve written any code, and by the time we’ve got running code thousands of people outside of Microsoft have provided input and influenced the feature set and design of Windows 7. As we’ve seen, the input from even a small set of customers can often represent a wide variety of choices–often in alignment, but just as often in opposition. As we’re developing the features for Windows 7 we work closely with PC makers, enterprise customers, and all types of customers across small business, education, enthusiasts, product reviewers and industry "thought leaders", and so on. We shape the overall "blueprint" of the release based on this wide variety of input. As we have design prototypes or code running, we have much more targeted and specific feedback by using tools such as usability tests, concept tests, benchmark studies, and other techniques to validate the implementation of this blueprint. Our goal with this level of feedback is for it to be representative of the broad set of Windows customers, even if we don’t have a 1:1 interaction with each and every customer.
Once upon a time, it might have made sense to get design feedback from a widely released beta. That is not true today. If you’re building an entire operating system just to get feedback on its design, you’re setting yourself up for failure. The Windows 7 design was largely set before the beta code was released and that design incorporated a lot of feedback.
Anyone who looks at the Windows 7 beta and expects that a design change request has a significant chance of being incorporated is living in the past. The purpose of the Windows 7 beta is to gather automated feedback data from millions of real world installations and identify issues that can be fixed.
Go read my piece and Paul’s response and tell me what you think.
PS: Paul and I will be sitting at the same table this Thursday for a very small meeting (just a handful of people) to talk with HP about their plans for Windows Home Server. As always, I’m looking forward to chatting with him about this and other topics.
Ed Bott (@ wordpress.com) 
I fully agree with you on this, Ed. I love Paul’s site and podcast and think he’s one of the best technology analysts out there, but he’s wrong on this topic. The purpose of a Beta is to get the product in real-world scenarios and fix bugs. Period. Only in rare circumstances should anything significant be changed once the Beta is released. Otherwise, you will significantly affect your release schedule. The problem comes from companies like Google misusing the word Beta (and from the way Microsoft apparently approached the product lifecycle in the past). A Beta should be a fully complete, fully internally tested product. The feedback has already been taken into account in the design process.
If you think about it, most companies approach their software products in the same manner. How many other software products have you, as an average user, been able to have a say in how the product was designed this late in the product lifecycle.
Moreover, the features that Paul is complaining about are already written the way Microsoft clearly intended to create them. Paul’s suggestions are very debatable and may come from being to close to the product (i.e. not opening one’s mind enough). I, personally, like the way the taskbar works and think it’s better in most ways than the previous design.
ed:
we made dcr’s in the early feedback program during and after the vista beta. i think most of these went unnoticed. so, whoever said it’s time for feedback for the next version of windows not win7, is sadly mistaken, in my opinion. it will fall through the cracks, too, just like the vista early feedback program.
It’s hilarious that you’re here playing the victim. Paul’s post was no more malicious than your four word response to his point of view on this topic. “Give me a break,” is pretty dismissive, wouldn’t you agree?
After reading your post, I wouldn’t say the ends justifies the means is an accurate way to summarize your position. That infers that you would have some objection to the means if it weren’t for the end result. You appear to have no objection to the means at all. That’s the argument you and Paul should have.
I am happy to have Microsoft ignore beta-tester suggestions.
I want bug fixes and nothing more. I want Windows 7 sooner rather than later. I would take it “as it is” in W7 build 7000. I find it works, it works well, and it works better than Vista on the machines where I have tested/used both operating systems.
I think that Sinofsky knows what he is doing better than the blogging community. He has an excellent record of delivering quality software on time.
I’m with Ed – and the others here.
Windows 7 could be shipped right now if Microsoft wanted to hit the shelves right away. The only issues I’m seeing with the beta code are some missing drivers and minor tweaks.
One could argue about design choices that have been made in Windows 7. But when the product is already a significant improvement over Windows Vista and XP, I can’t get excited about these comments.
It’s clearly time to resolve the driver and fit and finish items with the Windows 7 beta – and then move on to the next edition of Windows. Perhaps some of the feature suggestions from the bloggers and analysts can make it into that edition – while the rest of us enjoy Windows 7 when it hits the stores this Fall.
I’ve been reading Yours and Paul’s articles for a number of years and I think both of you do a good job most of the time. On this instance I totally agree with Ed and the great response we see for the Windows 7 beta against the disastrous response for Windows Vista shows it…
Ed,
I have to agree with you 100%. To use Paul Thurrott’s own terminology since I’m a frequent respondent on his site, Paul Thurrott has jumped the shark. To simply say that Windows users and all the feedback is being ignored is being very naive. Companies that are completely incapable of responding to consumers needs become dinosaurs. In a word, extinct. I’ve had several responses from Microsoft in my daily uses of the Windows 7 beta. I think the process has evolved from the honest failures in the Windows Vista beta program and the RTM reception of Vista.
Vista wasn’t a failure. A failed OS wouldn’t even boot. Vista boots and works. Obviously there was technical issues that created a public backlash. However, considering all that had to be done with XP SP 2 and SP3, Windows Server OSes, and other security issues, Vista didn’t have enough incubation time. Windows 7 is the remedy and and an ecosystem improvement. I don’t believe that if Windows Vista was such a colossal failure, that the same core would be used in Windows 7. The fact that Windows 7 is Vista with the proper incubation, necessary changes, improvements, and continues to evolve is a credit to the work in Vista.
However, Ed is very much correct that to many chef’s end up spoiling the pot. There’s a point when you have to make hard choices and I’m very proud of Sinofsky’s team for making them. Paul’s upset that his changes to the taskbar were not accepted by Microsoft. So far, the Windows 7 beta has been pretty damn good with the exception of the UAC flaw found by Rivera and Zheng. However, the change in stance by Microsoft did show that the beta input wasn’t being ignored because literally hundred’s of thousands of bloggers, respondents, testers, and commentators railed against the original decision. Microsoft changed its stance and people are eagerly awaiting the Release Candidate.
Ed, I’ve always loved your writing. Keep up the good work. Perhaps you need to have your own Podcast to give Thurrott a run for his money.
Peace.
Yupe I pretty much agree that asking “PLEASE PLEASE PLEASE INCLUDE THIS FEATURE” is pretty much retarded for a Windows beta tester (saw it in a comment right on this blog somewhere).
If there’s anything in your post that bugs me at all, it’s “Windows design is a series of compromises.” Certainly you don’t live to please everybody, and I have yet to try Windows 7 out (waiting for the real thing), but if I don’t know you any better, what you’re saying just further demonstrates the disadvantage Windows is at compared to Linux when it comes to being Free (as in free speech, freedom of expression). People will keep talking about how Microsoft locks you up inside your own computer and you have to be thankful whenever you’re given a choice. Hopefully, that wasn’t your intention and I didn’t say anything taboo.
I keep hearing from Microsoft how they incorporated feedback from “thousands of people outside of Microsoft” before they wrote a single line of code.
Who are these people? Microsoft won’t tell us.
The problem is that the product was effectively set in stone (“feature-complete”) before any sort of broad audience had the ability to provide feedback.
That’s how Apple does things.
I can has grammar? Spelling error you didn’t catch.
Honestly though, when this happens, you have to be flattered. To be read enough to have someone write about what you said, even in disagreement. And there is respect between people in the community; after all, none of the parties involved in this discourse are writing for Infoworld. 😛
C’mon guys, they just disagree. I don’t think that it was intended to insult or inflame. Reading Paul for a while now makes me think that he just wouldn’t do that, especially to another fanboy 😉
Doc
PS, I would bet that Win 7 will be a big hit. Can imagine revitalizing an old PC with a fast modern OS? People just may want to do it, instead of spending on a new one. I predict a lot of upgrades, even on some older machines that did not ship with Vista.
I like a good cat fight as much as anybody — God, how I used to enjoy wicked flame wars on Usenet — but I hope this settles into a simple recognition of disagreement between two reputable and talented journalists.
(PS: Tell HP for me that I love my MediaSmart EX470, but I’m still pissed about the memory…)
I think a Beta should be used to take input from users and make adjustments when a large majority of users see issue’s with the UI or any other aspect of the build. I know Paul made a few good points about the taskbar and I think his points were valid. I don’t think I’ll be able to use Messenger the way it is now, we should have the ability to turn those stupid notification feature’s off, reminds me of those stupid bouncing icons on the Mac. I feel like MS thinks the most important challenge with 7 is releasing it quickly, I’ll still buy it if it doesn’t change a few annoyances I have with it but I’d like to see it more customizable.
I agree with you Ed.
Microsoft has been getting feature and improvement feedback for Windows 7 since January 2007, that is to say since Vista’s public release. I agree with another commentator here, Google has bastardized the term beta. The only function of Windows 7 beta is to fix bugs, fix any problem-features that slipped through the cracks prior to the beta, and add minor refinements.
I also enjoy Paul’s site quite a bit but he is a bit of a bonehead at times.
Let’s be honest here: Microsoft knew that it had to fix Vista before they’d even released it.
Vista was such a troubled project from its earliest days as Longhorn, through PDC 2003 debacle, through the project reset shortly after PDC, through to a shaky delivery only partially remedied by SP1.
Luckily, Vista included a ton of telemetry catpure and Microsoft has REALLY good data on just how Vista is being used, how it performs on a wide variety of hardware and which apps crash most often and why.
This alone would be enough information to give the team 2 years’ work to fix many of the most troubling aspects of Windows. That they also decided to take a highly analytical approach to user experience is just a bonus!
Ed hits the nail on the head when he points out that Win7 is BY FAR the most stable, polished, responsive, consistent and powerful version of Windows ever shipped … and it’s not even at RTM yet!
Once Win7 is done and the platform has had some of its luster restored, the Windows team will be in a position to make Win8 something markedly special. If they’d not done what they’ve done for Win7, this would not be possible.
Massive props to the Windows team and to Sinofsky for having the courage to shun Allchin’s distracting over-exposure.
I think the issue here is one of the difference between
“Hey, send us your feedback on what you think”
and
“Hey, send us your feedback on problems and bugs you have”
There is a big difference between these two. If your prior message has been the first one, when you all of a sudden act like the 2nd, people have issues. I have no problem with either, and as others have said, I would rather they take the later road and get this thing finished cooking so I can buy it.
Scott
Vista was such a troubled project from its earliest days as Longhorn, through PDC 2003 debacle, through the project reset shortly after PDC, through to a shaky delivery only partially remedied by SP1.
Luckily, Vista included a ton of telemetry catpure and Microsoft has REALLY good data on just how Vista is being used, how it performs on a wide variety of hardware and which apps crash most often and why.
This alone would be enough information to give the team 2 years’ work to fix many of the most troubling aspects of Windows. That they also decided to take a highly analytical approach to user experience is just a bonus!
Ed hits the nail on the head when he points out that Win7 is BY FAR the most stable, polished, responsive, consistent and powerful version of Windows ever shipped … and it’s not even at RTM yet!