For more than a decade, some variation of this screen has been part of the Windows Disk Defragmenter utility:
Well, no more. The colorful progress window has been banished in Windows Vista. The Disk Defragmenter runs in the background, and the only interface available is the one to change the time it runs or to manually start or stop a defrag.
I remember people who actually used to sit and watch the Windows 95 Defrag progress window, in which little boxes of different colors were shifted around. We’ve come a long way.
Technorati tags: disk defragmenter, Windows Vista
Ed Bott (@ wordpress.com) 
Damn. There go my Satuday nights.
I resemble that remark! I still watch the display, or at least glance at it, while I’m working on someone’s laptop and my own PC at the same time. It’s sort of mesmerizing.
I noticed this while compiling the features for my ongoing guide to Vista features, and I wasn’t sure if I liked it or not. For one, the display did provide some visual cue as to how far along the defrag was — but now that the system sets the defrag to happen once a week in the background during off-peak hours, I’m not sure it’s really that critical anymore. (And if I want detailed defrag information, I can always run the command-line version of the defrag app.)
I wrote about this a little while ago… I think it’s one of Microsoft’s dumber design decisions. They should have left the old UI for power users and made the default the dumbed-down version.
Link
I think the people at Diskeeper must be behind this…Frankly, I like the display. It even has the same effect on me that my cell phone has on my 20 month old niece. 🙂 I also like reading the report to make sure that it actually defragged all of the defraggable files.
I am thinking that when they went back to renegotiate with Diskeeper on the neutered XP defragger, they wanted more money and some intern at Microsoft said “I can write that!” So they did.
The XP defragger sometimes lies to save its reputation. It will say you have no fragged files in the final report, but if you exit, restart it, and do another analysis there will be fragmented files.
One reason is that XP can’t defrag files that are open; I suspect that still applies in Vista. Of course, the files you have open are often the ones most desperately in need of defragging, such as your mail folders. It really should tell you that!
Vista’s home-grown UI-less defrag eliminated the need to lie and saved the trouble of explaining. Novices are oblivious, Microsoft doesn’t have to pay Diskeeper royalties, and experts will go out and buy Diskeeper or some other product. Everyone wins! 🙂
I really don’t like the new UI, because you can’t see any real progress, I agree with robert, the should left the old interface in for more advanced users…
i;m with robert.leave the ui alone as it was.i like to see what is going on.and can see for myself how well it went.there are a lot of things on vista that should give you the option of how you want to view it.like xp. did.guess i’m becoming an old fart and will adjust.and don’t leave to many tabs open.ect .ect.
I had thought they would have improved the defrag, not dumb-ed it down further. I was hoping to get the ability to move around the swap file and or defrag it. I have NO idea if this n00b friendly defragger actually does any of that.
I guess perhaps the commandline is more useful if it will give me more info. Robert is right. There’s no reason for ths thing to be dumbed down for power-users.
At least their releasing a monad-esque power-user commandline tool…
Wow, another reason I don’t like Vista. While I understand that watching it 100% of the time whiel it runs is idiotic, the GUI is a lot nicer to look at when you are checking to see how long it has to go to finish.
If no one liked the GUI display then none of the third party defrag programs would come with one. Microsoft is really killing themselves with these stupid decisions.
Andrew, why do you need to know when it’s finished? It’ll finish when it’s finished, and it’ll back out of the way when you want to do something.
So the upgrade I just bought is basically for my XP, if I read you guys right. Vista will take care of itself??
If I recall, last time I checked, in RC1, that display is still available if you go to the Computer Management console (still available from right-clicking Computer, then Manage, IIRC) and selecting Disk Defragmenter in the left-hand pane.
I think it was RC1 – it might have been an earlier beta or CTP.
Mike, it’s no longer there. It was there in XP and you might be right that it was there in earlier builds, but definitely not in any RCs.
Has anyone noticed the neat icon of Ed you get when you shortcut his blog to your Desktop?
(Nothing to due with defrag, I just watched too many of the 95 ones years ago.)
Simple, if you are specifically defraging to complete a system optimization for someone it is nice to see how far it has to go and the GUI is nicer to look at then some percentage bar. I like to clean and defrag a system before working on it so it is responsive before I touch it and the GUI is a very pleasing way to check on the progress of the defrag.
I mean why did they add the GUI in the first place? Why do third party defrags come with them to begin with? Yes I agree automatic in the background defragmenting is the way to go but the nice GUI is there for power users who want to look at it. I also prefer it in combination to detailed reports when looking at the fragmentation level of a drive.
Right now Diskeeper is on Automatic but at any time I can pull up the GUI and get a quick visual of how fragmented the drive is. This is a really nice feature. I also always look at the GUI when working on client machines before looking at the text report. It is quick, easy and satisfying. I mean why all the hoopla of the new 3D interface? Why not go back to a text console? Come on, stripping the GUI out for no reason is a really bad decision. This is not progress but going back in time.
Pretty soon we will have one big button that you press on the desktop and you press it and wait……………………………………………………….
Eventually it will say “Finished” and you will have no idea what happened.
All this is going to do is sell more copies of third party defragmenters.
No offense but this is the absolute worst thing I have heard about Vista yet.
Andrew, if that’s the absolute worst thing you’ve heard about Vista, then I predict a few Microsoft developers and executives will be thrilled.
I’ll address the substance of your complaints in a follow-up post. But one quick answer to your question “why did they add the GUI in the first place?”
Because they licensed a piece of software from Executive Software, and Executive Software built their Express version with a GUI.
Ed,
Every third party defrag comes with a GUI, every version of defrag for Windows, hell even DOS came with one. It is one thing not having it on by default but having no access to it at all iis unacceptable. Am I correct to say that it gives no detailed reporting either?
Just because Executive Software now Diskeeper Corp BTW came with one does not mean they were the first. Norton Defrag had one too.
How many people have you heard complain that their third party defrag comes with a GUI or that their is even one for Windows? I’ve never heard it.
Why not make a more detailed GUI like Diskeeper comes with if the old one was too hard for people to read? But yeah I read through the Microsoft Vista Defrag FAQ so Ed if you are going to make a post answering these questions with that don’t bother, I still think it is the poor man way out for Microsoft. Instead of fixing the problem they simply removed something else.
PerfectDisk is the defragger i use for more than 6 years.
The defragmenter over the years hasn’t scaled well compared to increased in hdd capacities…It takes a hell lot of time….So people defrag less often.
Andrew, I said I’d respond in a separate post. And no, I’ve never regurgitated FAQs from Microsoft here, so why should I start now?
Based on what you’ve written here, it appears there are a few fun facts you don’t know about defragmenting in general, about the history of defragmenting in Windows and DOS, and about the operation of the Vista defragger. So I think you’ll find this follow-up post interesting.
After reading most of the comments posted here, I get the impression that most of the people who will even find this a problem are seeing the wrong end of the picture.
I posted earlier that the command-line defrag tool can provide you with all the stats you really need, and the pros will be savvy enough to run it. Frankly, I’m not sure it’s important to make anyone else care; all they care about is if the system runs well. As long as I still have access to the command-line tool (which now has more options than it did before in XP), then I’m fine. The lack of a GUI in my defragmenter is not a deal-breaker.
I’m more disturbed by the common image that “pros” are simply happy with a command line true. That’s a load of crap and avoiding the more important and fundmental flaw: Why can’t we simply design (for a change) a GOOD interface that displays everything we need to know?
The tech is there, and the information is clearly available. Why make the user jump through hoops for this? This IS 2006 people!! Kiss those damned terminals good-bye! It’s time to take back the night on user interface design.
The defragger isn’t the deal-breaker. But it’s a sad state of affairs when in this day and age of computers you STILL have to resort to the command prompt to get BASIC information out of a tool that could EASILY be made available to the pros as well as the n00bs via a GUI.
Greetings all,
I find it interresting that:
1.) People seem to want a GUI for defrag and Microsoft has not provided it. (Why not an available command line switch for a progress bar?)
2.) Maybe I am about to leave for the weekend and I want to allow Windows to finish Defragging… Can I see how long I need to wait to finish so I can shut off the PC and go home?
Monday I want to restart the PC and immediately go back to work with the system maintenance already done.
OK, I assume I could tell the machine to shut down when it is done? (Yes, I realise Windows Updates may become available durring the weekend and Monday I will likely see them)
I have seen too many customer’s machines FAIL to shut down automatically in XP, Why should I expect Vista to be able to pull it off EVERY time it is told to?
3.) Can I assume maybe some 3rd party or Microsoft will make a Performance Console Snap-in for monitoring Defrag in a GUI?…. or I would hope I would be able to myself without having to learn a new interface.
Don’t get me wrong, at least Microsoft is still giving us a defrag tool for Vista, for that I thank them.
Off Topic: Does anyone know if installing a 3rd party video driver for improved performance in games/video editting will break DVD playback, HiDef or standard?
Part of my job requires me to edit videos and I use a self optimized driver for my video card. (My editting software uses the Video Card for rendering live/faster than live from the timeline and it must be able to flush the rendered frames to system ram)
So, Source is hard drive (DV video) it’s sent to the video card for rendering and is piped back to Ram, then is saved to Hard Disk. Compression/transcoding optional.
I am hoping this won’t be too close to reading from hidef DVD, rendering, and saving… aka. “ripping”
Personally I take VERY good care of my optical discs, and have no need for even backups… rarely do I ever watch a disc more than 2 times… (Brain Rot to me)
Thanks for the information,
Eric
Here is Microsoft making excuses:
http://blogs.technet.com/filecab/articles/440717.aspx
“Interestingly enough, one of the biggest and consistent complaints we had from users (broad sample here from home users to experienced IT Pros) in the past was that a vast majority of them had no idea what the detailed fragmentation statistics they saw meant. The Windows XP graphical view also had some limitations and inaccuracies that prevented it from being included in Windows Vista. If you really want to keep a close eye on fragmentation, I’d recommend using the command-line tool Defrag.exe.”
Wow talk about idiotic! Why not FIX the GUI to display properly and clearly explain what is going on? Because that makes too much sense!
Anyone who says the command line is a substitute should be shot.
Ed, don’t take this the wrong way but their is absolutely NOTHING you can say that will excuse removing the GUI and progress bar for when you manually defrag except laziness and incompetence on Microsoft’s part.
I read the fact and found it full of sad and pathetic excuses. The absolute lack of information in Vista’s defrag is insulting. No reporting, no progress bar, no GUI. And no the command line is NOT a substitute for any of this.
The absolute ONLY thing good about it, is the automatic scheduling option. Microsoft really seems to be screwing over enthusiasts, first with their licensing which still screws over OEM users and now this.
Vista is going to become the OS for Dummies and those stupid enough to pay $400 for the “ultimate edition”.
Andrew, don’t take this the wrong way, but why don’t you wait until you READ what I have to write. What you just wrote is a pretty profound insult, pal.
I’m trying to finish a book right now (in fact, the reason this came up is I just finished a whole chapter on this and other tools) and haven’t the time to write about this for at least a few more days. But I will get to it.
I’m posting a summation of my own feelings about this issue on my own blog (see URL).
It probably won’t — DVD playback is usually controlled through a filter driver that is decoupled from the video driver itself. The only time I can see this happening is if you install a video driver that comes specifically with some kind of DVD decoder driver, but those are almost always a third-party program that you have to install separately.
Ed, Well don’t take it personally but if you are going to defend this in some way, that is how I feel. I was definitely not insulting you rather Microsoft. When third party defragmenters are able to give me everything I am asking for with clear information and detailed meaningful statistics then I cannot excuse Microsoft’s half-assed attempt.
Serdar, You can always tell if a drive is at 0% fragmentation level or not. Which means you can always tell if the drive is fragmented and create an accurate system for determining how badly it is. Everything you mentioned Third Party Defragmenters have been doing for years. You can then have seperate metrics for free space and like Diskeeper does a performance indicator of how much the fragmentation will effect you. Now I am simply asking for the fragmentation level and an accurate GUI with a progree bar.
The real problem with the progress bar is that Diskeeper still refuses to have a full single pass manual defrag option available. They can add one in a second but refuse to. I agree with them on multi-pass vs. time and resouces used but when you do a manual defrag you want it 100% complete, even if it takes some time. Since Disk Defragmenter is based on Diskeeper code this is the real problem. Even still Diskeeper still gives you some sort of progress bar with the manual option you get with Diskeeper.
Don’t get me wrong I understand why they did it but it was the wrong move for enthusiasts and treats us like the rest of the sheep.
Andrew, stop putting words in my mouth. Where did I say I was going to “defend” Microsoft’s decision? I said I’m going to do a follow-up post. And I also said you obviously are missing a whole bunch of facts about this. It’s no wonder, because you’re not even willing to listen to information that doesn’t fit with your opinion already.
Must be wonderful to be able to make decisions without having to listen to facts. Makes for a much less complicated life, I guess.
First of all I did not put any words in your mouth. I simply stated “if you are going to…”
So far you haven’t stated any “facts” that you claim I am missing. So how can I not listen to them?
All I have is one FAQ from Microsoft on this that is filled with a bunch of excuses. Here are the words from Microsoft:
“While I agree that having no progress is bad…”
They admit it is bad! But make excuses for it that since they consider it “inaccurate” they simply removed it, instead of fixing it? You are telling me in six years this is the best millions of dollars can give us?
I also have your post here “Adios, Defrag Display” that unless I am reading it wrong is happy that the GUI is gone. “We’ve come a long way.”
You’re reading it wrong.
Maybe that is the problem?
Anyway it turns out the Defrag Program in Vista is written by Microsoft. Notice all the complaints about no GUI:
http://blogs.technet.com/filecab/pages/447168.aspx
Sounds like we are Beta testing for Microsoft again and the Defrag that comes with Vista is experimental.
A scary thought:
“Q: Do you have a kind of “I-FAAST” technologies (Intelligent File Access Acceleration Sequencing) in the built in defragmenter ?
A: I’m unfortunately not familiar with this technology.”
The program manager for Vista’s Defrag program is not familiar with Diskeeper? The Diskeeper 10 Professional Premier Edition with I-FAAST has been out for over a year with this. That gives me ALOT of confidence in the Vista Defragmenter.
Third party defragmenters are overrated, especially if you have lots of RAM. As I understand it, this will be even more true with Vista than with previous operating systems.
The only thing that matters regarding defragmentation is how quickly a file on the hard disk loads into RAM (including the system cache). A defragmented file loads faster than a fragmented file. In this regard, a file defragmented by the native defragger will load into RAM just as fast as a file defragmented by a third party defragger.
The real performance gain comes from the file data and code remaining in physical memory (including the system cache) once you load it from the disk. Using a utility such as Task Manager, you have lots of control over how your computer actually manages memory and CPU usage. And again, this will be even more so under Vista’s Superfetch feature than under XP.
There is much more to it than that. The built-in Disk Defragmenter with XP is resource intensive and cannot be run while you are using the computer unless you want to reduce performance. It is also slow and not automatic. The reporting features for third party defragmenters are also more extensive and accurate. There are many, many more reasons that make Third Party Defragmenters much better than the native one.
Superfetch is simply another high speed cache for commonly used applications using flash memory devices like USB memory sticks. It has nothing to do with fragmentation. Fragmentation is simply something that will always have to be dealt with so long as we use magnetic hard disks. While it is true that proper caching and file system placement can reduce the negative performance effects imposed by fragmentation, it is still there and negatively effects the life span of the HD regardless.
Andrew, you need to do some more research.
“Superfetch is simply another high speed cache for commonly used applications using flash memory devices like USB memory sticks.”
Wrong. That’s ReadyBoost. SuperFetch is a form of smart memory caching and is indeed related to disk access. It has nothing to do with USB flash memory keys.
Andrew, the obvious solution is to run the native defrag program when you know you will be away from your computer.
And again, defragmentation affects only how quickly a file is loaded from a hard disk to RAM. Once it is in RAM, as long as it stays there, the computer can access it at lighting quick rates. This is why multitasking works so well on XP systems with lots of RAM. You can monitor memory usage with tools such as Task Manager or the Performance console. The time you spend reading defragmentation reports is about 100 times better spent simply understanding and tracking how your system uses physical memory.
Ed:
Are you getting prepared for all of those e-mails and blog comments complaining that Vista is a memory hog because it uses up almost all “free” memory? 🙂
Ken:
I’ve already dealt with that one. At some point I’m going to make a post in my own blog about the metaphor I used to try and help people wrap their minds around the subject. There’s a lot of really confused people out there.
Serdar:
I read your blog entry (and immediately subscribed to your blog) regarding defrag in Vista. I tend to agree pretty much 100% with everything you said, although I am still using XP.
Incidentally, in your post No. 42, are you talking about defragmentation or memory management? If the latter, can you quickly elaborate? TIA.
Ken: Memory management, actually, but I’ve had to do some explaining of both in different circumstances. And thanks for signing up! I hope you like the ongoing Vista-for-XP-users feature I’ve been doing; there’s many more installments to come.
Ken: Sorry, I forgot to elaborate. I think the problem is that many people are still working from the mentality that was very much in vogue when PCs had 64MB of RAM. Vista’s memory consumption doesn’t seem to be all the more than XP right now, but I have been running it on a machine with a relatively light application load; I’d need to run it on my production system to really get a feel for that (and that may not happen for a bit yet).
Serdar:
I haven’t read the Vista for XP users feature yet, but I will (along with most of the rest of your blog entries). I decided not to try to be a beta tester for Vista, but I do intend to get it and I have been reading lots about it.
Ed,
Actaully I have done only 2 minutes of research on any of those techs for Vista. Yep I got those backwards thanks for clearing that up. I wouldn’t want to get it wrong like I did about you liking the removal of the GUI in the Vista Defrag. But from what I did read it says Superfetch is based on application usage patterns and mentions nothing about disk access.
Ken,
If that was true than the Vista team would not have spent so much time making sure the new Vista Defrag runs in low IO mode and backs off when you use your PC. But you just proved my point, Diskeeper runs as soon as the system is idle all the time, any time. The XP one can only be scheduled and does not back off. So if you miss the schedule your screwed. The Vista one at least trys to finish the next time you are idle.
That is only part of the problem. My complaints are with the Reporting and progress bars for the manual defrag.
No one is spending all their time looking at the reports but I want the ability to be able to. I want more info then some stupid icon saying defrag now or not. If the info is available from the command line then they should make it availabe from a GUI.
Andrew,
“But from what I did read it says Superfetch is based on application usage patterns and mentions nothing about disk access.”
Where do you think it “fetches” the data from? The whole point is to anticipate disk access, get the data off the disk before it’s needed while the system is relatively idle, and have it in fast RAM when you’re ready for it.
Caches are ALL about disk access.
Andrew:
Have you ever attempted to measure how long it takes for a known fragmented file to load into RAM? And have you tried comparing this time with the CPU time required to run a third party program like Diskeeper (measurable using Task Manager), much less the time required to operate the program and read the reports? And how many files that you regularly load into RAM are even defragmented in the first place?
I regularly defragment my own hard drives using the native XP defragger. I probably do it way more often than really necessary from an overall performance standpoint. But I don’t think that third party defraggers can withstand a vigorous cost-benefit analysis, not in terms of time and certainly not in terms of money. Unfortunately, I had to learn this the hard way.
Ken: I’m more or less in agreement with this view. I’m not sure that there’s enough provable, sustainable benefit in the real world from a third-party defragger, including things like optimized file-placement techniques, to make it worth the effort. Too much of this stuff is too hard to nail down and reproduce reliably. The one thing that is clear is that more free space is always better, so if you’re on a drive that’s 70-75% full, moving to one that’s not will provide far more substantive a performance boost than a defrag app.
But it clearly states it is to optimize memory usage by keeping frequently used applications in Memory based on application usage patterns where Ready Boost is specifically targeting disk access. Obviously everything is loading off the HD but Superfetch just sounds like something Windows should have been doing to begin with. Neither has anything to do with reducing Disk Fragmentation.
Ken,
There is no time involved in using Diskeeper because it runs in real-time automatically as soon as the system is idle. There is effectively no negative time wasted “using it”. However at any time I can pull up detailed reports which give useful and detailed information. If I wanted to run a manual defrag it will give me a progress bar.
Now I find it funny that you claim to “regularly” defragment your HD and are the one claiming it is a waste of time or rather the benefits do not outweigh the time wasted to do it. While I have not manually defraged my HD in three years and have wasted exactly ZERO minutes on it.
It is a moot point as to whether it improves performance since even Vista is coming with a Defragmenter. But the argument of exactly how much it will benefit someone is impossible to say because there are so many variables involved which include the person’s usage patterns and the hardware involved.
More importantly than anything else is that intelligent defragmenting will improve the life of your HD.
Andrew:
I think we have pretty much already covered your previous points. On your last point about HD life, defragmenting may reduce the overall number of accesses on the HD. This is because the HD treats a fragmented file as two or more physical files (depending on number of fragments) that it must access separately. But this applies only to accesses to fragmented files.
In theory, more accesses means a shorter HD lifespan, although even that is a guess rather than known fact. In practice, most of the files we regularly access will be defragmented. For example, I use Outlook every day. All of files used to load Outlook are defragmented. When I update Outlook (or Windows generally) at Microsoft Update, I immediately defragment my HD. And so on for every other program or data file that I regularly use. And because I leave my computer on the time, most of these files are already in the system cache (or working memory) constantly, meaning that I only rarely need to access the hard drive anyway. Incidentally, this is why more RAM will give you a much bigger performance boost than the money you spend on third party defraggers and other so-called “optimization” software.
One last point, which I learned from Ed a while ago: HD’s have a long life. They will almost certainly outlast your computer, with or without regular defragmentation. And again, we aren’t really arguing over regular defragmentation. I do it at least weekly (which is probably excessive, but I used to be even more obsessive about it than I am now). We are arguing only over whether to use the native defragger or a third party program with more bells and whistles.
Ken,
Actually you haven’t covered any of my points. Unless of course you take your opinions of them as “coverage”. I don’t, I consider them your opinions.
I work for a system OEM and know full well the real life span of HDs which can vary greatly. Nothing against Ed but he writes books and has no way to know the average length of HDs unless he deals with them on a volume level. They can fail immediately or last ten or more years. Only a very few things have been proven to extend their life, more memory and defragmenting. HD tech has generally improved but system memory has also, which I have seen to be the greatest factor in extending their life. Systems with more RAM have fewer drive failures. The same with defragmenting. The systems that use Diskeeper have fewer drive failures then the ones that do not. There is no way to ignore the data, I see the RMA numbers myself.
Your usage patterns have nothing to do with how the average person uses their system. And what is cached in memory has nothing to do with the fragmentation level of the file before it was read into memory. It is absolutely ridiculous to make decisions based on your personal usage patterns as to whether something is effective or not.
Don’t confuse Disk Defragmenters with other nonsense “optimizers”. I can tell you obvious never deal with end user systems on any remote level. I get systems in so poorly maintained, including fragmented that access and load times for some applications were cut in half.
If you think that HDs will outlast your computer, I have a bridge to sell you. HDs are the number one most failure prone component of your PC.
Andrew:
It doesn’t surprise me that regular defragmentation and lots of RAM can extend the life of a HD. Both mean fewer disk accesses, and fewer disk accesses should extend HD life (other things being equal).
You act like I disagree with you on the benefits of regular defragmentation. I don’t. Our disagreement concerns the relative merits of third party defraggers such as Diskeeper over the native Windows defragger.
If I am dealing with a novice user using a system with relatively low RAM, I might recommend Diskeeper (or perhaps PerfectDisk, with its scheduling feature enabled). That’s not me and it is decidedly not my computers.
The merits are with Diskeeper that it is automatic in real-time when your system goes idle. This is the most efficient way to defrag, since there is no scheduling. If you schedule the built-in Disk Defragmenter in XP it must be when you are not using your system due to it’s high resource usage. But it is not easy for the average user to even figure out how to schedule the built-in one out of the box. Vista improves this but it still is scheduled instead of efficiently running in real-time at idle. It only does that if you miss the schedule. Diskeeper is also much faster than the built-in Disk Defragmenter in XP and defragments more files more efficiently.
Almost no one manually defrags except tech related people, this is an easy to prove fact. Ask any place that does PC service. Which is why an automatic solution is essential.
What does a novice user have to do with anything? Why do you want to manually defrag your system when it can do it automatically? Seems like a waste of time to me.
Andrew:
In answer to your questions:
The novice user is much less likely to defragment, much less on a regular basis. He is much less likely to do other regular HD maintenance. He is much more likely to minimize running programs (which increases paging to disk even on systems with lots of RAM — weird but true, check it out). He is much less likely to multitask efficiently to maximize efficient use of memory and minimize HD I/O. He is much more likely to turn his computer on and off on at least a daily basis.
I do manual defragmentation from the Computer Management console when also checking Event Viewer and Disk Management (to make sure I don’t have disk errors). It is a weekly maintenance routine that on average takes me only 1-2 minutes (closer to 1 than 2). Meanwhile, I don’t have yet another process and related CPU cycles running on my system. I don’t need to worry about periodically updating any third party software. I don’t need to read Diskeeper or PerfectDisk related Event Viewer messages. I typically run Disk Defragmentation at a time when I know I am then going to step away from the computer. So, by my reckoning I actually end up saving time, or at least break even.
Ken, can you give me a citation about minimizing running apps causing more paging to disk? I’d like to find out more about that.
Serdar:
I don’t have a citation (although see the last paragraph below). I observed it myself and discovered it almost by accident. Here is how you can observe the same thing in XP Professional (which I run on my two computers).
First, run the Performance counsel and set up the following counters in System Monitor: Memory/ pages sec, Memory/ Pages Output/Sec, Process/Working Set_Total, Paging File/% Usage, and Memory/Available Memory MBytes (or KBytes, any of them will do).
Second, load Outlook (any application will do, but Outlook makes more and better use of physical memory than most applications). Read a few e-mails, look at the calender, whatever. The more operations you do, the better or more clear the test results will be. Get the working set for Outlook sufficiently high to make the following test meaningful. After a few minutes, minimize Outlook.
Third, immediately look at System Monitor. You can observe the following as it happens:
(1) A sharp drop in working set memory.
(2) A gradual increase in the size of the paging file, coupled with a similar gradual rise in available physical memory.
(3) A sharp increase in the Pages Sec and Page Output Sec readings (indicating that working set memory is being paged to disk).
Finally, when these effects end after a few seconds or maybe a minute, exit Outlook. The hard drive indicator lights up as the Outlook working set memory previously paged to disk is released. The page file will come back down to almost its original size (although not all the way in my experience).
One more thing. You would think that the working set memory is being paged to the modified page list or standby list (depending on whether the Outlook working set data was “dirty” or “clean”). But I don’t think that’s what actually happens. If it was happening, you would expect to see the System Cache number rise gradually in Task Manager. But the two or three times I tested this (including just now), that didn’t happen. Apparently the working set memory gets paged to disk and then the pages are transferred to the free list or zero demand list. This would explain why available memory increases while the System Cache does not. Again, if that is what is actually happening (I may have not tested this enough).
While I didn’t find a citation to this, I did find a citation to an article that noticed everything except the last paragraph. The author seemed to think that this was a great way to get additional memory. He even calls it his “tip of the year.” I strongly disagree. This is what “memory optimizers” actually do. Instead of sending working set memory to the System Cache (i.e. the standby list or the modified-file list), it goes straight to the much slower HD. But here is a link to the article (scroll down to the end). It may load slowly, so be patient:
http://www.expresscomputeronline.com/20030915/techspace01.shtml
One more thing: to see the actual results, you will need to adjust the scales on System Monitor. Otherwise, all the memory graphs will stay at the very top of the screen.
None of which has anything to do with the fragmentation level of the drive or tha lack of defragmenting. An automatic solution is the best which even the Vista team has found to be essential for maintaining optimal performance. Absolutely everything essential for security and maintenance needs to be automatic on a novice users machine. But it makes no sense for anyone to manually do something that can be done automatically.
I never manually defrag, ever and my drive stays defragmented. I never have to check for updates (which are rare with Diskeeper) because it does so automatically. Processes in memory do not consume CPU cycles unless they are being used – that is absolutely irrelevant. I never have to check event viewer messages because it defaults to nothing more then the diskeeper service was started. You can add more detailed messages to event viewer if you like, the option is there in Diskeeper but it is unnecessary. I don’t have to ever worry about running it, it runs automatically when the system is idle and I am not using the disk. I don’t do anything and my HDs stay defragmented automatically without ever interfering with me using the computer. That is better than any solution you have stated.
Andrew:
I guess my reaction here is more one of misplaced emphasis than real disagreement. In the grand scheme of things, fragmentation may account for a very small percentage of drags on performance. This is because of how XP processes multiple threads and manages memory.
This is why my favorite performance tool is Task Manager (or the Performance console), not Disk Defragmenter or its third party substitutes. This is something I first learned from Ed here over a year ago. I have since taken the “graduate” course — Mark Russonivitch’s book on Windows Internals.
If you prefer automatic defragging to manual defragging, I agree that Diskeeper (or maybe PerfectDisk) is the tool for you. I no longer care enough to spend the money (even though I have spent the money on past versions). I simply manually defrag the drive when it needs it (still more often than what the program itself recommends) as part of a weekly routine maintenance program. I do it all (except for Disk Cleanup) from the Computer Management console. My hard drive runs like a bat out of hell, just like the rest of my system.
This reminds me of a favorite Barry Switzer saying. He supposedly once told a recruit that the University of Oklahoma will win national championships with him and it will win national championships without him. This pretty much sums up my attitude about having Diskeeper or PerfectDisk on my system.
I am not arguing there are other ways to improve performance. I fully agree on this and that Defragmenting is not the main one either. Defragmenting to me is a maintenance task that can be automated.
How much defragmenting helps IMO is completely variable based on the system and how the person uses it. I deal with so many end users that I see alot of the variables. I also see alot of the extremes. You have not seen a slow system until you have seen one infected with over 5000 malware entries! For many it does help simply because they never do any maintenance. And there are others that it will offer no noticeable improvement. Overall I feel better recommending something that prevents one potential type of performance problem. And I completely agree that with NTFS and XP defragmenting is not as important as it was with FAT.
I also think you are looking at this from being approached by the salesman type. Diskeeper and other third party defragmenters are not the performance tool that will solve all your performance problems. Nor is it the most important or only tool you should have. It is simply one other thing you can do to maximize the performance of your system. I approach everything as someone who cannot buy a new PC and wants to get every last drop of performance out of their system.
A program like Diskeeper is absolutely the only piece of maintenance software I have ever recommended anyone even spend money on and that includes Anti-Virus and Anti-Spyware (which you can both get for free). You can get the home edition for $29 and honestly that is all most people need, the other features in the better versions like Fragshield and I-FAAST are nice but not worth the price IMO. $99 for I-FAAST? Not worth it for the average user IMO. Hell Diskeeper releases a new version every year and you definitely don’t need to buy the new version each time. v9 from two years ago will get the job done. Though I would recommend the new version every 3 years at least, simply for the new reporting features and performance improvements.
Funny thing is a I would recommend people spend $30 on Diskeeper Home than on Norton or any other security software. Now say you don’t want to spend money at all? Then I recommend scheduling the built-in Disk Defragmenter with a free program called StartDefrag. But in XP it is not the best solution IMO. And I know you manually defrag every week but for $30 every three years not to ever have to do that again? Works for me.
Before I begin – a brief intro: I’m the Product Manager at Diskeeper. I host a blog where Andrew, who I’ve never spoken with, wrote and pointed me here. For the record, I have spoken with Serdar by phone and email in the past.
My first comment is that I understand what Microsoft did with Windows Vista, and I agree with their basic “marketing” direction. On that note, and I realize it was said in jest, Diskeeper Corporation did not play any role in development of the Vista product nor do we receive any royalties.
Ideally defragmentation, anti-spyware, anti-virus, and any other “utility” should just RUN and no end user should ever have to deal with them. However, there are two Windows markets with different needs, the consumer and the corporate. As Ken noted, most consumers do not have the wherewithal to manage a computer, nor should they. I believe Vista has done a good job of addressing basic consumer needs with its utilities. You all know that Windows competes with Mac in the consumer space, and the Mac is a very user-friendly product with many utilities embedded or less significant as an “issue”. IMO Windows is mimicking what Mac has offered in respect to defrag. Mac offers a great user experience and Windows is trying to match that.
Someday, given their current direction, Microsoft will probably arrive at the point of doing everything (or near–everything) related to the OS (utilities, security, etc…).
In corporate IT, all utilities must allow for complete control by a system administrator. As Ken alluded to earlier, everything has some cost associated with it. With respect to corporate IT, I regularly state “there is no such things as free software” (my take on the “free lunch” adage). IT professionals need to be able to manage/control apps, and the overhead, coupled with applications that are not complete or perfect, incurs overhead.
While MS may state that the defrag backs off (low priority I/O), I strongly suggest that before forwarding this message as true that it be confirmed. We’ve (Diskeeper) tested it and yes, it is significantly better than XP, but it is not as transparent as suggested. Now granted, that doesn’t necessarily mean it will be an issue for someone (if resources used for a period of time are a non-issue). For that matter, XP has always been relatively easy to schedule, given that you have off-time you can run it within.
The second point I want to make is really just an opinion based on my personal experience in technology.
I work behind a desk all day, and while I do speak with enterprise customers throughout the week, I am somewhat removed from the day to day of what a System Administrator goes through. For many years I was at the front lines as a support engineer (traveling to customers 3-4 months of the year) and formerly as an IT consultant in private practice. On that, Andrew raises an excellent point. People like myself can theorize all day long about technology, from behind a desk, but the truth isn’t known until it proves itself out in the real world. I can even test in a lab, on my personal computers or even my company’s computers, but I’ll never be able recreate all the things that people do with computers. I could never say Diskeeper (or defrag) guarantees 20% performance improvement based on my narrow evaluation because I’ve only seen it in limited scope.
Just like the media or an industry analyst, my job is to represent the end user. I do not believe that I or anyone else can pretend to be an expert about what an end-user goes through without asking, visiting, and experiencing the reality of the end-user. For me to speak for “all customers” would be asinine, even if I’d spoken with hundreds of them. What Ken is saying and doing is real for Ken. What Andrew is doing works for him. While no one in this thread is suggesting the other is wrong, the important thing is what is or isn’t important is relevant to the person’s circumstances (I think that is already agreed upon, but I felt it appropriate to restate anyways).
If anyone is speculative on the value of defrag in the real world go out there and look – talk to people, visit their company. As a vendor, I would ask two things of a potential customer (IT Professional), the first is to look for themselves. It’s the vendor’s responsibility to ethically educate the customer on the issues (the media is a huge help here), provide information on the solutions (the media is a huge help here again) and in some cases even provide tools to make the evaluation easier. Some people evaluate correctly and buy, some evaluate correctly and don’t buy. I’m content either way, because the person has made the right choice for them!
The only other thing I ask of an IT professional is to evaluate correctly. Some people evaluate incorrectly and buy, some evaluate incorrectly and don’t buy. That is a case of garbage-in garbage-out. They won’t get reliable results either way (except out of sheer dumb luck that they made the right decision) because they evaluated wrong.
Technology buying behavior for a consumer is often driven by their perception of value and/or recommendation by a trusted authority. Technical Media represents the authority figure very well, as do tech savvy friends, relatives, etc… It is up to the product to deliver a perception of value (I say perception only because the true value may elude a less-technical buyer who may not know how or what to look for). Apart from the fact that these driving factors can help the unaware buyer avoid technology scams (like memory optimizers), they aren’t ideal as each user’s realized value can vary. Anti-virus software, for example, is far more valuable to a consumer that does not know to not open .vbs files emailed from unknown sources. Ideally consumers are wholly capable of making the right technical decisions (as IT professionals should be), but that won’t happen any time soon.
When a corporation buys tens or hundreds of thousands of dollars of software (which happens daily at Diskeeper Corporation) you can be sure that it wasn’t because of some pretty advertisement or warm-fuzzy using the software on their own PC. The senior technology analysts at these corporations evaluate for months/years in their “real world”. They make decisions based on fact, not emotive best-guess.
If anyone is interested I wrote a paper that goes into a more detailed technical, albeit largely theoretical, explanation of disk performance issues, including some of the discussions brought up earlier: http://files.diskeeper.com/pdf/Stability_WhitePaper.pdf
Michael:
Thanks for your comments, which were very interesting. I particularly agree with one of your points, which is that performance or value is relative to the user. What is good for me may not be good for someone else in different circumstances.
Ken: Very true. One of the things I found after doing my own investigation was that the kind of defrag that works well on a server may not work well on a desktop PC by dint of being too intrusive.
Thanks Michael. I can honestly say I have never had a single complaint ever about Diskeeper in all the years I’ve been using it. Serdar, I use Diskeeper on all the servers I setup and all the client’s desktops. Nothing instrusive about it and each version just keeps getting better. You never even know version 2007 is running.
I’d agree with Michael that DK does an awesome job, with no issues and keeps the servers running smooth.
We checked out the 07 version and its cool!