This is probably just me not comprehending this, but when you said "The confusion is that, there is also a /p option that can be provided to defrag, like so:"
Did you mean the confusion is that they are both** valid, or that one is valid and one is not? ** eseutil /p mydb.edb and eseutil /d mydb.edb /p -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brett Shirley Sent: Thursday, August 18, 2005 10:27 AM To: [email protected] Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat I am actually a programmer for ESE (you know the database under Exchange, once know as JET Blue ... ) ... yes, it may come as a shock to some of you that building 7 garage door operator is not my only job duty at msft ... Anyway, I'd like to clear up some confusion and mistatements ... 1. The /p switch ... There was some confusion on the /p switch ... There are two different operations being discussed below that one can perform on an ESE database. One is called defrag and the other repair. It is CRITICAL everyone understand the difference between the two, because one is dangerous and destructive under certain circumstances. Defrag: eseutil /d mydb.edb Repair: eseutil /p mydb.edb The confusion is that, there is also a /p option that can be provided to defrag, like so: Defrag (w/o instate): eseutil /d mydb.edb /p The original mail understood this, but some subsequent mails got it mixed up, just wanted to make it explicit. I loathe ESEUtil's command syntax, BTW. :P Subcommands should always be full words, like repadmin. ;) 2. Repair (/p) is destructive. Repair is really an unfortunate term for this functionality, because like when you repair a car, it works again! That may not be the case after ESE's repair. The command should've been called "salvage". The command basically throws out any data that ESE doesn't understand due to physical or ESE logical corruptions in your database, basically salvaging what's left. The defrag w/o instate ("eseutil /d mydb.edb /p") is NOT destructive. Repair is dangerous. I always try to steer people away from repair. If though somehow the database has been corrupted, there is irreplaceable data it can really save you. If the database is in perfect order, both physically ("eseutil /k" checks this) and ESE logically ("eseutil /g" checks that), then in theory repair is safe. But that idea gives me the hee-bee-jee-bees. It is possible for a disk to return valid data on one read, and invalid data on a 2nd read, so it could never be perfectly safe. Did I mention I try to steer people away from repair. 2.a. Aside: NEVER run repair on an AD database. Off the subject of this mail, but it bears repeating. As you may or may not know, the ESE database engine is used in both Windows for Active Directory's database/ntds.dit and in Exchange for mailbox stores. In the Windows version of the ESE database engine, eseutil.exe is called esentutl.exe. Even though these binaries are similar, and based off similar sources, the versions are different, and compiled with the Esentutl.exe and eseutil.exe are Never run repair on an AD database. In fact in Win2k3 SP1, we disabled that functionality in esentutl.exe for AD databases. Ok, we're really offtopic for the thread, moving back to ... 3. Defrag (how it works) ... I'd like to go over very approximately the steps that ESE (offline) defrag goes through, because it will make some of the comments in the next point more poinaintly clear. Defrag works like this: Step 1 - Open the "source" database. ESE opens for reading the source or target database, that you've asked specified as the first non-flag argument after the /d sub- command. i.e. "mydb.edb" above. Step 2 - Create a "destination" database with a temporary name. By default the destination or temporary DB, is created in the same directory as the source database. Step 3 - Move the data table by table to the destination database. Enumerate over each table in the "source" database, and move each row of data to the "destination" database. This is why I call them source and destination. However, usually, eseutil and docs call the destingation the "temp. database". You'll see why in step 4. And indices are recreated in the process too of course. Step 4 - Move the destination database to the source database. ESE moves the destination/temp database name, to the source database name. This is the step that specifying "/p" to defrag skips. Note: You may specify the destination database name for step 2, in this process by adding an argument like "/tE:\mytempdrive\emailstuff.db" to the defrag command line. Also there is another option "/b", that makes a backup copy. "/bD:\mydb.backup.edb" That I think (83% sure) pretty much inserts a step 3.5, which just moves the source database to this backup copy name, before step 4. I of course also hate the term defrag for this command, because it is easy to confuse with "online defrag", which is different of course. But for the whole of this mail, I've been assuming we're all talking about offline defrag. Sometimes it is listed as the term compaction for the "/d" command, but even this lacks, because the database can actually GROW during offline defrag! Bet you didn't know that, eh? Maybe a good time to move on to ... 4. Space Usage ... Douglas M. Long quotes this .... anyone know where he got this information from? > "Run ESEUTIL with the /p switch to configure ESEUTIL to create the new > defragmented database on an alternate location (for example, to a > location on a different hard disk). This switch lets you preserve your > original defragmented database (which lets you revert back to your > original database if necessary). This switch also significantly > reduces the amount of time it takes to defragment a database, because > you are rebuilding to a new location, * > rather then rebuilding the database in place." The last line is incorrect, but only slightly. Obviously from the steps above, defrag never rebuilds "in place". Repair however, DOES operate in place. So I'd like this to be made very clear whereever this text is, if it is in Microsoft documentation. Defrag can require up to 2.1x times the database size. I.e. our friend with the 91 GB database, should have like 100 GBs free to defrag the database. That said, the .1 extra is I a "cover our asses" number to cover some extra temporary storage required during offline defrag and to cover the fact that the database can increase in size. That said, I've only ever seen a database grow from a test case I was doing. In all deployment usages I've ever heard of the database shrinks. Technically the available disk space required will be "a new database with only the raw data move in" + some temporary storage required. So if this 91 GB database only had 1 GB of raw data (I'd say 1 GB of mail, but mail single instancing and several other things make it more complicated), then you'd see a new database of 1 GB + whatever is needed for database overhead (maybe another 100-900 MBs, depending). 5. White Space ... There are multiple "kinds of space usage" that can be recovered in an active Exchange database. This is because there are multiple levels and kinds of garbage collection, to collect data that isn't being used, or is white space. You can see the 10 Exchange specific levels or kinds here: http://blogs.msdn.com/jeremyk/archive/2004/06/12/154283.aspx After those levels are done, then there is potentially "actual ESE white space" in the database, so the 11th level mentioned in that article is ONLINE defrag, which is basically whitespace consolidation. Yes, ESE online defrag _GOES TO ELEVEN!_ Eleven is more than ten! (pop culture joke) Anyway, online defrag does white space consolidation, which you could think of as white space reclamation _to the database level_, but not to the file level (which would result in file shrinkage). To get some fairly low level info on ESE white space, check this article out.: http://blogs.msdn.com/jeremyk/archive/2004/06/12/154283.aspx Note that that the first 10 levels only happen if the store.exe is online, and maintaining the database. Doing an offline defrag gets none of the space those 10 tasks can get back. ESE thinks that expired indexes (to pick the first task) are still valid indexes, because it is the store/Exchange which decides when it tells ESE to delete the indexes. So I think the last thing to mention is that white space comes in two varieties in ESE. There is lets say, "fragmented white space", and "available white space". If the store ran online defrag long enough to take a pass at the whole database, all white space would be of the later/available white space variety. If online defrag had not finished, it is possible to have "fragmented white space". Fragmented white space is tricky, because the white space report and "eseutil /ms mydb.edb" do not show the size of this fragmented white space. 5. Miscellany Someone used the term "hard repair". There is no such term, there is "hard recovery" and "soft recovery". Lets not even start discussing them, they have nothing to do with repair, other than they all start with "re", but they're very different. As someone said, the defrag rate is completely dependant upon the disk hardware. Our sucky, old, anchient SAN seems to get about 20 GBs / hour. Eric Fleischman right now is playing with some newer SAN hardware, and getting something like ~40 GBs / hour for initial Exchange database create (creating like 1 or 1.2 TBs of databases), so I imagine that defrag on that SAN would be probably at least 50% faster (though I just completely made that up). With the /t option mentioned, you can at least make the source and destination databases be on different spindles/disks or volumes, to increase the IO rate. Michael B. Smith, I'm glad you'd never recommend a tool that would recommend offline defrag as standard maintance! :) I generally don't recommand regular offline defrags myself, believing if that becomes necessary it is an issue that shoudl be fixed in ESE or whatever app (Exch/AD) is using ESE. But people definately fall into two camps on this aspect. Unfortunately, the people in the offline camps, are not in a completely defenseless position. They do have some points. None that I consider hard points though. Hopefully, I didn't make any mistakes, wouldn't want to mislead anyone ... Cheers, BrettSh [msft] ESE Developer On Wed, 17 Aug 2005, Michael B. Smith wrote: > It rather depends on what you want. If you want table-level usage > reports, then eseutil /ms does just a fine job. If you want management > reports, then I think the MOM Management Pack does a great job. For > even more detailed (and configurable) information, I think the Quest > MessageStats for Exchange (for management) and SpotLight on Exchange > (for administrators) are both pretty good. > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Medeiros, Jose > Sent: Wednesday, August 17, 2005 8:14 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > Hi Michael, > > Until they add the feature to compact ( Or defrag ) the Information > store while online, it's probably not some thing we would have an > interest in. As far as other solutions for reporting, do you have any > recommendations as to which tools you prefer? > > Jose :-) > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Michael B. Smith > Sent: Wednesday, August 17, 2005 4:29 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > > If you choose to purchase the product, I would certainly be interested > in knowing your perspective on it after a couple of months. From my > perspective, it gives some pretty reports (which other packages do > just as well); and it puts a nice GUI in front of something that you > shouldn't be doing very often anyway. > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Medeiros, Jose > Sent: Wednesday, August 17, 2005 5:01 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > Hi Michael, > > I just got off the phone with the Sales rep, I am mistaken. GoExchange > will not have the additional functionally to do a full defrag of the > data base with out taking the server offline until end of year. My > apologies. > > However it does give excellent reporting capability on the condition > of the data base. > > The demo is free, and although the demo will not let you actually > repair the data base it will give you a full report as to it's present > state. > > Jose > > ---------------------------------------------------- > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Michael B. Smith > Sent: Wednesday, August 17, 2005 12:00 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > > This tool, I believe, runs eseutil in the background. > > The message store that goExchange is working on must be offline while > it's being "maintained": > > http://www.lucid8.com/faq/faq_general6.asp > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Medeiros, Jose > Sent: Wednesday, August 17, 2005 2:32 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > I am not sure I understand your point.. if he is trying to fix his > bloat issue, this tool will do the same thing as Esutuil in compacting > the database with out having to take down his exchange servers. > > Last time I ran Esutuil on a 30gb data base it took nearly 10 hours to finish. > > Jose > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Michael B. Smith > Sent: Wednesday, August 17, 2005 11:08 AM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > > I would never recommend a tool that does offline defragmentation as > preventive maintenance. > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Medeiros, Jose > Sent: Wednesday, August 17, 2005 1:56 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > Want a simpler method? Try http://www.goexchange.com/, ( GOexchange is > painless to use and saves you time by running automatic expert > preventive maintenance while you attend to more important things ) > > You won't even have to take your Exchange servers offline to defrag > the information and public folder stores. > > Jose :-) > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of [EMAIL PROTECTED] > Sent: Wednesday, August 17, 2005 10:28 AM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > > KB192185 has good info on this. You are on the right path, IMO. > > > Sincerely, > > Dèjì Akómöláfé, MCSE+M MCSA+M MCP+I > Microsoft MVP - Directory Services > www.readymaids.com - we know IT > www.akomolafe.com > Do you now realize that Today is the Tomorrow you were worried about > Yesterday? -anon > > ________________________________ > > From: [EMAIL PROTECTED] on behalf of Douglas M. Long > Sent: Wed 8/17/2005 10:06 AM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > > > I guess I was thinking of using the /p switch because of a paragraph" > > "Run ESEUTIL with the /p switch to configure ESEUTIL to create the new > defragmented database on an alternate location (for example, to a > location on a different hard disk). This switch lets you preserve your > original defragmented database (which lets you revert back to your > original database if necessary). This switch also significantly > reduces the amount of time it takes to defragment a database, because > you are rebuilding to a new location, rather then rebuilding the > database in place." > > > I was thinking...hmmm, it takes less time, and I have a little more > protection from something going wrong...sounds good to me. Comments? > > > And now that I think of it, my command probably needed a PATH for the > new DB unless there is a default, but I won't know that till I run > eseutil. > > > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Peter Johnson > Sent: Wednesday, August 17, 2005 12:17 PM > To: [email protected] > Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat > > Well IIRC /p is a hard repair whilst /d is for defrag. If you have a > working, mountable store that you want to defrag you don't need the /p > switch. > > Anyone got any comment? > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Douglas M. Long > Sent: 17 August 2005 18:13 > To: [email protected] > Subject: [ActiveDir] OT:Exchange 2003 SP1 bloat > > OK, so I have scheduled a window this weekend to run an offline defrag > of the mailbox store. Now I am looking for the best way to do this, > and since this list seems to have more/better experience with Exchange > than the Exchange list, I am looking for comments. > > Here are the steps I am planning on taking: > > 1. Full backup of "General" store > 2. Dismount "General" store > 3. > eseutil /d /p F:\Exchsrvr\mdbdata\General.edb > *Is it a good idea to use the /p switch (I have enough > space) > **If using the /p switch, what is the mounting procedure for > the new DB? > > 4. Full backup of new DB > > > Any comments are very welcome, and appreciated. > > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: > http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: > http://www.mail-archive.com/activedir%40mail.activedir.org/ > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
