Both are valid, but one is destructive
(repair) as in "2. Repair (/p) is destructive." and "The defrag
w/o instate ("eseutil /d mydb.edb /p") is NOT destructive."
Better? Or did I confuse it worse?
This is
probably just me not comprehending this, but when you said
"The confusion is that, there is also a /p option that can be provided to
defrag, like so:"
Did you mean the confusion is that they are both** valid, or that one is valid
and one is not?
** eseutil /p mydb.edb and eseutil /d mydb.edb /p
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Brett Shirley
Sent: Thursday, August 18, 2005 10:27 AM
To: [email protected]
Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
I am actually a programmer for ESE (you know the database under Exchange,
once know as JET Blue ... ) ... yes, it may come as a shock to some of you
that building 7 garage door operator is not my only job duty at msft ...
Anyway, I'd like to clear up some confusion and mistatements ...
1. The /p switch ...
There was some confusion on the /p switch ...
There are two different operations being discussed below that one can
perform on an ESE database. One is called defrag and the other
repair.
It is CRITICAL everyone understand the difference between the two, because
one is dangerous and destructive under certain circumstances.
Defrag:
eseutil /d mydb.edb
Repair:
eseutil /p mydb.edb
The confusion is that, there is also a /p option that can be provided to
defrag, like so:
Defrag (w/o instate):
eseutil /d mydb.edb /p
The original mail understood this, but some subsequent mails got it mixed
up, just wanted to make it explicit. I loathe ESEUtil's command syntax,
BTW. :P Subcommands should always be full words, like repadmin. ;)
2. Repair (/p) is destructive.
Repair is really an unfortunate term for this functionality, because like
when you repair a car, it works again! That may not be the case after
ESE's repair. The command should've been called
"salvage". The command
basically throws out any data that ESE doesn't understand due to physical
or ESE logical corruptions in your database, basically salvaging what's
left.
The defrag w/o instate ("eseutil /d mydb.edb /p") is NOT destructive.
Repair is dangerous. I always try to steer people away from repair.
If
though somehow the database has been corrupted, there is irreplaceable
data it can really save you.
If the database is in perfect order, both physically ("eseutil /k"
checks
this) and ESE logically ("eseutil /g" checks that), then in theory
repair
is safe. But that idea gives me the hee-bee-jee-bees. It is
possible for
a disk to return valid data on one read, and invalid data on a 2nd read,
so it could never be perfectly safe. Did I mention I try to steer people
away from repair.
2.a. Aside: NEVER run repair on an AD database.
Off the subject of this mail, but it bears repeating.
As you may or may not know, the ESE database engine is used in both
Windows for Active Directory's database/ntds.dit and in Exchange for
mailbox stores. In the Windows version of the ESE database engine,
eseutil.exe is called esentutl.exe.
Even though these binaries are similar, and based off similar sources, the
versions are different, and compiled with the Esentutl.exe and eseutil.exe
are
Never run repair on an AD database. In fact in Win2k3 SP1, we disabled
that functionality in esentutl.exe for AD databases. Ok, we're really
offtopic for the thread, moving back to ...
3. Defrag (how it works) ...
I'd like to go over very approximately the steps that ESE (offline) defrag
goes through, because it will make some of the comments in the next point
more poinaintly clear.
Defrag works like this:
Step 1 - Open the "source" database.
ESE opens for reading the source or
target database, that you've
asked specified as the first
non-flag argument after the /d sub-
command. i.e.
"mydb.edb" above.
Step 2 - Create a "destination" database with a temporary
name.
By default the destination or
temporary DB, is created in the
same directory as the source
database.
Step 3 - Move the data table by table to the destination database.
Enumerate over each table in the
"source" database, and move
each row of data to the
"destination" database. This is why
I call them source and
destination. However, usually, eseutil
and docs call the destingation the
"temp. database". You'll
see why in step 4. And indices
are recreated in the process too
of course.
Step 4 - Move the destination database to the source database.
ESE moves the destination/temp
database name, to the source
database name.
This is the step that specifying
"/p" to defrag skips.
Note: You may specify the destination database name for step 2, in this
process by adding an argument like
"/tE:\mytempdrive\emailstuff.db"
to the defrag command line.
Also there is another option "/b", that makes a backup copy.
"/bD:\mydb.backup.edb"
That I think (83% sure) pretty much inserts a step 3.5, which just moves
the source database to this backup copy name, before step 4.
I of course also hate the term defrag for this command, because it is easy
to confuse with "online defrag", which is different of course.
But for
the whole of this mail, I've been assuming we're all talking about offline
defrag. Sometimes it is listed as the term compaction for the
"/d"
command, but even this lacks, because the database can actually GROW
during offline defrag! Bet you didn't know that, eh? Maybe a good
time
to move on to ...
4. Space Usage ...
Douglas M. Long quotes this ....
anyone know where he got this information
from?
> "Run ESEUTIL with the /p switch to configure ESEUTIL to create
the new
> defragmented database on an alternate location (for example, to a
> location on a different hard disk). This switch lets you preserve
your
> original defragmented database (which lets you revert back to your
> original database if necessary). This switch also significantly
> reduces the amount of time it takes to defragment a database,
because
> you are rebuilding to a new location,
* > rather then rebuilding the database in place."
The last line is incorrect, but only slightly. Obviously from the steps
above, defrag never rebuilds "in place". Repair however, DOES
operate in
place. So I'd like this to be made very clear whereever this text is, if
it is in Microsoft documentation.
Defrag can require up to 2.1x times the database size. I.e. our friend
with the 91 GB database, should have like 100 GBs free to defrag the
database. That said, the .1 extra is I a "cover our asses"
number to
cover some extra temporary storage required during offline defrag and to
cover the fact that the database can increase in size.
That said, I've only ever seen a database grow from a test case I was
doing. In all deployment usages I've ever heard of the database
shrinks.
Technically the available disk space required will be "a new database with
only the raw data move in" + some temporary storage required.
So if this 91 GB database only had 1 GB of raw data (I'd say 1 GB of mail,
but mail single instancing and several other things make it more
complicated), then you'd see a new database of 1 GB + whatever is needed
for database overhead (maybe another 100-900 MBs, depending).
5. White Space ...
There are multiple "kinds of space usage" that can be recovered in an
active Exchange database. This is because there are multiple levels and
kinds of garbage collection, to collect data that isn't being used, or is
white space.
You can see the 10 Exchange specific levels or kinds here:
http://blogs.msdn.com/jeremyk/archive/2004/06/12/154283.aspx
After those levels are done, then there is potentially "actual ESE white
space" in the database, so the 11th level mentioned in that article is
ONLINE defrag, which is basically whitespace consolidation. Yes, ESE
online defrag _GOES TO ELEVEN!_ Eleven is more than ten! (pop culture
joke) Anyway, online defrag does white space consolidation, which you
could think of as white space reclamation _to the database level_, but not
to the file level (which would result in file shrinkage).
To get some fairly low level info on ESE white space, check this article
out.:
http://blogs.msdn.com/jeremyk/archive/2004/06/12/154283.aspx
Note that that the first 10 levels only happen if the store.exe is online,
and maintaining the database. Doing an offline defrag gets none of the
space those 10 tasks can get back. ESE thinks that expired indexes (to
pick the first task) are still valid indexes, because it is the
store/Exchange which decides when it tells ESE to delete the indexes.
So I think the last thing to mention is that white space comes in two
varieties in ESE. There is lets say, "fragmented white space",
and
"available white space". If the store ran online defrag long
enough to
take a pass at the whole database, all white space would be of the
later/available white space variety. If online defrag had not finished,
it is possible to have "fragmented white space". Fragmented
white space
is tricky, because the white space report and "eseutil /ms mydb.edb"
do
not show the size of this fragmented white space.
5. Miscellany
Someone used the term "hard repair". There is no such term,
there is
"hard recovery" and "soft recovery". Lets not even
start discussing them,
they have nothing to do with repair, other than they all start with
"re",
but they're very different.
As someone said, the defrag rate is completely dependant upon the disk
hardware. Our sucky, old, anchient SAN seems to get about 20 GBs /
hour.
Eric Fleischman right now is playing with some newer SAN hardware, and
getting something like ~40 GBs / hour for initial Exchange database create
(creating like 1 or 1.2 TBs of databases), so I imagine that defrag on
that SAN would be probably at least 50% faster (though I just completely
made that up).
With the /t option mentioned, you can at least make the source and
destination databases be on different spindles/disks or volumes, to
increase the IO rate.
Michael B. Smith, I'm glad you'd never recommend a tool that would
recommend offline defrag as standard maintance! :) I generally don't
recommand regular offline defrags myself, believing if that becomes
necessary it is an issue that shoudl be fixed in ESE or whatever app
(Exch/AD) is using ESE. But people definately fall into two camps on this
aspect. Unfortunately, the people in the offline camps, are not in a
completely defenseless position. They do have some points. None
that I
consider hard points though.
Hopefully, I didn't make any mistakes, wouldn't want to mislead anyone ...
Cheers,
BrettSh [msft]
ESE Developer
On Wed, 17 Aug 2005, Michael B. Smith wrote:
> It rather depends on what you want. If you want table-level usage
> reports, then eseutil /ms does just a fine job. If you want management
> reports, then I think the MOM Management Pack does a great job. For
> even more detailed (and configurable) information, I think the Quest
> MessageStats for Exchange (for management) and SpotLight on Exchange
> (for administrators) are both pretty good.
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Medeiros, Jose
> Sent: Wednesday, August 17, 2005 8:14 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> Hi Michael,
>
> Until they add the feature to compact ( Or defrag ) the Information
> store while online, it's probably not some thing we would have an
> interest in. As far as other solutions for reporting, do you have any
> recommendations as to which tools you prefer?
>
> Jose :-)
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On
Behalf Of Michael B. Smith
> Sent: Wednesday, August 17, 2005 4:29 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
>
> If you choose to purchase the product, I would certainly be interested
> in knowing your perspective on it after a couple of months. From my
> perspective, it gives some pretty reports (which other packages do
> just as well); and it puts a nice GUI in front of something that you
> shouldn't be doing very often anyway.
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Medeiros, Jose
> Sent: Wednesday, August 17, 2005 5:01 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> Hi Michael,
>
> I just got off the phone with the Sales rep, I am mistaken. GoExchange
> will not have the additional functionally to do a full defrag of the
> data base with out taking the server offline until end of year. My
> apologies.
>
> However it does give excellent reporting capability on the condition
> of the data base.
>
> The demo is free, and although the demo will not let you actually
> repair the data base it will give you a full report as to it's present
> state.
>
> Jose
>
> ----------------------------------------------------
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On
Behalf Of Michael B. Smith
> Sent: Wednesday, August 17, 2005 12:00 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
>
> This tool, I believe, runs eseutil in the background.
>
> The message store that goExchange is working on must be offline while
> it's being "maintained":
>
> http://www.lucid8.com/faq/faq_general6.asp
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Medeiros, Jose
> Sent: Wednesday, August 17, 2005 2:32 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> I am not sure I understand your point.. if he is trying to fix his
> bloat issue, this tool will do the same thing as Esutuil in compacting
> the database with out having to take down his exchange servers.
>
> Last time I ran Esutuil on a 30gb data base it took nearly 10 hours to
finish.
>
> Jose
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On
Behalf Of Michael B. Smith
> Sent: Wednesday, August 17, 2005 11:08 AM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
>
> I would never recommend a tool that does offline defragmentation as
preventive maintenance.
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Medeiros, Jose
> Sent: Wednesday, August 17, 2005 1:56 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> Want a simpler method? Try http://www.goexchange.com/,
( GOexchange is
> painless to use and saves you time by running automatic expert
> preventive maintenance while you attend to more important things )
>
> You won't even have to take your Exchange servers offline to defrag
> the information and public folder stores.
>
> Jose :-)
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On
Behalf Of [EMAIL PROTECTED]
> Sent: Wednesday, August 17, 2005 10:28 AM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
>
> KB192185 has good info on this. You are on the right path, IMO.
>
>
> Sincerely,
>
> Dèjì Akómöláfé, MCSE+M MCSA+M MCP+I
> Microsoft MVP - Directory Services
> www.readymaids.com - we know IT
> www.akomolafe.com
> Do you now realize that Today is the Tomorrow you were worried about
Yesterday? -anon
>
> ________________________________
>
> From: [EMAIL PROTECTED] on behalf of Douglas M. Long
> Sent: Wed 8/17/2005 10:06 AM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
>
>
> I guess I was thinking of using the /p switch because of a paragraph"
>
> "Run ESEUTIL with the /p switch to configure ESEUTIL to create the
new
> defragmented database on an alternate location (for example, to a
> location on a different hard disk). This switch lets you preserve your
> original defragmented database (which lets you revert back to your
> original database if necessary). This switch also significantly
> reduces the amount of time it takes to defragment a database, because
> you are rebuilding to a new location, rather then rebuilding the
> database in place."
>
>
> I was thinking...hmmm, it takes less time, and I have a little more
> protection from something going wrong...sounds good to me. Comments?
>
>
> And now that I think of it, my command probably needed a PATH for the
> new DB unless there is a default, but I won't know that till I run
> eseutil.
>
>
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]
On Behalf Of Peter Johnson
> Sent: Wednesday, August 17, 2005 12:17 PM
> To: [email protected]
> Subject: RE: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> Well IIRC /p is a hard repair whilst /d is for defrag. If you have a
> working, mountable store that you want to defrag you don't need the /p
> switch.
>
> Anyone got any comment?
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]
On Behalf Of Douglas M. Long
> Sent: 17 August 2005 18:13
> To: [email protected]
> Subject: [ActiveDir] OT:Exchange 2003 SP1 bloat
>
> OK, so I have scheduled a window this weekend to run an offline defrag
> of the mailbox store. Now I am looking for the best way to do this,
> and since this list seems to have more/better experience with Exchange
> than the Exchange list, I am looking for comments.
>
> Here are the steps I am planning on taking:
>
> 1. Full backup of "General" store
> 2. Dismount "General" store
> 3. > eseutil /d /p
F:\Exchsrvr\mdbdata\General.edb
>
*Is it a good idea to use the /p switch (I have enough
> space)
>
**If using the /p switch, what is the mounting procedure for the new DB?
>
> 4. Full backup of new DB
>
>
> Any comments are very welcome, and appreciated.
>
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
