activedir  

RE: [ActiveDir] Stress testing and performance analysis of domain controllers

Eric Fleischman
Mon, 06 Dec 2004 12:50:41 -0800

Tremendous amt of churn on this thread. Let me see if I can pull it all
together.

One of the things we do internally on the ESE level is caching of pages
of the DIT from disk. The perf benefit is clear, and measurable.

In 2003 on 32bit hardware, the /3gb switch begins to make sense when
your dit is in the neighborhood of 2gb and you have >2GB of physical
memory. At that point we might hit the max cache size, and to grow
beyond that /3gb will help. Max cache size is in the neighborhood of 2.6
or 2.7gb when /3gb is used.
On 64bit, our max cache size is 2^48bytes if memory serves me correctly.
If you have that much ram on a 64bit box, call me. I want to see your
box. :)

I should note that /3gb does not come w/o a cost. I would be careful in
using this setting this value on machines which are not just DCs, as it
does have a perf impact on your system more generally. Without going too
far off topic, I'll say it will yield a scenario where you have fewer
resources for kernel data structures, like non-paged pool and system
PTEs. If you are interested in the details, this is a question best
fielded by a book like Inside Windows 2000 I'd think.

There was discussion around the amt of benefit (I think someone tossed
out a phrase like "a factor of 5"). The reality is that the benefit
depends greatly upon your workload. If you have a workload which can be
optimized through server-side indexes, to accurately measure the benefit
of 64bit you probably want to compare a 32bit box with heavy indexes,
custom tailored to your environment, vs. 64bit with either comparable or
no indexes (your choice) and a _warm_ cache. I say it in this way as
really, you want to compare max perf you can get on 32bit with max you
can get on 64bit. That might mean enabling some indexes, as that can
help with perf even w/o loading everything in memory (probably
intuitive, but wanted to draw special attention to it).

Note my usage of the word "warm" to describe the cache. I say warm cache
as out of the box, we won't preload your DIT in to memory, even if you
have the physical memory for it (32bit or 64bit). Rather, we cache
things as they are fetched. So if you issue a query which need traverse
a series of pages not yet cached, we still take the same I/O hit. It is
when they are in memory and you try to use them a second time that you
get the benefit, as we don't need to fetch them again.
This yields the fact that some customers that run 64bit write a little
script to "walk" their database. They do this to warm the cache and get
most everything preloaded in to memory.

There was also discussion around how large the cache is. In essence,
like most software which caches stuff, we have algorithms for it. :) Joe
eluded to it, but basically we have a series of elements we look at to
help decide what movements in cache size should be done. I won't go in
to the details of such things for the sake of brevity.

Hope that helps.

~Eric






-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of joe
Sent: Monday, December 06, 2004 1:24 PM
To: [EMAIL PROTECTED]
Subject: RE: [ActiveDir] Stress testing and performance analysis of
domain controllers

Brett is fun. :o)

He isn't so much the type that will fish for you or even teach you to
fish,
he will throw you a fishing line and let you figure it all out. This can
be
troublesome though if you lived in a desert and had never even seen
water
let alone a fish. Understanding Tech involved with the AD code
specifically,
he is very strong. Understanding Tech as it has to be used in some sites
and
locations and operational support concerns, not always so strong. He is
very
much like the rest of the Dev team guys which is mostly working in the
Dev
this is what you should shoot for world versus the ditches and puddles
that
many people end up having to work in. Overall a good guy though.
Extremely
entertaining guy to talk to. 

On the beefy DC side of things. I don't know, I don't really consider
2GB to
be exceptionally beefy or even 4GB. Not when we have workstations now
coming
from the factory with 1GB and 2GB and options to do 4GB. Exceeding 4GB
RAM
gets a little unusual and you truly get beefy based on proc
Architectures
(say multiproc opteron versus athlon or on the intel side the Xeon
versus
the non-Xeon's, etc) and disk subsystems with heavy duty hardware RAID
solutions with oodles of cache and RAID type offerings. We need to go to
64
bit for no better reason than the cost of memory is consistently
dropping
and we need good easy ways of dealing with more than 4GB of RAM that
doesn't
depend on goofy paging mechanisms. 

Finally, I don't recommend /3gb unless you truly need it and all of the
software on the machine properly supports it. It has been long while
(years)
but I have seen some odd /3GB failures with apps that didn't properly
implement that functionality due to memory addressing issues. Also
obviously
you can't use /3GB with 2K standard, that could cause some fun things to
happen as well, collectively termed as undefined results. No reason to
force
the kernel to live in 1GB unless it is required for some other reason
which
if I recall can impact some video drivers and other kernel apps that may
need to grab a chunk of address space for some reason.

  joe



-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Renouf, Phil
Sent: Monday, December 06, 2004 1:59 PM
To: [EMAIL PROTECTED]
Subject: RE: [ActiveDir] Stress testing and performance analysis of
domain
controllers

> > Gotcha, then yeah the /3gb switch would help with performance. 
> > I've learned something new, thanks :)
> 
> Maybe. It depends on the DIT size as well as what else needs memory. 
> From what I understand based on old conversations, the DIT caching 
> routines are sensitive to memory pressure and will not page DIT cache,

> it will release memory instead.
> Again if you have a DIT of 200MB, you can use /3gb and most likely 
> wouldn't see a benefit.

You might not see a benefit with a small DIT size, but then again why go
with such a beefed up DC if your DIT size is that small (unless you are
planning for it to grow substantially). Adding the /3GB switch shouldn't
cause any issues even if the DIT is small enough to not get much benefit
from it, unless the OS is effected by being reduced to 1GB of virtual
address space.

> Hopefully ~Eric will pop along shortly with some info as I know he 
> loves this stuff. In the meanwhile, you can be pretty sure BrettSh 
> generally knows what he is talking about with AD. Not saying he can't 
> be wrong, but all things being equal concerning a bet on AD internals,

> I would bet with Brett.
> Unless he was betting against Will, Dmitri, ~Eric, Dean or some of 
> those guys and then I would simply put my wallet away, pull out some 
> popcorn, and watch the show.

I'm definitely interested to see what they have to say :) I certainly
wasn't
implying Brett didn't know what he was talking about, but showing me the
size of a DIT really didn't tell me much without the information that
LSASS
is large address aware. Now it makes sense ;)

Anyway, looking forward to some more information on this and its effect
on
performance.

Phil
List info   : http://www.activedir.org/mail_list.htm
List FAQ    : http://www.activedir.org/list_faq.htm
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/

List info   : http://www.activedir.org/mail_list.htm
List FAQ    : http://www.activedir.org/list_faq.htm
List archive:
http://www.mail-archive.com/activedir%40mail.activedir.org/
List info   : http://www.activedir.org/mail_list.htm
List FAQ    : http://www.activedir.org/list_faq.htm
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/