I thought about that for a little while... I'd like to ask the question a little differently -- if I were to look at any two counters that was going to indicate to me that I MIGHT need to do an offline defrag -- which counters would those be and what would they tell me?
:-) -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brett Shirley Sent: Thursday, April 28, 2005 10:44 PM To: ActiveDir@mail.activedir.org Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? That would take too long ... I'll tell you want, YOU can ask about any two of them, that have a similar name prefix (i.e. start w/ the same word) ... and I'll try to answer today or tomorrow ... Cheers, -BrettSh [msft] On Thu, 28 Apr 2005, Michael B. Smith wrote: > Heh. Since YOU said that, for Exchange: > > [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ESE\Performance] > "Squeaky Lobster"=dword:00000001 > > (Given that the name of the feature was "Squeaky Lobster" and that it > was message-store related -- the deduction was fairly obvious.) > > Now - a description of those values - many of which have VERY > interesting names, would rock. > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Brett Shirley > Sent: Thursday, April 28, 2005 9:51 PM > To: ActiveDir@mail.activedir.org > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > Ignoring Joe's cruel comment about my categorically challenged blog ... > lets get back to something useful ... > > So this link gets you started enabling Windows ESE perf counters ... > > http://www.microsoft.com/resources/documentation/Windows/2000/server/r > es kit/en-us/Default.asp?url=/resources/docume$ > BUT ... > > Enabling Advanced Perf Counters: > During step 3, also add this registry value in that same > ...\Services\ESENT\Performance regkey: > Squeaky Lobster = REG_DWORD 00000001 Now, technically this reg > value also works, and is the professional equivalent of the above value: > Show Advanced Counters = REG_DWORD 00000001 But Eric and I try > to promote Squeaky Lobster usage whenever possible to ensure that ESE > will honor the reg value forever. > > > Then you can get started on the looking at all the fascinating > database perf counters ... oooh, I just saw one of our DC's pop to > 405k latches / second, that's cool. > > > Setting up adv perf counters is I think similar for Exchange, but for > a slight different registry key. > > > BTW, does anyone know why does that web page say copy esentprf.dll to > a seperate directory? I've never done that, seems to work fine. Huh. > > Cheers, > BrettSh [msft] > > posting "as is", don't blame me if you hose your DCs. ;) > > > > On Thu, 28 Apr 2005, joe wrote: > > > Hey Brett... I've seen your blog, how about you tell ~Eric the story > > and he can blog it. :o) > > > > <evilgrin> > > > > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Brett > > Shirley > > Sent: Thursday, April 28, 2005 8:32 PM > > To: ActiveDir@mail.activedir.org > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > The dev who put it in, is what I like to call "my boss" ... he has > > no child, I can guarantee it had nothing to do with that ... > > > > Email me directly the Exch product manager's name, and I'll try to > > light a fire under them ... if they don't product something, I'll > > produce something on my blog (when it is up) and send it around ... > > > > Cheers, > > BrettSh > > > > > > On Thu, 28 Apr 2005, Michael B. Smith wrote: > > > > > One of the Exchange Product Managers said today that she is > > > preparing a blog on Squeaky Lobster, including a picture of the > > > original Squeaky. I also asked about the KB and was told, simply, > > > that "it isn't currently publicly available". > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf Of joe > > > Sent: Thursday, April 28, 2005 7:38 PM > > > To: ActiveDir@mail.activedir.org > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > Try - http://www.realcooltoys.com/squeakylobster.html > > > > > > Squeaky Lobster is a magic reg key to enable special "Squeaky > Lobster" > > > ESE counters. It first came to being, I believe, with Exchange 5.5 > > > where I heard two different stories, the first being that the dev > > > guy who put it in had a kid who had a squeaky lobster toy (or he > > > had > > > > it) and the other is that it was thought up after lunch. I would > > > tend to go with the first explanation myself... Anyway, it was > > > carried through and is available on AD, or at least it was on 2K > > > AD which is the last time I used it a couple of years ago. > > > > > > There used to be a KB out there that talked about what it made > > > available but I don't see it anywhere which sucks because if I > > > need it again I will have to go dig through 8 GB of PSTs and > > > notepad docs. :o) > > > > > > I want to say that I think I heard they changed (or were changing) > > > the name of this reg entry to something like "show advanced > > > counters" or something like that but I don't think I can point at > > > any references for that. > > > > > > As far as I know, this key wasn't supposed to be hidden or secret, > > > though it appears it might have gone underground. I don't think I > > > will post any more on it and let ~Eric or Brett put out in the > > > public whatever they think should be available. > > > > > > > > > joe > > > > > > > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf Of Isenhour, > > > Joseph > > > Sent: Thursday, April 28, 2005 1:31 PM > > > To: ActiveDir@mail.activedir.org > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > This has been a great thread. I've really enjoyed reading it. > > > > > > This question is going to illustrate my extreme ignorance; > > > however, the answer is worth it. What is "Squeaky Lobster"? > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf Of Brett > > > Shirley > > > Sent: Wednesday, April 27, 2005 3:42 PM > > > To: ActiveDir@mail.activedir.org > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > >>From ESE's advanced perf counters exist, that tell you on a > > > >non-per-search > > > basis: > > > - Database Pages Transferred/sec > > > - Database Page Latches/sec > > > > > > IIRC, the first is rate of pages being transferred from disk, and > > > the 2nd is the rate at wich you are "making a read of something on > > > a > > > > page in the cache" > > > (that will include the read right after a page is transferred, BTW). > > > > It doesn't give you the per query stats you were discussing, but > > > it does give you an idea of how much disk the DC is requiring ... > > > > > > If you were to isolate a DC from load, except your query, it could > > > give a _rough_ idea for a paticular query, but remember latches > > > aren't unique references, so if a single query internally has to > > > read a page several times, that will be several latch counts. > > > > > > ... > > > > > > Cheers, > > > -BrettSh > > > > > > On Wed, 27 Apr 2005, joe wrote: > > > > > > > I waffled on posting that at all. I am not sure I can properly > > > > illustrate why I think it would be good for educational info. > > > > Maybe just to see from the outside the deltas in speeds of the > > > > same query when things are in cache versus not, etc. Overall it > > > > is > > > > > just another stat to help understand how your directory is > performing. > > > > > > > > joe > > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of Eric > > > > Fleischman > > > > Sent: Wednesday, April 27, 2005 2:14 AM > > > > To: ActiveDir@mail.activedir.org > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > Correcting myself inline (full of that today aren't I?)..... > > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of Eric > > > > Fleischman > > > > Sent: Tuesday, April 26, 2005 10:41 PM > > > > To: ActiveDir@mail.activedir.org > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > I think it would be kind of interesting if the STATS control > > > > > could tell you what % of the result set came from cache or > > > > > something like that > > > > > > > > Actually, that's not really what you want. If I may, let me > > > > change > > > > > your ask in to what I think you really would like.... > > > > What you really want is the % of pages touched to service the > > > > query that were in the cache. It doesn't matter if those pages > > > > are > > > > > returned or not, it only matters that you needed the pages to > > > > effective service > > > > > > > the search. As that's what defines the amt of time it takes to > > > > service > > > it. > > > > [Efleis] - I shouldn't say this, it isn't quite true. What I > > > > meant > > > > > was, this defines the amt of time that we would spend on I/O, > > > > should those pages not be in memory. Other things might > > > > necessitate more time > > > spent on the search. > > > > > > > > That said, assuming you got what you really want, I'm not > > > > totally sold > > > > > > > of the value. What will you learn? > > > > 1) More db cache -> inefficient searches are faster > > > > 2) Better search filter optimization -> better index selection > > > > -> faster searches with less cache needed and less I/O needed > > > > > > > > Searches that hit infrequently used indexes will have a lower % > > > > of > > > > > pages in memory, but still be faster than inefficient ones that > > > > hit many pages in memory. And the avg IT admin will wonder why. > > > > :) > > > > > > > > Inefficient searches are still inefficient, and are still going > > > > to > > > > > require a large db cache to service them in any sort of timely > manner. > > > > How much cache? As much as you have dataset that need be > > > > traversed > > > > > for > > > > > > > the inefficient search in question. Whatever that dataset might > be. > > > > > > > > Sell me on the learning opportunity here? Sorry, I'm just not > > > > seeing > > > it. > > > > I like the idea on paper, and would be more than happy to file > > > > the > > > bug. > > > > I'm just not seeing what you think you can do better with this > > > > data point than you can today. > > > > > > > > ~Eric > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of joe > > > > Sent: Tuesday, April 26, 2005 9:11 PM > > > > To: ActiveDir@mail.activedir.org > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > Thanks ~Eric. I think it would be kind of interesting if the > > > > STATS > > > > > control could tell you what % of the result set came from cache > > > > or > > > > > something like that. How feasible would something like that be? > > > > Possibly the results of that would only be for educational > > > > reasons > > > > > but > > > > > > > I, at least, would find that info interesting. > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of Eric > > > > Fleischman > > > > Sent: Tuesday, April 26, 2005 8:01 PM > > > > To: ActiveDir@mail.activedir.org > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > You beat me to the reply, thanks Brett. > > > > > > > > A better way to think of this Joe is that a subset of the DIT is > > > > in RAM, as much as we can fit, assuming 1) we don't run out of > > > > memory to use 2) we don't have pressure to back off. And we try > > > > and pick the best pages to cache ("best" definition omitted for > now). > > > > > > > > The one thing we can't do today is that we can't proactively > > > > cache > > > > > something. Though I've thought a lot about whether or not it is > > > > something that I should personally be pushing Brett's team to > > > > work > on. > > > > There's good and bad, but the bottom line today is that you can > "warm" > > > > the cache. In the absence of memory pressure, this warming > > > > technique will help get things in the first time. But there are > > > > some things it doesn't do > > > > 1) It doesn't let you tell buffer manager to keep something in > > > > the > > > > > cache no matter what, if you think you're smarter than the > > > > buffer manager. I would point out, almost never are you smarter > > > > than buffer manager, even when you think you are. But that > > > > doesn't mean > > > > > you won't complain that we don't have a mechanism for it. > > > > 2) You can't really guarantee that something is in the cache > > > > with these sorts of warming techniques. You can get close, but > > > > you can't (for > > > > example) say "please prefetch this index". But warming the cache > > > > can do the big stuff, like walking ancestry and pulling in the > > > > mass of the > > > data table. > > > > > > > > ~Eric > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of Brett > > > > Shirley > > > > Sent: Tuesday, April 26, 2005 4:46 PM > > > > To: ActiveDir@mail.activedir.org > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > Joe, > > > > > > > > When you say > > > > > the actual DIT isn't cached in RAM, the tables, indexes, and > such > > > > > are cached. > > > > I'd take issue with that ... that isn't a good way to explain > > > > what > > > > > is really happening. > > > > > > > > The DIT is most definately cached in RAM, it is cached directly > > > > 1 or more pages at a time. Where a page is an 8k chunk for > > > > Active Directory. We do not extrude the tables and indexes from > > > > those pages, > > > > > > > they stay in the pages, and we "take a latch" on that page's > > > > memory when we want to update the page ... then later we write > > > > that 8k chunk directly from that memory to the offest (based on > > > > it's pgno) of the > > > DIT file it belongs at. > > > > > > > > Now, it is true, not all of the DIT may be cached, we'll only > > > > cache what we need, and it will not pull in free space pages > > > > into memory (at > > > > > > > least in most circumstances ...? I'm thinking of prefetching > > > > might > ... > > > but lets ignore). > > > > > > > > I _think_ _online_ defrag (I know we're talking offline defrag > > > > below, but mentioning online defrag is important, it is what > > > > makes > > > > > offline defrag unnecessay ... online defrag is frequently > > > > abbreviated > > OLD ... > > > > which of course would be the acronym of offline defrag if it had > > > > one, trust me OLD is online defrag (at least as far as the ESE > > > > devs are > > > concerned) ... > > > > poor > > > > taste for a TLA in my opinion ... that was a long aside), > > > > actually > > > > > logs an event on how much free space there is in the database ... > > > > I'm 57% sure that "the DIT size" - "that free size", is the > > > > approximate size of the non-empty data pages (i.e. pages with > > > > data) in > > the DIT ... > > > > > > > due to underflow of a record size on a page, the actual data > > > > size is almost assuredly even less than that ... I just made > > > > that up w/o looking at the code, so I may take that back later ... > > > > > > > > You can see exactly how many bytes of the DIT file + Temp DB* > > > > are in RAM with perfmon, counters, by using perfmon ... first > > > > set the > > > "Squeaky Lobster" > > > > registry key to get the advanced ESE performance counter, then > > > > use > > > > > the > > > > > > > "Database" performance object the "Database Cache Size" counter. > > > > > > > > Also look at the "Database Cache % Clean", b/c you should > > > > multiply > > > > > those by each other to get real data pages currently in memory. > > > > > > > > * Temp DB ... so the database cache is global, so any temporary > > > > sorts we needed to do, during LDAP queries may be taking up some > > > > of the database cache ... I think it's like tmp.edb next to the > > > > ntds.dit file. There'd be no technical way to subtract one from > > > > the other, but > > > > > > > maybe just subtract the whole tmp database size, because that > > > > gives you a lower bound on what is definately ntds.dit. > > > > > > > > ( watch for usage of offline and online here ... ) I agree you > > > > shouldn't worry about offline defrag, but you should make sure > > > > that online defrag is completing every now and then or the space > > > > wastage will grow towards (I'll make a number range here) 3-5x > > > > what it could be. Online defrag ensures that useful data is > > > > collected onto the same > > > > > > > page when it can be, such that the number of non-empty data > > > > pages is really quite close to what you'd get if you did an > > > > offline > defrag. > > > > THOUGH, you'd have free pages in the database in the online > > > > defrag > > > > > case, that offline defrag would give you back in the form of a > > > > smaller > > > DIT file. > > > > So for memory purposes, joe is right, don't worry about offline > > > > defrag, unless there are disk space issues ... but do look for > > > > the > > > > > successful online defrag event. > > > > Note: There was an issue where online defrag was never > > > completing. > > > > > > > > Both online defrag and offline defrag basically scrunch all the > > > > data closer to where it belongs (on a per table, per index, etc > > > > basis), with the online version leaving white space in between > "places" ... > > > > BUT all that said, there is technically one difference between > > > > online defrag and offline defrag data layout ... the offline > > > > defrag will reorder burst long values, into a order that matches > > > > the rows in the database ... I don't feel lik delving into that > yet ... > > > > > > > > That's off the top of my head, I'll check facts, and try to > > > > write more > > > > > > > later ... > > > > > > > > Cheers, > > > > Brett Shirley [msft] > > > > > > > > > > > > posting is as is, but ... > > > > > > > > On Tue, 26 Apr 2005, joe wrote: > > > > > > > > > Possibly Eric will see my response to this and come on and > > > > > smack > > > > > > me > > > > but I > > > > > think your PSS guy may be less than accurate. It is entirely > > > > > my > > > > opinion > > > > > though. > > > > > > > > > > Reducing the physical size of the DIT I don't believe will > > > > > increase > > > > the perf > > > > > of your queries. As Carlos mentioned, the actual DIT isn't > > > > > cached in > > > > RAM, > > > > > the tables, indexes, and such are cached. The empty spaces in > > > > > the DIT physical file should have little if any impact on > > > > > those tables in > > > > memory > > > > > unless you start looking at things like how long does it take > > > > > the head > > > > to > > > > > get from the physical location on the spindle of one entry of > > > > > the > > > > table to > > > > > the next which again, once in memory, shouldn't come into play. > > > > > > > > > > > The big bene of offline defrag that I am aware of is simply to > > > > > reduce > > > > DIT > > > > > bloat and bring it down to a smaller size. You can accomplish > > > > > the same > > > > with > > > > > a dcpromo demote and repromote and you can automate that with > > > > > an > > > > unattended > > > > > script. :o) But honestly, unless you are having disk space > > > > > issues, I > > > > don't > > > > > know many people who worry overly much about doing offline > defrags. > > > > > > > > > > Even once you enable the counters, I am not sure if you will > > > > > know > > > > whether or > > > > > not the whole DB is cached or not simply because the DIT size > > > > > may not accurately reflect how much data you really have due > > > > > to free space in > > > > the > > > > > DIT. > > > > > > > > > > I saw go out and buy a 64 bit machine, load 64 bit Windows > > > > > Server > > > > > 2003 > > > > on it > > > > > and buy RAM = 4GB+2xDIT size and you can be pretty sure your > > > > > entire DB > > > > is > > > > > cached. ;o) > > > > > > > > > > >>From the numbers Wook posted on his slide deck between poems > > > > > >>and > > > > haiku's at > > > > > the most recent DEC you should see a remarkable increase in > perf. > > > > > > > > > > joe > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] > > > > > [mailto:[EMAIL PROTECTED] On Behalf Of > > > > > Fugleberg, > > > > David A > > > > > Sent: Monday, April 18, 2005 11:36 AM > > > > > To: ActiveDir@mail.activedir.org > > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > > The reason I asked was out of curiosity, not because of any > problem. > > > > A MS > > > > > engineer told us that if the DIT is small enough in relation > > > > > to the > > > > amount > > > > > of RAM in the DC, the entire DIT would be cached, increasing > > > > > directory > > > > query > > > > > performance. I was just curious if there was a way to > > > > > objectively > > > > measure > > > > > this. It's always interesting to measure things to see how > > > > > changes > > > > affect > > > > > performance. For example, if I delete a large number of > > > > > objects > > > > > > and > > > > wait > > > > > for the tombstones to age out, I know I could shrink the DIT > > > > > with an > > > > offline > > > > > defrag. Would doing so have any measurable effect on > > > > > perfomance > ? > > > > > I > > > > don't > > > > > know, but it would be interesting to do some before and after > > > > measurements > > > > > to find out. > > > > > > > > > > By the way, the context of the conversation was that the > > > > > engineer was recommending offline defrags after removing a > > > > > large > > > > > > number of objects > > > > (and > > > > > waiting the requisite time for garbage collection). I have no > > > > argument with > > > > > that, but it's nice to be able to measure what if anything > > > > > it's buying > > > > > > > > > (besides a smaller DIT file). Some of us are just funny that > > > > > way, I guess.... > > > > > > > > > > Dave > > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] > > > > > [mailto:[EMAIL PROTECTED] On Behalf Of > > > > > Carlos > > > > Magalhaes > > > > > Sent: Friday, April 15, 2005 4:27 AM > > > > > To: ActiveDir@mail.activedir.org > > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > > > > > > > Well none of the actually DIT is cached (into the RAM), IMO. > > > > > The > > > > engine > > > > > might cache regular/common lookups, indexes etc but none to > > > > > the > > > > actually > > > > > DC's RAM. But then again you have to define but what you mean > > > > > by > > > > > > "into > > > > RAM". > > > > > > > > > > Nathan is quite right with "Checking the working set size of > > > > > LSASS is > > > > not > > > > > reliable." There are many more processes that the LSASS is > > > > > taking care > > > > of. > > > > > You could dump the LSASS process and take a look and then > > > > > determine > > > > from > > > > > there what is happening. > > > > > > > > > > But now I am curious why you asking :P Do you have a hungry > > > > > LSASS > > > > process? > > > > > If you do what Patch/Service Pack level do you have on that box? > > > > > > > > > > Carlos Magalhaes > > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] > > > > > [mailto:[EMAIL PROTECTED] On Behalf Of > > > > > Nathan Muggli > > > > > Sent: 15 April 2005 06:10 AM > > > > > To: ActiveDir@mail.activedir.org > > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > > Checking the working set size of LSASS is not reliable. > > > > > There's > > > > process > > > > > overhead for things like lsa session handles and other stuff > > > > > related > > > > to the > > > > > security sub system. > > > > > > > > > > The most accurate method is to enable the ESE Database > > > > > performance > > > > counters > > > > > and look at "Cache Size". To enable the DB counters, install > > > > > Server Performance Advisor, or check out > > > > > > > > > http://www.microsoft.com/resources/documentation/Windows/2000/se > > > > rv > > > > er > > > > /r > > > > es > > > > > > > > > kit/en-us/Default.asp?url=/resources/documentation/Windows/2000/ > > > > se > > > > rv > > > > er > > > > /r > > > > > eskit/en-us/distrib/dsbm_mon_pzgc.asp > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] > > > > > [mailto:[EMAIL PROTECTED] On Behalf Of Roger > > > > Seielstad > > > > > Sent: Thursday, April 14, 2005 8:45 PM > > > > > To: ActiveDir@mail.activedir.org > > > > > Subject: RE: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > > By checking the working set size of by LSASS? > > > > > > > > > > -------- > > > > > Roger Seielstad > > > > > E-mail Geek > > > > > > > > > > > -----Original Message----- > > > > > > From: [EMAIL PROTECTED] > > > > > > [mailto:[EMAIL PROTECTED] On Behalf Of > > > > > > Fugleberg, David A > > > > > > Sent: Thursday, April 14, 2005 2:22 PM > > > > > > To: activedir@mail.activedir.org > > > > > > Subject: [ActiveDir] How much of the DIT is cached in RAM ? > > > > > > > > > > > > How can I determine how much of the DIT is being cached in > > > > > > RAM > > > > > > > on a given DC ? > > > > > > > > > > > > Dave > > > > > > List info : http://www.activedir.org/List.aspx > > > List FAQ : http://www.activedir.org/ListFAQ.aspx > > > List archive: > > > http://www.mail-archive.com/activedir%40mail.activedir.org/ > > > List info : http://www.activedir.org/List.aspx > > > List FAQ : http://www.activedir.org/ListFAQ.aspx > > > List archive: > > > http://www.mail-archive.com/activedir%40mail.activedir.org/ > > > > > > > List info : http://www.activedir.org/List.aspx > > List FAQ : http://www.activedir.org/ListFAQ.aspx > > List archive: > > http://www.mail-archive.com/activedir%40mail.activedir.org/ > > > > List info : http://www.activedir.org/List.aspx > > List FAQ : http://www.activedir.org/ListFAQ.aspx > > List archive: > > http://www.mail-archive.com/activedir%40mail.activedir.org/ > > > > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: > http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx > List FAQ : http://www.activedir.org/ListFAQ.aspx > List archive: > http://www.mail-archive.com/activedir%40mail.activedir.org/ > List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
