If asking about the BLOB structure definition, absolutely. I actually needed no special access into MS to figure this one out. The hardest part was recognizing a new data structure was defined and realizing that the format of the data coming from AD by default wouldn't work with it.
Seeing that definition made me realize that there HAD to be a way to get that info out. It wasn't necessarily documented but it had to exist. So when I went back to the RFCs I found a modifier (officially termed an attribute option) that you can append to an attribute to get it to return in an alternate format. You are probably already familiar with a modifier/option, it is ;range=x-y which is used to return a range of members of a multivalue attribute. This other modifier I found for this is ;binary. I had previously seen this years ago but it made no sense to me because the exampole that was given was around a user certificate which is already returned in an octet string, specifying ;binary does nothing to the output format. However with the replication metadata info it normally comes back as an XML stream such as C:\WINDOWS\ADAM>adfind -rootdse msDS-ReplQueueStatistics AdFind V01.31.00cpp Joe Richards ([EMAIL PROTECTED]) March 2006 Using server: 2k3dc01.joe.com:389 Directory: Windows Server 2003 dn: >msDS-ReplQueueStatistics: <DS_REPL_QUEUE_STATISTICS> <ftimeCurrentOpStarted>1601-01-01T00:00:00Z</ftimeCurrentOpStarted> <cNumPendingOps>0</cNumPendingOps> <ftimeOldestSync>160 1-01-01T00:00:00Z</ftimeOldestSync> <ftimeOldestAdd>1601-01-01T00:00:00Z</ftimeOldestAdd> <ftimeOldestDel>1601-01-01T00:00:00Z</ftimeOldestDel> <ftimeOldestMod>1601-01-01T00:00: 00Z</ftimeOldestMod> <ftimeOldestUpdRefs>1601-01-01T00:00:00Z</ftimeOldestUpdRefs> </DS_REPL_QUEUE_STATISTICS> 1 Objects returned Appending the ;binary forces it to come back instead in a BLOB C:\WINDOWS\ADAM>adfind -rootdse msDS-ReplQueueStatistics;binary AdFind V01.31.00cpp Joe Richards ([EMAIL PROTECTED]) March 2006 Using server: 2k3dc01.joe.com:389 Directory: Windows Server 2003 dn: >msDS-ReplQueueStatistics;binary: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 1 Objects returned Which can be cast into the BLOB data structure ds_repl_queue_statisticsw_blob and then the info can be directly pulled out of data structure with code like dwCurrentPendingOps=blob.cNumPendingOps; Instead of having to parse the XML string. What are the benefits here? First off, less processing at the DC to return the info. If you are doing this a lot, you could get a decent savings in DC side processing as it doesn't have to generate the XML. That savings is further increased because you have far less data coming across the wire, it is only half the data at the most. In the example above the binary is 26*2 or 52 bytes in size. For the XML 52 bytes only gets you the string "<DS_REPL_QUEUE_STATISTICS> <ftimeCurrentOpStarted>" which doesn't even have any real data... It takes 445 bytes to send the XML stream.... Finally you don't have to parse up the XML at the client. XML is not usable as it is. Some people may feel it is usable directly because the language or framework they use hides the details but it isn't, you have to process the strings and break them up into the various fields to do anything with it. For one off items you don't have to worry much about that processing because it is neglible and I wouldn't think twice about it, in those cases you go for easy over fast or efficient - that is what .NET and VB are all about. If you are pulling this info a lot, it becomes considerable load very quickly. String processing is and I expect always will be expensive in relation to any straight integer processing. The longer you can avoid string processing the better off you are in terms of speed. Now you could possibly handle this in vbscript, I don't think I would probably want to. Perl shouldn't be too difficult though I have never tried it. For c/c++ this is trivial. At some point I will look into switching adqueueloop over to using this mechanism as it side steps the RPC connectivity requirement and will help with ADAM as I have been having trouble with using DsBindByInstance which is required to make the RPC connection to ADAM. I didn't have time to do the switching prior to DEC as I worked this all while working on the DEC slides. This info is actually all in the DEC slides too btw. The brief initial mention of it was in the LDAP portion of the talk and I figured best to dust over it versus talk about how cool it was as not everyone seemed to be extremely interested in the minutia of LDAP. There is also a slide in the appendix with a little info. joe -- O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, April 06, 2006 4:04 AM To: [email protected] Subject: RE: [ActiveDir] repadmin info oddity That's very cool (and frustrating?), considering the effort you (joe) have made to write something which can collate repl stats. Is this something you take advantage of in adqueueloop or some other utility? neil -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: 05 April 2006 23:04 To: [email protected] Subject: RE: [ActiveDir] repadmin info oddity > I think they need to rethink the choices they've made and just call it > The Dean and Joe Show. Ah we are just the outside pieces of the whole show. There are a ton of some amazing people back in the shadows of the halls of MS that are the real heros. I am sure there are many from those areas that would like to come out and play and post but I have to say that it is a touchy thing to post publicly when you have information that may or may not be publicly available yet. There are often times where I will not respond to some question that I absolutely know the answer to because I am not quite sure if I can write something without disclosing information that I shouldn't and I (and Dean) have just tip of the iceberg access for the most part to that info; if I had full access like the folks "on the inside" it would be VERY difficult for me to post anything I imagine because I would be constantly worrying if what I "know" and wrote was not supposed to be known to anyone else yet. To be more specific, the smallest little slip could give away quite a lot of info if the right people see it. I recently saw a data structure definition in MSDN that I had never heard of / seen before and it kicked in my mind that there must be something else that goes with it just based on how I know things work. I ended up chasing it in the RFCs and found functionality in AD that I had no clue existed and I can't find any public documentation for it related to AD though it is mentioned in the LDAP RFCs. If I listed the data structure definition most people (>99% probably) would just look at it and go "uh yeah, and what about it you dork?". The web page that had me scrambling? http://windowssdk.msdn.microsoft.com/library/default.asp?url=/library/en -us/ AD/ad/ds_repl_queue_statisticsw_blob.asp All that being said, renaming ADAM was a horrible decision. So horrible in fact that I refuse to accept it. Whose with me????? joe -- O'Reilly Active Directory Third Edition - http://www.joeware.net/win/ad3e.htm -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Scott Klassen Sent: Wednesday, February 22, 2006 9:55 AM To: [email protected] Subject: RE: [ActiveDir] repadmin info oddity Very handy and helpful. Thank you so much for the time you've put into this Dean. If I ever get to meet you, it's worth a few at the bar. If you didn't see a couple of days ago, MS announced that it was changing the names for parts of AD. I think they need to rethink the choices they've made and just call it The Dean and Joe Show. Has a bit of a ring to it. :) Scott Klassen -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dean Wells Sent: Wednesday, February 22, 2006 8:17 AM To: Send - AD mailing list Subject: RE: [ActiveDir] repadmin info oddity As I'd hoped, joe does indeed have another way ... one that does the encoding for you. There's a prefix that can be supplied within the filter that extends to any attribute of your choosing (this instructs ADfind to manipulate the byte ordering and related structure); {{GUID=<octet string>}}. The query below exploits that feature permitting repadmin's string>GUID format to be supplied directly. C:\>adfind -config -binenc -f (retiredReplDSASignatures=*{{GUID:6cc4a8e0-2019-4e4f-81cd-f35926de38a3}} *)" -dn -- Dean Wells MSEtechnology * Email: [EMAIL PROTECTED] http://msetechnology.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dean Wells Sent: Tuesday, February 21, 2006 7:09 PM To: Send - AD mailing list Subject: RE: [ActiveDir] repadmin info oddity Hmmm, I would guess he's probably adding a new switch to deal with this particular thread. Anyway, since he's not responded, I'll take a stab at what ADfind can or cannot do here (not really ADfind's problem if my lazy research is accurate). The attribute in question's syntax is a single-valued "octet string" which can typically be filtered against assuming the correct notation is supplied. This particular attribute, however, will often contain multiple GUIDs within the flat value (a pack of them) making it difficult to successfully construct a reliable and/or optimal filter (remember, medial queries are painful without the necessary index). To further complicate the issue, the byte ordering is maintained differently internally to the way it's displayed. Since ADfind AFAIK cannot yet decode "retiredReplDSASignatures", in order to query against it we have to reorder it ourselves. Here's an example of how to convert repadmin's display format to the internally maintained byte ordering (this is a little painful) - repadmin's output = 6cc4a8e0-2019-4e4f-81cd-f35926de38a3 internal structure = E0 A8 C4 6C 19 20 4F 4E 81 CD F3 59 26 DE 38 A3 ... now trim the hyphens and pad repadmin's output to pair up the bytes - repadmin's output = 6c c4 a8 e0 20 19 4e 4f-81 cd f3 59 26 de 38 a3 (padded & trimmed) internal structure = E0 A8 C4 6C 19 20 4F 4E 81 CD F3 59 26 DE 38 A3 ... now, re-order the 1st 4 octets, then the next 2 octets and again the next 2 octets. I've added extra spaces for legibility (essentially, you're re-ordering the first double-word, the next word, the next word and the rest remains as is ... this is known as "network" or "pretty" byte ordering) - repadmin's output = e0 a8 c4 6c 19 20 4f 4e 81 cd f3 59 26 de 38 a3 (re-ordered) internal structure = E0 A8 C4 6C 19 20 4F 4E 81 CD F3 59 26 DE 38 A3 OK, having done all of that, you now have two possible options: option 1) use a fairly concise query and parse the output as follows ... ... create a string of 8 words (or 8 octet pairs if you prefer) to match ADfind's output format - resulting structure = E0A8 C46C 1920 4F4E 81CD F359 26DE 38A3 ... then use the following syntax - C:\>adfind -config -f "&(objectcategory=ntdsdsa)(retiredReplDSASignatures=*)" -csv -nocsvheader retiredReplDSASignatures | findstr "E0A8 C46C 1920 4F4E 81CD F359 26DE 38A3" ... this returns the DN of the "NTDS Settings" object of the DC that owns the retired invocation ID. If no results are returned, one of two things occurred; 1) you fat-fingered it or 2) the DC no longer exists. option 2) submit the following v. expensive query (note, it's a medial query) - C:\>adfind -config -f "retiredReplDSASignatures=*\E0\A8\C4\6C\19\20\4F\4E\81\CD\F3\59\26\DE\38 \A3* " retiredReplDSASignatures Fingers crossed that Joe will have a hidden switch to do the decoding for you, until then, this is it I'm afraid. -- Dean Wells MSEtechnology * Email: [EMAIL PROTECTED] http://msetechnology.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Scott Klassen Sent: Tuesday, February 21, 2006 5:11 PM To: [email protected] Subject: RE: [ActiveDir] repadmin info oddity Too bad Joe picked today to be MIA. Scott Klassen -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dean Wells Sent: Tuesday, February 21, 2006 7:59 AM To: Send - AD mailing list Subject: RE: [ActiveDir] repadmin info oddity The GUIDs returned in this scenario are not used by the directory in the traditional manner and, as such, using a GUID-based binding string won't locate the owning object. The invocation IDs (which are indeed GUIDs but not objectGUIDs) are maintained on the DC's NTDSDSA instance (its NTDS Settings object) by the "invocationId" property ... retired invocation IDs are maintained by retiredReplDSASignatures. ADfind can likely hit these ... but the GUIDS needs to be expressed as part of the query filter, not the base. I'll leave joe (why does he insist on using a little "j"?) to provide the ADfind syntax (it seems that no matter how hard I try, joe will always have a better switch ... and if he doesn't have one, I'm fairly certain he quickly adds it, quietly releases the new binary with the same version number, posts the reply, ridicules my uneducated attempt at using a "real tool" (joe's words, not mine) and professes his innocence :0). -- Dean Wells MSEtechnology * Email: [EMAIL PROTECTED] http://msetechnology.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Thommes, Michael M. Sent: Tuesday, February 21, 2006 7:44 AM To: [email protected] Subject: RE: [ActiveDir] repadmin info oddity Adfind (http://www.joeware.net/win/free/tools/adfind.htm) to the rescue! I recently had to do this and got it accomplished with the following syntax (with a little help from joe :) ): adfind -default -binenc -f objectGUID={{GUID:0B3F5BC4-5713-4611-8F6A-752A3B0DE664}} dn ("adfind /???" For lots of good info!) Mike Thommes -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of SCOTT KLASSEN Sent: Monday, February 20, 2006 8:56 PM To: [email protected] Subject: [ActiveDir] repadmin info oddity I try to keep up on new or updated MS KB articles and often check to see how they correlate with my environment. I noticed that 875495, dealing with USN rollbacks, was updated earlier this month. As I've experienced two AD issues, both of which needed PSS involvement (one dealing with sysvol inconsistency and the other which wound up being the RID master going on temporary strike) I figured that I'd do a quick check as described in the article. On the good side, the USN's are consistent between controllers. On the disconcerting side, I got a little more information than I was expecting. Besides my DC's, I also got USN listings for several GUIDs. I assume these are leftovers from DC demotions and only remain in the form of historical data. Do I need to worry about these (especially the DC1 (retired) listing) and is there a way I can resolve the GUIDs to names, find where this info is hiding, and clear them out? Thanks, Scott Klassen >repadmin /showutdvec dc1 dc=domain,dc=com Caching GUIDs. .. Default-First-Site-Name\DC2 @ USN 455091 @ Time 2006-02-20 20:08:20 2c92760e-e8fc-4418-947e-3b1016ab8514 @ USN 1012381 @ Time 2005-08-04 00:02:34 6e129965-56c3-469e-b70a-f1fdfb8bb2cc @ USN 969931 @ Time 2004-07-24 11:53:16 Default-First-Site-Name\DC1 @ USN 1717571 @ Time 2006-02-20 20:10:50 Default-First-Site-Name\DC1 (retired) @ USN 1298674 @ Time 2005-08-05 06:36:16 e2199f22-f1dd-4d1c-90a6-0e8bb874f355 @ USN 744173 @ Time 2004-12-28 20:52:04 ff0d7d50-214f-4bc1-96b6-55ac6ef317f0 @ USN 852323 @ Time 2005-06-08 14:29:20 List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ PLEASE READ: The information contained in this email is confidential and intended for the named recipient(s) only. If you are not an intended recipient of this email please notify the sender immediately and delete your copy from your system. You must not copy, distribute or take any further action in reliance on it. Email is not a secure method of communication and Nomura International plc ('NIplc') will not, to the extent permitted by law, accept responsibility or liability for (a) the accuracy or completeness of, or (b) the presence of any virus, worm or similar malicious or disabling code in, this message or any attachment(s) to it. If verification of this email is sought then please request a hard copy. Unless otherwise stated this email: (1) is not, and should not be treated or relied upon as, investment research; (2) contains views or opinions that are solely those of the author and do not necessarily represent those of NIplc; (3) is intended for informational purposes only and is not a recommendation, solicitation or offer to buy or sell securities or related financial instruments. NIplc does not provide investment services to private customers. Authorised and regulated by the Financial Services Authority. Registered in England no. 1550505 VAT No. 447 2492 35. Registered Office: 1 St Martin's-le-Grand, London, EC1A 4NP. A member of the Nomura group of companies. List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/ List info : http://www.activedir.org/List.aspx List FAQ : http://www.activedir.org/ListFAQ.aspx List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
