RE: [ActiveDir] Another new joeware tool - GCChk
Agreed on the tuple part. Too bad the AD or the Engine wasn't better at that insertion. If you aren't guaranteeing uniqueness then the user shouldn't really feel that impact. On the CNF part, I am a little confused by what you write. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. It sounds like you start to say the CNF's aren't consistent then are then aren't. It has been my experience that the CNF objects replicate just like normal objects around the rest of the domain (and GCs). This would be correct behavior actually since they are simply an object, doesn't matter who created the object, be it the system or a person. I agree that an additional attribute to flag these would be nice as Robbie indicated. Especially since these aren't ever something good and most likely not expected. The fact that these are handled poorly by most MS apps including ldifde helps point out, I think, that they are special and pretty much unexpected. MS sort of fixed this in K3by fixing the output of distinguishedName to return \0A instead of \n but they missed cn and name. So anyway, doing a search on one dsa for all CNF: objects should catch all of them within the normal rules of loose consistency. Are you saying that this may not be true? If you bump the timeout value in adfind with -t, timeout shouldn't be an issue as I set that on the page retrieval as well as the search init call.If you have a large directory with very few CNFs you could make the search page of 1 record length for return and still have an issue without modifying timeout values. joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric FleischmanSent: Monday, July 12, 2004 10:03 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Monday, July 12, 2004 8:11 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. Kind of would be interesting to have a "bad" things service that watched for "bad" things in the directory and would flag them out when it found them. These objects would be good things to flag, what else could be flagged? Objects w/o GUIDs? What else? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robbie AllenSent: Sunday, July 11, 200
RE: [ActiveDir] Another new joeware tool - GCChk
Are you saying that this may not be true? I am saying it may not be true if there are other, not understood issue. For example, assume you have a lingering object on GC1 but not GC2. Then when each gets an update from DC1 (who say has a writeable copy of the NC) you have a CNF on one DC and not the other. So they should be uniform in the face of perfectly healthy environment. But thats not always the case is all Im saying. ~Eric From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Tuesday, July 13, 2004 11:51 AM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Agreed on the tuple part. Too bad the AD or the Engine wasn't better at that insertion. If you aren't guaranteeing uniqueness then the user shouldn't really feel that impact. On the CNF part, I am a little confused by what you write. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. It sounds like you start to say the CNF's aren't consistent then are then aren't. It has been my experience that the CNF objects replicate just like normal objects around the rest of the domain (and GCs). This would be correct behavior actually since they are simply an object, doesn't matter who created the object, be it the system or a person. I agree that an additional attribute to flag these would be nice as Robbie indicated. Especially since these aren't ever something good and most likely not expected. The fact that these are handled poorly by most MS apps including ldifde helps point out, I think, that they are special and pretty much unexpected. MS sort of fixed this in K3by fixing the output of distinguishedName to return \0A instead of \n but they missed cn and name. So anyway, doing a search on one dsa for all CNF: objects should catch all of them within the normal rules of loose consistency. Are you saying that this may not be true? If you bump the timeout value in adfind with -t, timeout shouldn't be an issue as I set that on the page retrieval as well as the search init call.If you have a large directory with very few CNFs you could make the search page of 1 record length for return and still have an issue without modifying timeout values. joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman Sent: Monday, July 12, 2004 10:03 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Monday, July 12, 2004 8:11 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time
RE: [ActiveDir] Another new joeware tool - GCChk
Noted in my ideas folder Thank.you.very.much. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robbie AllenSent: Monday, July 12, 2004 11:12 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Don't even get me started on medial searches, which in my mind wasone of the glaringdeficiencieswith W2K AD compared to the other LDAP-based directories I'm familiar (e.g., iPlanet/SunOne/Java whatever). With W2K, you might as well not even try them. Horrible performance. In a 50k object domain I've seen medial searches tack on another 10 seconds to the query time (compared to the same query but remove the leading star). Allowing users to configuretuple indexes in W2K3 is fine, but IMO tuple indexing should be the norm for common attributes. Sync'ing objects to another directory for the sole purpose of finding conflict objects sounds like an overcomplicated solution to me. How about if MS just flagged conflict objects as being in conflict via some attribute:-? Telling people to install ADAM and download the AD/ADAM synchronizer is going to sound too much like work to do something as (conceptually) simple as finding conflict objects. Joe, here are the types of objects I consider to be "bad": - conflict objects - lingering objects - objects w/o guids - objects in the LostAndFound container - user objects w/dup SIDs - user objects w/dup UPNs Then there are a bunch of data maintenance related things I consider "not optimal": - missing subnet objects (requires parsing the system event log on DCs) - sites with no subnets (or site links) - computer objects for Windows 2000 and higher computers that have a password age of 6 months or more - groups with no members - GPOs that aren't linked - etc. I'm sure there are manyothers people can think of. Robbie Allen http://www.rallenhome.com/ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric FleischmanSent: Monday, July 12, 2004 10:03 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Monday, July 12, 2004 8:11 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. Kind of would be interesting to have a "bad" things service
RE: [ActiveDir] Another new joeware tool - GCChk
Ah I see, a lingering object spawning the creation of yet another lingering object. Sounds like another good reason for MS to have a nice easy public method of finding lingering objects or maybe a generic database cleanup/audit tool... :o) joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric FleischmanSent: Tuesday, July 13, 2004 2:00 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Are you saying that this may not be true? I am saying it may not be true if there are other, not understood issue. For example, assume you have a lingering object on GC1 but not GC2. Then when each gets an update from DC1 (who say has a writeable copy of the NC) you have a CNF on one DC and not the other. So they should be uniform in the face of perfectly healthy environment. But thats not always the case is all Im saying. ~Eric From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Tuesday, July 13, 2004 11:51 AMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Agreed on the tuple part. Too bad the AD or the Engine wasn't better at that insertion. If you aren't guaranteeing uniqueness then the user shouldn't really feel that impact. On the CNF part, I am a little confused by what you write. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. It sounds like you start to say the CNF's aren't consistent then are then aren't. It has been my experience that the CNF objects replicate just like normal objects around the rest of the domain (and GCs). This would be correct behavior actually since they are simply an object, doesn't matter who created the object, be it the system or a person. I agree that an additional attribute to flag these would be nice as Robbie indicated. Especially since these aren't ever something good and most likely not expected. The fact that these are handled poorly by most MS apps including ldifde helps point out, I think, that they are special and pretty much unexpected. MS sort of fixed this in K3by fixing the output of distinguishedName to return \0A instead of \n but they missed cn and name. So anyway, doing a search on one dsa for all CNF: objects should catch all of them within the normal rules of loose consistency. Are you saying that this may not be true? If you bump the timeout value in adfind with -t, timeout shouldn't be an issue as I set that on the page retrieval as well as the search init call.If you have a large directory with very few CNFs you could make the search page of 1 record length for return and still have an issue without modifying timeout values. joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric FleischmanSent: Monday, July 12, 2004 10:03 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Monday, July 12, 2004 8:11 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has
RE: [ActiveDir] Another new joeware tool - GCChk
Where is STEWART KWAN when you need him to chime in about this. Trying to see if the key word notification system works. Joe, I am running one more scan, I will send you the results and we can discuss it more. I will share with the rest once I am complete my analysis. Thanks, Todd From: joe [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 13, 2004 3:20 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Ah I see, a lingering object spawning the creation of yet another lingering object. Sounds like another good reason for MS to have a nice easy public method of finding lingering objects or maybe a generic database cleanup/audit tool... :o) joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman Sent: Tuesday, July 13, 2004 2:00 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Are you saying that this may not be true? I am saying it may not be true if there are other, not understood issue. For example, assume you have a lingering object on GC1 but not GC2. Then when each gets an update from DC1 (who say has a writeable copy of the NC) you have a CNF on one DC and not the other. So they should be uniform in the face of perfectly healthy environment. But thats not always the case is all Im saying. ~Eric From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Tuesday, July 13, 2004 11:51 AM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Agreed on the tuple part. Too bad the AD or the Engine wasn't better at that insertion. If you aren't guaranteeing uniqueness then the user shouldn't really feel that impact. On the CNF part, I am a little confused by what you write. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. It sounds like you start to say the CNF's aren't consistent then are then aren't. It has been my experience that the CNF objects replicate just like normal objects around the rest of the domain (and GCs). This would be correct behavior actually since they are simply an object, doesn't matter who created the object, be it the system or a person. I agree that an additional attribute to flag these would be nice as Robbie indicated. Especially since these aren't ever something good and most likely not expected. The fact that these are handled poorly by most MS apps including ldifde helps point out, I think, that they are special and pretty much unexpected. MS sort of fixed this in K3by fixing the output of distinguishedName to return \0A instead of \n but they missed cn and name. So anyway, doing a search on one dsa for all CNF: objects should catch all of them within the normal rules of loose consistency. Are you saying that this may not be true? If you bump the timeout value in adfind with -t, timeout shouldn't be an issue as I set that on the page retrieval as well as the search init call.If you have a large directory with very few CNFs you could make the search page of 1 record length for return and still have an issue without modifying timeout values. joe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman Sent: Monday, July 12, 2004 10:03 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page
RE: [ActiveDir] Another new joeware tool - GCChk
Joe,
Thanks a Bunch!
Todd
From: joe
[mailto:[EMAIL PROTECTED]
Sent: Sunday, July 11, 2004 5:14
PM
To: [EMAIL PROTECTED]
Subject: [ActiveDir] Another new
joeware tool - GCChk
Ok, those of you who have been following Todd's recent
adventures may have come to the same conclusion I did that it would be nice to
have a tool that could check a GC and see if it has lingering objects on it.
Well I threw together something quick and dirty this
afternoon that should do that. It is not extremely efficient in how it does the
work but should theoretically work. I didn't test with lingering objects but I
did test with renames and deletes with a slow replication link to the GC and it
always found the objects that were different on the GC partial NC than in the
full NC on the proper DCs.
Basically you tell the program to check a specific GC. It
will then enumerate the partitions and find the nearest (per normal nearest
methodology) DC for each partial NC and then enumerate every object in every NC
(getys DN and objectGUID)and do a lookup of that object against the
proper DC. If it does not find the exact DN, it will then do a objectguid
search in case the object was renamed, moved,or deleted and that hasn't
replicated to the GC yet. If it finds the object, it will display the new DN
and the whenChanged timestamp. Note that by default you must be an Admin to
find/display objects that have been deleted so if you aren't an admin you will
not know the whole story if it finds objects on the GC that it can't locate on
the DCs. Note though, the first pass could be with a normal userid and if you
have objects it can't findyou could rerun as an admin or check deleted objects
yourself. Oh one thing to note that if the object has been moved to another
domain then this won't find it either, I didn't feel it was worth trying to
chase in that case because that requires a GC and the GC just told me where it
thought it was at and I looked there...
In order to really check a forest you will need to check at
least two GCs. This is because the tool will not check the default NC of the GC
you select. So if you have say 5 domains in your forest named joe.com,
child1.joe.com,child2.joe.com,child3.joe.com,and child4.joe.com running the
tool against a GC for joe.com would check all of the children domains but not
joe.com. Running against one of the children domains would check joe.com and
all but that one child domain.
If you find issues with this tool, let me know. If you find
lingering objects with it I WOULD LOVE to hear about that. I can then say for
sure that someone has used it and found lingering objects.
Here is a sample run showing it finding some deleted
objects... Note that once the replication occurred to the GC, the deleted
objects would no longer show up in the report.
[Sun 07/11/2004
16:28:57.36]
F:\DEV\cpp\GCChkgcchk -gc 2k3dc10
GCChk
V01.00.00cpp Joe Richards ([EMAIL PROTECTED]) July 2004
Collecting NCs
from partitions container...
Locating DCs for other NCs...
NC: DC=joe,DC=com -- 2k3dc01.joe.com
Skipping local NC DC=child1,DC=joe,DC=com
Comparing DC=joe,DC=com
.
Objects
Checked : 524
Object Issues : 0
Objects Not Found: 0
The command
completed successfully.
[Sun 07/11/2004
16:30:11.65]
F:\DEV\cpp\GCChkadfind -b
ou=admodundeletetest,ou=testou,dc=joe,dc=com -f
((objectcategory=computer)) -dsq |admod -del
AdMod
V01.00.00cpp Joe Richards ([EMAIL PROTECTED]) July 2004
DN Count: 5
Using server: 2k3dc01.joe.com
Deleting specified objects...
DN: cn=undel-a1,ou=admodundeletetest,ou=testou,dc=joe,dc=com...
DN: cn=undel-a2,ou=admodundeletetest,ou=testou,dc=joe,dc=com...
DN: cn=undel-a3,ou=admodundeletetest,ou=testou,dc=joe,dc=com...
DN: cn=undel-a4,ou=admodundeletetest,ou=testou,dc=joe,dc=com...
DN: cn=undel-a5,ou=admodundeletetest,ou=testou,dc=joe,dc=com...
The command
completed successfully
[Sun 07/11/2004
16:30:16.49]
F:\DEV\cpp\GCChkgcchk -gc 2k3dc10
GCChk
V01.00.00cpp Joe Richards ([EMAIL PROTECTED]) July 2004
Collecting NCs
from partitions container...
Locating DCs for other NCs...
NC: DC=joe,DC=com -- 2k3dc01.joe.com
Skipping local NC DC=child1,DC=joe,DC=com
Comparing DC=joe,DC=com
ERROR: GC DN for object not found:
CN=undel-a1,OU=AdModUndeleteTest,OU=TestOU,DC=joe,DC=com
-- {DB12BE55-CDAC-4C06-A9FB-280CF445AE73}
Found this:
CN=undel-a1\0ADEL:db12be55-cdac-4c06-a9fb-280cf445ae73,CN=Deleted
Objects,DC=joe,DC=com -- 20040711203016.0Z
ERROR: GC DN
for object not found:
CN=undel-a2,OU=AdModUndeleteTest,OU=TestOU,DC=joe,DC=com
-- {3DA8AB38-D8A5-40F0-8CC2-13A43BAD59B7}
Found this:
CN=undel-a2\0ADEL:3da8ab38-d8a5-40f0-8cc2-13a43bad59b7,CN=Deleted
Objects,DC=joe,DC=com -- 20040711203016.0Z
ERROR: GC DN
for object not found:
RE: [ActiveDir] Another new joeware tool - GCChk
Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. Kind of would be interesting to have a "bad" things service that watched for "bad" things in the directory and would flag them out when it found them. These objects would be good things to flag, what else could be flagged? Objects w/o GUIDs? What else? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robbie AllenSent: Sunday, July 11, 2004 9:56 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Looks interesting, I'll check it out. On a somewhat related topic, I've always wanted an "efficient"toolfor findingconflict objects. Most domains I've looked at have more than a few hanging around; especially computer objects. The problem is the only way I know of to find conflict objects is to use a terribly inefficient search filter like (cn=*\0ACNF:*). This can easily timeout in large domains. But other than the name, I'm not aware of any other way an object is tagged as beinginconflict. Robbie Allen http://www.rallenhome.com/ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Sunday, July 11, 2004 5:14 PMTo: [EMAIL PROTECTED]Subject: [ActiveDir] Another new joeware tool - GCChk Ok, those of you who have been following Todd's recent adventures may have come to the same conclusion I did that it would be nice to have a tool that could check a GC and see if it has lingering objects on it. Well I threw together something quick and dirty this afternoon that should do that. It is not extremely efficient in how it does the work but should theoretically work. I didn't test with lingering objects but I did test with renames and deletes with a slow replication link to the GC and it always found the objects that were different on the GC partial NC than in the full NC on the proper DCs. Basically you tell the program to check a specific GC. It will then enumerate the partitions and find the nearest (per normal nearest methodology) DC for each partial NC and then enumerate every object in every NC (getys DN and objectGUID)and do a lookup of that object against the proper DC. If it does not find the exact DN, it will then do a objectguid search in case the object was renamed, moved,or deleted and that hasn't replicated to the GC yet. If it finds the object, it will display the new DN and the whenChanged timestamp. Note that by default you must be an Admin to find/display objects that have been deleted so if you aren't an admin you will not know the whole story if it finds objects on the GC that it can't locate on the DCs. Note though, the first pass could be with a normal userid and if you have objects it can't findyou could rerun as an admin or check deleted objects yourself. Oh one thing to note that if the object has been moved to another domain then this won't find it either, I didn't feel it was worth trying to chase in that case because that requires a GC and the GC just told me where it thought it was at and I looked there... In order to really check a forest you will need to check at least two GCs. This is because the tool will not check the default NC of the GC you select. So if you have say 5 domains in your forest named joe.com, child1.joe.com,child2.joe.com,child3.joe.com,and child4.joe.com running the tool against a GC for joe.com would check all of the children domains but not joe.com. Running against one of the children domains would check joe.com and all but that one child domain. If you find issues with this tool, let me know. If you find lingering objects with it I WOULD LOVE to hear about that. I can then say for sure that someone has used it and found lingering objects. Here is a sample run showing it finding some deleted objects... Note that once the replication occurred to the GC, the deleted objects would no longer show up in the report. [Sun 07/11/2004 16:28:57.36]F:\DEV\cpp\GCChkgcchk -gc 2k3dc10 GCChk V01.00.00cpp Joe Richards ([EMAIL PROTECTED]) July 2004 Collecting
RE: [ActiveDir] Another new joeware tool - GCChk
Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Monday, July 12, 2004 8:11 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. Kind of would be interesting to have a bad things service that watched for bad things in the directory and would flag them out when it found them. These objects would be good things to flag, what else could be flagged? Objects w/o GUIDs? What else? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robbie Allen Sent: Sunday, July 11, 2004 9:56 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Looks interesting, I'll check it out. On a somewhat related topic, I've always wanted an efficienttoolfor findingconflict objects. Most domains I've looked at have more than a few hanging around; especially computer objects. The problem is the only way I know of to find conflict objects is to use a terribly inefficient search filter like (cn=*\0ACNF:*). This can easily timeout in large domains. But other than the name, I'm not aware of any other way an object is tagged as beinginconflict. Robbie Allen http://www.rallenhome.com/ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Sunday, July 11, 2004 5:14 PM To: [EMAIL PROTECTED] Subject: [ActiveDir] Another new joeware tool - GCChk Ok, those of you who have been following Todd's recent adventures may have come to the same conclusion I did that it would be nice to have a tool that could check a GC and see if it has lingering objects on it. Well I threw together something quick and dirty this afternoon that should do that. It is not extremely efficient in how it does the work but should theoretically work. I didn't test with lingering objects but I did test with renames and deletes with a slow replication link to the GC and it always found the objects that were different on the GC partial NC than in the full NC on the proper DCs. Basically you tell the program to check a specific GC. It will then enumerate the partitions and find the nearest (per normal nearest methodology) DC for each partial NC and then enumerate every object in every NC (getys DN and objectGUID)and do a lookup of that object against the proper DC. If it does not find the exact DN, it will then do a objectguid search in case the object was renamed, moved,or deleted and that hasn't replicated to the GC yet. If it finds the object, it will display the new DN and the whenChanged timestamp. Note
RE: [ActiveDir] Another new joeware tool - GCChk
Don't even get me started on medial searches, which in my mind wasone of the glaringdeficiencieswith W2K AD compared to the other LDAP-based directories I'm familiar (e.g., iPlanet/SunOne/Java whatever). With W2K, you might as well not even try them. Horrible performance. In a 50k object domain I've seen medial searches tack on another 10 seconds to the query time (compared to the same query but remove the leading star). Allowing users to configuretuple indexes in W2K3 is fine, but IMO tuple indexing should be the norm for common attributes. Sync'ing objects to another directory for the sole purpose of finding conflict objects sounds like an overcomplicated solution to me. How about if MS just flagged conflict objects as being in conflict via some attribute:-? Telling people to install ADAM and download the AD/ADAM synchronizer is going to sound too much like work to do something as (conceptually) simple as finding conflict objects. Joe, here are the types of objects I consider to be "bad": - conflict objects - lingering objects - objects w/o guids - objects in the LostAndFound container - user objects w/dup SIDs - user objects w/dup UPNs Then there are a bunch of data maintenance related things I consider "not optimal": - missing subnet objects (requires parsing the system event log on DCs) - sites with no subnets (or site links) - computer objects for Windows 2000 and higher computers that have a password age of 6 months or more - groups with no members - GPOs that aren't linked - etc. I'm sure there are manyothers people can think of. Robbie Allen http://www.rallenhome.com/ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric FleischmanSent: Monday, July 12, 2004 10:03 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joeSent: Monday, July 12, 2004 8:11 PMTo: [EMAIL PROTECTED]Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. Kind of would be interesting to have a "bad" things service that watched for "bad" things in the directory and would flag them out when it found them. These objects would be good things to flag, what else could be flagged? Objects w/o GUIDs? What else? From: [EMAIL PROTECTED
RE: [ActiveDir] Another new joeware tool - GCChk
So then go to 2k03 and use medial substring indexes. J Thats certainly on the table if you want it. Expensive, but it greatly improves performance. Is there a reason that doesnt do what you need? Again, I dont think this is a good usage scenario for such an expensive index, but if you are really that interested in super fast perf on these sorts of searches, it is a way to make them faster. Another way: cache the entire DIT in memory. ~Eric From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robbie Allen Sent: Monday, July 12, 2004 10:12 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Don't even get me started on medial searches, which in my mind wasone of the glaringdeficiencieswith W2K AD compared to the other LDAP-based directories I'm familiar (e.g., iPlanet/SunOne/Java whatever). With W2K, you might as well not even try them. Horrible performance. In a 50k object domain I've seen medial searches tack on another 10 seconds to the query time (compared to the same query but remove the leading star). Allowing users to configuretuple indexes in W2K3 is fine, but IMO tuple indexing should be the norm for common attributes. Sync'ing objects to another directory for the sole purpose of finding conflict objects sounds like an overcomplicated solution to me. How about if MS just flagged conflict objects as being in conflict via some attribute:-? Telling people to install ADAM and download the AD/ADAM synchronizer is going to sound too much like work to do something as (conceptually) simple as finding conflict objects. Joe, here are the types of objects I consider to be bad: - conflict objects - lingering objects - objects w/o guids - objects in the LostAndFound container - user objects w/dup SIDs - user objects w/dup UPNs Then there are a bunch of data maintenance related things I consider not optimal: - missing subnet objects (requires parsing the system event log on DCs) - sites with no subnets (or site links) - computer objects for Windows 2000 and higher computers that have a password age of 6 months or more - groups with no members - GPOs that aren't linked - etc. I'm sure there are manyothers people can think of. Robbie Allen http://www.rallenhome.com/ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric Fleischman Sent: Monday, July 12, 2004 10:03 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. Whether this is of interest or not would be related to the # of times the search is run. The more often you plain on doing said search, the easier this is to justify. It should be noted, however, that tuple indexes are one of the most expensive types in AD. A string of length N would yield N-2 index entries where N=3.. 3.Have some sort of sinking tool that just watched for those objects and when it found them, synced them to another directory and you could just pull them out of there. This statement comes with the assumption that all CNFs are consistently found on all dsas throughout the forest as if this is not true, looking at one DSAs CNFs does not mean you know the CNFs found on another DSA. I think time has told us that this is an unfair assumption. (think lingering objects) If you did want to do this, however, I think this is a good ADAM usage scenario. Use the new AD syncher tool up on www.microsoft.com/adam (currently beta) and do it against ADAM. Light weight, and zero incremental cost on top of the server it sits on. You can also medial substring index it up in ADAM and eat the pef there, probably not a big deal given usage of this dsa. For the timeout problem, have you tried to use a paged search, and just keep requesting the next page as you get the one before it (despite amt of time the page took to deliver)? Does that help the timeout problem at all? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of joe Sent: Monday, July 12, 2004 8:11 PM To: [EMAIL PROTECTED] Subject: RE: [ActiveDir] Another new joeware tool - GCChk Hmm I can't think of a single way that is more efficient to get that info... Worse yet that is a medial search and I'm betting no one has set their cn index to be a tuple index. The only things I can think of are 1. Use a standard LDAP query and crank the timeout value through the roof (-t option in adfind). 2. Have a program that keeps track of USN's when it does its searches so that it can have the last USN that was in place when it did its last search. That would drammatically limit the number of objects. However if you pointed at a new DC or had to rebuild the DC or the first time you ran it it would have to start at the beginning anyway. 3.Have some sort of sinking tool that just
