>>>For the lingering object I specified, the output lists a GUID ("Originating
>>>DC") that doesn't exist any more
If it shows the GUID that database instance (that is what the invocation ID
identifies) does not exist anymore and most probably the DC does not exist
anymore because it was demoted or its metadata was cleaned
>>>An "Originating DC" is also the "owner" of the object, right?
nope. the "originating DC" is the DC where the change was made or was created
(originated!) it is part of the replication mechanism of AD
>>>The member DC/GCs) of the domain that once hosted this "Originating DC"
>>>produce a different output from the repadmin /showobjmeta command than the
>>>other GCs - namely "Directory Object not found"
"directory object not found" can mean two things...
(1) the object was created somewhere, but has not yet replicated to several
DCs/GCs
(2) it is a lingering object (which by the way can be identified by event ID
1388 or 1988 if an attempt is made to replicate a lingering object)
>>>If a DC is demoted, the object would be "owned" by one of the remaining DCs.
>>> But, if the "owner" is no longer around, the object is garbage. Right?
nope...
again... the "originating DC" is the DC where the change was made or was
created (originated!) it is part of the replication mechanism of AD
the object was "born" on a DC. just because the DC is gone does not mean the
object is crap or worthless. look into it from another point of view. you were
born as a child from your mom and dad. just because your mom and/or dad is/are
gone someday does NOT mean you should not exist. Definitely not true! I think
you get the point ;-))
....
Lingering objects are a real PITA!!!
Lingering objects can only be detected if:
(1) something of a lingering objects is changed and it tries to replicate
(event ID 1388 if you loose consistency replication and event ID 1988 if you
strict consistency replication)
(2) you go in and look for them using REPADMIN (support tools) or GCCHK
(joeware).
REPADMIN in advisory mode will report event IDs 1938 (starting detection
summary), 1946 (for each lingering object detected) and 1942 (final detection
summary) are registered in the DS event log
REPADMIN in cleanup mode will report event IDs 1937 (starting removal summary),
1945 (for each lingering object detected and removed) and 1939 (final removal
summary) are registered in the DS event log
(3) you have issues in your directory. mail not being delivered, address book,
replication, etc...
I believe it is possible to automate the detection and cleanup, but that
requires some scripting...
the following is some info I posted about a week or two ago...
#########################
"Lingering objects" are a PITA! Lingering objects on DCs/GCs occur when one or
more objects are deleted on some DC while that DC is disconnected for more than
the tombstone lifetime (which depends on the first DC in the forest and its OS
or some manual configuration). When an object is deleted the object is
"transformed" into a tombstone. Because the DC is disconnected where the
deletion occured it is not able to replicate the tombstone to other DCs. After
the tombstone lifetime the tombstone objects are garbage collected. After that
moment one or more DCs don't have any trace of the deleted object or tombstones
while other DCs do because the tombstone was never replicated to them. As soon
as the connection is reestablished after the tombstone lifetime, the lingering
objects replicate to the other DCs again (at least they try to, but that
depends on how (A) is configured).
More information about lingering objects and possible ways to remove lingering
objects:
(1)
http://technet2.microsoft.com/WindowsServer/en/Library/4a1f420d-25d6-417c-9d8b-6e22f472ef3c1033.mspx
(2) http://support.microsoft.com/?id=314282
"non-existent objects in GCs". These could also be called lingering objects,
but that is a definition thing I'm not getting into right now. "Non-existent
objects" can occur in GCs when a domain in a MULTI DOMAIN FOREST has been fully
restored using backups (with this I mean that all DCs, or some DCs while others
are being rebuild, have been restored). Because the domain was restored using
backups (and backups are always older than the current situation), the domain
went "back in time" (it went to the state when the backups were created).
However, GCs in other domains are still up and running and those were not
restored or did not go to the same state as the writable version of the domain
did. Because of that the GCs know more than the DCs of the domain itself and
THAT is NOT correct!
A possible way to remove "non-existent objects in GCs" (or the read-only
version of objects in the GCs that correspond to the writable version of
objectson DCs but do not exist):
(1) http://support.microsoft.com/?id=314282
Something else that can happen is that the version of attributes from objects
in the restored domain can be lower than the version of attributes of the same
objects in the GCs.
The only way to solve both is to rebuild the particular read only partition on
ALL GCs that corresponds with the writable domain that has been fully restored
using REPADMIN /UNHOST <GCs> <DN of partition>
Replication protections....
W2K DCs have one protection concerning lingering objects (A)
W2K3 DCs have two protections concerning lingering objects (A) & (B)
(A) Strict and loose replication
When strict replication is enabled the replication of lingering objects is
halted. When loose replication is enabled lingering objects replicate to the
DCs (which have loose replication enabled)
(B) Replication time exceeds the tombstone lifetime
When a DC does not replicate at least once within the tombstone lifetime,
replication is halted with the DC that has been disconnected for more than the
tombstone lifetime.
Also see:
http://blogs.dirteam.com/blogs/jorge/archive/2005/11/24/153.aspx
When STRICT replication is ENABLED:
* Event ID 1988 is registered in the DS event log of the DC NOT containing the
lingering (while replication of the lingering object was prevented), which
specifies the "source DC" (the DC containing the lingering object) (e.g. Source
DC (Transport-specific network address):
ed0c6501-28c1-47e9-b3db-5dcf281e9e31._msdcs.ADCORP.LAN), the "object" (the
lingering object) (e.g. CN=JORGE DE ALMEIDA
PINTO,OU=Users,OU=ORG,DC=ADCORP,DC=LAN), the "object GUID" (the GUID of the
lingering object) (e.g. Object GUID: a46ccb2a-2214-475e-b59a-c2aa3b357196)
to detect lingering objects (but not remove): REPADMIN /REMOVELINGERINGOBJECTS
<FQDN of DC with lingering objects> <objectGUID of DC with correct data> <DN of
partition> /ADVISORY_MODE
--> on the DC containing the lingering objects the event IDs 1938 (starting
detection summary), 1946 (for each lingering object detected) and 1942 (final
detection summary) are registered in the DS event log
to detect AND remove lingering objects: REPADMIN /REMOVELINGERINGOBJECTS <FQDN
of DC with lingering objects> <objectGUID of DC with correct data> <DN of
partition>
--> on the DC containing the lingering objects the event IDs 1937 (starting
removal summary), 1945 (for each lingering object detected and removed) and
1939 (final removal summary) are registered in the DS event log
When LOOSE replication is ENABLED:
* Event ID 1388 is registered in the DS event log of the DC NOT containing the
lingering (while replication of the lingering object was NOT prevented), which
specifies the "source DC" (the DC containing the lingering object) (e.g. Source
DC (Transport-specific network address):
ed0c6501-28c1-47e9-b3db-5dcf281e9e31._msdcs.ADCORP.LAN), the "object" (the
lingering object) (e.g. CN=JORGE DE ALMEIDA
PINTO,OU=Users,OU=ORG,DC=ADCORP,DC=LAN), the "object GUID" (the GUID of the
lingering object) (e.g. Object GUID: a46ccb2a-2214-475e-b59a-c2aa3b357196), the
"Directory partition" (the partition were the lingering object was hosted)
(e.g. Directory partition: DC=ADCORP,DC=LAN ), the "Destination highest
property USN" (the highest commited USN on the DC NOT containing the lingering
object) (e.g. Destination highest property USN: 14221)
REPADMIN /REMOVELINGERINGOBJECTS <FQDN of DC with lingering objects>
<objectGUID of DC with correct data> <DN of partition> /ADVISORY_MODE --> WILL
NOT DETECT LINGERING OBJECTS because the data between "DC with lingering
objects" and "DC with correct data" is the same
REPADMIN /REMOVELINGERINGOBJECTS <FQDN of DC with lingering objects>
<objectGUID of DC with correct data> <DN of partition> --> WILL NOT DETECT AND
NOT REMOVE LINGERING OBJECTS because the data between "DC with lingering
objects" and "DC with correct data" is the same
So if STRICT replication is ENABLED, possible ways to detect lingering objects:
* REPADMIN /REMOVELINGERINGOBJECTS with the /ADVISORY_MODE option
* scan DCs for event ID 1988 which contain information about the lingering
objects (e.g. with EVENTCOMBMT)
* GCCHK from joeware.net
So if LOOSE replication is ENABLED, possible ways to detect lingering objects:
* scan DCs for event ID 1388 which contain information about the lingering
objects (e.g. with EVENTCOMBMT)
MY EUR0.02:
* don't enable LOOSE replication (because of security issues with re-introduced
objects that should have been deleted)
* be carefull using a mixture of loose replication and strict replication as
that may lead to inconsistencies and/or security issues
* Enable STRICT replication
* for now detect and cleanup lingering objects
* Make sure you MONITOR end-to-end AD and SYSVOL replication between DCs!!!
(<-- should be a MUST on your priority list!!!)
* Make sure you MONITOR DC/BH/GC/FSMO availability and detect disconnected DCs
ASAP!!! (<-- should be a MUST on your priority list!!!)
* Monitoring is a preventive measure that helps you NOT getting the headaches
you may have now concerning lingering objects!
#########################
Met vriendelijke groeten / Kind regards,
Ing. Jorge de Almeida Pinto
Senior Infrastructure Consultant
MVP Windows Server - Directory Services
LogicaCMG Nederland B.V. (BU RTINC Eindhoven)
( Tel : +31-(0)40-29.57.777
( Mobile : +31-(0)6-26.26.62.80
* E-mail : <see sender address>________________________________ From: [EMAIL PROTECTED] on behalf of Thommes, Michael M. Sent: Wed 2006-05-03 17:28 To: [email protected] Subject: RE: [ActiveDir] which GC answers? Hi Jorge, I don't mean to hijack this thread but I have also been having an issue with lingeringobjects. I ran your repadmin command shown below on one of the lingering objects I have. For the lingering object I specified, the output lists a GUID ("Originating DC") that doesn't exist any more. An "Originating DC" is also the "owner" of the object, right? The member DC/GCs) of the domain that once hosted this "Originating DC" produce a different output from the repadmin /showobjmeta command than the other GCs - namely "Directory Object not found". If a DC is demoted, the object would be "owned" by one of the remaining DCs. But, if the "owner" is no longer around, the object is garbage. Right? My question is this - why are lingeringobjects such a "bear" to clean out? It seems to me an admin should be able to use a "repadmin /removelingeringobjects GC: <DN of lingering object>" -type of syntax to take care of all of the GCs at the same time. My TAM has indicated the existence of a "replfix" tool, but I'm not sure how it works. Thoughts/comments? Mike Thommes Ps. For any MS folks out there, it would really be helpful to include examples within the repadmin help considering how powerful this command can be. Pps. I think "lingeringobjects" are synonymous with "headache". ________________________________ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Almeida Pinto, Jorge de Sent: Wednesday, May 03, 2006 9:21 AM To: [email protected] Subject: RE: [ActiveDir] which GC answers? a way to check this is: REPADMIN /SHOWOBJMETA GC: <DN of lingering object> > OUTPUT.TXT GC: targets ALL GCs in the forest For each GC: * you get the metadata of the object if it exists on the GC OR * you get "Directory object not found" if the object does not exist in addition to this you can wrap a script around this that takes away some manual stuff you must do. Met vriendelijke groeten / Kind regards, Ing. Jorge de Almeida Pinto Senior Infrastructure Consultant MVP Windows Server - Directory Services LogicaCMG Nederland B.V. (BU RTINC Eindhoven) * Tel : +31-(0)40-29.57.777 * Mobile : +31-(0)6-26.26.62.80 * E-mail : <see sender address> ________________________________ From: [EMAIL PROTECTED] on behalf of [EMAIL PROTECTED] Sent: Wed 2006-05-03 14:44 To: [email protected] Subject: [ActiveDir] which GC answers? When I use ldp and I found a user (lingering) how can I know which GC of many of them has that copy of the object? I use ADSIEDT, but I have many GC´s. is there a easier way to discover in which of them it is? Thanks Adrião F Ramos This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
<<winmail.dat>>
