Hi all,
I'd like to report on some questions, their answers and some conclusions
reached while Trustin and I were talking on IM about Mitosis:
======================= Start =========================
1). When adding an entry that was already marked entryDeleted true, why
does mitosis perform a modify operation on the entry instead?
Answer: I don't remember maybe I did it this way because of a lack of
knowledge.
2). In the OperationFactory.newAdd(CSN,LdapDN,Attributes) method the
first thing you do is call checkBeforeAdd(LdapDN) whose short body is
printed out below:
private void checkBeforeAdd( LdapDN newEntryName ) throws
NamingException
{
if ( nexus.hasEntry( newEntryName ) )
{
throw new NameAlreadyBoundException(
newEntryName.toString() + " already exists." );
}
}
On add operations where the entry is marked entryDeleted=true, this
method always throws an exception and shorts the progress of the add()
operation. I have modified it to check if the entry is marked as
deleted first before throwing an exception like so:
private void checkBeforeAdd( LdapDN newEntryName ) throws
NamingException
{
if ( nexus.hasEntry( newEntryName ) )
{
Attributes entry = nexus.lookup( newEntryName );
Attribute deleted = entry.get( Constants.ENTRY_DELETED );
Object value = deleted == null ? null : deleted.get();
/*
* Check first if the entry has been marked as deleted before
* throwing an exception and delete the entry if so and return
* without throwing an exception.
*/
if ( value != null && "true".equalsIgnoreCase(
value.toString() ) )
{
return;
}
throw new NameAlreadyBoundException(
newEntryName.toString() + " already exists." );
}
}
Now is this the right thing to do?
Answer: Yes this is good.
3). After making this change I still have some problems. Let me
describe the sequence of events in terms of 2 replicas A and B and an
add operation:
(i) add entry (1) to replica A
Everything is fine up to now. Replication of the add propagates just
fine. I can see the entry in both replica A and B.
(ii) delete entry (1) from replica A
Everything is fine up to now. Replication of the del propagates just
fine. I can not see the entry in either replica A and B. Pulling up
the diagnostic gui I can see that the entry is in the partition but
marked entryDeleted = true.
(iii) add entry (1) back to replica A with same DN and same contents
Here I get a *FAILURE*. The error response I get from the server is:
06:06:49 PM: Failed to add new entry ou=neworganizationalUnit,
dc=example,dc=com Root error: [LDAP: error code 80 - failed to add entry
ou=neworganizationalUnit,dc=example,dc=com: Unexpected exception.]
In fact I get this error if I try to add to either replica A or B. At
this point I started a debug session to determine exactly what is going
on which prevents this 2nd add operation from working.
3a). I noticed that for add you build a CompositeOperation but really
you're only putting into it a single AddEntryOperation. Did you intend
to do this or is it a simple bug?
Answer: Yeah you are right we don't need to wrap it.
3b). To the entry to be added you remove any entryDeleted and entryUUID
attributes and add a entryDelete (false), entryUUID and an entryCSN
attribute. Then execute() is called eventually for the
AddEntryOperation. Here you're checking if the entry is updateable by
calling EntryUtil.isEntryUpdatable. Here's the code for it:
http://svn.apache.org/repos/asf/directory/trunks/apacheds/mitosis/src/main/java/org/apache/directory/mitosis/operation/support/EntryUtil.java
Basically you lookup the entry with a call to the nexus. Then you
return true if the entry does not exist. Is this valid? Meaning if the
entry does not exist in the store how can it be updatable?
You also check if there is no CSN attribute. If there is no attribute
then you return true that the entry is updatable. What does this mean?
It would help if you clarify the meaning of these methods in EntryUtils
with a bit more doco. Also it would be nice if you elaborate more on
how mitosis does not delete entries, and why it does this as well as
when and how it cleans up later. This is all critical stuff for the
architecture that was omitted.
Answer:
(20:52:07) trustin: first, if there's no entryCSN attribute, it means that
(20:52:24) trustin: an entry is added before Mitosis is in action.
(20:52:55) akarasulu: ahh ok
(20:53:02) trustin: so it should be updatable and needs to get the CSN
assigned by modification
(20:53:09) akarasulu: ok
(20:53:23) trustin: and then it compares the CSN
(20:53:37) akarasulu: why do you return true if entry == null ?
(20:54:28) trustin: Semantics of update includes both add and modify..
so it's updatable if there's no such entry.
(20:54:42) akarasulu: ahhh ok so add is considered an update
(20:54:58) trustin: yeah update is all operations that affects the DIT.
3c). The code goes past these checks for this scenario since the entry
does exist and is marked entryDeleted: true and it does have a CSN
attribute. Finally you compare whether this old CSN is older than the
new CSN. If so then you return true.
What is the meaning of this? Is it just an undefined situation where
you should not be changing an entry with CSN that is older than the CSN
of the original entry? Why then return true or false for
isEntryUpdatable? Should you not just throw an exception if this
is just an undefined situtation?
Answer: We need to decide between two CSN's, the existing one or the new
operation at hand.
3d). Next you create glue entries. This is not very clear to me. It
was not in your architecture document. I think it is very important to
define exactly what this is for to understand mitosis.
Inside EntryUtils.createGlueEntries() you are using getSuffix() to walk
a DN. How do you intend to walk the DN? From the top down like so for
example?
ou=system
ou=A,ou=system
ou=B,ou=A,ou=system
For each one you are trying to create a glue entry I think. There was
yet no need for a glue entry but we have to understand what it is and
make sure they don't violate schema or we'll bomb out. Can you
elaborate a little on glue entries?
Answer: will put it into another iteration of the mitosis architecture doco.
3e). In an effort to modify the entry you are removing all attributes
that the two old and new entries do not have in common. However you are
removing these from the cloned version of the old entry. Then you never
use this old entry that has attributes removed from it.
Is this another bug?
Answer: Yes
4) If I restart a server after shuting it down I get the following error
and the server dies:
Caused by: java.net.BindException: Address already in use
We found a way to stop this from happening with the LDAP port used by
the server. Can
we make the mitosis replication service port also behave similarly so
this does not
happen?
Answer: config.setReuseAddress( true );
5). Looks like you have not finished implementing the move operation as
is indicated
here in OperationFactory line 188 in newMove():
if ( !deleteOldRn )
{
throw new OperationNotSupportedException( "deleteOldRn must
be true." );
}
Is it that you did not implement this or that it was not possible
because of some issue?
6). Can you please add some internal documentation to the mitosis code?
Answer: Yep
7). Can you update the archtecture documentation to include information
about glue entries, not deleting and cleaning up stuff?
Answer: Yep
8). Could you also elaborate more on the log and how it is used?
Answer: Yep
9). Can we detect when we are out of sync and email administrators?
(22:46:27) akarasulu: yes right ?
(22:46:33) trustin: yes.
===========================================
Notes:
(1) We need to log when we have dropped operations.
(2) We need to email administrators when we have dropped ops and/or when
we are out
of sync.
Decided it is best to recursively delete entries marked as deleted when
re-adding the
entry from scratch.
Alex
