Le 20/09/2018 à 14:18, Howard Chu a écrit : > Emmanuel Lécharny wrote: >> Hi ! >> Now, there is more work to do : >> o rollback hasn't been tested yet >> o we need to forbid concurrent writes while a transaction is being >> executed - which means that a transaction is seen as one single giant >> write - with the consequence that a rogue client starting a transaction >> but never finishing it will potentially lock the server. >> o a mechanism to protect the server against such rogues client has to be >> implemented (could be a limitation in the number of update per >> transaction and/or a time limit for a transaction). > > In OpenLDAP we simply queue the incoming ops until the Commit is received, > then execute them all at once. And yes, there's a limit on the number of > ops allowed in a txn.
In ApacheDS, that would not save a lot of CPU/time, as the work is done
when updates are injected in the in-memory pages and flushed on disk.
Stacking them and injecting them in one shot will not save anything.
Now, it may allow some other updates to be executed before the
endTransaction is committed, but the risk is that the bigger transaction
is rollbacked due to some errors caused by the updates received during
the txn. This is a choice we could refine later, as the wall are not yet
painted ;-)
>
>> Note that while a
>> transaction is being conducted, nothing is flushed on disk.
>> o more tests are to be conducted (do we see entries being updated during
>> a transaction if we do a search aside the transaction ?)
>
> The RFC doesn't require searches to honor transactions. The spec really
> just allows you to perform multiple write ops as a single atomic operation.
With JDBM, I don't know how well searches would be executed while a
transaction is ongoing. With Mavibot, it's totally safe, as with MDB, as
a search will simply use teh latest committed version, and teh
transaction creates a new revision. But you already know that, just
mentionning for clarity.
>
>> o controls are not returned back atm, as they should
>>
>> Otherwise, I have met some issues in other parts of the server whild
>> implementing transactions :
>>
>> o for most of the operations, controls aren't sent back to the client
>> (except for the Bind Operation)
>> o We should write a <Operation>Decorator instead of a request/response
>> implementation, to avoid a call to a method that is a big switch doing
>> the exact same thing. This is a bit cleaner.
>>
>> Also note that if the partition does not support transactions, then the
>> transaction requests are simply ignored.
>>
>> I will commit the code when the previous listed points have been
>> fixed/implemented.
>>
>> I do think we should leverage this extension when injecting a huge
>> number of entries into the server, like when using Studio to import a
>> LDIF file. I would suggest to gather updates in block of 100 to get a
>> good boost (x3) without blocking the writes for other clients for too
>> long (300ms).
>
> Yes, in OpenLDAP 2.5 we can use txns to batch replication updates, also for
> performance reasons.
I'm not absolutely sure it's a good idea ('not sure', aka 'I have not
yet a solid opinion') : we generally don't know how many updates we are
going to have to replicate, so arbitrarily decide that updates are going
to be sent by batches of N updates is likely to be problematic (but I
may be wrong). But something similar to the Nagle algorithm applied to
replication could certainly work, that's for sure. At worse, in this
case, you may differ replication by a few seconds to take the chance to
speedup the average replication. This is worthfull when you know that
some periodic updates are scheduled (like daily updates). This is a
frequent use case in many companies.
Ideally speaking, this should be driven by a control...
Thanks for yout insights Howard, interesting, as always !
--
Emmanuel Lecharny
Symas.com
directory.apache.org
pEpkey.asc
Description: application/pgp-keys
