reddycharan opened a new issue #1343: Advisory (optimistic) write close 
   1. Please describe the feature you are requesting.
   With entrylog per ledger 
feature will be introduced. With this feature there will be dedicated entrylog 
for each ledger. With the <sub-task6> of issue-570 
EntryLogManagerForEntryLogPerLedger will be introduced. 
   Since there is going to be entrylog per ledger, with the current Bookie 
implementation there is no way to know when the entrylog is writeclosed and the 
entrylog for the active ledger can be rotated. So in <sub-task6> of issue-57, 
as a first step of handling this, expireAfterAccess (each entrylog should be 
automatically rotated from the current active entrylog list once a fixed 
duration has elapsed after the  last access of entrylog for addEntry) will be 
implemented and also the maximum number of entrylogs that can be active at a 
given time (this is for limiting number of entrylogs/filedescriptors open at a 
given time).
   The above mentioned approaches are preliminary ways of handling but it would 
be ideal to have explicit call to EntryLogger when the write for this ledger is 
done so that it can rotate the entrylog as soon as it is done writing. This 
will minimize the number of entrylogs/file descriptors that are open/active and 
also it will make progress in leastUnflushedLogId, so that 
GarbageCollectorThread can consider these entrylogs for garbage collection. So 
as part of next step, following can be done
   1. have explicit write close request. This write close request should be 
sent to the current ensemble when the write handle is closed. This should be 
just optimistic write close operation and callback of this operation should be 
just logger, saying if it is succeeded / partially succeeded / failed. This 
should be done asynchronously and the write handle close operation should not 
be blocked for this response.
   2. In the case of ledger recover open, readrequest (ReadEntryProcessorV3) 
with fence flag, can take care of calling appropriate methods in 
ledgerdescriptor / entry logger to rotate the entrylog for the ledger
   3.  in the case of auto-replication case, LedgerFragmentReplicator already 
uses bookieclient for addEntry (bkc.getBookieClient().addEntry), the same 
bookieClient instance can be used to call explicit writeClose method in  
   4. in the case of any write failure in bookie for an entry, then before 
sending error response to the client do entrylog rotation
   2. Indicate the importance of this issue to you (blocker, must-have, 
should-have, nice-to-have). Are you currently using any workarounds to address 
this issue?
   This is should-have feature for entrylogperledger 
   3. If there are some sub-tasks using -[] for each subtask and create a 
corresponding issue to map to the sub task:
   - [ ] [sub-task1]: Introduce new writeclose request (proto/message) and 
implement the bookie server side of the write close. Integrate 
ReadEntryProcessorV3 readrequest with fence flag to bookie side of writeClose 
   - [ ] [sub-task2]: client side of write handle close request, and 
integration of calling appropriate writeclose method of bookieclient in 
LedgerFragmentReplicator (replication scenario)
   - [ ] [sub-task3]: consider the optimizations of rotating entrylog in the 
case of any write failure in bookie for an entry before sending write error 
response to the client. (This needs more discussions and this is not well 
defined as of now)

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

Reply via email to