[
https://issues.apache.org/jira/browse/DERBY-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564353#action_12564353
]
V.Narayanan commented on DERBY-3359:
------------------------------------
Implementation Nuances
----------------------
Here, I will be explaining the changes that will be required to the appropriate
classes for
implementating the Dynamic log shipping functionality as detailed above.
org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer
----------------------------------------------------------------------
This class will contain the functionality for
1) calculating the Fill Information (*FI*)
2) For notifying the log shipper each time a *log buffer element* becomes full.
1) calculating the Fill Information (*FI*)
------------------------------------------
FI will be an integer with a value ranging between 0-100
FI = (dirtyBuffers.size()/DEFAULT_NUMBER_LOG_BUFFERS)*100
where,
dirtyBuffers is a LinkedList containing the log buffers elements with unsent
log.
DEFAULT_NUMBER_LOG_BUFFERS The total number of log buffers elements (filled and
unfilled).
A method,
public int getFillInformation();
will be added that will return the value of FI as calculated above.
2) For notifying the log shipper each time a log buffer element becomes full.
----------------------------------------------------------------------------
Each time a log shipper element becomes full we call *switchDirtyBuffer()* in
order to
Append the currentDirtyBuffer to dirtyBuffers, and make a fresh buffer element
from
freeBuffers.
We will add logic in the switchDirtyBuffer() method for notifying the Log
Shipper.
For more information on the LogShipper please refer to,
1) http://issues.apache.org/jira/browse/DERBY-2926
2) http://issues.apache.org/jira/browse/DERBY-3051
org.apache.derby.impl.services.replication.master.LogShipper (interface)
------------------------------------------------------------------------
This interface will contain the definition of the method that will be called by
the
log buffer.
Will add a method workToDo that will be used by the Log Buffer to inform the
log shipper
whenever a Log Buffer Element becomes full.
org.apache.derby.impl.services.replication.master.AsynchronousLogShipper
------------------------------------------------------------------------
This class will contain the functionality for managing load in the Log buffer
(by shipping log records)
,by using the FI information, during each buffer element full notification that
is received.
Two constants
-------------
MID = 50ms
MAX = 550ms (MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID)
attribute nextShippingTime
--------------------------
will store the time at which the next log ship will happen and will be
calculated as System.currentTimeMillis() +
shippingInterval in the run method.
methods
-------
constructor
-----------
shippingInterval will be initialized to MID in the beginning.
nextShippingTime will be initialized in the constructor initially to
System.currentTimeMills() + shippingInterval (i.e. MID).
public void workToDo();
-----------------------
Will be added that will be called by the log buffer each time a buffer element
becomes full.
The following steps will be used in this method
a) Get FI from log buffer
b) If FI >= 80
b.1) call shipALogChunk()
b.2) repeat b) until FI < 80
b.3) notify the log shipper thread.
c) If FI > 10 && FI < 80
c.1) calculate the time elapsed since last log ship.
(For this before calling wait we need to store the
time at which next transmit will happen.)
c.2) If the time remaining for the next ship is less than
MID then do not do anything.
c.3) If the time elapsed since last ship is greater than
MID notify the log shipper thread. (here we could have
simply not shipped and waited for timeout to happen, but
this would be a risk and a better option would be to ship)
will check if the forceFlush method is required at all since all of the work
done by the forceFlush
method can now be done by the workToDo method (see changes to MasterController).
public void calculateWTfromFI() (WT - wait time)
-------------------------------
will be used to calculate the wait time using the FI from the buffer.
shippingInterval will be assigned to the return value from this method before
calling
wait(shippingInterval) in the run method. *If this method returns -1 however we
should
not call wait but just continue with the shipping.*
The following steps will be used in this method
a) FI >= 80 return -1 (*we should not call wait if the return value is -1*)
b) FI > 10 and FI < 80 return MID
c) FI <= 10 return MAX
run()
-----
This method will be modified to
1) shippingInterval will be assigned the return value
of calculateWTfromFI().
2) calculate the next shipping time which will be
System.currentTimeMillis() + shippingInterval.
For more information on the log shipper please refer to
http://issues.apache.org/jira/browse/DERBY-3064
org.apache.derby.impl.services.replication.master.MasterController
------------------------------------------------------------------
When a LogBufferFullException is thrown we will call workToDo instead of
forceFlush.
For more information on the MasterController please refer to
http://issues.apache.org/jira/browse/DERBY-2977
> The log shipper needs to be modified to vary the shipping intervals
> dynamically (based on load)
> -----------------------------------------------------------------------------------------------
>
> Key: DERBY-3359
> URL: https://issues.apache.org/jira/browse/DERBY-3359
> Project: Derby
> Issue Type: Sub-task
> Reporter: V.Narayanan
> Assignee: V.Narayanan
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
