[
https://issues.apache.org/jira/browse/DERBY-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512423
]
Jørgen Løland commented on DERBY-2926:
--------------------------------------
Hi Mike
There is a better description of what we intend to create in DERBY-2872. Jira
does not allow multiple layers of subtasks, so I had to use the "is part of"
link.
Basically, the goal is to provide asynchronous replication, i.e. replication
where log shipment to the slave is completely decoupled from transactions on
the master. This is even looser synchronization than your third alternative,
which I believe is also known as 1-safe replication.
The asynchronous replication strategy may result in some lost transactions when
the slave performs fail-over. The amount of lost transactions is, of course,
closely related to how often log shipment is performed. As you mention, there
is a trade off between how tight the master/slave synchronization is and the
incurred performance degradation. Since log shipping in asynchronous
replication is completely decoupled from the transactions, this strategy should
have less performance impact than the alternatives.
Although the plan is to add asynchronous replication now, replication with
tighter synchronization should be kept in mind. If possible within reasonable
increased work, the architecture should easily extend to 1-safe or 2-safe (your
second alternative) replication later.
When it comes to the core replication functionality, there are two things the
replication master must know about: 1) log writes and 2) log flush. The log
records must (sooner or later) be sent to the slave, hence 1). What to do with
flush calls is up to the replication strategy. For the planned asynchronous
replication, flush calls can be ignored. For 1-safe and 2-safe replication,
flush calls require log shipment as you describe in alternatives 2 and 3.
The current plan for using the log buffer is to append log records to it
somewhere in LogToFile.appendLogRecord. This is the same method used to append
log records to logOut (output stream to the log file; class type
LogAccessFile). LogAccessFile is implemented with a number byte[] buffers
(LogAccessFileBuffer), which are ordered in a linked list. LogToFile is the
only entry point for log writes, and is therefore easily modifiable for our
purpose.
At the other end of the log buffer, a log shipping service will consume log
records. The service should, as you suggest, run as a separate thread. I think
DaemonFactory could be useful to create this thread, but that is just a guess.
In the current code, the flush-methods in LogToFile are the only entry points
for transactions to force a log flush (e.g. at commit). Hence, adding forced
log shipment to achieve 1 or 2 safe replication later can be easily put in
these methods. In the planned asynchronous strategy, log shipment may, e.g., be
based on a timeout; flush calls can be ignored altogether.
I hope this clarifies most of your concerns. Does this architecture fit into
your idea of "tie into the existing log writing code" since log records are
added to the buffer from the log factory? The single entry point for both log
writes and flushes makes this a good place for modifications (for both
asynchronous and the x-safe strategies) in my opinion, but there may be good
reasons for doing otherwise.
> Replication: Add a log buffer for log records that should be shipped to the
> slave
> ---------------------------------------------------------------------------------
>
> Key: DERBY-2926
> URL: https://issues.apache.org/jira/browse/DERBY-2926
> Project: Derby
> Issue Type: Sub-task
> Components: Store
> Affects Versions: 10.4.0.0
> Reporter: Jørgen Løland
> Assignee: Jørgen Løland
>
> When a Derby instance has the master role for a database, log records are
> shipped to the slave to keep it up to date. A buffer is needed because the
> log records should not be shipped one at a time. Also, writing the log
> records to a buffer instead of sending them immediately removes the network
> communication from the critical path for the transaction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
