Hi all,
I am currently working on a project to implement a file based store for
Message Broker. Currently the basic implementations are carried out as a
proof of concept using LevelDB. As there is no direct transaction support
for LevelDB, a transaction layer should be implemented separately. The
transaction implementation is mainly based on WriteBatch functionality
provided by LevelDB. That is mainly because WriteBatch can hold a sequence
of operations without actually apply it to the database. In LevelDB, the
database cannot be reverted when the operations are applied.
*LevelDB WriteBatch (Batch) :*
The WriteBatch holds a sequence of operations which are to be applied to
the database, and these operations within the batch are applied atomically
in order. This can be explained using following example.
WriteBatch batch = db.createWriteBatch();
try {
batch.delete(bytes("c"));
batch.put(bytes("a"), bytes("1"));
batch.put(bytes("b"), bytes("2"));
db.write(batch);
} finally {
batch.close();
}
In the code above, one “delete” operation and two “put” operations are
added to the batch. And all of those three operations are guaranteed to be
executed completely from the db.write(batch); command. There will be no
partial executions as WriteBatch class atomically applies the set of
updates [1].
*Implementation :*
The transaction consists of a batch (WriteBatch) which can accumulate “put”
and “delete” operations as explained above. When committing the
transaction, the batch is simply executed and applies updates to the
database. As WriteBatch guarantees the atomicity, there will be no partial
executions of accumulated operations. When rollbacked, the batch is simply
discarded and the changes will not be applied to the database, as it
remains in its previous state.
*Problem :*
The major problem in this implementation is that the transaction cannot see
its own changes and updates. As the batch only accumulates operations and
not applies them to the database, the changes will not visible. In MySQL
type transactions, the transaction is actually making changes to the
database, but they are not visible to other transactions until it is
committed. A similar approach is impossible in LevelDB, as the database
cannot be reverted after the changes are applied.
*Solution :*
In order to overcome this problem, a hashmap is implemented within the
transaction to keep track of intermediate changes and updates. During the
execution of a transaction, these stored values in the hashmap should be
used.
Hence, the current implementation of the transaction has two major
components.
1.
“Updates” - HashMap
2.
Write Batch
Following is a visualization of the execution of a transaction. [image:
Screenshot from 2017-09-06 09-29-50.png]
The initial state of the database is displayed below.
[image: Screenshot from 2017-09-06 09-32-41.png]
Assume it is required to add three more messages to “queueA”
transactionally. In this case a transaction should be initiated and the
operations should be inserted. Here, the message count for “queueA”,
“queueA.count” should be incremented by one for each message insertion. But
as explained above the value cannot be written to the database as it is not
revertible. Hence it should be applied to the “Updates” hashmap to keep
tracking and make intermediate updates. The initialization of the
transaction can represented as below.
[image: Screenshot from 2017-09-06 09-40-17.png]
For each inserted operation to the batch, the transaction stores the
incremented the message count (“queueA.count”) value in the “Updates”
hashmap.
When committed the changes will be applied to the database. [image:
Screenshot from 2017-09-06 09-43-16.png]
If rollbacked, the batch is discarded while the database remains in its
previous state.
[image: Screenshot from 2017-09-06 09-43-34.png]
An implementation of the transaction with examples is available in the
GitHub [2]. Feedbacks regarding current implementation will be much useful
in making improvements furthermore.
[1] https://github.com/google/leveldb/blob/master/doc/index.md#
atomic-updates
[2] https://github.com/Wishmitha/LevelDBTransactions
Best Regards,
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
