[
https://issues.apache.org/jira/browse/HDDS-11366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-11366:
-------------------------------
Description:
Currently OM uses WriteBatch which only allows write on the batch. Therefore in
OM Response, we should not call DB#get directly since the latest write for a
particular key might be within the current batch that is yet to be flushed by
the OM double buffer.
We might want to replace WriteBatch with WriteBatchIndex which allows us to
read the latest writes from the write batch only (WriteBatchIndex#getFromBatch)
or also with the DB (WriteBatchIndex#getFromBatchAndDB).
Perhaps WriteBatchIndex remove the need of additional bookeeping in
RDBBatchOperation#opCache by setting overWriteKey flag. However, we might still
need to account for JNI call overhead of using WriteBatchIndex (multiple JNI
calls) vs RDBBatchOperationo#opCache (single JNI call).
Related resources
[https://www.cockroachlabs.com/blog/cockroachdb-on-rocksd/#blitzing-through-more-rocksdb-features#Indexed-Batches]
[https://github.com/facebook/rocksdb/wiki/Write-Batch-With-Index]
[https://rocksdb.org/blog/2015/02/27/write-batch-with-index.html]
was:
Currently OM uses WriteBatch which only allows write on the batch. Therefore in
OM Response, we should not call DB#get directly since the latest write for a
particular key might be within the current batch that is yet to be flushed by
the OM double buffer.
We might want to replace WriteBatch with WriteBatchIndex which allows us to
read the latest writes from the write batch only (WriteBatchIndex#getFromBatch)
or also with the DB (WriteBatchIndex#getFromBatchAndDB).
Perhaps WriteBatchIndex remove the need of additional bookeeping in
RDBBatchOperation#opCache by setting overWriteKey flag. However, we might still
need to account for JNI call overhead of using WriteBatchIndex vs
RDBBatchOperationo#opCache.
Related resources
[https://www.cockroachlabs.com/blog/cockroachdb-on-rocksd/#blitzing-through-more-rocksdb-features#Indexed-Batches]
[https://github.com/facebook/rocksdb/wiki/Write-Batch-With-Index]
https://rocksdb.org/blog/2015/02/27/write-batch-with-index.html
> Consider Replacing WriteBatch with WriteBatchWithIndex
> ------------------------------------------------------
>
> Key: HDDS-11366
> URL: https://issues.apache.org/jira/browse/HDDS-11366
> Project: Apache Ozone
> Issue Type: New Feature
> Components: OM
> Reporter: Ivan Andika
> Priority: Major
>
> Currently OM uses WriteBatch which only allows write on the batch. Therefore
> in OM Response, we should not call DB#get directly since the latest write for
> a particular key might be within the current batch that is yet to be flushed
> by the OM double buffer.
> We might want to replace WriteBatch with WriteBatchIndex which allows us to
> read the latest writes from the write batch only
> (WriteBatchIndex#getFromBatch) or also with the DB
> (WriteBatchIndex#getFromBatchAndDB).
> Perhaps WriteBatchIndex remove the need of additional bookeeping in
> RDBBatchOperation#opCache by setting overWriteKey flag. However, we might
> still need to account for JNI call overhead of using WriteBatchIndex
> (multiple JNI calls) vs RDBBatchOperationo#opCache (single JNI call).
> Related resources
> [https://www.cockroachlabs.com/blog/cockroachdb-on-rocksd/#blitzing-through-more-rocksdb-features#Indexed-Batches]
> [https://github.com/facebook/rocksdb/wiki/Write-Batch-With-Index]
> [https://rocksdb.org/blog/2015/02/27/write-batch-with-index.html]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
