[
https://issues.apache.org/jira/browse/IGNITE-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875168#comment-17875168
]
Aleksandr Polovtsev commented on IGNITE-19395:
----------------------------------------------
New benchmark results (switched to Single Shot Mode and added benchmarks for
separate addWrite and commitWrite operations):
{noformat}
Benchmark Mode Cnt Score Error
Units
CommitManyWritesBenchmark.addAndCommitManyWrites ss 11.079
s/op
CommitManyWritesBenchmark.addManyWrites ss 3.468
s/op
CommitManyWritesBenchmark.commitManyWrites ss 10.165
s/op
Benchmark Mode Cnt Score Error
Units
CommitManyWritesBenchmark.addAndCommitManyWrites ss 5.427
s/op
CommitManyWritesBenchmark.addManyWrites ss 2.748
s/op
CommitManyWritesBenchmark.commitManyWrites ss 1.571
s/op
{noformat}
> Reduce write amplification for RocksDB partition storage
> --------------------------------------------------------
>
> Key: IGNITE-19395
> URL: https://issues.apache.org/jira/browse/IGNITE-19395
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Aleksandr Polovtsev
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Currently, the "commit" operation in rocksdb storage looks like this:
> {code:java}
> val data = db.read(writeIntentKey);
> db.remove(writeIntentKey);
> db.write(committedKey, data);{code}
> This is wasteful, we end up writing everything twice. There's another
> solution, we may add a level of indirection to the data:
> {code:java}
> // RowId index.
> [ TableId?? | PartId | RowId | Timestamp ] -> [ DataId ]
> [ TableId?? | PartId | RowId ] -> [ DataId | TxId | CommitTableId |
> CommitPartId ]
> // Data.
> [ DataId ] -> [ Payload ]{code}
> {{DataId}} must be unique. I don't like the idea of auto-incrementing key (we
> should always persist latest value), there must be another way.
> The main idea is that DataId doesn't change while committing the data,
> meaning that it can be generated using RowId and TxId.
> For example, {{RowId ++ beginTimestamp(TxId)}} seems like a unique value
> (with mandatory partition ID prefix and probably a table ID prefix)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
