[
https://issues.apache.org/jira/browse/HDDS-16730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chu Cheng Li updated HDDS-16730:
--------------------------------
Status: Patch Available (was: Open)
> [hsync] FinalizeBlock does not persist the last chunk merge
> -----------------------------------------------------------
>
> Key: HDDS-16730
> URL: https://issues.apache.org/jira/browse/HDDS-16730
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Chu Cheng Li
> Assignee: Chu Cheng Li
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> HDDS-11014 adds a merge step to {{BlockManagerImpl#finalizeBlock}}. For a
> block with an incremental chunk list, this step moves the last chunk from the
> {{last_chunk_info}} table to the {{block_data}} table.
> The code commits the batch before it adds the merge operations to the batch:
> {code:java}
> db.getStore().getFinalizeBlocksTable().putWithBatch(batch, blockKey, localID);
> db.getStore().getBatchHandler().commitBatchOperation(batch);
> mergeLastChunkForBlockFinalization(blockId, db, kvContainer, batch, localID);
> {code}
> No code commits the batch again. {{RDBBatchOperation#close}} discards the
> merge operations. Thus, the {{last_chunk_info}} row stays, and the
> {{block_data}} row does not get the last chunk.
> The datanode logs these warnings for each finalized block that has an
> incremental chunk list:
> {noformat}
> WARN db.RDBBatchOperation: discarding changes Batch-2: block_data
> batchSize=94 B, discarded: 0 (0 B), #put=1, #del=0
> WARN db.RDBBatchOperation: discarding changes Batch-2: last_chunk_info
> batchSize=1 B, discarded: 0 (0 B), #put=0, #del=1
> {noformat}
> All releases since 2.0.0 have this problem.
> h3. Effect
> * Only a client with {{ozone.client.incremental.chunk.list=true}} writes an
> incremental chunk list. The default is {{false}}.
> * Reads return the full block. {{getBlock}} merges the {{last_chunk_info}}
> row at read time.
> * Some code reads the {{block_data}} rows directly. This code does not see
> the last chunk. Thus, the container scanner does not verify the checksum of
> the last chunk, and the container merkle tree does not include it.
> * Container reconciliation can make the block unreadable on a datanode. When
> reconciliation repairs a chunk of the block, it writes the full chunk list to
> {{block_data}}. It does not delete the {{last_chunk_info}} row. Thus, the
> next read adds the last chunk a second time and fails with
> {{IllegalStateException: chunk offset does not match}}.
> h3. Fix
> Add the merge operations to the batch before the commit. Thus, RocksDB gets
> the finalize marker and the merge in one write.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
