[
https://issues.apache.org/jira/browse/HDDS-16874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16874:
----------------------------------
Labels: pull-request-available (was: )
> FilePerChunk overwrite check misses chunks after the first in a block
> ---------------------------------------------------------------------
>
> Key: HDDS-16874
> URL: https://issues.apache.org/jira/browse/HDDS-16874
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Chi-Hsuan Huang
> Assignee: Andy Huang
> Priority: Minor
> Labels: pull-request-available
>
> {{FilePerChunkStrategy.writeChunk}} decides whether a write is an overwrite
> with {{info.getOffset() < chunkFile.length()}}. Since HDDS-2717 the chunk
> offset is relative to the block, but in the FILE_PER_CHUNK layout each chunk
> has its own file. For any chunk after the first, the offset is at least one
> chunk length, so a rewrite of an existing chunk is not detected.
> I found two effects with unit tests on {{TestFilePerChunkStrategy}} (second
> chunk, offset = len):
> # A COMBINED retry counts the chunk twice in container {{bytesUsed}}
> (expected 20, actual 40).
> # A repeated COMMIT_DATA, when the chunk file exists and the tmp file is
> gone, fails with {{NoSuchFileException}}.
> The first chunk (offset 0) behaves correctly in both cases.
> On datanode restart, {{ContainerStateMachine.applyTransaction}} replays
> WriteChunk entries after the last snapshot as COMMIT_DATA, and
> {{HddsDispatcher}} does not skip already applied entries. So effect 2 could
> fail the apply and trigger a pipeline close. I have not reproduced this end
> to end.
> *Expected behavior*
> A rewrite of any existing chunk in a FILE_PER_CHUNK container should be
> treated as an overwrite.
> *Notes*
> FILE_PER_CHUNK is deprecated for new containers (HDDS-11753), so this only
> affects existing containers in that layout. Found while reviewing HDDS-16844,
> which keeps the current behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
