[
https://issues.apache.org/jira/browse/HDDS-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-15791:
----------------------------------
Labels: pull-request-available (was: )
> EC reconstructed RECOVERING container while still idle can be deleted by SCM
> and recreated as new partial OPEN container
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-15791
> URL: https://issues.apache.org/jira/browse/HDDS-15791
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Aswin Shakil
> Assignee: Aswin Shakil
> Priority: Major
> Labels: pull-request-available
>
> This issue has been identified by [~deveshsingh] and [~umamahesh]
> During EC offline reconstruction, the coordinator creates a *RECOVERING*
> replica on a spare DataNode and writes blocks over time. If reconstruction
> stalls (e.g. slow block-group processing) longer than
> `ozone.recovering.container.timeout` (~20m, measured from creation, not last
> write), the DN scrubber marks the replica *UNHEALTHY* and SCM force-deletes
> it.
> The coordinator may still be running. The next `WriteChunk` or `PutBlock`
> finds no container; `HddsDispatcher` auto-creates a new OPEN container
> instead of failing. The coordinator continues and closes that replica shortly
> after, leaving a partial container (fewer blocks/chunks than the reference)
> that SCM can accept as *CLOSED* — silent data loss.
> *Logic flow*
> {code:java}
> Coordinator Phase 1
> createRecoveringContainer → RECOVERING on spare DN
> (slow / sleep — no writes for extended period)
> StaleRecoveringContainerScrubbingService
> creationTime + timeout exceeded → mark UNHEALTHY
> SCM / Replication Manager
> force-delete UNHEALTHY replica → DELETED on spare DN
> Coordinator resumes writes
> container == null
> HddsDispatcher auto-create → OPEN (not RECOVERING)
> Coordinator completes
> closeContainer → CLOSING / CLOSED
> Result: CLOSED replica with partial blocks/chunks at EC index → data loss
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
