smengcl opened a new pull request, #11880: URL: https://github.com/apache/ozone/pull/11880
Generated-by: Claude Code (Opus 5.5) ## What changes were proposed in this pull request? With `ozone.client.hbase.enhancements.allowed=true` and `ozone.client.key.write.concurrency` other than 1 (HBase WAL sync threads run with -1), calls on one `KeyOutputStream` can overlap. When one call fails for good, `markStreamClosed` drops the buffered data and the block list (`BlockOutputStreamEntryPool.cleanup`) and sets `closed`. `write`, `flush` and `hsync` check `closed` only on entry, so a second call that was already in progress runs to its end and returns normally. For `hsync`, `BlockOutputStreamEntryPool.hsyncKey` then builds the block list from the cleared pool and sends OM an hsync with the stream length and no blocks. OM stores that in the key table (`OmKeyInfo.updateLocationInfoList` replaces the block list), so data acknowledged by this and by earlier `hsync` calls is no longer readable, and lease recovery closes the file with length 0. This PR: - Records in `KeyOutputStream` when `close` has committed the key. A stream that was closed without that has failed. - Rechecks for a failed stream with the write lock held: `write` before it touches the block pool, `flush` and `hsync` before they report success. `hsync` does it in the same critical section as `hsyncKey`, so no block list is sent to OM for a failed stream. - Always closes the stream with the write lock held, so the check is exact: `handleException` marks the stream closed before it releases the lock when the handling fails, and `handleFlushOrClose` takes the lock for `markStreamClosed`. Behavior: a call in progress when the stream fails now gets "Stream is closed", also when its own data had already reached the datanodes. Nothing changes without a failure. A call in progress when another thread's `close` commits the key behaves as before (for example `hflush` with the default configuration returns normally). Not in scope: an OM side check against an hsync with a data size and no blocks (needs its own discussion), and a related hang when the replacement block fails as well (HDDS-16979). ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16980 ## How was this patch tested? Added to `TestKeyOutputStream` (mock datanodes and mock OM, every write answered with `CLOSED_CONTAINER_IO`, the first caller held with latches until the second is blocked behind it, no fixed sleeps): - `testCallInProgressFailsWhenStreamFails`: 5 cases (`hsync` held in flush or in failure handling, without retries, `flush`, `write`). Also checks that an `hsync` before the failure succeeded and its data stays readable. All 5 fail without the fix ("Expecting code to raise a throwable") and pass with it. - `testFlushInProgressSucceedsWhenStreamIsClosed`: default configuration, `hflush` against `close`, passes with and without the fix. The reproduction attached to the JIRA passes (defect present) on current master and 5 of its 6 cases fail with this patch, the single caller control still passes. `TestKeyOutputStream` passes and checkstyle is clean for `ozone-client`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
