[
https://issues.apache.org/jira/browse/HDDS-15886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127301#comment-18127301
]
Chung-En Lee commented on HDDS-15886:
-------------------------------------
Hi [~szetszwo] .Sorry for the confusion earlier. My initial description was
inaccurate, and I've updated it. Please take a look at the updated description
and see if it makes sense.
> Client unable to read newly synced data due to outdated block length
> --------------------------------------------------------------------
>
> Key: HDDS-15886
> URL: https://issues.apache.org/jira/browse/HDDS-15886
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Chung-En Lee
> Assignee: Chung-En Lee
> Priority: Critical
>
> With `ozone.client.stream.read.block` enabled, a reader that opens a file
> that is still being written can't read all the data the writer has hsync'ed.
> It hits EOF at the block length OM last recorded. With the setting disabled,
> the same read succeeds.
> Why OM's block length is out of date
> To keep OM load down, BlockOutputStreamEntryPool#hsyncKey only sends the new
> length to OM when the last block has changed. Later hsyncs that stay within
> the same block write the data to the datanodes but don't update OM. So for
> the last block of an hsync'ed file, OM's length can be shorter than what is
> actually on the datanode.
> The read path expects this. KeyInputStream marks the last block of an
> hsync'ed file as under construction (setUnderConstruction(true)).
> BlockInputStream then gets the real length from the datanode.
> StreamBlockInputStream doesn't do this. It takes blockInfo.getLength() from
> OM through BlockInputStreamFactoryImpl, stores it in a final field, and never
> checks isUnderConstruction().
> This issue was found by `TestContractCreate#testSyncable` . It fails with
> streamReadBlock enabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
