[
https://issues.apache.org/jira/browse/HADOOP-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521184#comment-16521184
]
Steve Loughran commented on HADOOP-15074:
-----------------------------------------
My mistake: the builder API is for creating files, its for opening on the todo
list.
If HDFS added an option like "hdfs:update-length-on-hflush", it could be picked
up and used by sequence file.
I'm a bit reluctant to go this way though, as there's a risk it gets used
widely, and that may be bad for NN performance.
I've seen code in the timeline service which does this by remembering the last
length of a file, then opening it, attempting to seek past, and, if it can do
that, read in the new stuff. Ugly, but works today
> SequenceFile#Writer flush does not update the length of the written file.
> -------------------------------------------------------------------------
>
> Key: HADOOP-15074
> URL: https://issues.apache.org/jira/browse/HADOOP-15074
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Mukul Kumar Singh
> Assignee: Shashikant Banerjee
> Priority: Major
>
> SequenceFile#Writer flush does not update the length of the file. This
> happens because as part of the flush, {{UPDATE_LENGTH}} flag is not passed to
> the DFSOutputStream#hsync.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
