[
https://issues.apache.org/jira/browse/HADOOP-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518779#comment-16518779
]
Harish Jaiprakash commented on HADOOP-15074:
--------------------------------------------
[~arpitagarwal], SequenceFileWriter depends upon FSDataOutputStream object
returned by FileSystem.create/append, which does not expose the
hsync(EnumSet<SyncFlag> syncFlags). The DFSOutputStream gets wrapped into a
FSDataOutputStream so it's not possible to fix in SequenceFile.
This bug makes it hard to implement producer / consumer using sequence files.
We are a bit stuck on this. When would length get persisted, if hsync is never
called with UPDATE_LENGTH? Is there like a periodic update of length or udpate
when a block is full and written or only when close is called?
> SequenceFile#Writer flush does not update the length of the written file.
> -------------------------------------------------------------------------
>
> Key: HADOOP-15074
> URL: https://issues.apache.org/jira/browse/HADOOP-15074
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Mukul Kumar Singh
> Assignee: Shashikant Banerjee
> Priority: Major
>
> SequenceFile#Writer flush does not update the length of the file. This
> happens because as part of the flush, {{UPDATE_LENGTH}} flag is not passed to
> the DFSOutputStream#hsync.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
