Sverre Bakke created HDFS-7941:
----------------------------------
Summary: SequenceFile.Writer.hsync() not working?
Key: HDFS-7941
URL: https://issues.apache.org/jira/browse/HDFS-7941
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.6.0
Environment: HDP 2.2 running on Redhat
Reporter: Sverre Bakke
Priority: Minor
When using SequenceFile.Writer and appending+syncing to file repeatedly, the
sync does not appear to work other than:
- once after writing headers
- when closing.
Imagine the following test case:
http://pastebin.com/Y9xysCRX
This code would append a new record every second and then immediately sync it.
One would also imagine that the file would grow for every append, however, this
does not happen.
After watching the behavior I have noticed that it only syncs the headers at
the very beginning (providing a file of 164 bytes) and then never again until
its closed. This despite it is asked to hsync() after every append.
Looking into the debug logs, this also claims the same behavior (executed the
provided code example and grepped for "sync"):
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
2015-03-17 15:55:14 DEBUG ProtobufRpcEngine:253 - Call: fsync took 11ms
This was the only time the code ran fsync throughout the entire execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
