Daryn Sharp created HDFS-12142:
----------------------------------
Summary: Files may be closed before streamer is done
Key: HDFS-12142
URL: https://issues.apache.org/jira/browse/HDFS-12142
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 2.8.0
Reporter: Daryn Sharp
We're encountering multiple cases of clients calling updateBlockForPipeline on
completed blocks. Initial analysis is the client closes a file, completeFile
succeeds, then it immediately attempts recovery. The exception is swallowed on
the client, only logged on the NN by checkUCBlock.
The problem "appears" to be benign (no data loss) but it's unproven if the
issue always occurs for successfully closed files. There appears to be very
poor coordination between the dfs output stream's threads which leads to races
that confuse the streamer thread – which probably should have been joined
before returning from close.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
