Max Schmidt created STORM-3698:
----------------------------------
Summary: AbstractHdfsBolt does not sync Writers that are purged
Key: STORM-3698
URL: https://issues.apache.org/jira/browse/STORM-3698
Project: Apache Storm
Issue Type: Bug
Components: storm-hdfs
Affects Versions: 1.2.2
Reporter: Max Schmidt
We just discovered when using a SequenceFileBolt (although it might happen with
other implementations as well) that the writers it uses, held in the map
AbstractHdfsBolt.writers are not closed/synced when they are removed from the
map by the removeEldestEntry method.
This leads to data loss.
Can be reproduced by creating a SequenceFileBolt.withMaxOpenFiles(1) and
writing to just two different files.
One will have a size of zero, the other has the data in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
