Max Schmidt created STORM-3698: ---------------------------------- Summary: AbstractHdfsBolt does not sync Writers that are purged Key: STORM-3698 URL: https://issues.apache.org/jira/browse/STORM-3698 Project: Apache Storm Issue Type: Bug Components: storm-hdfs Affects Versions: 1.2.2 Reporter: Max Schmidt
We just discovered when using a SequenceFileBolt (although it might happen with other implementations as well) that the writers it uses, held in the map AbstractHdfsBolt.writers are not closed/synced when they are removed from the map by the removeEldestEntry method. This leads to data loss. Can be reproduced by creating a SequenceFileBolt.withMaxOpenFiles(1) and writing to just two different files. One will have a size of zero, the other has the data in it. -- This message was sent by Atlassian Jira (v8.3.4#803005)