Shoeb created STORM-2745:
----------------------------
Summary: Hdfs Open Files problem
Key: STORM-2745
URL: https://issues.apache.org/jira/browse/STORM-2745
Project: Apache Storm
Issue Type: New Feature
Components: storm-hdfs
Affects Versions: 2.0.0, 1.x
Reporter: Shoeb
Fix For: 2.0.0, 1.x
Issue:
Problem exists when there are multiple HDFS writers in writersMap. Each writer
keeps an open hdfs handle to the file. Incase of Inactive writer(i.e. one which
is not consuming any data from long period), the files are not closed and
always remain in open state.
Ideally, these files should get closed and Hdfs writers removed from the
WritersMap.
Solution:
Implement a ClosingFilesPolicy that is based on Tick tuple intervals. At each
tick tuple all Writers are checked and closed if they exist for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
