[
https://issues.apache.org/jira/browse/FLUME-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852091#comment-13852091
]
Hari Shreedharan commented on FLUME-2265:
-----------------------------------------
On idle or roll time based close, the bucket writer is closed and the bucket
writer is removed from sfWriters map.
The patch introduces a new exception, which is thrown when an append tries to
write to a bucket writer that is closed based on time (roll or idle).
The sink then catches this exception, creates a new bucket writer and writes to
that one.
> Closed bucket writers should be removed from sfwriters map
> ----------------------------------------------------------
>
> Key: FLUME-2265
> URL: https://issues.apache.org/jira/browse/FLUME-2265
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Attachments: FLUME-2265.patch
>
>
> If we don't remove the bucket writer from the map, the reference is kept in
> the map and not removed till the maxOpenFiles limit is hit, even though the
> bucket writer is closed. This leads to HDFS buffers sticking around for a
> long time after the bucket writer is closed.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
