[
https://issues.apache.org/jira/browse/FLUME-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117824#comment-14117824
]
Brock Noland commented on FLUME-2450:
-------------------------------------
bq. I had 45gigs of data parked in the file channel , with the patch flume
took about 25 mins to figure itself out
Could you share how many events were in the queue? Also, was that for a full
replay? Are you using backup checkpoints?
bq. The frustration right now for us is that our flume nodes are basically
'down' until this recovery completes.
Are your nodes performing a full recovery often? Are you using backup
checkpoints? Unless the checkpoint and backpoint checkpoints are gone, a replay
should be quite fast.
bq. Make a new config option to run the version that requires extending the
amount of JVM memory
This actually would not improve recovery much.
> Improve replay index insertion speed.
> -------------------------------------
>
> Key: FLUME-2450
> URL: https://issues.apache.org/jira/browse/FLUME-2450
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2450.patch
>
>
> Insertion into the replay index can take long sometimes because we use a file
> based index and tree set. We should switch this out for a memory mapped db
> and a hash set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
