[
https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173062#comment-14173062
]
Johny Rufus edited comment on FLUME-2197 at 10/15/14 10:20 PM:
---------------------------------------------------------------
Given that Xmx is set to 6G, byteCapacity=0 is not correct, right ?
- I think thats true, if byte capacity is set to 0, then it defaults to
200GB, and if that much memory is not available, then it may lead to OOM
(assuming that the max. no of events (capacity) has not yet been reached).
Shall we set byteCapacity to 50% of 6G or smaller ?
- setting this to 50% of 6G implies, you are only allocating 3G to be used by
the total set of events [header+body] in the memory channel
- if this is set, then this should be set to how much space you need for both
[headers+body] for the total number of events expected to be in the channel at
any point of time
- if this is not set, then it defaults to the heap size allocated to the jvm
- Also the byteCapacityBufferPercentage should be set close to the ratio of
header-size/[header-size+body-size]. Setting this to 50% means that every event
is expected to have equal header and body size
Hope this helps
was (Author: jrufus):
Given that Xmx is set to 6G, byteCapacity=0 is not correct, right ?
- I think thats true, if byte capacity is set to 0, then it defaults to
200GB, and if that much memory is not available, then it may lead to OOM
- (assuming that the max. no of events (capacity) has not yet been reached).
Shall we set byteCapacity to 50% of 6G or smaller ?
- setting this to 50% of 6G implies, you are only allocating 3G to be used by
the total set of events [header+body] in the memory channel
- if this is set, then this should be set to how much space you need for both
[headers+body] for the total number of events expected to be in the channel at
any point of time
- if this is not set, then it defaults to the heap size allocated to the jvm
- Also the byteCapacityBufferPercentage should be set close to the ratio of
header-size/[header-size+body-size]. Setting this to 50% means that every event
is expected to have equal header and body size
Hope this helps
> Memory Channel has GC issues
> ----------------------------
>
> Key: FLUME-2197
> URL: https://issues.apache.org/jira/browse/FLUME-2197
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Roshan Naik
> Attachments: HA_result.jpg, mem ch - mem alloc.png, spill ch - mem
> alloc.png
>
>
> Due to the fact that we use a LinkedBlockingDeque as the backing queue for
> the MemoryChannel, we end up hitting GC issues more often than we should.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
