gautham varada created FLUME-2465:
-------------------------------------
Summary: Protections against OOM
Key: FLUME-2465
URL: https://issues.apache.org/jira/browse/FLUME-2465
Project: Flume
Issue Type: New Feature
Components: Sinks+Sources
Affects Versions: v1.5.0
Environment: centos 6.3, open jdk 7
Reporter: gautham varada
Priority: Minor
I wanted to propose some kind of a mechanism in the Sinkrunner to prevent
processing large messages to avoid OOM. Invariably we all end up configuring
multiple sinks to read messages in batches to drain the channel faster, this
can potentially lead to OOM depending on the size of the message.
Can we implement some kind of a cyclic barrier (not sure if it needs to be in
the sink runner) for the sink threads to wait once we hit a preconfigured
threshold?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
