Marimuthu Ponnambalam created FLUME-2405:
--------------------------------------------
Summary: Flume stops working on arbitrary Frame Size more than 16
MB on ScribeSource
Key: FLUME-2405
URL: https://issues.apache.org/jira/browse/FLUME-2405
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.5.0, v1.4.0
Reporter: Marimuthu Ponnambalam
Priority: Critical
Fix For: v1.5.0.1, v1.6.0, v1.5.0, v1.4.0
Flume stops receiving data on scribe source if there is an arbitrary size of
data send from scribe, beyond 16MB.
TFrameTransport throws exception, due to frame size is more than 16 MB, which
is default size on TFrameTransport.
This can be resolved by adding one more property "thriftFrameSize", which will
allow custom frame size be configured, which will be used to create
TFrameTransport.
This happens while scribe lag is being cleared, scribe sends Frame size larger
than 16MB.
http://stackoverflow.com/questions/24150439/flume-scribesource-how-to-set-thrift-frame-size
2014-06-10 19:40:40,405 WARN org.apache.thrift.server.THsHaServer: Exception
while invoking!
org.apache.thrift.transport.TTransportException: Frame size (23757404) larger
than max length (16384000)!
at
org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:137)
at
org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
at org.apache.thrift.server.Invocation.run(Invocation.java:18)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
