Hi, I think I found the problem. In SparkFlumeEvent the readExternal method use in.read(bodyBuff) which read the first 1020 bytes, but no more. The code should make sure to read everything. The following change will fix the problem: in.read(bodyBuff) to: in.readFully(bodyBuff)
I attached a patch. If somebody can point me in the right direction, I'll gladly do the required legwork to submit the patch upstream. Have a nice day. /D -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-using-Flume-body-size-limitation-tp6127p6314.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
