Please find the configuration file attached
ThanksDheeraj
Sent from Yahoo Mail on Android
On Mon, 17 Oct, 2016 at 18:23, Attila Simon<s...@cloudera.com> wrote: Hi,
It might've been my mobile client for gmail but couldn't find the flume
conf attached. Could you please double check?
Cheers,
Attila
On Monday, 17 October 2016, Dheeraj Rokade <dheerajrok...@yahoo.com.invalid>
wrote:
> Hello Team,
>
> My development activity is stuck due to below error in the Apache agent.
>
> *org.apache.flume.ChannelException: Unable to put batch on required
> channel: FileChannel file-channel-1 *
>
> I have the following configuration
> One Source - *Kafka*
> Three channels - *File channel*
> Four Sinks - *HDFS Sinks*
>
> Attached is the flume agent config file and below is the error message
>
> Things tried
> 1) Lowering or increasing the batch size for Kafka source and Hive HDFS
> sink
> 2) Lowering or increasing the TransactionCapacity and Capacity for File
> channel
> 3) With some help from the net, tried keeping File channel
> TransactionCapacity to be 5x of Kafka and HDFS Batch size but in vain.
>
> Please help advise
>
> Error message in detail :
> 16/10/17 18:06:08 ERROR kafka.KafkaSource: KafkaSource EXCEPTION, {}
> org.apache.flume.ChannelException: Unable to put batch on required
> channel: FileChannel file-channel-1 { dataDirs:
> [/home/dheeraj.rokade/flume/.flume/file-channel1/data] }
> at org.apache.flume.channel.ChannelProcessor.processEventBatch(
> ChannelProcessor.java:200)
> at org.apache.flume.source.kafka.KafkaSource.process(
> KafkaSource.java:123)
> at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(
> PollableSourceRunner.java:139)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.flume.channel.file.proto.ProtosFactory$
> FlumeEventHeader$Builder.setValue(ProtosFactory.java:7415)
> at org.apache.flume.channel.file.Put.writeProtos(Put.java:85)
> at org.apache.flume.channel.file.TransactionEventRecord.
> toByteBuffer(TransactionEventRecord.java:174)
> at org.apache.flume.channel.file.Log.put(Log.java:642)
> at org.apache.flume.channel.file.FileChannel$
> FileBackedTransaction.doPut(FileChannel.java:468)
> at org.apache.flume.channel.BasicTransactionSemantics.put(
> BasicTransactionSemantics.java:93)
> at org.apache.flume.channel.BasicChannelSemantics.put(
> BasicChannelSemantics.java:80)
> at org.apache.flume.channel.ChannelProcessor.processEventBatch(
> ChannelProcessor.java:189)
> ... 3 more
> 16/10/17 18:06:13 ERROR kafka.KafkaSource: KafkaSource EXCEPTION, {}
> org.apache.flume.ChannelException: Unable to put batch on required
> channel: FileChannel file-channel-1 { dataDirs:
> [/home/dheeraj.rokade/flume/.flume/file-channel1/data] }
> at org.apache.flume.channel.ChannelProcessor.processEventBatch(
> ChannelProcessor.java:200)
> at org.apache.flume.source.kafka.KafkaSource.process(
> KafkaSource.java:123)
> at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(
> PollableSourceRunner.java:139)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.flume.channel.file.proto.ProtosFactory$
> FlumeEventHeader$Builder.setValue(ProtosFactory.java:7415)
> at org.apache.flume.channel.file.Put.writeProtos(Put.java:85)
> at org.apache.flume.channel.file.TransactionEventRecord.
> toByteBuffer(TransactionEventRecord.java:174)
> at org.apache.flume.channel.file.Log.put(Log.java:642)
> at org.apache.flume.channel.file.FileChannel$
> FileBackedTransaction.doPut(FileChannel.java:468)
> at org.apache.flume.channel.BasicTransactionSemantics.put(
> BasicTransactionSemantics.java:93)
> at org.apache.flume.channel.BasicChannelSemantics.put(
> BasicChannelSemantics.java:80)
> at org.apache.flume.channel.ChannelProcessor.processEventBatch(
> ChannelProcessor.java:189)
> ... 3 more
>
>
--
*Attila Simon*
Software Engineer
Email: s...@cloudera.com
[image: Cloudera Inc.]
