Hello Team,
My development activity is stuck due to below error in the Apache agent.
org.apache.flume.ChannelException: Unable to put batch on required channel: 
FileChannel file-channel-1 

I have the following configurationOne Source - KafkaThree channels - File 
channelFour Sinks - HDFS Sinks
Attached is the flume agent config file and below is the error message
Things tried1) Lowering or increasing the batch size for Kafka source and Hive 
HDFS sink2) Lowering or increasing the TransactionCapacity and Capacity for 
File channel3) With some help from the net, tried keeping File channel 
TransactionCapacity to be 5x of Kafka and HDFS Batch size but in vain.

Please help advise

Error message in detail : 
16/10/17 18:06:08 ERROR kafka.KafkaSource: KafkaSource EXCEPTION, {}
org.apache.flume.ChannelException: Unable to put batch on required channel: 
FileChannel file-channel-1 { dataDirs: 
[/home/dheeraj.rokade/flume/.flume/file-channel1/data] }
        at 
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
        at 
org.apache.flume.source.kafka.KafkaSource.process(KafkaSource.java:123)
        at 
org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at 
org.apache.flume.channel.file.proto.ProtosFactory$FlumeEventHeader$Builder.setValue(ProtosFactory.java:7415)
        at org.apache.flume.channel.file.Put.writeProtos(Put.java:85)
        at 
org.apache.flume.channel.file.TransactionEventRecord.toByteBuffer(TransactionEventRecord.java:174)
        at org.apache.flume.channel.file.Log.put(Log.java:642)
        at 
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doPut(FileChannel.java:468)
        at 
org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
        at 
org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80)
        at 
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
        ... 3 more
16/10/17 18:06:13 ERROR kafka.KafkaSource: KafkaSource EXCEPTION, {}
org.apache.flume.ChannelException: Unable to put batch on required channel: 
FileChannel file-channel-1 { dataDirs: 
[/home/dheeraj.rokade/flume/.flume/file-channel1/data] }
        at 
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)
        at 
org.apache.flume.source.kafka.KafkaSource.process(KafkaSource.java:123)
        at 
org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at 
org.apache.flume.channel.file.proto.ProtosFactory$FlumeEventHeader$Builder.setValue(ProtosFactory.java:7415)
        at org.apache.flume.channel.file.Put.writeProtos(Put.java:85)
        at 
org.apache.flume.channel.file.TransactionEventRecord.toByteBuffer(TransactionEventRecord.java:174)
        at org.apache.flume.channel.file.Log.put(Log.java:642)
        at 
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doPut(FileChannel.java:468)
        at 
org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
        at 
org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80)
        at 
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
        ... 3 more

Reply via email to