Hi All,
We are developing a tool which embeds Flume. I have some questions about
configuring Flume.
The Avro sink emits error level log messages every few seconds when it loses
connection to the next hop Avro source.
Using a file channel before the Avro sink guarantees the transactional behavior
and 100% delivery. So the messages will be delivered when the next hop Avro
source comes online.
Is there a way to configure Flume to only log a warning level log message or
only log this error once?
The log message:
2012-08-16 14:33:26,192 [SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR
org.apache.flume.SinkRunner - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at org.apache.flume.sink.AvroSink.process(AvroSink.java:325)
~[flume-ng-core-1.2.0.jar:1.2.0]
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
~[flume-ng-core-1.2.0.jar:1.2.0]
at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
~[flume-ng-core-1.2.0.jar:1.2.0]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_31]
Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host:
localhost, port: 11113 }: RPC connection error
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
~[flume-ng-sdk-1.2.0.jar:1.2.0]
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
~[flume-ng-sdk-1.2.0.jar:1.2.0]
at
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
~[flume-ng-sdk-1.2.0.jar:1.2.0]
at
org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
~[flume-ng-sdk-1.2.0.jar:1.2.0]
at
org.apache.flume.sink.AvroSink.createConnection(AvroSink.java:182)
~[flume-ng-core-1.2.0.jar:1.2.0]
at
org.apache.flume.sink.AvroSink.verifyConnection(AvroSink.java:222)
~[flume-ng-core-1.2.0.jar:1.2.0]
at org.apache.flume.sink.AvroSink.process(AvroSink.java:282)
~[flume-ng-core-1.2.0.jar:1.2.0]
... 3 common frames omitted
Caused by: java.io.IOException: Error connecting to /127.0.0.1:11113
at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
~[avro-ipc-1.6.3.jar:1.6.3]
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
~[avro-ipc-1.6.3.jar:1.6.3]
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
~[avro-ipc-1.6.3.jar:1.6.3]
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
~[flume-ng-sdk-1.2.0.jar:1.2.0]
... 9 common frames omitted
Caused by: java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.6.0_31]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
~[na:1.6.0_31]
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
~[netty-3.2.7.Final.jar:na]
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
~[netty-3.2.7.Final.jar:na]
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
~[netty-3.2.7.Final.jar:na]
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
~[na:1.6.0_31]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
~[na:1.6.0_31]
... 1 common frames omitted
Thank you for your help.
Regards,
Rudolf
Rudolf Rakos
Morgan Stanley | ISG Technology
[email protected]<mailto:[email protected]>
Be carbon conscious. Please consider our environment before printing this email.
--------------------------------------------------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or
views contained herein are not intended to be, and do not constitute, advice
within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and
Consumer Protection Act. If you have received this communication in error,
please destroy all electronic and paper copies and notify the sender
immediately. Mistransmission is not intended to waive confidentiality or
privilege. Morgan Stanley reserves the right, to the extent permitted under
applicable law, to monitor electronic communications. This message is subject
to terms available at the following link:
http://www.morganstanley.com/disclaimers. If you cannot access these links,
please notify us by reply message and we will send the contents to you. By
messaging with Morgan Stanley you consent to the foregoing.
