Hi Rudolf,
Unfortunately, the sink will try to connect to the source and when it fails it
will log this error message. The sink would back off for a few seconds(it
increases with each failure till a max of 5 seconds I think), but each time it
tries to connect it will log this error. Please file a jira if you want this to
be reduced to a WARN, so it can be discussed with other devs and if a consensus
is reached, the current code can be patched to make this a warn.
Thanks,
Hari
--
Hari Shreedharan
On Thursday, August 16, 2012 at 5:34 AM, Rakos, Rudolf wrote:
> Hi All,
>
> We are developing a tool which embeds Flume. I have some questions about
> configuring Flume.
>
> The Avro sink emits error level log messages every few seconds when it loses
> connection to the next hop Avro source.
> Using a file channel before the Avro sink guarantees the transactional
> behavior and 100% delivery. So the messages will be delivered when the next
> hop Avro source comes online.
> Is there a way to configure Flume to only log a warning level log message or
> only log this error once?
>
> The log message:
> 2012-08-16 14:33:26,192 [SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR
> org.apache.flume.SinkRunner - Unable to deliver event. Exception follows.
> org.apache.flume.EventDeliveryException: Failed to send events
> at org.apache.flume.sink.AvroSink.process(AvroSink.java:325)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> at java.lang.Thread.run(Thread.java:662) [na:1.6.0_31]
> Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host:
> localhost, port: 11113 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> ~[flume-ng-sdk-1.2.0.jar:1.2.0]
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> ~[flume-ng-sdk-1.2.0.jar:1.2.0]
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> ~[flume-ng-sdk-1.2.0.jar:1.2.0]
> at
> org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
> ~[flume-ng-sdk-1.2.0.jar:1.2.0]
> at org.apache.flume.sink.AvroSink.createConnection(AvroSink.java:182)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> at org.apache.flume.sink.AvroSink.verifyConnection(AvroSink.java:222)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> at org.apache.flume.sink.AvroSink.process(AvroSink.java:282)
> ~[flume-ng-core-1.2.0.jar:1.2.0]
> ... 3 common frames omitted
> Caused by: java.io.IOException: Error connecting to /127.0.0.1:11113
> at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
> ~[avro-ipc-1.6.3.jar:1.6.3]
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
> ~[avro-ipc-1.6.3.jar:1.6.3]
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
> ~[avro-ipc-1.6.3.jar:1.6.3]
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ~[flume-ng-sdk-1.2.0.jar:1.2.0]
> ... 9 common frames omitted
> Caused by: java.net.ConnectException: Connection refused: no further
> information
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.6.0_31]
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> ~[na:1.6.0_31]
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
> ~[netty-3.2.7.Final.jar:na]
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
> ~[netty-3.2.7.Final.jar:na]
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
> ~[netty-3.2.7.Final.jar:na]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ~[na:1.6.0_31]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> ~[na:1.6.0_31]
> ... 1 common frames omitted
>
> Thank you for your help.
>
> Regards,
> Rudolf
>
> Rudolf Rakos
> Morgan Stanley | ISG Technology
> [email protected] (mailto:[email protected])
>
>
> Be carbon conscious. Please consider our environment before printing this
> email.
>
>
> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions
> or views contained herein are not intended to be, and do not constitute,
> advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform
> and Consumer Protection Act. If you have received this communication in
> error, please destroy all electronic and paper copies and notify the sender
> immediately. Mistransmission is not intended to waive confidentiality or
> privilege. Morgan Stanley reserves the right, to the extent permitted under
> applicable law, to monitor electronic communications. This message is subject
> to terms available at the following link:
> http://www.morganstanley.com/disclaimers. If you cannot access these links,
> please notify us by reply message and we will send the contents to you. By
> messaging with Morgan Stanley you consent to the foregoing.
