It'd be great if you could submit the patch upstream to Flume! :-) -- Hari Shreedharan
On Tuesday, October 30, 2012 at 4:06 PM, Paul Chavez wrote: > Thank you, I was unsure if that was desired behavior or not. I updated the > class as a POC to add exit codes for each of those catch blocks and it works > great. > > Thanks, > Paul > > > From: Hari Shreedharan [mailto:[email protected]] > Sent: Tuesday, October 30, 2012 11:47 AM > To: [email protected] (mailto:[email protected]) > Subject: Re: Help determining exit code from flume avro-client > > Hi Paul. > > Thanks for reporting this. It does look like Flume does not return non-zero > error code even if there was an exception. I filed > https://issues.apache.org/jira/browse/FLUME-1670 to address this. > > Thanks, > Hari > > -- > Hari Shreedharan > > > On Tuesday, October 30, 2012 at 11:25 AM, Paul Chavez wrote: > > > I am working on getting the flume avro-client functionality working on > > Windows, and am currently stuck on how to determine if the file was sent > > successfully. > > > > I used the blog post at > > http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to guide > > me and have built the 1.4.0 snapshot successfully. I can manually send > > files along to flume using the > > ''org.apache.flume.client.avro.AvroCLIClient" class. > > > > The full command I am using is invoked via batch file with the following > > single line: > > "C:\Program Files (x86)\Java\jdk1.6.0_37\bin\java.exe" -Xmx20m > > -Dlog4j.configuration=file:///C:\apache-flume-1.4.0-SNAPSHOT\conf\log4j.properties > > -cp "C:\apache-flume-1.4.0-SNAPSHOT\lib\*" > > org.apache.flume.client.avro.AvroCLIClient -H 10.20.2.8 -p 6240 -F%1 > > > > My intention is to invoke from Powershell like so: > > .\flume.bat C:\IISLogs\W3SVC100\u_ex121029.log > > > > This is working well and I am able to successfully send IIS and our app > > logs to flume and then onto HDFS without issues. Now I am trying to > > automate the process, and need to know if the operation failed for some > > reason. If I simply turn off the flume source I can trigger an error which > > is dumped to console like so (using flume.root.logger=DEBUG,console in the > > log4j file): > > > > > > 2012-10-30 11:14:22,185 (main) [ERROR - > > org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)] > > Unable t > > o open connection to Flume. Exception follows. > > org.apache.flume.FlumeException: NettyAvroRpcClient { host: x.x.x.x, port: > > 6240 }: RPC connection error > > at > > org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117) > > at > > org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93) > > at > > org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507) > > at > > org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169) > > at > > org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168) > > at > > org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68) > > Caused by: java.io.IOException: Error connecting to /x.x.x.x:6240 > > at > > org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261) > > at > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203) > > at > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152) > > at > > org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106) > > ... 5 more > > Caused by: java.net.ConnectException: Connection refused: no further > > information > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > > at > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599) > > at > > org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java: > > 401) > > at > > org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipeli > > neSink.java:370) > > at > > org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292) > > > > > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > 2012-10-30 11:14:22,185 (main) [DEBUG - > > org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:81)] > > Exiting > > > > Despite the ERROR message being logged to the console, java.exe still exits > > with code 0, indicating success. Is there a way to use the console appender > > to trigger java.exe to send a non-0 exit code if an error occurs so the > > rest of my automation tooling can detect the error? I can parse the output > > with Powershell and have it send a non-0 exit code itself, but looking for > > a more 'native' way to do this. > > > > thanks, > > Paul Chavez > > > > > > > >
