Hi all, I am new to flume and all that logging stuff and probably many things are unclear to me despite I read the docs. I want to run Flume on windows server to create proof of concept solution. I was able to compile it successfully. I was able to make successful test with netcat. Next test that i wanted to do is with http source but unfortunately it did not pass. Here is what I do:
my server config (httppost.conf): agent1.sources = r1 agent1.channels = memoryChannel agent1.sinks = k1 agent1.sources.r1.type = avro agent1.sources.r1.bind = localhost agent1.sources.r1.port = 9003 agent1.sinks.k1.type = file_roll agent1.sinks.k1.sink.directory = C:\\Projects\\nickyk\\apache-flume-1.3.1-bin\\log agent1.channels.memoryChannel.type = memory agent1.channels.memoryChannel.capacity = 10000 agent1.channels.memoryChannel.transactionCapacity = 1000 agent1.sources.r1.channels = memoryChannel agent1.sinks.k1.channel = memoryChannel I run the server with this bat file: set FLUME_HOME=C:\Projects\nickyk\apache-flume-1.3.1-bin\ set JAVA="C:\Java\jdk1.6.0_45\bin\java.exe" set JAVA_OPTS=-Xmx60m set CONF=%FLUME_HOME%\conf\httppost.conf set AGENT=agent1 %JAVA% %JAVA_OPTS% -Dlog4j.configuration=file:///%FLUME_HOME%\conf\log4j.properties -cp "%FLUME_HOME%\lib\*" org.apache.flume.node.Application -f %CONF% -n %AGENT% netstat does not show any listener on port 9003? Is this normal or there must be a listener? I run avro client with this bat file: set FLUME_HOME=C:\Projects\nickyk\apache-flume-1.3.1-bin set JAVA="C:\Java\jdk1.6.0_45\bin\java.exe" set JAVA_OPTS=-Xmx60m set AVRO_HOST=localhost set AVRO_PORT=9003 set SENDFILE=%FLUME_HOME%\logs\data.log set HEADFILE=%FLUME_HOME%\conf\header.txt %JAVA% %JAVA_OPTS% -Dlog4j.configuration=file:///%FLUME_HOME%\conf\log4j.properties -cp "%FLUME_HOME%\lib\*" org.apache.flume.client.avro.AvroCLIClient -H %AVRO_HOST% -p %AVRO_PORT% -F %SENDFILE% --headerFile %HEADFILE% Then I get this output: C:\Projects\nickyk\apache-flume-1.3.1-bin>"C:\Java\jdk1.6.0_45\bin\java.exe" -Xmx60m -Dlog4j.configuration=file:///C:\Projects\nickyk\apache-flume-1.3.1-bin\conf\log4j.properties -cp "C:\Projects\nickyk\apache-flume-1.3.1-bin\lib\*" org.apache.flume.client.avro.AvroCLIClient -H localhost -p 9003 -F C:\Projects\nickyk\apache-flume-1.3.1-bin\logs\data.log --headerFile C:\Projects\nickyk\apache-flume-1.3.1-bin\conf\header.txt 2013-06-20 17:55:58,285 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.parseHeaders(AvroCLIClient.java:101)] Inserting Header Key [filename] header value [datalog] 2013-06-20 17:55:58,287 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.parseHeaders(AvroCLIClient.java:101)] Inserting Header Key [host] header value [zhouhh] 2013-06-20 17:55:58,295 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)] Batch size string = 5 2013-06-20 17:55:59,358 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:79)] Unable to open connection to Flume. Exception follows. org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 9003 }: RPC connection error at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117) at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93) at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507) at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169) at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180) at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71) Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:9003 at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261) at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203) at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152) at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106) ... 5 more Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) 2013-06-20 17:55:59,369 (main) [DEBUG - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:84)] Exiting My question is what I am doing wrong and what I need to test in order to fix this situatioin. Thanks in advance. best regards,