[
https://issues.apache.org/jira/browse/FLUME-2831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991591#comment-14991591
]
Pramod commented on FLUME-2831:
-------------------------------
Gonzalo ,
Thanks a lot for the assistance . I was able to start Flume .
But now i have got stuck in another issue. This time, Flume is failing to make
connection between agents i.e agent1 (Windows machine) and agent2(Linux
machine).
Avro source avroIn is running on Linux. I have also tried giving different
ports.
Path > mc1 channel > Agent1> mc1 channel > Agent2 > HDFS
Kindly assist.
Agent:1 in windows:
source_agent.sources = weblogic_server
source_agent.sources.weblogic_server.type = spooldir
source_agent.sources.weblogic_server.spoolDir = C:/logs/success
source_agent.sources.weblogic_server.batchSize = 1
source_agent.sources.weblogic_server.channels = memoryChannel
#source_agent.sources.weblogic_server.interceptors = itime ihost itype
source_agent.sources.weblogic_server.interceptors.itime.type = timestamp
source_agent.sources.weblogic_server.interceptors.ihost.type = host
source_agent.sources.weblogic_server.interceptors.ihost.useIP = false
source_agent.sources.weblogic_server.interceptors.ihost.hostHeader = host
source_agent.sources.weblogic_server.interceptors.itype.type = static
source_agent.sources.weblogic_server.interceptors.itype.key = log_type
source_agent.sources.weblogic_server.interceptors.itype.value =
apache_access_combined
source_agent.channels = memoryChannel
source_agent.channels.memoryChannel.type = memory
source_agent.channels.memoryChannel.capacity = 10000
source_agent.sinks = avro_sink
source_agent.sinks.avro_sink.type = avro
source_agent.sinks.avro_sink.channel = memoryChannel
source_agent.sinks.avro_sink.hostname = XXX.XXX.80.104
source_agent.sinks.avro_sink.port = 4141
Agent:2 on Linux:
collector.sources = AvroIn
collector.sources.AvroIn.type = avro
collector.sources.AvroIn.bind = localhost
collector.sources.AvroIn.port = 4141
collector.sources.AvroIn.channels = mc1
collector.channels = mc1
collector.channels.mc1.type = memory
collector.channels.mc1.capacity = 100
#collector.channels.mc2.type = memory
#collector.channels.mc2.capacity = 100
collector.sinks = HadoopOut
collector.sinks.HadoopOut.type = hdfs
collector.sinks.HadoopOut.channel = mc1
collector.sinks.HadoopOut.hdfs.path =hdfs://xxx.80.104:9000/user/logs/
collector.sinks.HadoopOut.hdfs.callTimeout = 150000
collector.sinks.HadoopOut.hdfs.fileType = DataStream
collector.sinks.HadoopOut.hdfs.writeFormat = Text
collector.sinks.HadoopOut.hdfs.rollSize = 0
collector.sinks.HadoopOut.hdfs.rollCount = 10000
collector.sinks.HadoopOut.hdfs.rollInterval = 600
Error Message :
05 Nov 2015 17:14:26,982 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AbstractRpcSink.createConnection:206) - Rpc sink
avro_sink: Building RpcClient with hostname: 172.19.80.104, port: 4141
05 Nov 2015 17:14:26,982 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AvroSink.initializeRpcClient:126) - Attempting to
create Avro Rpc client.
05 Nov 2015 17:14:26,982 WARN [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.api.NettyAvroRpcClient.configure:634) - Using default
maxIOWorkers
05 Nov 2015 17:14:32,530 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver event.
Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host:
172.19.80.104, port: 4141 }: RPC connection error
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:182)
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:121)
at
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)
at
org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:89)
at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
at
org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:211)
at
org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:272)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:349)
... 3 more
Caused by: java.io.IOException: Error connecting to /172.19.80.104:4141
at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:168)
... 10 more
Caused by: java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:496)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:452)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
05 Nov 2015 17:14:37,535 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AbstractRpcSink.createConnection:206) - Rpc sink
avro_sink: Building RpcClient with hostname: 172.19.80.104, port: 4141
05 Nov 2015 17:14:37,535 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AvroSink.initializeRpcClient:126) - Attempting to
create Avro Rpc client.
05 Nov 2015 17:14:37,535 WARN [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.api.NettyAvroRpcClient.configure:634) - Using default
maxIOWorkers.
> Flume 0.9.4 : Windows client : Unable to start HTTP server
> -----------------------------------------------------------
>
> Key: FLUME-2831
> URL: https://issues.apache.org/jira/browse/FLUME-2831
> Project: Flume
> Issue Type: IT Help
> Components: Node
> Affects Versions: v0.9.4
> Environment: Node OS : Windows 8
> Master OS : CentOS 6.4
> Reporter: Pramod
>
> We are trying to collect log files collected on a Windows machine into CentOS
> collector. Flume master is placed on the CentOS machine.
> While starting the flume node on Windows node, we are getting the below
> messages :
> 1. com.cloudera.util.InternalHttpServer$InternalHttpServerException: Unable
> to start HTTP server
> 2. 15/10/29 11:46:13 INFO agent.MultiMasterRPC: No active master RPC
> connection
> 15/10/29 11:46:14 WARN agent.MultiMasterRPC: Could not connect to any master
> nodes (tried 1: [localhost:35872])
> Kindly assist
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
