[
https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421161#comment-13421161
]
Will McQueen commented on FLUME-1392:
-------------------------------------
Test 2
======
Tested with this config file:
agent.channels = c1
agent.sources = r1
agent.sinks = k1
agent.channels.c1.type = MEMORY
agent.channels.c2.type = MEMORY
agent.sources.r1.channels = c1
agent.sources.r1.type = NETCAT
agent.sources.r1.bind = 0.0.0.0
agent.sources.r1.port = 41414
agent.sinks.k1.channel = c2
agent.sinks.k1.type = LOGGER
...and got this:
[will@will-laptop apache-flume-1.3.0-SNAPSHOT]$ bin/flume-ng agent -c conf -f
conf/flume.conf -n agent
+ exec /usr/java/latest/bin/java -Xmx20m -cp
'/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/conf:/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f conf/flume.conf -n
agent
2012-07-23 21:15:05,370 (main) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 1
2012-07-23 21:15:05,372 (main) [INFO -
org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node starting -
agent
2012-07-23 21:15:05,374 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)]
Node manager starting
2012-07-23 21:15:05,374 (lifecycleSupervisor-1-2) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)]
Configuration provider starting
2012-07-23 21:15:05,377 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)]
Starting lifecycle supervisor 10
2012-07-23 21:15:05,377 (lifecycleSupervisor-1-0) [DEBUG -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)]
Node manager started
2012-07-23 21:15:05,378 (lifecycleSupervisor-1-2) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)]
Configuration provider started
2012-07-23 21:15:05,378 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)]
Checking file:conf/flume.conf for changes
2012-07-23 21:15:05,378 (conf-file-poller-0) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)]
Reloading configuration file:conf/flume.conf
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:904)]
Added sinks: k1 Agent: agent
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)]
Processing:k1
2012-07-23 21:15:05,382 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:994)]
Created context for k1: type
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)]
Processing:k1
2012-07-23 21:15:05,383 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)]
Starting validation of configuration for agent: agent, initial-configuration:
AgentConfiguration[agent]
SOURCES: {r1={ parameters:{port=41414, channels=c1, type=NETCAT, bind=0.0.0.0}
}}
CHANNELS: {c1={ parameters:{type=MEMORY} }, c2={ parameters:{type=MEMORY} }}
SINKS: {k1={ parameters:{type=LOGGER, channel=c2} }}
2012-07-23 21:15:05,387 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)]
Created channel c1
2012-07-23 21:15:05,402 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:651)]
Creating sink: k1 using LOGGER
2012-07-23 21:15:05,405 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)]
Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic
syntactical validation was performed[agent]
SOURCES: {r1={ parameters:{port=41414, channels=c1, type=NETCAT, bind=0.0.0.0}
}}
CHANNELS: {c1={ parameters:{type=MEMORY} }}
AgentConfiguration created with Configuration stubs for which full validation
was performed[agent]
SINKS: {k1=ComponentConfiguration[k1]
CONFIG:
CHANNEL:c2
}
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)]
Channels:c1
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)]
Sinks k1
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)]
Sources r1
2012-07-23 21:15:05,406 (conf-file-poller-0) [INFO -
org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:122)]
Post-validation flume configuration contains configuration for agents: [agent]
2012-07-23 21:15:05,406 (conf-file-poller-0) [INFO -
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:249)]
Creating channels
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG -
org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:68)]
Creating instance of channel c1 type MEMORY
2012-07-23 21:15:05,443 (conf-file-poller-0) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.<init>(MonitoredCounterGroup.java:68)]
Monitoried counter group for type: CHANNEL, name: c1, registered successfully.
2012-07-23 21:15:05,443 (conf-file-poller-0) [INFO -
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:273)]
created channel c1
2012-07-23 21:15:05,443 (conf-file-poller-0) [DEBUG -
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)]
Creating instance of source r1, type NETCAT
2012-07-23 21:15:05,456 (conf-file-poller-0) [INFO -
org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:70)]
Creating instance of sink: k1, type: LOGGER
2012-07-23 21:15:05,458 (conf-file-poller-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:106)]
Starting new configuration:{ sourceRunners:{r1=EventDrivenSourceRunner: {
source:org.apache.flume.source.NetcatSource@5d5bdc50 }}
sinkRunners:{k1=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } }}
channels:{c1=org.apache.flume.channel.MemoryChannel@6a8c436b} }
2012-07-23 21:15:05,458 (conf-file-poller-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:113)]
Starting Channel c1
2012-07-23 21:15:05,459 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:82)]
Component type: CHANNEL, name: c1 started
2012-07-23 21:15:05,459 (conf-file-poller-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:141)]
Starting Sink k1
2012-07-23 21:15:05,459 (conf-file-poller-0) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)]
Starting Source r1
2012-07-23 21:15:05,460 (lifecycleSupervisor-1-3) [INFO -
org.apache.flume.source.NetcatSource.start(NetcatSource.java:147)] Source
starting
2012-07-23 21:15:05,460 (lifecycleSupervisor-1-0) [ERROR -
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)]
Unable to start SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at
org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-07-23 21:15:05,468 (lifecycleSupervisor-1-3) [INFO -
org.apache.flume.source.NetcatSource.start(NetcatSource.java:161)] Created
serverSocket:sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:41414]
2012-07-23 21:15:05,468 (lifecycleSupervisor-1-3) [DEBUG -
org.apache.flume.source.NetcatSource.start(NetcatSource.java:180)] Source
started
2012-07-23 21:15:05,468 (Thread-1) [DEBUG -
org.apache.flume.source.NetcatSource$AcceptHandler.run(NetcatSource.java:258)]
Starting accept handler
2012-07-23 21:15:08,463 (lifecycleSupervisor-1-1) [ERROR -
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)]
Unable to start SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at
org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-07-23 21:15:11,464 (lifecycleSupervisor-1-3) [ERROR -
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)]
Unable to start SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at
org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
^C2012-07-23 21:15:12,742 (node-shutdownHook) [INFO -
org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping -
agent
2012-07-23 21:15:12,742 (node-shutdownHook) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:78)]
Stopping lifecycle supervisor 9
2012-07-23 21:15:12,744 (node-shutdownHook) [INFO -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:91)]
Configuration provider stopping
2012-07-23 21:15:12,744 (node-shutdownHook) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:97)]
Waiting for file watcher to terminate
2012-07-23 21:15:12,744 (node-shutdownHook) [DEBUG -
org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:107)]
Configuration provider stopped
2012-07-23 21:15:12,744 (node-shutdownHook) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop(DefaultLogicalNodeManager.java:215)]
Node manager stopping
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:68)]
Shutting down configuration: { sourceRunners:{r1=EventDrivenSourceRunner: {
source:org.apache.flume.source.NetcatSource@5d5bdc50 }}
sinkRunners:{k1=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } }}
channels:{c1=org.apache.flume.channel.MemoryChannel@6a8c436b} }
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:72)]
Stopping Source r1
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)]
Stopping component: EventDrivenSourceRunner: {
source:org.apache.flume.source.NetcatSource@5d5bdc50 }
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:186)] Source
stopping
2012-07-23 21:15:12,745 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:191)] Stopping
accept handler thread
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:195)] Waiting for
accept handler to finish
2012-07-23 21:15:12,746 (Thread-1) [DEBUG -
org.apache.flume.source.NetcatSource$AcceptHandler.run(NetcatSource.java:281)]
Accept handler exiting
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:205)] Stopped
accept handler thread
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:220)] Waiting for
handler service to stop
2012-07-23 21:15:12,747 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:235)] Handler
service stopped
2012-07-23 21:15:12,747 (node-shutdownHook) [DEBUG -
org.apache.flume.source.NetcatSource.stop(NetcatSource.java:238)] Source
stopped. Event metrics:{ name:null counters:{open.attempts=1} }
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:82)]
Stopping Sink k1
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)]
Stopping component: SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{
name:null counters:{} } }
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:92)]
Stopping Channel c1
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)]
Stopping component: org.apache.flume.channel.MemoryChannel@6a8c436b
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.instrumentation.MonitoredCounterGroup.stop(MonitoredCounterGroup.java:87)]
Component type: CHANNEL, name: c1 stopped
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO -
org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:78)]
Stopping lifecycle supervisor 9
2012-07-23 21:15:12,748 (node-shutdownHook) [DEBUG -
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop(DefaultLogicalNodeManager.java:221)]
Node manager stopped
....where IllegalStateException keeps getting displayed every 2 or 3 secs, so
if you could please revise the patch to catch this case, that would be great.
Thank you.
Cheers,
Will
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
