Chen if you can send jstack output that could help. Or do: sudo kill -3 <pid> on Flume and send the output from the Flume stdout, which is the same thing.
Regards Mike On Mon, Oct 8, 2012 at 12:37 AM, Lichen <[email protected]> wrote: > Hi Mike,**** > > ** ** > > I’ve tried hadoop-1.0.3 which doesn’t work as before. Nothing happens after > **** > > [INFO - > org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:70)] > Creating instance of sink: hdfs-sink, type: hdfs**** > > I can execute $HADOOP_HOME/bin/hadoop classpath. Don’t know why the HDFS > sink is not working.**** > > ** ** > > Chen**** > > ** ** > > On Sat, Sep 30, 2012 at 10:49 AM, Mike Percy <[email protected]> wrote::** > ** > > ** ** > > Hi,**** > > Don't use vanilla hadoop 0.20.2 ... you will not get durability.**** > > ** ** > > You must use hadoop 0.20.205, hadoop 1.0.x, hadoop 2.0.x, or CDH3/4 with > Flume NG.**** > > ** ** > > Regards,**** > > Mike**** > > ** ** > > On Sat, Sep 29, 2012 at 12:57 AM, Lichen <[email protected]> wrote:*** > * > > Hi. I’m using flume 1.2 with hadoop-0.20.2 but unable to use HDFS sink. > Flume stuck during creating HDFS sink, which is OK using file-roll or > logger sinks. I’ve included $HADOOP_HOME and $HADOOP_HOME/lib in > FLUME_CLASSPATH. Is it because my hadoop version not compatible?**** > > **** > > Configuration file:**** > > agent_foo.sources = netcat-source**** > > agent_foo.sinks = hdfs-sink**** > > agent_foo.channels = mem-channel-1**** > > **** > > # properties of netcat-source**** > > agent_foo.sources.netcat-source.channels = mem-channel-1**** > > agent_foo.sources.netcat-source.type = netcat**** > > agent_foo.sources.netcat-source.bind = localhost**** > > agent_foo.sources.netcat-source.port = 44444**** > > **** > > # properties of hdfs-sink**** > > agent_foo.sinks.hdfs-sink.channel = mem-channel-1**** > > agent_foo.sinks.hdfs-sink.type = hdfs**** > > agent_foo.sinks.hdfs-sink.hdfs.path = hdfs://hadoop0:9000/flume/**** > > **** > > # properties of mem-channel-1**** > > agent_foo.channels.mem-channel-1.type = memory**** > > agent_foo.channels.mem-channel-1.capacity = 1000**** > > agent_foo.channels.mem-channel-1.transactionCapacity = 100**** > > **** > > Log:**** > > $ bin/flume-ng agent -c conf/ -f conf/test.conf -n agent_foo**** > > 2012-09-29 15:19:47,765 (main) [INFO - > org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] > Starting lifecycle supervisor 1**** > > 2012-09-29 15:19:47,768 (main) [INFO - > org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node > starting - agent_foo**** > > 2012-09-29 15:19:47,772 (lifecycleSupervisor-1-1) [INFO - > org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:187)] > Node manager starting**** > > 2012-09-29 15:19:47,772 (lifecycleSupervisor-1-1) [INFO - > org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] > Starting lifecycle supervisor 11**** > > 2012-09-29 15:19:47,774 (lifecycleSupervisor-1-2) [INFO - > org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)] > Configuration provider starting**** > > 2012-09-29 15:19:47,776 (conf-file-poller-0) [INFO - > org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)] > Reloading configuration file:conf/test.conf**** > > 2012-09-29 15:19:47,780 (conf-file-poller-0) [INFO - > org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:988)] > Processing:hdfs-sink**** > > 2012-09-29 15:19:47,788 (conf-file-poller-0) [INFO - > org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:988)] > Processing:hdfs-sink**** > > 2012-09-29 15:19:47,788 (conf-file-poller-0) [INFO - > org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:902)] > Added sinks: hdfs-sink Agent: agent_foo**** > > 2012-09-29 15:19:47,789 (conf-file-poller-0) [INFO - > org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:988)] > Processing:hdfs-sink**** > > 2012-09-29 15:19:47,816 (conf-file-poller-0) [INFO - > org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:122)] > Post-validation flume configuration contains configuration for agents: > [agent_foo]**** > > 2012-09-29 15:19:47,816 (conf-file-poller-0) [INFO - > org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:249)] > Creating channels**** > > 2012-09-29 15:19:47,858 (conf-file-poller-0) [INFO - > org.apache.flume.instrumentation.MonitoredCounterGroup.<init>(MonitoredCounterGroup.java:68)] > Monitoried counter group for type: CHANNEL, name: mem-channel-1, registered > successfully.**** > > 2012-09-29 15:19:47,859 (conf-file-poller-0) [INFO - > org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:273)] > created channel mem-channel-1**** > > 2012-09-29 15:19:47,879 (conf-file-poller-0) [INFO - > org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:70)] > Creating instance of sink: hdfs-sink, type: hdfs**** > > **** > > There is no more output after the last line.**** > > ** ** >
