Hi,
I am running flume in Virtual Machines with DHCP network.
-- cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Intel Corporation 82575EB Gigabit Network Connection
DEVICE=eth0
BOOTPROTO=dhcp
DHCPCLASS=
HWADDR=5C:F3:FC:A6:5D:C0
ONBOOT=yes
-- The system information:
Linux version 2.6.18-194.el5 ([email protected]) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Mar 16 21:52:39 EDT 2010
Following is all the configuration about zookeeper in flume-conf.xml:
<property>
<name>flume.master.store</name>
<value>zookeeper</value>
<description>How the Flume Master stores node configurations. Must
be either 'zookeeper' or 'memory'.</description>
</property>
<property>
<name>flume.master.zk.logdir</name>
<value>/flume/flume-${user.name}-zk</value>
<description>The base directory in which the ZBCS stores data.</description>
</property>
<property>
<name>flume.master.zk.use.external</name>
<value>true</value>
</property>
<property>
<name>flume.master.zk.servers</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
At 2012-05-10 13:58:40,"alo alt" <[email protected]> wrote:
>Add your zookeeper config too. Do you run them in a VM or bare metal? DHCP
>network?
>
>
>--
>Alexander Lorenz
>http://mapredit.blogspot.com
>German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>On May 10, 2012, at 3:46 AM, lulynn_2008 wrote:
>
>> Yes, sure.
>>
>>
>>
>> At 2012-05-09 18:40:31,"alo alt" <[email protected]> wrote:
>> >Did you add
>> >
>> ><property>
>> ><name>flume.master.zk.use.external</name>
>> ><value>true</value>
>> ></property>
>> >
>> >into the flume master config?
>> >
>> >--
>> >Alexander Lorenz
>> >http://mapredit.blogspot.com
>> >German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >
>> >On May 9, 2012, at 12:22 PM, lulynn_2008 wrote:
>> >
>> >> Hi,
>> >> After remove ";" in < ;/configuration>, and restart flume based on
>> >> https://cwiki.apache.org/FLUME/troubleshooting-faq.html#TroubleshootingFAQ-Ihaveencountereda%2522Couldnotincrementversioncounter%2522errormessage.
>> >> There is still the same error during starting flume master.
>> >>
>> >>
>> >>
>> >> At 2012-05-09 16:39:03,"alo alt" <[email protected]> wrote:
>> >> >Hi,
>> >> >
>> >> >Please remove the ";" in
>> >> >> < ;/configuration>
>> >> >
>> >> >That could be a zookeeper issue on virtual machines:
>> >> >https://cwiki.apache.org/FLUME/troubleshooting-faq.html#TroubleshootingFAQ-Ihaveencountereda%2522Couldnotincrementversioncounter%2522errormessage.
>> >> >
>> >> >- Alex
>> >> >
>> >> >--
>> >> >Alexander Lorenz
>> >> >http://mapredit.blogspot.com
>> >> >German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >> >
>> >> >On May 9, 2012, at 10:31 AM, lulynn_2008 wrote:
>> >> >
>> >> >> Hi All,
>> >> >> Please help to check why this happened. thank you.
>> >> >> -- I am using external zookeeper to run flume. During start flume
>> >> >> master node, I got following error information in flumemaster.out. And
>> >> >> we can find flume master has been started by pid files.
>> >> >> -- But when run "refreshAll" in flume shell, the command failed
>> >> >> with "Could not increment version counter...".
>> >> >> -- When run flume without external zookeeper, flume can work
>> >> >> normally. So I assume this is caused by zookeeper part.
>> >> >> Here is the zookeeper configuration: node1 is hostname and
>> >> >> node2/node3 is ip.
>> >> >> <property>
>> >> >> <name>flume.master.zk.servers</name>
>> >> >> <value>node1:2181,node2:2181,node3:2181</value>
>> >> >> </property>
>> >> >> < ;/configuration>
>> >> >>
>> >> >> ERROR INFORMATION:
>> >> >> 2012-05-09 01:09:39,418 [main] ERROR master.ConfigManager: ConfigStore
>> >> >> init threw IOException
>> >> >> java.io.IOException: Unexpected exception in loadConfigs
>> >> >> at
>> >> >> com.cloudera.flume.master.ZooKeeperConfigStore.loadConfigs(ZooKeeperConfigStore.java:267)
>> >> >> at
>> >> >> com.cloudera.flume.master.ZooKeeperConfigStore$1.success(ZooKeeperConfigStore.java:138)
>> >> >> at
>> >> >> com.cloudera.flume.master.ZKClient$1.doTry(ZKClient.java:169)
>> >> >> at com.cloudera.util.RetryHarness.attempt(RetryHarness.java:64)
>> >> >> at com.cloudera.flume.master.ZKClient.init(ZKClient.java:178)
>> >> >> at
>> >> >> com.cloudera.flume.master.ZooKeeperConfigStore.connect(ZooKeeperConfigStore.java:145)
>> >> >> at com.c
>> >> >> loudera.flume.master.ZooKeeperConfigStore.init(ZooKeeperConfigStore.java:110)
>> >> >> at
>> >> >> com.cloudera.flume.master.ConfigManager.start(ConfigManager.java:375)
>> >> >> at
>> >> >> com.cloudera.flume.master.flows.FlowConfigManager.start(FlowConfigManager.java:291)
>> >> >> at
>> >> >> com.cloudera.flume.master.TranslatingConfigurationManager.start(TranslatingConfigurationManager.java:391)
>> >> >> at
>> >> >> com.cloudera.flume.master.FlumeMaster.serve(FlumeMaster.java:295)
>> >> >> at
>> >> >> com.cloudera.flume.master.FlumeMaster.main(FlumeMaster.java:556)
>> >> >> Caused by: java.io.IOException: Not a data file.
>> >> >> at
>> >> >> org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:105)
>> >> >> at org.apache.avro.file.DataFileStream.<init>(DataFileStre
>> >> >> am.java:84)
>> >> >> at
>> >> >> com.cloudera.flume.master.ZooKeeperConfigStore.deserializeConfigs(ZooKeeperConfigStore.java:190)
>> >> >> at
>> >> >> com.cloudera.flume.master.ZooKeeperConfigStore.loadConfigs(ZooKeeperConfigStore.java:265)
>> >> >> ... 11 more
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >
>>
>>
>>
>
