Re: Cluster Configuration Issues

2009-10-23 Thread Patrick Hunt
Usually this indicates that either a firewall is blocking one or more of 
the ports, or the server isn't started on all the hosts. First verify 
connectivity, I usually use telnet, see detail here:


https://help.maximumasp.com/KB/a445/connectivity-testing-with-ping-telnet-tracert-and-pathping-.aspx

Patrick

Mark Vigeant wrote:

Ok, so now the clientPort is all set, but I'm running into another problem.

When I try to start up zookeeper server, the error I'm getting now is that it 
can't open the channel to the other machine:

2009-10-223 10:39:11,941 - WARN 
[QuorumPeer:/0:0:0:0:0:0:0:0:2181:quorumcnxmana...@336] - Cannot open channel 
to 2 at
election address leela/10.7.21.216:3888
java.net.ConnectException: Connection refused

I've checked to see if ports 2888 or 3888 are in use on either machine and they are not. Also, there is an open ssh between the 2 
of them. I've tried on a few other ports as well and it's giving me the same issue. Do I need to tamper with port permissions?

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Friday, October 23, 2009 1:23 AM

To: zookeeper-user@hadoop.apache.org; Mark Vigeant
Subject: Re: Cluster Configuration Issues

That usually indicates that you have a process that's already using a 
port that you've specified zookeeper to use. Try something like:


netstat -a -n -p tcp

and look at 2181/2888/3888 on the hosts you are using. Either use a 
different port or stop the process that's currently using.


Patrick

Mark Vigeant wrote:

Haha well I'm an idiot. Thank you.

I would end there but a new error is being thrown:

2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected 
exception, exiting abnormally
Java.net.BindException: Address already in use


-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com] 
Sent: Thursday, October 22, 2009 6:01 PM

To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

yeah - thought this was it: you've missed the forward slash on
home/mark/zookeeper (this turned up on your exception message).

On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


Yeah I just figured out the problem with zoocfg.py

I am running as the same user who created myid. Here's my config:

zoo.cfg

tickTime-2000
dataDir=home/mark/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1= hermes:2888:3888
server.2= leela:2888:3888

on the machines hermes and leela I've put myid files in
/home/mark/zookeeper
with the numbers 1 and 2 respectively
-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com]
Sent: Thursday, October 22, 2009 5:43 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Hi Mark -

The Python error relates to not being able to find the zoocfg module - is
zoocfg.py in the same directory as zkconf.py?

Another couple of questions - are you running zookeeper as the same user
who
created myid? Can you post your entire configuration file please - copy and
paste?

Henry

On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


Before I make the Jira, I am trying to go with Ted's advice to use the
python script.

Unfortunately I'm relatively unfamiliar with python so I'm having trouble
running it.

When I execute Python zkconf.py on the command line it tells me:
Traceback (most recent call last):
  File zkconf.py, line 27, in module
   from zoocfg import zoocfg
ImportError: No module named zoocfg

The same error comes when I try to call zkcfg.py from the python

interface

and when I try running
Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
gathered from the Usage). Any suggestions?

Also, I've been using zookeeper 3.2.1
-Original Message-
From: Ted Dunning [mailto:ted.dunn...@gmail.com]
Sent: Thursday, October 22, 2009 4:33 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Try Patrick's utility for creating the config files and compare the

result

to your hand-made files.

On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


The file contains the number 1 and nothing else. My other node has the
number 2 (I only have 2 machines right now, I know it makes more sense

to

run an odd number of zookeeper nodes but I just want to make sure it

works

first). Any suggestions?



--
Ted Dunning, CTO
DeepDyve



Re: Cluster Configuration Issues

2009-10-23 Thread Patrick Hunt

Great to hear. Good luck.

Patrick

Mark Vigeant wrote:

Oh awesome! Thanks Patrick!

Everything works now. Thanks a lot everyone for all your help!

-Mark

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Friday, October 23, 2009 11:58 AM

To: Mark Vigeant
Cc: Patrick Hunt; zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Usually this indicates that either a firewall is blocking one or more of 
the ports, or the server isn't started on all the hosts. First verify 
connectivity, I usually use telnet, see detail here:


https://help.maximumasp.com/KB/a445/connectivity-testing-with-ping-telnet-tracert-and-pathping-.aspx

Patrick

Mark Vigeant wrote:

Ok, so now the clientPort is all set, but I'm running into another problem.

When I try to start up zookeeper server, the error I'm getting now is that it 
can't open the channel to the other machine:

2009-10-223 10:39:11,941 - WARN 
[QuorumPeer:/0:0:0:0:0:0:0:0:2181:quorumcnxmana...@336] - Cannot open channel 
to 2 at
election address leela/10.7.21.216:3888
java.net.ConnectException: Connection refused

I've checked to see if ports 2888 or 3888 are in use on either machine and they are not. Also, there is an open ssh between the 2 
of them. I've tried on a few other ports as well and it's giving me the same issue. Do I need to tamper with port permissions?

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Friday, October 23, 2009 1:23 AM

To: zookeeper-user@hadoop.apache.org; Mark Vigeant
Subject: Re: Cluster Configuration Issues

That usually indicates that you have a process that's already using a 
port that you've specified zookeeper to use. Try something like:


netstat -a -n -p tcp

and look at 2181/2888/3888 on the hosts you are using. Either use a 
different port or stop the process that's currently using.


Patrick

Mark Vigeant wrote:

Haha well I'm an idiot. Thank you.

I would end there but a new error is being thrown:

2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected 
exception, exiting abnormally
Java.net.BindException: Address already in use


-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com] 
Sent: Thursday, October 22, 2009 6:01 PM

To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

yeah - thought this was it: you've missed the forward slash on
home/mark/zookeeper (this turned up on your exception message).

On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


Yeah I just figured out the problem with zoocfg.py

I am running as the same user who created myid. Here's my config:

zoo.cfg

tickTime-2000
dataDir=home/mark/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1= hermes:2888:3888
server.2= leela:2888:3888

on the machines hermes and leela I've put myid files in
/home/mark/zookeeper
with the numbers 1 and 2 respectively
-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com]
Sent: Thursday, October 22, 2009 5:43 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Hi Mark -

The Python error relates to not being able to find the zoocfg module - is
zoocfg.py in the same directory as zkconf.py?

Another couple of questions - are you running zookeeper as the same user
who
created myid? Can you post your entire configuration file please - copy and
paste?

Henry

On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


Before I make the Jira, I am trying to go with Ted's advice to use the
python script.

Unfortunately I'm relatively unfamiliar with python so I'm having trouble
running it.

When I execute Python zkconf.py on the command line it tells me:
Traceback (most recent call last):
  File zkconf.py, line 27, in module
   from zoocfg import zoocfg
ImportError: No module named zoocfg

The same error comes when I try to call zkcfg.py from the python

interface

and when I try running
Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
gathered from the Usage). Any suggestions?

Also, I've been using zookeeper 3.2.1
-Original Message-
From: Ted Dunning [mailto:ted.dunn...@gmail.com]
Sent: Thursday, October 22, 2009 4:33 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Try Patrick's utility for creating the config files and compare the

result

to your hand-made files.

On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


The file contains the number 1 and nothing else. My other node has the
number 2 (I only have 2 machines right now, I know it makes more sense

to

run an odd number of zookeeper nodes but I just want to make sure it

works

first). Any suggestions?


--
Ted Dunning, CTO
DeepDyve



RE: Cluster Configuration Issues

2009-10-22 Thread Mark Vigeant
So I'm trying to get zookeeper to run again, but even though I created a file 
entitled myid and saved in the specified DataDir, the error I get when trying 
to run it is

2009-10-22 15:58:31, 830 FATAL [main:quorumpeerm...@82] - Invalid config, 
exiting abnormally
then it lists the methods/classes affected
Caused by: java.lang.IllegalArgumentException: home/hadoop/zookeeper/myid file 
is missing

The file contains the number 1 and nothing else. My other node has the number 2 
(I only have 2 machines right now, I know it makes more sense to run an odd 
number of zookeeper nodes but I just want to make sure it works first). Any 
suggestions?

-Mark
-Original Message-
From: Mark Vigeant [mailto:mark.vige...@riskmetrics.com] 
Sent: Wednesday, October 21, 2009 9:43 AM
To: zookeeper-user@hadoop.apache.org
Subject: RE: Cluster Configuration Issues

Thank you guys for all of your immediate help!

My machines are currently down for other reasons so I'll be able to try things 
again, send out the errors I'm getting and such in a day or so (hopefully!)

thanks again!

-Mark

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Tuesday, October 20, 2009 1:55 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

You might try my ZooKeeper configuration generator if you have python 
handy: http://bit.ly/mBEcF

The main issue that I see with your config is that each config file 
needs to contain a list of all the servers in the ensemble:

...
syncLimit=2
server.1=host1...
server.2=host2...
server.3=host3...
server.4=host4...

where the myid file in the data dir for each hostX corresponds to it's 
server id (so myid=1 on host1, myid=2 on host2, etc...)

Patrick

Mark Vigeant wrote:
 Hey-
 
 So I'm trying to run hbase on 4 nodes, and in order to do that I need to run 
 zookeeper in replicated mode (I could have hbase run the quorum for me, but 
 it's suggested that I don't).
 
 I have an issue though.  For some reason the id I'm assigning each server in 
 the file myid in the assigned data directory is not getting read. I feel 
 like another id is being created and put somewhere else. Does anyone have any 
 tips on starting a zookeeper quorum? Do I create the myid file myself or do I 
 edit one once it is created by zookeeper?
 
 This is what my  config looks like:
 ticktime=2000
 dataDir=/home/hadoop/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1=hadoop1:2888:3888
 
 The name of my machine is hadoop1, with user name hadoop. In 
 /home/hadoop/zookeeper I've created a myid file with the number 1 in it.
 
 Mark Vigeant
 RiskMetrics Group, Inc.
 
 


Re: Cluster Configuration Issues

2009-10-22 Thread Patrick Hunt
I would suggest that you create a jira with the config files and your 
data directories attached as a compressed archive. We can take a look 
and pinpoint the issue. Also detail the release and command line you are 
using to start the cluster.


https://issues.apache.org/jira/browse/ZOOKEEPER

Patrick

Mark Vigeant wrote:

So I'm trying to get zookeeper to run again, but even though I created a file entitled 
myid and saved in the specified DataDir, the error I get when trying to run 
it is

2009-10-22 15:58:31, 830 FATAL [main:quorumpeerm...@82] - Invalid config, 
exiting abnormally
then it lists the methods/classes affected
Caused by: java.lang.IllegalArgumentException: home/hadoop/zookeeper/myid file 
is missing

The file contains the number 1 and nothing else. My other node has the number 2 
(I only have 2 machines right now, I know it makes more sense to run an odd 
number of zookeeper nodes but I just want to make sure it works first). Any 
suggestions?

-Mark
-Original Message-
From: Mark Vigeant [mailto:mark.vige...@riskmetrics.com] 
Sent: Wednesday, October 21, 2009 9:43 AM

To: zookeeper-user@hadoop.apache.org
Subject: RE: Cluster Configuration Issues

Thank you guys for all of your immediate help!

My machines are currently down for other reasons so I'll be able to try things 
again, send out the errors I'm getting and such in a day or so (hopefully!)

thanks again!

-Mark

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Tuesday, October 20, 2009 1:55 PM

To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

You might try my ZooKeeper configuration generator if you have python 
handy: http://bit.ly/mBEcF


The main issue that I see with your config is that each config file 
needs to contain a list of all the servers in the ensemble:


...
syncLimit=2
server.1=host1...
server.2=host2...
server.3=host3...
server.4=host4...

where the myid file in the data dir for each hostX corresponds to it's 
server id (so myid=1 on host1, myid=2 on host2, etc...)


Patrick

Mark Vigeant wrote:

Hey-

So I'm trying to run hbase on 4 nodes, and in order to do that I need to run 
zookeeper in replicated mode (I could have hbase run the quorum for me, but 
it's suggested that I don't).

I have an issue though.  For some reason the id I'm assigning each server in the file 
myid in the assigned data directory is not getting read. I feel like another 
id is being created and put somewhere else. Does anyone have any tips on starting a 
zookeeper quorum? Do I create the myid file myself or do I edit one once it is created by 
zookeeper?

This is what my  config looks like:
ticktime=2000
dataDir=/home/hadoop/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=hadoop1:2888:3888

The name of my machine is hadoop1, with user name hadoop. In 
/home/hadoop/zookeeper I've created a myid file with the number 1 in it.

Mark Vigeant
RiskMetrics Group, Inc.




Re: Cluster Configuration Issues

2009-10-22 Thread Ted Dunning
Try Patrick's utility for creating the config files and compare the result
to your hand-made files.

On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:

 The file contains the number 1 and nothing else. My other node has the
 number 2 (I only have 2 machines right now, I know it makes more sense to
 run an odd number of zookeeper nodes but I just want to make sure it works
 first). Any suggestions?




-- 
Ted Dunning, CTO
DeepDyve


Re: Cluster Configuration Issues

2009-10-22 Thread Henry Robinson
Hi Mark -

The Python error relates to not being able to find the zoocfg module - is
zoocfg.py in the same directory as zkconf.py?

Another couple of questions - are you running zookeeper as the same user who
created myid? Can you post your entire configuration file please - copy and
paste?

Henry

On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:

 Before I make the Jira, I am trying to go with Ted's advice to use the
 python script.

 Unfortunately I'm relatively unfamiliar with python so I'm having trouble
 running it.

 When I execute Python zkconf.py on the command line it tells me:
 Traceback (most recent call last):
   File zkconf.py, line 27, in module
from zoocfg import zoocfg
 ImportError: No module named zoocfg

 The same error comes when I try to call zkcfg.py from the python interface
 and when I try running
 Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
 gathered from the Usage). Any suggestions?

 Also, I've been using zookeeper 3.2.1
 -Original Message-
 From: Ted Dunning [mailto:ted.dunn...@gmail.com]
 Sent: Thursday, October 22, 2009 4:33 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues

 Try Patrick's utility for creating the config files and compare the result
 to your hand-made files.

 On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:

  The file contains the number 1 and nothing else. My other node has the
  number 2 (I only have 2 machines right now, I know it makes more sense to
  run an odd number of zookeeper nodes but I just want to make sure it
 works
  first). Any suggestions?
 



 --
 Ted Dunning, CTO
 DeepDyve



Re: Cluster Configuration Issues

2009-10-22 Thread Patrick Hunt
bummer, donno what to say, what version of python are you using? works 
for me in 2.5 and 2.6 (linux jaunty)


$ ls
LICENSE.txt  README.html start.py  start.pyc   stop.py 
stop.pyc   test3  zoocfg.py  zoocfg.pyc
ptestREADME.textile  start.py.bak  start.tmpl  stop.py.bak 
stop.tmpl  zkconf.py  zoocfg.py.bak  zoocfg.tmpl



$ python2.5 zkconf.py ../t/zookeeper-3.2.1 ptest
$ ls ptest/
cli.sh  localhost:2181  localhost:2182  localhost:2183  log4j.jar 
log4j.properties  start.sh  status.sh  stop.sh  zookeeper.jar

$ rm -fr ptest
$ python2.6 zkconf.py ../t/zookeeper-3.2.1 ptest
$ ls ptest/
cli.sh  localhost:2181  localhost:2182  localhost:2183  log4j.jar 
log4j.properties  start.sh  status.sh  stop.sh  zookeeper.jar



Perhaps you can try something like:

PYTHONPATH=. python zkconf.py ../t/zookeeper-3.2.1 ptest

that help at all?

Patrick

Mark Vigeant wrote:
Before I make the Jira, I am trying to go with Ted's advice to use the python script. 


Unfortunately I'm relatively unfamiliar with python so I'm having trouble 
running it.

When I execute Python zkconf.py on the command line it tells me:
Traceback (most recent call last):
   File zkconf.py, line 27, in module
from zoocfg import zoocfg
ImportError: No module named zoocfg

The same error comes when I try to call zkcfg.py from the python interface and 
when I try running
Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I 
gathered from the Usage). Any suggestions?

Also, I've been using zookeeper 3.2.1
-Original Message-
From: Ted Dunning [mailto:ted.dunn...@gmail.com] 
Sent: Thursday, October 22, 2009 4:33 PM

To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Try Patrick's utility for creating the config files and compare the result
to your hand-made files.

On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:


The file contains the number 1 and nothing else. My other node has the
number 2 (I only have 2 machines right now, I know it makes more sense to
run an odd number of zookeeper nodes but I just want to make sure it works
first). Any suggestions?







RE: Cluster Configuration Issues

2009-10-22 Thread Mark Vigeant
Yeah I just figured out the problem with zoocfg.py

I am running as the same user who created myid. Here's my config:

zoo.cfg

tickTime-2000
dataDir=home/mark/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1= hermes:2888:3888
server.2= leela:2888:3888

on the machines hermes and leela I've put myid files in /home/mark/zookeeper
with the numbers 1 and 2 respectively
-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com] 
Sent: Thursday, October 22, 2009 5:43 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

Hi Mark -

The Python error relates to not being able to find the zoocfg module - is
zoocfg.py in the same directory as zkconf.py?

Another couple of questions - are you running zookeeper as the same user who
created myid? Can you post your entire configuration file please - copy and
paste?

Henry

On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:

 Before I make the Jira, I am trying to go with Ted's advice to use the
 python script.

 Unfortunately I'm relatively unfamiliar with python so I'm having trouble
 running it.

 When I execute Python zkconf.py on the command line it tells me:
 Traceback (most recent call last):
   File zkconf.py, line 27, in module
from zoocfg import zoocfg
 ImportError: No module named zoocfg

 The same error comes when I try to call zkcfg.py from the python interface
 and when I try running
 Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
 gathered from the Usage). Any suggestions?

 Also, I've been using zookeeper 3.2.1
 -Original Message-
 From: Ted Dunning [mailto:ted.dunn...@gmail.com]
 Sent: Thursday, October 22, 2009 4:33 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues

 Try Patrick's utility for creating the config files and compare the result
 to your hand-made files.

 On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:

  The file contains the number 1 and nothing else. My other node has the
  number 2 (I only have 2 machines right now, I know it makes more sense to
  run an odd number of zookeeper nodes but I just want to make sure it
 works
  first). Any suggestions?
 



 --
 Ted Dunning, CTO
 DeepDyve



Re: Cluster Configuration Issues

2009-10-22 Thread Leonard Cuff
Missing slash on the front of home/mark/zookeeper


On 10/22/09 2:55 PM, Mark Vigeant mark.vige...@riskmetrics.com wrote:

 Yeah I just figured out the problem with zoocfg.py
 
 I am running as the same user who created myid. Here's my config:
 
 zoo.cfg
 
 tickTime-2000
 dataDir=home/mark/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1= hermes:2888:3888
 server.2= leela:2888:3888
 
 on the machines hermes and leela I've put myid files in /home/mark/zookeeper
 with the numbers 1 and 2 respectively
 -Original Message-
 From: Henry Robinson [mailto:he...@cloudera.com]
 Sent: Thursday, October 22, 2009 5:43 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues
 
 Hi Mark -
 
 The Python error relates to not being able to find the zoocfg module - is
 zoocfg.py in the same directory as zkconf.py?
 
 Another couple of questions - are you running zookeeper as the same user who
 created myid? Can you post your entire configuration file please - copy and
 paste?
 
 Henry
 
 On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:
 
 Before I make the Jira, I am trying to go with Ted's advice to use the
 python script.
 
 Unfortunately I'm relatively unfamiliar with python so I'm having trouble
 running it.
 
 When I execute Python zkconf.py on the command line it tells me:
 Traceback (most recent call last):
   File zkconf.py, line 27, in module
from zoocfg import zoocfg
 ImportError: No module named zoocfg
 
 The same error comes when I try to call zkcfg.py from the python interface
 and when I try running
 Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
 gathered from the Usage). Any suggestions?
 
 Also, I've been using zookeeper 3.2.1
 -Original Message-
 From: Ted Dunning [mailto:ted.dunn...@gmail.com]
 Sent: Thursday, October 22, 2009 4:33 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues
 
 Try Patrick's utility for creating the config files and compare the result
 to your hand-made files.
 
 On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:
 
 The file contains the number 1 and nothing else. My other node has the
 number 2 (I only have 2 machines right now, I know it makes more sense to
 run an odd number of zookeeper nodes but I just want to make sure it
 works
 first). Any suggestions?
 
 
 
 
 --
 Ted Dunning, CTO
 DeepDyve
 

-- 

Leonard Cuff
lc...@valueclick.com

³This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use of the
intended recipient(s).  Any disclosure, copying, distribution, posting, or
use of the information contained in or attached to this email is prohibited
unless permitted by the sender.  If you have received this email in error,
please immediately notify the sender via return e-mail, telephone, or fax
and destroy this original transmission and its included files without
reading or saving it in any manner. Thank you.²






This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use
of the intended recipient(s).  Any disclosure, copying, distribution,
posting, or use of the information contained in or attached to this
email is prohibited unless permitted by the sender.  If you have
received this email in error, please immediately notify the sender
via return email, telephone, or fax and destroy this original transmission
and its included files without reading or saving it in any manner.
Thank you.



Re: Cluster Configuration Issues

2009-10-22 Thread Henry Robinson
yeah - thought this was it: you've missed the forward slash on
home/mark/zookeeper (this turned up on your exception message).

On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:

 Yeah I just figured out the problem with zoocfg.py

 I am running as the same user who created myid. Here's my config:

 zoo.cfg

 tickTime-2000
 dataDir=home/mark/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1= hermes:2888:3888
 server.2= leela:2888:3888

 on the machines hermes and leela I've put myid files in
 /home/mark/zookeeper
 with the numbers 1 and 2 respectively
 -Original Message-
 From: Henry Robinson [mailto:he...@cloudera.com]
 Sent: Thursday, October 22, 2009 5:43 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues

 Hi Mark -

 The Python error relates to not being able to find the zoocfg module - is
 zoocfg.py in the same directory as zkconf.py?

 Another couple of questions - are you running zookeeper as the same user
 who
 created myid? Can you post your entire configuration file please - copy and
 paste?

 Henry

 On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:

  Before I make the Jira, I am trying to go with Ted's advice to use the
  python script.
 
  Unfortunately I'm relatively unfamiliar with python so I'm having trouble
  running it.
 
  When I execute Python zkconf.py on the command line it tells me:
  Traceback (most recent call last):
File zkconf.py, line 27, in module
 from zoocfg import zoocfg
  ImportError: No module named zoocfg
 
  The same error comes when I try to call zkcfg.py from the python
 interface
  and when I try running
  Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
  gathered from the Usage). Any suggestions?
 
  Also, I've been using zookeeper 3.2.1
  -Original Message-
  From: Ted Dunning [mailto:ted.dunn...@gmail.com]
  Sent: Thursday, October 22, 2009 4:33 PM
  To: zookeeper-user@hadoop.apache.org
  Subject: Re: Cluster Configuration Issues
 
  Try Patrick's utility for creating the config files and compare the
 result
  to your hand-made files.
 
  On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
  mark.vige...@riskmetrics.comwrote:
 
   The file contains the number 1 and nothing else. My other node has the
   number 2 (I only have 2 machines right now, I know it makes more sense
 to
   run an odd number of zookeeper nodes but I just want to make sure it
  works
   first). Any suggestions?
  
 
 
 
  --
  Ted Dunning, CTO
  DeepDyve
 



RE: Cluster Configuration Issues

2009-10-22 Thread Mark Vigeant
Haha well I'm an idiot. Thank you.

I would end there but a new error is being thrown:

2009-10-22 18:00:17,178 - FATAL [main:quorumpeerm...@86] - Unexpected 
exception, exiting abnormally
Java.net.BindException: Address already in use


-Original Message-
From: Henry Robinson [mailto:he...@cloudera.com] 
Sent: Thursday, October 22, 2009 6:01 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

yeah - thought this was it: you've missed the forward slash on
home/mark/zookeeper (this turned up on your exception message).

On Thu, Oct 22, 2009 at 2:55 PM, Mark Vigeant
mark.vige...@riskmetrics.comwrote:

 Yeah I just figured out the problem with zoocfg.py

 I am running as the same user who created myid. Here's my config:

 zoo.cfg

 tickTime-2000
 dataDir=home/mark/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1= hermes:2888:3888
 server.2= leela:2888:3888

 on the machines hermes and leela I've put myid files in
 /home/mark/zookeeper
 with the numbers 1 and 2 respectively
 -Original Message-
 From: Henry Robinson [mailto:he...@cloudera.com]
 Sent: Thursday, October 22, 2009 5:43 PM
 To: zookeeper-user@hadoop.apache.org
 Subject: Re: Cluster Configuration Issues

 Hi Mark -

 The Python error relates to not being able to find the zoocfg module - is
 zoocfg.py in the same directory as zkconf.py?

 Another couple of questions - are you running zookeeper as the same user
 who
 created myid? Can you post your entire configuration file please - copy and
 paste?

 Henry

 On Thu, Oct 22, 2009 at 2:32 PM, Mark Vigeant
 mark.vige...@riskmetrics.comwrote:

  Before I make the Jira, I am trying to go with Ted's advice to use the
  python script.
 
  Unfortunately I'm relatively unfamiliar with python so I'm having trouble
  running it.
 
  When I execute Python zkconf.py on the command line it tells me:
  Traceback (most recent call last):
File zkconf.py, line 27, in module
 from zoocfg import zoocfg
  ImportError: No module named zoocfg
 
  The same error comes when I try to call zkcfg.py from the python
 interface
  and when I try running
  Python zkconf.py -help /home/hadoop/zookeeper-3.2.1/ /home/hadoop (as I
  gathered from the Usage). Any suggestions?
 
  Also, I've been using zookeeper 3.2.1
  -Original Message-
  From: Ted Dunning [mailto:ted.dunn...@gmail.com]
  Sent: Thursday, October 22, 2009 4:33 PM
  To: zookeeper-user@hadoop.apache.org
  Subject: Re: Cluster Configuration Issues
 
  Try Patrick's utility for creating the config files and compare the
 result
  to your hand-made files.
 
  On Thu, Oct 22, 2009 at 1:04 PM, Mark Vigeant
  mark.vige...@riskmetrics.comwrote:
 
   The file contains the number 1 and nothing else. My other node has the
   number 2 (I only have 2 machines right now, I know it makes more sense
 to
   run an odd number of zookeeper nodes but I just want to make sure it
  works
   first). Any suggestions?
  
 
 
 
  --
  Ted Dunning, CTO
  DeepDyve
 



RE: Cluster Configuration Issues

2009-10-21 Thread Mark Vigeant
Thank you guys for all of your immediate help!

My machines are currently down for other reasons so I'll be able to try things 
again, send out the errors I'm getting and such in a day or so (hopefully!)

thanks again!

-Mark

-Original Message-
From: Patrick Hunt [mailto:ph...@apache.org] 
Sent: Tuesday, October 20, 2009 1:55 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Cluster Configuration Issues

You might try my ZooKeeper configuration generator if you have python 
handy: http://bit.ly/mBEcF

The main issue that I see with your config is that each config file 
needs to contain a list of all the servers in the ensemble:

...
syncLimit=2
server.1=host1...
server.2=host2...
server.3=host3...
server.4=host4...

where the myid file in the data dir for each hostX corresponds to it's 
server id (so myid=1 on host1, myid=2 on host2, etc...)

Patrick

Mark Vigeant wrote:
 Hey-
 
 So I'm trying to run hbase on 4 nodes, and in order to do that I need to run 
 zookeeper in replicated mode (I could have hbase run the quorum for me, but 
 it's suggested that I don't).
 
 I have an issue though.  For some reason the id I'm assigning each server in 
 the file myid in the assigned data directory is not getting read. I feel 
 like another id is being created and put somewhere else. Does anyone have any 
 tips on starting a zookeeper quorum? Do I create the myid file myself or do I 
 edit one once it is created by zookeeper?
 
 This is what my  config looks like:
 ticktime=2000
 dataDir=/home/hadoop/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1=hadoop1:2888:3888
 
 The name of my machine is hadoop1, with user name hadoop. In 
 /home/hadoop/zookeeper I've created a myid file with the number 1 in it.
 
 Mark Vigeant
 RiskMetrics Group, Inc.
 
 


Re: Cluster Configuration Issues

2009-10-20 Thread Henry Robinson
Hi Mark -

You should create the myid file yourself, as you have done. What errors are
you seeing that lead you to think the id is not being read correctly?

cheers,
Henry

On Tue, Oct 20, 2009 at 10:12 AM, Mark Vigeant mark.vige...@riskmetrics.com
 wrote:

 Hey-

 So I'm trying to run hbase on 4 nodes, and in order to do that I need to
 run zookeeper in replicated mode (I could have hbase run the quorum for me,
 but it's suggested that I don't).

 I have an issue though.  For some reason the id I'm assigning each server
 in the file myid in the assigned data directory is not getting read. I
 feel like another id is being created and put somewhere else. Does anyone
 have any tips on starting a zookeeper quorum? Do I create the myid file
 myself or do I edit one once it is created by zookeeper?

 This is what my  config looks like:
 ticktime=2000
 dataDir=/home/hadoop/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1=hadoop1:2888:3888

 The name of my machine is hadoop1, with user name hadoop. In
 /home/hadoop/zookeeper I've created a myid file with the number 1 in it.

 Mark Vigeant
 RiskMetrics Group, Inc.




Re: Cluster Configuration Issues

2009-10-20 Thread Mahadev Konar
HI Mark,
 ZooKeeper does not create the myid file in the data directory.

Looking at the config file it looks like it is missing the quorum
configuration for other servers.

Please take alook at

http://hadoop.apache.org/zookeeper/docs/r3.2.1/zookeeperAdmin.html#sc_zkMuli
tServerSetup


You will need to add config options for other servers in the quorum in the
config file.


Thanks
mahadev


On 10/20/09 10:12 AM, Mark Vigeant mark.vige...@riskmetrics.com wrote:

 Hey-
 
 So I'm trying to run hbase on 4 nodes, and in order to do that I need to run
 zookeeper in replicated mode (I could have hbase run the quorum for me, but
 it's suggested that I don't).
 
 I have an issue though.  For some reason the id I'm assigning each server in
 the file myid in the assigned data directory is not getting read. I feel
 like another id is being created and put somewhere else. Does anyone have any
 tips on starting a zookeeper quorum? Do I create the myid file myself or do I
 edit one once it is created by zookeeper?
 
 This is what my  config looks like:
 ticktime=2000
 dataDir=/home/hadoop/zookeeper
 clientPort=2181
 initLimit=5
 syncLimit=2
 server.1=hadoop1:2888:3888
 
 The name of my machine is hadoop1, with user name hadoop. In
 /home/hadoop/zookeeper I've created a myid file with the number 1 in it.
 
 Mark Vigeant
 RiskMetrics Group, Inc.
 



Re: Cluster Configuration Issues

2009-10-20 Thread Patrick Hunt
You might try my ZooKeeper configuration generator if you have python 
handy: http://bit.ly/mBEcF


The main issue that I see with your config is that each config file 
needs to contain a list of all the servers in the ensemble:


...
syncLimit=2
server.1=host1...
server.2=host2...
server.3=host3...
server.4=host4...

where the myid file in the data dir for each hostX corresponds to it's 
server id (so myid=1 on host1, myid=2 on host2, etc...)


Patrick

Mark Vigeant wrote:

Hey-

So I'm trying to run hbase on 4 nodes, and in order to do that I need to run 
zookeeper in replicated mode (I could have hbase run the quorum for me, but 
it's suggested that I don't).

I have an issue though.  For some reason the id I'm assigning each server in the file 
myid in the assigned data directory is not getting read. I feel like another 
id is being created and put somewhere else. Does anyone have any tips on starting a 
zookeeper quorum? Do I create the myid file myself or do I edit one once it is created by 
zookeeper?

This is what my  config looks like:
ticktime=2000
dataDir=/home/hadoop/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=hadoop1:2888:3888

The name of my machine is hadoop1, with user name hadoop. In 
/home/hadoop/zookeeper I've created a myid file with the number 1 in it.

Mark Vigeant
RiskMetrics Group, Inc.