Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
I'm contemplating an upcoming zookeeper rollout and was wondering what the zookeeper brain trust here thought about a network deployment question: Is it generally considered bad practice to just deploy zookeeper on our existing hdfs/MR nodes? Or is it better to run zookeeper instances on

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Patrick Hunt
See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency requirements it should be fine. If the machine

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
On 03/08/2010 02:21 PM, Patrick Hunt wrote: See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Ted Dunning
I have used 5 and 3 in different clusters. Moderate amounts of sharing is reasonable, but sharing with less intensive applications is definitely better. Sharing with the job tracker, for instance is likely fine since it doesn't abuse disk so much. The namenode is similar, but not quite as nice.