Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Ted Dunning
I have used 5 and 3 in different clusters. Moderate amounts of sharing is reasonable, but sharing with less intensive applications is definitely better. Sharing with the job tracker, for instance is likely fine since it doesn't abuse disk so much. The namenode is similar, but not quite as nice.

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
On 03/08/2010 02:21 PM, Patrick Hunt wrote: See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency requirem

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Patrick Hunt
See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency requirements it should be fine. If the machine wer

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Mahadev Konar
Hi David, Sharing the cluster with HDFS and Map reduce might cause significant problems. Mapreduce is very IO intensive and this might cause lot of unnecessary hiccups in your cluster. I would suggest atleast providing something like this, if you really want to share the nodes. - atleast conside

Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
I'm contemplating an upcoming zookeeper rollout and was wondering what the zookeeper brain trust here thought about a network deployment question: Is it generally considered bad practice to just deploy zookeeper on our existing hdfs/MR nodes? Or is it better to run zookeeper instances on thei