Zookeeper behind load balancer

2010-04-14 Thread Toader, Sebastian
Hi ZooKeeper Dev Team,

We are thinking of putting our ZooKeeper server behind load balancer. The 
client applications would connect to assemble through the load balancer.

Can you let us know if there are any drawbacks in putting the assemble behind a 
load balancer?

Thank you,
Seb

--
NOTICE: If received in error, please destroy, and notify sender. Sender does 
not intend to waive confidentiality or privilege. Use of this email is 
prohibited when received in error. We may monitor and store emails to the 
extent permitted by applicable law.


Re: Zookeeper behind load balancer

2010-04-14 Thread Flavio Junqueira
Hi Seb, I was wondering how you're thinking about performing session  
management interposing a load balancer between clients and zookeeper  
servers. I'm probably missing some important piece of your design.


Thanks,
-Flavio


On Apr 14, 2010, at 3:48 PM, Toader, Sebastian wrote:


Hi ZooKeeper Dev Team,

We are thinking of putting our ZooKeeper server behind load  
balancer. The client applications would connect to assemble through  
the load balancer.


Can you let us know if there are any drawbacks in putting the  
assemble behind a load balancer?


Thank you,
Seb

--
NOTICE: If received in error, please destroy, and notify sender.  
Sender does not intend to waive confidentiality or privilege. Use of  
this email is prohibited when received in error. We may monitor and  
store emails to the extent permitted by applicable law.




Re: Zookeeper behind load balancer

2010-04-14 Thread Patrick Hunt

Two main issues I can think of:

1) unnecessary -- clients already randomize their connect string to 
balance the sessions across servers
2) unless you turn on some sort of session tracking in the ballancer 
client's won't be able to use the service. ZK sessions use long lived 
TCP connections to maintain a session with a particular server.


See session docs for more detail. http://bit.ly/db90Y5

Regards,

Patrick

On 04/14/2010 06:48 AM, Toader, Sebastian wrote:

Hi ZooKeeper Dev Team,

We are thinking of putting our ZooKeeper server behind load balancer. The 
client applications would connect to assemble through the load balancer.

Can you let us know if there are any drawbacks in putting the assemble behind a 
load balancer?

Thank you,
Seb

--
NOTICE: If received in error, please destroy, and notify sender. Sender does 
not intend to waive confidentiality or privilege. Use of this email is 
prohibited when received in error. We may monitor and store emails to the 
extent permitted by applicable law.



Re: Zookeeper behind load balancer

2010-04-14 Thread Patrick Hunt


On 04/14/2010 09:33 AM, Mekaraj, Prashant wrote:

Many thanks for the quick reply.


1) unnecessary -- clients already randomize their connect string
to balance the sessions across servers


The problems we are trying to solve are 1. updating the ensemble to
add new servers implies finding an updating every client who is
connecting to it(which implies we know who the clients are)


That's true. However the only real reason to change the ensemble size is
to increase reliability. We typically suggest 5 servers for on-line
production serving environments. This provides high reliability and also
allows for things like bringing down a server for maint, but still being
able to handle the case where another server may fail unexpectedly (but
the service will still stay up). Over 5 servers there is no significant
benefit to reliability, only the negative performance impact of having a
larger quorum.


2. if we deploy zookeeper in a region and put observers in other
regions, load balancers make it easy to preferably connect to a local
regional ip and failover to a remote one if the local ip is
unavailable


Yes, this is a good point. Also if you change the name/ip of a server
(say one fails and you need to replace)


3. load balancers offer more sophisticated balancing algo's - load
based for example.


2) unless you turn on some sort of session tracking in the
ballancer client's won't be able to use the service. ZK sessions
use long lived TCP connections to maintain a session with a
particular server.


The load balancer tracks sessions and makes sure that a client will
stick to a particular backend zookeeper server.


This is a trick question. ;-) One additional thing you would need to
ensure is that if the zk server fails the load balancer will take it out
of rotation.

I don't have any experience using LBs for this. It should be easy enough 
for you to test in your environment/lbs though. I'd be interested to 
know you results - if possible please post your findings!


Regards,

Patrick


Flavio,

Hi Seb, I was wondering how you're thinking about performing
session management interposing a load balancer between clients
and zookeeper servers. I'm probably missing some important piece of
your design.


Load balancer stickiness should handle it if I understand your point
correctly.


-prashant





See session docs for more detail. http://bit.ly/db90Y5

Regards,

Patrick

On 04/14/2010 06:48 AM, Toader, Sebastian wrote:

Hi ZooKeeper Dev Team,

We are thinking of putting our ZooKeeper server behind load
balancer.

The client applications would connect to assemble through the load
balancer.


Can you let us know if there are any drawbacks in putting the

assemble behind a load balancer?


Thank you, Seb

-





-

NOTICE: If received in error, please destroy, and notify sender.

Sender does not intend to waive confidentiality or privilege. Use
of this email is prohibited when received in error. We may monitor
and store emails to the extent permitted by applicable law.




--


NOTICE: If received in error, please destroy, and notify sender. Sender 
does not intend to waive confidentiality or privilege. Use of this email 
is prohibited when received in error. We may monitor and store emails to 
the extent permitted by applicable law.


Re: Zookeeper behind load balancer

2010-04-14 Thread Patrick Hunt


On 04/14/2010 10:09 AM, Mekaraj, Prashant wrote:

We did test this and it seems to work fine, but ymmv depending on the
load balancer product I suppose.


Yes, that's def true. Please highlight this in your results writeup. If 
you can provide the name, config, etc.. of the lb you used that would be 
helpful.



We've done some tests and can post results. Do we just file a jira with the 
text of the findings ?


Awesome. If you don't have any issues (bugs/features) I'd suggest you 
put it on the wiki directly, something like:


http://wiki.apache.org/hadoop/ZooKeeper/LoadBalancer

Feel free to put that on the ZooKeeper main wiki page.

You might also consider doing a blog post about this. It would be 
interesting for our users!


Regards,

Patrick



-prashant



Regards,

Patrick


Flavio,

Hi Seb, I was wondering how you're thinking about performing
session management interposing a load balancer between clients
and zookeeper servers. I'm probably missing some important piece of
your design.


Load balancer stickiness should handle it if I understand your point
correctly.


-prashant





See session docs for more detail. http://bit.ly/db90Y5

Regards,

Patrick

On 04/14/2010 06:48 AM, Toader, Sebastian wrote:

Hi ZooKeeper Dev Team,

We are thinking of putting our ZooKeeper server behind load
balancer.

The client applications would connect to assemble through the load
balancer.


Can you let us know if there are any drawbacks in putting the

assemble behind a load balancer?


Thank you, Seb

---

--





-

NOTICE: If received in error, please destroy, and notify sender.

Sender does not intend to waive confidentiality or privilege. Use
of this email is prohibited when received in error. We may monitor
and store emails to the extent permitted by applicable law.




-

-




NOTICE: If received in error, please destroy, and notify sender. Sender
does not intend to waive confidentiality or privilege. Use of this
email
is prohibited when received in error. We may monitor and store emails
to
the extent permitted by applicable law.


--
NOTICE: If received in error, please destroy, and notify sender. Sender does 
not intend to waive confidentiality or privilege. Use of this email is 
prohibited when received in error. We may monitor and store emails to the 
extent permitted by applicable law.


Re: Zookeeper behind load balancer

2010-04-14 Thread Mahadev Konar
Hi Prashant,
  Just one more thing. It would be great if you could update the powered by
page at:

http://wiki.apache.org/hadoop/ZooKeeper/PoweredBy


With your usage model as well once you start using ZooKeeper.


Thanks
mahadev


On 4/14/10 10:38 AM, Patrick Hunt ph...@apache.org wrote:

 
 On 04/14/2010 10:09 AM, Mekaraj, Prashant wrote:
 We did test this and it seems to work fine, but ymmv depending on the
 load balancer product I suppose.
 
 Yes, that's def true. Please highlight this in your results writeup. If
 you can provide the name, config, etc.. of the lb you used that would be
 helpful.
 
 We've done some tests and can post results. Do we just file a jira with the
 text of the findings ?
 
 Awesome. If you don't have any issues (bugs/features) I'd suggest you
 put it on the wiki directly, something like:
 
 http://wiki.apache.org/hadoop/ZooKeeper/LoadBalancer
 
 Feel free to put that on the ZooKeeper main wiki page.
 
 You might also consider doing a blog post about this. It would be
 interesting for our users!
 
 Regards,
 
 Patrick
 
 
 -prashant
 
 
 Regards,
 
 Patrick
 
 Flavio,
 Hi Seb, I was wondering how you're thinking about performing
 session management interposing a load balancer between clients
 and zookeeper servers. I'm probably missing some important piece of
 your design.
 
 Load balancer stickiness should handle it if I understand your point
 correctly.
 
 
 -prashant
 
 
 
 
 See session docs for more detail. http://bit.ly/db90Y5
 
 Regards,
 
 Patrick
 
 On 04/14/2010 06:48 AM, Toader, Sebastian wrote:
 Hi ZooKeeper Dev Team,
 
 We are thinking of putting our ZooKeeper server behind load
 balancer.
 The client applications would connect to assemble through the load
 balancer.
 
 Can you let us know if there are any drawbacks in putting the
 assemble behind a load balancer?
 
 Thank you, Seb
 
 ---
 --
 
 
 -
 NOTICE: If received in error, please destroy, and notify sender.
 Sender does not intend to waive confidentiality or privilege. Use
 of this email is prohibited when received in error. We may monitor
 and store emails to the extent permitted by applicable law.
 
 
 -
 -
 
 
 NOTICE: If received in error, please destroy, and notify sender. Sender
 does not intend to waive confidentiality or privilege. Use of this
 email
 is prohibited when received in error. We may monitor and store emails
 to
 the extent permitted by applicable law.
 
 --
 NOTICE: If received in error, please destroy, and notify sender. Sender does
 not intend to waive confidentiality or privilege. Use of this email is
 prohibited when received in error. We may monitor and store emails to the
 extent permitted by applicable law.