problems on EC2?

2009-04-14 Thread Ted Dunning
We have been using EC2 as a substrate for our search cluster with zookeeper as our coordination layer and have been seeing some strange problems. These problems seem to manifest around getting lots of anomalous disconnects and session expirations even though we have the timeout values set to 2 sec

Re: problems on EC2?

2009-04-14 Thread Nitay
Hi Ted, Fellow user coming from HBase. We were recently seeing lots of SessionExpired events as well. Check out this mail thread: http://markmail.org/search/?q=SessionExpired#query:SessionExpired+page:1+mid:gt4c2kn4n4f5s5kw+state:results Perhaps this might have something to do with what you're s

Re: problems on EC2?

2009-04-14 Thread Ted Dunning
Very good pointer. Thanks. Are you still having your problems? On Tue, Apr 14, 2009 at 6:09 PM, Nitay wrote: > Hi Ted, > > Fellow user coming from HBase. We were recently seeing lots of > SessionExpired events as well. Check out this mail thread: > > > http://markmail.org/search/?q=SessionExpi

Re: problems on EC2?

2009-04-14 Thread Nitay
Yes, we are. We currently don't handle SessionExpired very well at all in HBase. There are two things going on in parallel to fix it: 1) Reinitialize the ZooKeeper handler (and everything else that depends on it) on the node in question when a SessionExpired event occurs. 2) Reduce the number of S

Re: problems on EC2?

2009-04-14 Thread Mahadev Konar
Hi Ted, > These problems seem to manifest around getting lots of anomalous disconnects > and session expirations even though we have the timeout values set to 2 > seconds on the server side and 5 seconds on the client side. > Your scenario might be a little differetn from what Nitay (Hbase) is s