Xid out of order. Got 8 expected 7

2010-05-12 Thread Jordan Zimmerman
We've just started seeing an odd error and are having trouble determining the cause. "Xid out of order. Got 8 expected 7" Any hints on what can cause this? Any ideas on how to debug? We're using ZK 3.3.0. The error occurs in ClientCnxn.java line 781 -Jordan

Re: Xid out of order. Got 8 expected 7

2010-05-12 Thread Jordan Zimmerman
ers? > > > Thanks > mahadev > > > On 5/12/10 10:49 AM, "Jordan Zimmerman" wrote: > >> We've just started seeing an odd error and are having trouble determining the >> cause. >> "Xid out of order. Got 8 expected 7" >

Re: Xid out of order. Got 8 expected 7

2010-05-12 Thread Jordan Zimmerman
? (having the server + client > logs will help alot) > > Patrick > > On 05/12/2010 11:08 AM, Jordan Zimmerman wrote: >> Sure - if you think it's a bug. >> >> We were using Zookeeper without issue. I then refactored a bunch of >> code and this new behavi

Re: Xid out of order. Got 8 expected 7

2010-05-12 Thread Jordan Zimmerman
int the client notices that >> getChildren (xid 7) got lost. >> >> Patrick >> >> On 05/12/2010 11:30 AM, Jordan Zimmerman wrote: >> >>> Oh, OK. When I get a moment I'll restart the 3.2.2 and post logs, >>> etc. >>>

Re: Xid out of order. Got 8 expected 7

2010-05-12 Thread Jordan Zimmerman
type. Then the server responds to the client for a >>> subsequent operation (xid 8), and at that point the client notices that >>> getChildren (xid 7) got lost. >>> >>> Patrick >>> >>> On 05/12/2010 11:30 AM, Jordan Zimmerman wrote:

Debugging help for SessionExpiredException

2010-06-09 Thread Jordan Zimmerman
We have a test system using Zookeeper. There is a single Zookeeper server node and 4 clients. There is very little activity in this system. After a day's testing we start to see SessionExpiredException on the client. Things I've tried: * Increasing the session timeout to 1 minute * Making sure

Re: Debugging help for SessionExpiredException

2010-06-10 Thread Jordan Zimmerman
>-XX:CMSInitiatingOccupancyFraction=60 >-XX:+UseCMSInitiatingOccupancyOnly > > Finally, you should always add options for lots of GC diagnostics: > >-XX:+PrintGCDetails >-XX:+PrintGCTimeStamps >-XX:+PrintTenuringDistribution > > On Thu, Jun 10, 2010 at

Re: Debugging help for SessionExpiredException

2010-06-10 Thread Jordan Zimmerman
On Jun 9, 2010, at 4:21 PM, Patrick Hunt wrote: > In particular you might look at GC/swapping on your clients, that's the most > common case we see for session expiration (apart from the obvious -- network > level connectivity failures). In one case I remember there was very heavy > network lo

Re: Debugging help for SessionExpiredException

2010-06-15 Thread Jordan Zimmerman
Z On Jun 9, 2010, at 11:36 AM, Jordan Zimmerman wrote: > We have a test system using Zookeeper. There is a single Zookeeper server > node and 4 clients. There is very little activity in this system. After a > day's testing we start to see SessionExpiredException on the client. T

Re: Debugging help for SessionExpiredException

2010-06-15 Thread Jordan Zimmerman
t; > On Tue, Jun 15, 2010 at 10:33 AM, Jordan Zimmerman < > jzimmer...@proofpoint.com> wrote: > >> More on this... >> >> I ran last night with verbose GC on our client. I analyzed the GC log in >> gchisto and 99% of the GCs are 1 or 2 ms. The longest gc is

Re: Debugging help for SessionExpiredException

2010-06-15 Thread Jordan Zimmerman
They're small instances. The thing is that these machines are doing next to no work. We're just running simple little tests. The session expiration has not happened while I've been watching. It tends to happen over night. -JZ On Jun 15, 2010, at 1:50 PM, Ted Dunning wrote: > As usual, the ZK t

Clarification on async calls in a cluster

2010-08-11 Thread Jordan Zimmerman
If I use an async version of a call in a cluster ("ensemble") what happens if the server I'm connected to goes down? Does ZK transparently resubmit the call to the next server in the cluster and call my async callback or is there something I need to do? The docs aren't clear on this and searchin