I'm not aware of sustained 1k/sec, Ben might know how long the 20k/sec test runs for (and for how long that rate is sustained). You'd definitely want to tune the GC, GC related pauses would be the biggest obstacle for this (assuming you are using a dedicated log device for the transaction logs).
Patrick On Tue, Oct 19, 2010 at 3:14 PM, Sandy Pratt <prat...@adobe.com> wrote: > Follow up question: does anyone have a production cluster that handles a > similar sustained rate of changes? > > -----Original Message----- > From: Benjamin Reed [mailto:br...@yahoo-inc.com] > Sent: Tuesday, October 19, 2010 2:53 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: zxid integer overflow > > we should put in a test for that. it is certainly a plausible scenario. in > theory it will just flow into the next epoch and everything will be fine, > but we should try it and see. > > ben > > On 10/19/2010 11:33 AM, Sandy Pratt wrote: > > Just as a thought experiment, I was pondering the following: > > > > ZK stamps each change to its managed state with a zxid ( > http://hadoop.apache.org/zookeeper/docs/r3.2.1/zookeeperInternals.html). > That ID consists of a 64 bit number in which the upper 32 bits are the > epoch, which changes when the leader does, and the bottom 32 bits are a > counter, which is incremented by the leader with every change. If 1000 > changes are made to ZK state each second (which is 1/20th of the peak rate > advertised), then the counter portion will roll over in 2^32 / (86400 * > 1000) = 49 days. > > > > Now, assuming that my math is correct, is this an actual concern? For > example, if I'm using ZK to provide locking for a key value store that > handles transactions at about that rate, am I setting myself up for failure? > > > > Thanks, > > > > Sandy > >
