Zookeeper does batched synchronous writes to disk. If you are IO bound, each write will take longer leading to a large number of queued up writes on the leader. This will affect zookeeper performance adversely and can even make it unavailable. That is one of the reasons, we deploy the zookeeper transaction log on a dedicated disk and monitor IO closely.
Thanks, Neha On Fri, Oct 12, 2012 at 4:34 PM, Guy Doulberg <guy.doulb...@conduit.com> wrote: > How can the zookeeper be IO bound? > > Isn't all its work in memory? > > Patricio Echagüe <patric...@gmail.com> כתב: > > > I'd recommend to run iostat -x while you do the test to rule that that your > are not io bound. > > On Thu, Oct 11, 2012 at 8:42 AM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > >> Guy, >> >> Connection reset messages are not particularly uncommon. Monitoring GC >> and io on Zookeeper is necessary, like Jay mentions, however if you >> are hitting any issues due to GC, you would see session expirations >> and timeouts. >> It will be helpful if you can send around the log4j files. >> >> Thanks, >> Neha >> >> On Thu, Oct 11, 2012 at 8:39 AM, Jay Kreps <jay.kr...@gmail.com> wrote: >> > Are you logging GC activity on the zk jvm? We had a lot of zk gc problems >> > before we got more scientific about our jvm settings (I think we added >> some >> > notes on the operations page). >> > >> > -Jay >> > >> > On Thu, Oct 11, 2012 at 7:48 AM, Guy Doulberg <guy.doulb...@conduit.com >> >wrote: >> > >> >> Hi guys, >> >> >> >> I am trying to understand a phenomena I am having in my cluster, >> >> >> >> My cluster consist of 3 zookeeprs (that are in the same machines as the >> >> brokers) >> >> >> >> Sometimes the zookeeprs freezes, which means, I cann't use new >> consumers, >> >> and I can't browse the znode using zookeeper browser. >> >> >> >> It disappears after a while, without doing anything pro-actively. >> >> >> >> A consumer that was already running is working alright - I guess since >> the >> >> zookeeper are not available the consumer doesn't report offsets. >> >> >> >> >> >> in my connection string I put all the 3 zookeeper instances. >> >> >> >> >> >> Have it ever happened to one of you? >> >> >> >> Thanks, >> >> >> >> Guy Doulberg >> >> Data Infrastructure engineer >> >> Conduit >> >> >> >> >> >> >>
