As you mentioned, enabling trace/debug might impact performance.

Here's the approach I'm using now.

   -

   Parse the Zookeeper transaction logs. The best repo I've found to do
   this is https://github.com/alenca/zklogtool. It's feature packed. The
   only downside is that it won't record the read queries.


-
zodvik

On Mon, Mar 5, 2018 at 5:42 PM, Andor Molnar <an...@cloudera.com> wrote:

> Hi Zod,
>
> Have you checked TRACE logging of client requests? (
> http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html#sc_authOptions
> )
> Not sure how much different is from what you refer as 'debug logs', but it
> might worth a try. It generates log messages like:
>
> TRACE [CommitProcWorkThread-1:ZooTrace@88][] -
> :Esessionid:0x8061f60cfd200000 type:getChildren2 cxid:0x6
> zxid:0xfffffffffffffffe txntype:unknown reqpath:/
> TRACE [CommitProcWorkThread-1:ZooTrace@88][] -
> :Esessionid:0x8061f60cfd200000 type:getData cxid:0x7
> zxid:0xfffffffffffffffe txntype:unknown reqpath:/andor2
>
> Which looks like quite easy to parse and contains all required information.
>
> Regards,
> Andor
>
>
> On Thu, Mar 1, 2018 at 5:33 AM, Zod <zod...@gmail.com> wrote:
>
> > Unfortunately, parsing debug log is no-go since it doesn't print the
> > complete command.
> >
> > I've 2 approached in mind as of now
> >
> > 1. Modify of org.apache.zookeeper.server.LogFormatter to support
> > "tailing".
> > 2. Use ngrep on my local to monitor the network traffic to remote ZK
> host &
> > port 2181
> >
> > ​Will get back with what works better.​
> >
> > -
> > zodvik
> >
> > On Thu, Mar 1, 2018 at 1:22 AM, Andor Molnar <an...@cloudera.com> wrote:
> >
> > > Hi Zod,
> > >
> > > I think there's no such thing in ZooKeeper currently other than debug
> > logs.
> > > You might want to attach some processing engine to the debug log feed
> and
> > > parse and output data which is important for you into MySQL or Redis
> > > database.
> > >
> > > The downside of this approach is debug logging has significant
> > performance
> > > impact on the entire ensemble, but it might be acceptable for you
> > analysis.
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
> > > On Tue, Feb 27, 2018 at 11:26 PM, Zod <zod...@gmail.com> wrote:
> > >
> > > > I want see all commands issued to my Zookeeper cluster, something
> like
> > > > general log in MySQL or "monitor" command in Redis. How can this be
> > done?
> > > >
> > > > The intent is to see how Storm uses Zookeeper (for state management &
> > > > ack-ing) and a good way to would be to run a sample topology and see
> > all
> > > > the commands issued to Zookeeper by Storm.
> > > >
> > > > I tried enabling debug log for Zookeeper, but that is insufficient
> has
> > a
> > > > lot of noise. For example, issue a create /node prints
> > > >
> > > > 2018-02-27 18:05:34 ZooKeeperMain [DEBUG] Processing create
> > > > -
> > > > zodvik
> > > >
> > >
> >
>

Reply via email to