ari [mailto:prateek...@gmail.com]
> Sent: Friday, June 8, 2018 15:00
> To: dev@samza.apache.org
> Subject: Re: Urgent : Help with latency / backlog / topic lag
>
> Just to clarify, when you say you tried single threaded mode, do you mean
> that you set job.container.thread.pool.siz
Maheshwari [mailto:prateek...@gmail.com]
Sent: Friday, June 8, 2018 15:00
To: dev@samza.apache.org
Subject: Re: Urgent : Help with latency / backlog / topic lag
Just to clarify, when you say you tried single threaded mode, do you mean that
you set job.container.thread.pool.size = 1, or that you set
t
> still only achieve ~700/sec
>
>
> -Original Message-
> From: Prateek Maheshwari [mailto:prateek...@gmail.com]
> Sent: Friday, June 8, 2018 13:54
> To: dev@samza.apache.org
> Subject: Re: Urgent : Help with latency / backlog / topic lag
>
> Hi Thunder,
>
>> What w
cy 0.2ms per message but still
only achieve ~700/sec
-Original Message-
From: Prateek Maheshwari [mailto:prateek...@gmail.com]
Sent: Friday, June 8, 2018 13:54
To: dev@samza.apache.org
Subject: Re: Urgent : Help with latency / backlog / topic lag
Hi Thunder,
> What we believe may be
Hi Thunder,
> What we believe may be happening is that most of the topics have no
backlog, but one topic has all the backlog (this is because one of the
topics accounts for ~60% of the total message rate). Could there be
something inducing extra latency on processing the one topic with a backlog
We have a new samza job which we just put into production. This job processes
many topics (~30) but the total rate is not that high (~1200/sec in aggregate).
I am unable to get above ~700/sec and have a growing backlog.
We are running samza 0.12 (I have an update to 0.14 that is not tested or
p
