another downside to client instrumentation (beyond the number of
client codebases one would need to cover) is that in a large
environments you'll have a very long tail of applications using older
clients to upgrade - it would be a long and disruptive process (as
opposed to updating broker-side instrumentation)
On Thu, Nov 8, 2018 at 11:04 AM Peter M. Elias <petermel...@gmail.com> wrote:
>
> I know we have a lot of use cases for this type of functionality at my
> enterprise deployment. I think it's helpful for maintaining reliability of
> the cluster especially and identifying clients that are not properly tuned
> and therefore applying excessive load to the brokers. Additionally, there
> is a bit of a dark spot without something like as currently. For example,
> if a client is not using a consumer group, there is no direct way to query
> the state of the consumer without looking at raw network connections to
> determine the extent of the traffic generated by that particular consumer.
>
> While client instrumentation can certainly help with this currently, given
> that Kafka is intended to be a shared service across a potentially very
> large surface area of clients, central observation of client activity is in
> my opinion an essential feature.
>
> Peter
>
> On Thu, Nov 8, 2018 at 12:13 PM radai <radai.rosenbl...@gmail.com> wrote:
>
> > bump.
> >
> > I think the proposed API (Observer) is useful for any sort of
> > multi-tenant environment for chargeback and reporting purposes.
> >
> > if no one wants to comment, can we initiate a vote?
> > On Mon, Nov 5, 2018 at 6:31 PM Lincong Li <andrewlinc...@gmail.com> wrote:
> > >
> > > Hi everyone. Here
> > > <
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-388%3A+Add+observer+interface+to+record+request+and+response
> > >
> > > is
> > > my KIP. Any feedback is appreciated.
> > >
> > > Thanks,
> > > Lincong Li
> >
