confluence access

2020-10-13 Thread Dániel Urbán
Hi,

I'd like to access the wiki and contribute to KIPs, please add me. My
username is urbandan.

Thanks in advance,
Daniel


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-10-08 Thread Dániel Urbán
Thank you for the votes!

The KIP passes with 3 binding votes (Manikumar, Mickael, Ismael) and 3
non-binding votes (Viktor, Kamal, David).

Daniel

Ismael Juma  ezt írta (időpont: 2020. okt. 8., Cs,
16:27):

> Thanks for the KIP, +1 (binding).
>
> On Mon, Jul 27, 2020 at 1:09 AM Dániel Urbán 
> wrote:
>
> > Hello everyone,
> >
> > I'd like to start a vote on KIP-635. The KIP enhances the GetOffsetShell
> > tool by enabling querying multiple topic-partitions, adding new filtering
> > options, and adding a config override option.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> >
> > The original discussion thread was named "[DISCUSS] KIP-308:
> > GetOffsetShell: new KafkaConsumer API, support for multiple topics,
> > minimize the number of requests to server". The id had to be changed as
> > there was a collision, and the KIP also had to be renamed, as some of its
> > motivations were outdated.
> >
> > Thanks,
> > Daniel
> >
>


Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

2020-10-01 Thread Dániel Urbán
Hi Viktor,

I think the current state of the proposal is flexible enough to support
use-cases where the response data is of interest to the auditor.
This part ensures that: "... doing the auditing before sending the response
back ...". Additionally, event classes could be extended with additional
data if needed.

Overall, the KIP looks good, thanks!

Daniel

Viktor Somogyi-Vass  ezt írta (időpont: 2020.
szept. 30., Sze, 17:24):

> Hi Daniel,
>
> I think in this sense we can use the precedence set with the
> KAfkaAdminClient. It has *Result and *Options classes which in this
> interpretation are similar in versioning and usage as they transform and
> convey the responses of the protocol in a minimalistic API.
> I've modified the KIP a bit and created some examples for these event
> classes. For now as the implementation I think we can treat this similarly
> to KIP-4 (AdminClient) which didn't push implementation for everything but
> rather pushed implementing everything to subsequent KIPs as the
> requirements become important. In this first KIP we can create the more
> important ones (listed in the "Default Implementation") section if that is
> fine.
>
> Regarding the response passing: to be honest I feel like that it's not that
> strictly related to auditing but I think it's a good idea and could fit
> into this API. I think that we should design this current API with this in
> mind. Did you have any specific ideas about the implementation?
>
> Viktor
>
> On Tue, Sep 22, 2020 at 9:05 AM Dániel Urbán 
> wrote:
>
> > An example I had in mind was the ProduceResponse - the auditor might need
> > access to the new end offset of the partitions.
> > The event-based approach sounds good - new events and fields can be added
> > on-demand. Do we need the same versioning strategy we use with the
> > requests/responses?
> >
> > Daniel
> >
> > Viktor Somogyi-Vass  ezt írta (időpont: 2020.
> > szept. 21., H, 14:08):
> >
> > > Hi Daniel,
> > >
> > > > If the auditor needs access to the details of the action, one could
> > argue
> > > that even the response should be passed down to the auditor.
> > > At this point I don't think we need to include responses into the
> > interface
> > > but if you have a use-case we can consider doing that.
> > >
> > > > Is it feasible to convert the Java requests and responses to public
> > API?
> > > Well I think that in this case we would need to actually transform a
> lot
> > of
> > > classes and that might be a bit too invasive. Although since the
> protocol
> > > itself *is* a public API it might make sense to have some kind of Java
> > > representation as a public API as well.
> > >
> > > > If not, do we have another option to access this info in the auditor?
> > > I think one option would be to do what the original KIP-567 was
> > > implemented. Basically we could have an AuditEvent interface that would
> > > contain request specific data. Its obvious drawback is that it has to
> be
> > > implemented for most of the 40 something protocols but on the upside
> > these
> > > classes shouldn't be complicated. I can try to do a PoC with this to
> see
> > > how it looks like and whether it solves the problem. To be honest I
> think
> > > it would be better than publishing the request classes as an API
> because
> > > here we're restricting access to only what is necessary.
> > >
> > > Thanks,
> > > Viktor
> > >
> > >
> > >
> > > On Fri, Sep 18, 2020 at 8:37 AM Dániel Urbán 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > If the auditor needs access to the details of the action, one could
> > argue
> > > > that even the response should be passed down to the auditor.
> > > > Is it feasible to convert the Java requests and responses to public
> > API?
> > > > If not, do we have another option to access this info in the auditor?
> > > > I know that the auditor could just send proper requests through the
> API
> > > to
> > > > the brokers, but that seems like an awful lot of overhead, and could
> > > > introduce timing issues as well.
> > > >
> > > > Daniel
> > > >
> > > >
> > > > Viktor Somogyi-Vass  ezt írta (időpont:
> 2020.
> > > > szept. 16., Sze, 17:17):
> > > >
> > > > > One more after-thought on your sec

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

2020-09-22 Thread Dániel Urbán
An example I had in mind was the ProduceResponse - the auditor might need
access to the new end offset of the partitions.
The event-based approach sounds good - new events and fields can be added
on-demand. Do we need the same versioning strategy we use with the
requests/responses?

Daniel

Viktor Somogyi-Vass  ezt írta (időpont: 2020.
szept. 21., H, 14:08):

> Hi Daniel,
>
> > If the auditor needs access to the details of the action, one could argue
> that even the response should be passed down to the auditor.
> At this point I don't think we need to include responses into the interface
> but if you have a use-case we can consider doing that.
>
> > Is it feasible to convert the Java requests and responses to public API?
> Well I think that in this case we would need to actually transform a lot of
> classes and that might be a bit too invasive. Although since the protocol
> itself *is* a public API it might make sense to have some kind of Java
> representation as a public API as well.
>
> > If not, do we have another option to access this info in the auditor?
> I think one option would be to do what the original KIP-567 was
> implemented. Basically we could have an AuditEvent interface that would
> contain request specific data. Its obvious drawback is that it has to be
> implemented for most of the 40 something protocols but on the upside these
> classes shouldn't be complicated. I can try to do a PoC with this to see
> how it looks like and whether it solves the problem. To be honest I think
> it would be better than publishing the request classes as an API because
> here we're restricting access to only what is necessary.
>
> Thanks,
> Viktor
>
>
>
> On Fri, Sep 18, 2020 at 8:37 AM Dániel Urbán 
> wrote:
>
> > Hi,
> >
> > Thanks for the KIP.
> >
> > If the auditor needs access to the details of the action, one could argue
> > that even the response should be passed down to the auditor.
> > Is it feasible to convert the Java requests and responses to public API?
> > If not, do we have another option to access this info in the auditor?
> > I know that the auditor could just send proper requests through the API
> to
> > the brokers, but that seems like an awful lot of overhead, and could
> > introduce timing issues as well.
> >
> > Daniel
> >
> >
> > Viktor Somogyi-Vass  ezt írta (időpont: 2020.
> > szept. 16., Sze, 17:17):
> >
> > > One more after-thought on your second point (AbstractRequest): the
> > reason I
> > > introduced it in the first place was that this way implementers can
> > access
> > > request data. A use case can be if they want to audit a change in
> > > configuration or client quotas but not just acknowledge the fact that
> > such
> > > an event happened but also capture the change itself by peeking into
> the
> > > request. Sometimes it can be useful especially when people want to
> trace
> > > back the order of events and what happened when and not just
> acknowledge
> > > that there was an event of a certain kind. I also recognize that this
> > might
> > > be a very loose interpretation of auditing as it's not strictly related
> > to
> > > authorization but rather a way of tracing the admin actions within the
> > > cluster. It even could be a different API therefore but because of the
> > > variety of the Kafka APIs it's very hard to give a method that fits
> all,
> > so
> > > it's easier to pass down the AbstractRequest and the implementation can
> > do
> > > the extraction of valuable info. So that's why I added this in the
> first
> > > place and I'm interested in your thoughts.
> > >
> > > On Wed, Sep 16, 2020 at 4:41 PM Viktor Somogyi-Vass <
> > > viktorsomo...@gmail.com>
> > > wrote:
> > >
> > > > Hi Mickael,
> > > >
> > > > Thanks for reviewing the KIP.
> > > >
> > > > 1.) I just wanted to follow the conventions used with the Authorizer
> as
> > > it
> > > > is built in a similar fashion, although it's true that in KafkaServer
> > we
> > > > call the configure() method and the start() in the next line. This
> > would
> > > be
> > > > the same in Auditor and even simpler as there aren't any parameters
> to
> > > > start(), so I can remove it. If it turns out there is a need for it,
> we
> > > can
> > > > add it later.
> > > >
> > > > 2.) Yes, this is a very good point, I will remove it, however in this
> > > case
> > >

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

2020-09-18 Thread Dániel Urbán
Hi,

Thanks for the KIP.

If the auditor needs access to the details of the action, one could argue
that even the response should be passed down to the auditor.
Is it feasible to convert the Java requests and responses to public API?
If not, do we have another option to access this info in the auditor?
I know that the auditor could just send proper requests through the API to
the brokers, but that seems like an awful lot of overhead, and could
introduce timing issues as well.

Daniel


Viktor Somogyi-Vass  ezt írta (időpont: 2020.
szept. 16., Sze, 17:17):

> One more after-thought on your second point (AbstractRequest): the reason I
> introduced it in the first place was that this way implementers can access
> request data. A use case can be if they want to audit a change in
> configuration or client quotas but not just acknowledge the fact that such
> an event happened but also capture the change itself by peeking into the
> request. Sometimes it can be useful especially when people want to trace
> back the order of events and what happened when and not just acknowledge
> that there was an event of a certain kind. I also recognize that this might
> be a very loose interpretation of auditing as it's not strictly related to
> authorization but rather a way of tracing the admin actions within the
> cluster. It even could be a different API therefore but because of the
> variety of the Kafka APIs it's very hard to give a method that fits all, so
> it's easier to pass down the AbstractRequest and the implementation can do
> the extraction of valuable info. So that's why I added this in the first
> place and I'm interested in your thoughts.
>
> On Wed, Sep 16, 2020 at 4:41 PM Viktor Somogyi-Vass <
> viktorsomo...@gmail.com>
> wrote:
>
> > Hi Mickael,
> >
> > Thanks for reviewing the KIP.
> >
> > 1.) I just wanted to follow the conventions used with the Authorizer as
> it
> > is built in a similar fashion, although it's true that in KafkaServer we
> > call the configure() method and the start() in the next line. This would
> be
> > the same in Auditor and even simpler as there aren't any parameters to
> > start(), so I can remove it. If it turns out there is a need for it, we
> can
> > add it later.
> >
> > 2.) Yes, this is a very good point, I will remove it, however in this
> case
> > I don't think we need to add the ApiKey as it is already available in
> > AuthorizableRequestContext.requestType(). One less parameter :).
> >
> > 3.) I'll add it. It will simply log important changes in the cluster like
> > topic events (create, update, delete, partition or replication factor
> > change), ACL events, config changes, reassignment, altering log dirs,
> > offset delete, group delete with the authorization info like who
> initiated
> > the call, was it authorized, were there any errors. Let me know if you
> > think there are other APIs I should include.
> >
> > 4.) The builder is there mostly for easier usability but actually
> thinking
> > of it it doesn't help much so I removed it. The AuditInfo is also a
> helper
> > class so I don't see any value in transforming it into an interface but
> if
> > I simplify it (by removing the builder) it will be cleaner. Would that
> work?
> >
> > I'll update the KIP to reflect my answers.
> >
> > Viktor
> >
> >
> > On Mon, Sep 14, 2020 at 6:02 PM Mickael Maison  >
> > wrote:
> >
> >> Hi Viktor,
> >>
> >> Thanks for restarting the discussion on this KIP. Being able to easily
> >> audit usage of a Kafka cluster is a very valuable feature.
> >>
> >> Regarding the API, I have a few of questions:
> >> 1) You introduced a start() method. I don't think any other interfaces
> >> have such a method. Users can do any setup they want in configure()
> >>
> >> 2) The first argument of audit is an AbstractRequest. Unfortunately
> >> this type is not part of the public API. But actually I'm not sure
> >> having the full request is really needed here. Maybe just passing the
> >> Apikey would be enough as we already have all the resources from the
> >> auditInfos field.
> >>
> >> 3) The KIP mentions a "LoggingAuditor" default implementation. What is
> >> it doing? Can you add more details about it?
> >>
> >> 4) Can fields of AuditInfo be null? I can see there's a constructor
> >> without an Errors and that sets the error field to None. However, with
> >> the builder pattern, if error is not set it's null.
> >>
> >> 5) Should AuditInfo be an interface?
> >>
> >> On Mon, Sep 14, 2020 at 3:26 PM Viktor Somogyi-Vass
> >>  wrote:
> >> >
> >> > Hi everyone,
> >> >
> >> > Changed the interface a little bit to accommodate methods better where
> >> > authorization happens for multiple operations so the implementer of
> the
> >> > audit interface will receive all authorizations together.
> >> > I'll wait a few more days to allow people to react or give feedback
> but
> >> if
> >> > there are no objections until then, I'll start a vote.
> >> >
> >> > Viktor
> >> >
> >> > On Tue, Sep 8, 2020 at 9:49 AM 

Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-08-31 Thread Dániel Urbán
Hello everyone,

I'd like to ask you to consider voting for this KIP, it only needs 2 more
binding votes.

This KIP focuses on the GetOffsetShell tool. The tool is useful for
monitoring and investigations as well.
Unfortunately, the tool misses one key, and some quality-of-life features:
- Because of lack of configuration options, it cannot be used with secure
clusters - which makes it unviable in many use-cases.
- Only supports a single topic - usability improvement, also in line with
other tools using pattern based matching.
- Still utilizes the deprecated "--broker-list" argument name.

Overall, I believe that this is a non-intrusive change - minor improvements
without breaking changes. But for some, this would be a great improvement
in using Kafka.

Thank you in advance,
Daniel


Dániel Urbán  ezt írta (időpont: 2020. aug. 27., Cs,
17:52):

> Hi all,
>
> Please vote if you'd like to see this implemented. This one fixes a
> long-time debt, would be nice to see it pass.
>
> Thank you,
> Daniel
>
> Dániel Urbán  ezt írta (időpont: 2020. aug. 18.,
> K, 14:06):
>
>> Hello everyone,
>>
>> Please, if you are interested in this KIP and PR, don't forget to vote.
>>
>> Thank you,
>> Daniel
>>
>> Dániel Urbán  ezt írta (időpont: 2020. aug. 13.,
>> Cs, 14:00):
>>
>>> Hi David,
>>>
>>> Thank you for the suggestion. KIP-635 was referencing the --broker-list
>>> issue, but based on your suggestion, I pinged the PR
>>> https://github.com/apache/kafka/pull/8123.
>>> Since I got no response, I updated KIP-635 to deprecate --broker-list.
>>> Will update the PR related to KIP-635 to reflect that change.
>>>
>>> Thanks,
>>> Daniel
>>>
>>> David Jacot  ezt írta (időpont: 2020. aug. 10., H,
>>> 20:48):
>>>
>>>> Hi Daniel,
>>>>
>>>> I was not aware of that PR. At minimum, I would add `--bootstrap-server`
>>>> to the list in the KIP for completeness. Regarding the implementation,
>>>> I would leave a comment in that PR asking if they plan to continue it.
>>>> If
>>>> not,
>>>> we could do it as part of your PR directly.
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On Mon, Aug 10, 2020 at 10:49 AM Dániel Urbán 
>>>> wrote:
>>>>
>>>> > Hi everyone,
>>>> >
>>>> > Just a reminder, please vote if you are interested in this KIP being
>>>> > implemented.
>>>> >
>>>> > Thanks,
>>>> > Daniel
>>>> >
>>>> > Dániel Urbán  ezt írta (időpont: 2020. júl.
>>>> 31., P,
>>>> > 9:01):
>>>> >
>>>> > > Hi David,
>>>> > >
>>>> > > There is another PR linked on KAFKA-8507, which is still open:
>>>> > > https://github.com/apache/kafka/pull/8123
>>>> > > Wasn't sure if it will go in, and wanted to avoid conflicts. Do you
>>>> think
>>>> > > I should do the switch to '--bootstrap-server' anyway?
>>>> > >
>>>> > > Thanks,
>>>> > > Daniel
>>>> > >
>>>> > > David Jacot  ezt írta (időpont: 2020. júl.
>>>> 30., Cs,
>>>> > > 17:52):
>>>> > >
>>>> > >> Hi Daniel,
>>>> > >>
>>>> > >> Thanks for the KIP.
>>>> > >>
>>>> > >> It seems that we have forgotten to include this tool in KIP-499.
>>>> > >> KAFKA-8507
>>>> > >> is resolved
>>>> > >> by this tool still uses the deprecated "--broker-list". I suggest
>>>> to
>>>> > >> include "--bootstrap-server"
>>>> > >> in your public interfaces as well and fix this omission during the
>>>> > >> implementation.
>>>> > >>
>>>> > >> +1 (non-binding)
>>>> > >>
>>>> > >> Thanks,
>>>> > >> David
>>>> > >>
>>>> > >> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
>>>> > >> kamal.chandraprak...@gmail.com> wrote:
>>>> > >>
>>>> > >> > +1 (non-binding), thanks for the KIP!
>>>> > >> >
>>>> > >> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar <
>>>> manikumar.re...@gmail

Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-08-27 Thread Dániel Urbán
Hi all,

Please vote if you'd like to see this implemented. This one fixes a
long-time debt, would be nice to see it pass.

Thank you,
Daniel

Dániel Urbán  ezt írta (időpont: 2020. aug. 18., K,
14:06):

> Hello everyone,
>
> Please, if you are interested in this KIP and PR, don't forget to vote.
>
> Thank you,
> Daniel
>
> Dániel Urbán  ezt írta (időpont: 2020. aug. 13.,
> Cs, 14:00):
>
>> Hi David,
>>
>> Thank you for the suggestion. KIP-635 was referencing the --broker-list
>> issue, but based on your suggestion, I pinged the PR
>> https://github.com/apache/kafka/pull/8123.
>> Since I got no response, I updated KIP-635 to deprecate --broker-list.
>> Will update the PR related to KIP-635 to reflect that change.
>>
>> Thanks,
>> Daniel
>>
>> David Jacot  ezt írta (időpont: 2020. aug. 10., H,
>> 20:48):
>>
>>> Hi Daniel,
>>>
>>> I was not aware of that PR. At minimum, I would add `--bootstrap-server`
>>> to the list in the KIP for completeness. Regarding the implementation,
>>> I would leave a comment in that PR asking if they plan to continue it. If
>>> not,
>>> we could do it as part of your PR directly.
>>>
>>> Cheers,
>>> David
>>>
>>> On Mon, Aug 10, 2020 at 10:49 AM Dániel Urbán 
>>> wrote:
>>>
>>> > Hi everyone,
>>> >
>>> > Just a reminder, please vote if you are interested in this KIP being
>>> > implemented.
>>> >
>>> > Thanks,
>>> > Daniel
>>> >
>>> > Dániel Urbán  ezt írta (időpont: 2020. júl.
>>> 31., P,
>>> > 9:01):
>>> >
>>> > > Hi David,
>>> > >
>>> > > There is another PR linked on KAFKA-8507, which is still open:
>>> > > https://github.com/apache/kafka/pull/8123
>>> > > Wasn't sure if it will go in, and wanted to avoid conflicts. Do you
>>> think
>>> > > I should do the switch to '--bootstrap-server' anyway?
>>> > >
>>> > > Thanks,
>>> > > Daniel
>>> > >
>>> > > David Jacot  ezt írta (időpont: 2020. júl.
>>> 30., Cs,
>>> > > 17:52):
>>> > >
>>> > >> Hi Daniel,
>>> > >>
>>> > >> Thanks for the KIP.
>>> > >>
>>> > >> It seems that we have forgotten to include this tool in KIP-499.
>>> > >> KAFKA-8507
>>> > >> is resolved
>>> > >> by this tool still uses the deprecated "--broker-list". I suggest to
>>> > >> include "--bootstrap-server"
>>> > >> in your public interfaces as well and fix this omission during the
>>> > >> implementation.
>>> > >>
>>> > >> +1 (non-binding)
>>> > >>
>>> > >> Thanks,
>>> > >> David
>>> > >>
>>> > >> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
>>> > >> kamal.chandraprak...@gmail.com> wrote:
>>> > >>
>>> > >> > +1 (non-binding), thanks for the KIP!
>>> > >> >
>>> > >> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar <
>>> manikumar.re...@gmail.com>
>>> > >> > wrote:
>>> > >> >
>>> > >> > > +1 (binding)
>>> > >> > >
>>> > >> > > Thanks for the KIP!
>>> > >> > >
>>> > >> > >
>>> > >> > >
>>> > >> > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán <
>>> urb.dani...@gmail.com
>>> > >
>>> > >> > > wrote:
>>> > >> > >
>>> > >> > > > Hi everyone,
>>> > >> > > >
>>> > >> > > > If you are interested in this KIP, please do not forget to
>>> vote.
>>> > >> > > >
>>> > >> > > > Thanks,
>>> > >> > > > Daniel
>>> > >> > > >
>>> > >> > > > Viktor Somogyi-Vass  ezt írta
>>> (időpont:
>>> > >> 2020.
>>> > >> > > > júl.
>>> > >> > > > 28., K, 16:06):
>>> > >> > > >
>>> > >> > > > > +1 from me (non-binding), thanks for the KIP.
>>> > >> > > > >
>>> > >> > > > > On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán <
>>> > >> urb.dani...@gmail.com
>>> > >> > >
>>> > >> > > > > wrote:
>>> > >> > > > >
>>> > >> > > > > > Hello everyone,
>>> > >> > > > > >
>>> > >> > > > > > I'd like to start a vote on KIP-635. The KIP enhances the
>>> > >> > > > GetOffsetShell
>>> > >> > > > > > tool by enabling querying multiple topic-partitions,
>>> adding
>>> > new
>>> > >> > > > filtering
>>> > >> > > > > > options, and adding a config override option.
>>> > >> > > > > >
>>> > >> > > > > >
>>> > >> > > > >
>>> > >> > > >
>>> > >> > >
>>> > >> >
>>> > >>
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
>>> > >> > > > > >
>>> > >> > > > > > The original discussion thread was named "[DISCUSS]
>>> KIP-308:
>>> > >> > > > > > GetOffsetShell: new KafkaConsumer API, support for
>>> multiple
>>> > >> topics,
>>> > >> > > > > > minimize the number of requests to server". The id had to
>>> be
>>> > >> > changed
>>> > >> > > as
>>> > >> > > > > > there was a collision, and the KIP also had to be
>>> renamed, as
>>> > >> some
>>> > >> > of
>>> > >> > > > its
>>> > >> > > > > > motivations were outdated.
>>> > >> > > > > >
>>> > >> > > > > > Thanks,
>>> > >> > > > > > Daniel
>>> > >> > > > > >
>>> > >> > > > >
>>> > >> > > >
>>> > >> > >
>>> > >> >
>>> > >>
>>> > >
>>> >
>>>
>>


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-08-18 Thread Dániel Urbán
Hello everyone,

Please, if you are interested in this KIP and PR, don't forget to vote.

Thank you,
Daniel

Dániel Urbán  ezt írta (időpont: 2020. aug. 13., Cs,
14:00):

> Hi David,
>
> Thank you for the suggestion. KIP-635 was referencing the --broker-list
> issue, but based on your suggestion, I pinged the PR
> https://github.com/apache/kafka/pull/8123.
> Since I got no response, I updated KIP-635 to deprecate --broker-list.
> Will update the PR related to KIP-635 to reflect that change.
>
> Thanks,
> Daniel
>
> David Jacot  ezt írta (időpont: 2020. aug. 10., H,
> 20:48):
>
>> Hi Daniel,
>>
>> I was not aware of that PR. At minimum, I would add `--bootstrap-server`
>> to the list in the KIP for completeness. Regarding the implementation,
>> I would leave a comment in that PR asking if they plan to continue it. If
>> not,
>> we could do it as part of your PR directly.
>>
>> Cheers,
>> David
>>
>> On Mon, Aug 10, 2020 at 10:49 AM Dániel Urbán 
>> wrote:
>>
>> > Hi everyone,
>> >
>> > Just a reminder, please vote if you are interested in this KIP being
>> > implemented.
>> >
>> > Thanks,
>> > Daniel
>> >
>> > Dániel Urbán  ezt írta (időpont: 2020. júl.
>> 31., P,
>> > 9:01):
>> >
>> > > Hi David,
>> > >
>> > > There is another PR linked on KAFKA-8507, which is still open:
>> > > https://github.com/apache/kafka/pull/8123
>> > > Wasn't sure if it will go in, and wanted to avoid conflicts. Do you
>> think
>> > > I should do the switch to '--bootstrap-server' anyway?
>> > >
>> > > Thanks,
>> > > Daniel
>> > >
>> > > David Jacot  ezt írta (időpont: 2020. júl. 30.,
>> Cs,
>> > > 17:52):
>> > >
>> > >> Hi Daniel,
>> > >>
>> > >> Thanks for the KIP.
>> > >>
>> > >> It seems that we have forgotten to include this tool in KIP-499.
>> > >> KAFKA-8507
>> > >> is resolved
>> > >> by this tool still uses the deprecated "--broker-list". I suggest to
>> > >> include "--bootstrap-server"
>> > >> in your public interfaces as well and fix this omission during the
>> > >> implementation.
>> > >>
>> > >> +1 (non-binding)
>> > >>
>> > >> Thanks,
>> > >> David
>> > >>
>> > >> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
>> > >> kamal.chandraprak...@gmail.com> wrote:
>> > >>
>> > >> > +1 (non-binding), thanks for the KIP!
>> > >> >
>> > >> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar <
>> manikumar.re...@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> > > +1 (binding)
>> > >> > >
>> > >> > > Thanks for the KIP!
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán <
>> urb.dani...@gmail.com
>> > >
>> > >> > > wrote:
>> > >> > >
>> > >> > > > Hi everyone,
>> > >> > > >
>> > >> > > > If you are interested in this KIP, please do not forget to
>> vote.
>> > >> > > >
>> > >> > > > Thanks,
>> > >> > > > Daniel
>> > >> > > >
>> > >> > > > Viktor Somogyi-Vass  ezt írta
>> (időpont:
>> > >> 2020.
>> > >> > > > júl.
>> > >> > > > 28., K, 16:06):
>> > >> > > >
>> > >> > > > > +1 from me (non-binding), thanks for the KIP.
>> > >> > > > >
>> > >> > > > > On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán <
>> > >> urb.dani...@gmail.com
>> > >> > >
>> > >> > > > > wrote:
>> > >> > > > >
>> > >> > > > > > Hello everyone,
>> > >> > > > > >
>> > >> > > > > > I'd like to start a vote on KIP-635. The KIP enhances the
>> > >> > > > GetOffsetShell
>> > >> > > > > > tool by enabling querying multiple topic-partitions, adding
>> > new
>> > >> > > > filtering
>> > >> > > > > > options, and adding a config override option.
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
>> > >> > > > > >
>> > >> > > > > > The original discussion thread was named "[DISCUSS]
>> KIP-308:
>> > >> > > > > > GetOffsetShell: new KafkaConsumer API, support for multiple
>> > >> topics,
>> > >> > > > > > minimize the number of requests to server". The id had to
>> be
>> > >> > changed
>> > >> > > as
>> > >> > > > > > there was a collision, and the KIP also had to be renamed,
>> as
>> > >> some
>> > >> > of
>> > >> > > > its
>> > >> > > > > > motivations were outdated.
>> > >> > > > > >
>> > >> > > > > > Thanks,
>> > >> > > > > > Daniel
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> >
>>
>


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-08-13 Thread Dániel Urbán
Hi David,

Thank you for the suggestion. KIP-635 was referencing the --broker-list
issue, but based on your suggestion, I pinged the PR
https://github.com/apache/kafka/pull/8123.
Since I got no response, I updated KIP-635 to deprecate --broker-list. Will
update the PR related to KIP-635 to reflect that change.

Thanks,
Daniel

David Jacot  ezt írta (időpont: 2020. aug. 10., H,
20:48):

> Hi Daniel,
>
> I was not aware of that PR. At minimum, I would add `--bootstrap-server`
> to the list in the KIP for completeness. Regarding the implementation,
> I would leave a comment in that PR asking if they plan to continue it. If
> not,
> we could do it as part of your PR directly.
>
> Cheers,
> David
>
> On Mon, Aug 10, 2020 at 10:49 AM Dániel Urbán 
> wrote:
>
> > Hi everyone,
> >
> > Just a reminder, please vote if you are interested in this KIP being
> > implemented.
> >
> > Thanks,
> > Daniel
> >
> > Dániel Urbán  ezt írta (időpont: 2020. júl. 31.,
> P,
> > 9:01):
> >
> > > Hi David,
> > >
> > > There is another PR linked on KAFKA-8507, which is still open:
> > > https://github.com/apache/kafka/pull/8123
> > > Wasn't sure if it will go in, and wanted to avoid conflicts. Do you
> think
> > > I should do the switch to '--bootstrap-server' anyway?
> > >
> > > Thanks,
> > > Daniel
> > >
> > > David Jacot  ezt írta (időpont: 2020. júl. 30.,
> Cs,
> > > 17:52):
> > >
> > >> Hi Daniel,
> > >>
> > >> Thanks for the KIP.
> > >>
> > >> It seems that we have forgotten to include this tool in KIP-499.
> > >> KAFKA-8507
> > >> is resolved
> > >> by this tool still uses the deprecated "--broker-list". I suggest to
> > >> include "--bootstrap-server"
> > >> in your public interfaces as well and fix this omission during the
> > >> implementation.
> > >>
> > >> +1 (non-binding)
> > >>
> > >> Thanks,
> > >> David
> > >>
> > >> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
> > >> kamal.chandraprak...@gmail.com> wrote:
> > >>
> > >> > +1 (non-binding), thanks for the KIP!
> > >> >
> > >> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar <
> manikumar.re...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > +1 (binding)
> > >> > >
> > >> > > Thanks for the KIP!
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán <
> urb.dani...@gmail.com
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Hi everyone,
> > >> > > >
> > >> > > > If you are interested in this KIP, please do not forget to vote.
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Daniel
> > >> > > >
> > >> > > > Viktor Somogyi-Vass  ezt írta
> (időpont:
> > >> 2020.
> > >> > > > júl.
> > >> > > > 28., K, 16:06):
> > >> > > >
> > >> > > > > +1 from me (non-binding), thanks for the KIP.
> > >> > > > >
> > >> > > > > On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán <
> > >> urb.dani...@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hello everyone,
> > >> > > > > >
> > >> > > > > > I'd like to start a vote on KIP-635. The KIP enhances the
> > >> > > > GetOffsetShell
> > >> > > > > > tool by enabling querying multiple topic-partitions, adding
> > new
> > >> > > > filtering
> > >> > > > > > options, and adding a config override option.
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> > >> > > > > >
> > >> > > > > > The original discussion thread was named "[DISCUSS] KIP-308:
> > >> > > > > > GetOffsetShell: new KafkaConsumer API, support for multiple
> > >> topics,
> > >> > > > > > minimize the number of requests to server". The id had to be
> > >> > changed
> > >> > > as
> > >> > > > > > there was a collision, and the KIP also had to be renamed,
> as
> > >> some
> > >> > of
> > >> > > > its
> > >> > > > > > motivations were outdated.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Daniel
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-08-10 Thread Dániel Urbán
Hi everyone,

Just a reminder, please vote if you are interested in this KIP being
implemented.

Thanks,
Daniel

Dániel Urbán  ezt írta (időpont: 2020. júl. 31., P,
9:01):

> Hi David,
>
> There is another PR linked on KAFKA-8507, which is still open:
> https://github.com/apache/kafka/pull/8123
> Wasn't sure if it will go in, and wanted to avoid conflicts. Do you think
> I should do the switch to '--bootstrap-server' anyway?
>
> Thanks,
> Daniel
>
> David Jacot  ezt írta (időpont: 2020. júl. 30., Cs,
> 17:52):
>
>> Hi Daniel,
>>
>> Thanks for the KIP.
>>
>> It seems that we have forgotten to include this tool in KIP-499.
>> KAFKA-8507
>> is resolved
>> by this tool still uses the deprecated "--broker-list". I suggest to
>> include "--bootstrap-server"
>> in your public interfaces as well and fix this omission during the
>> implementation.
>>
>> +1 (non-binding)
>>
>> Thanks,
>> David
>>
>> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
>> kamal.chandraprak...@gmail.com> wrote:
>>
>> > +1 (non-binding), thanks for the KIP!
>> >
>> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar 
>> > wrote:
>> >
>> > > +1 (binding)
>> > >
>> > > Thanks for the KIP!
>> > >
>> > >
>> > >
>> > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán 
>> > > wrote:
>> > >
>> > > > Hi everyone,
>> > > >
>> > > > If you are interested in this KIP, please do not forget to vote.
>> > > >
>> > > > Thanks,
>> > > > Daniel
>> > > >
>> > > > Viktor Somogyi-Vass  ezt írta (időpont:
>> 2020.
>> > > > júl.
>> > > > 28., K, 16:06):
>> > > >
>> > > > > +1 from me (non-binding), thanks for the KIP.
>> > > > >
>> > > > > On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán <
>> urb.dani...@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hello everyone,
>> > > > > >
>> > > > > > I'd like to start a vote on KIP-635. The KIP enhances the
>> > > > GetOffsetShell
>> > > > > > tool by enabling querying multiple topic-partitions, adding new
>> > > > filtering
>> > > > > > options, and adding a config override option.
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
>> > > > > >
>> > > > > > The original discussion thread was named "[DISCUSS] KIP-308:
>> > > > > > GetOffsetShell: new KafkaConsumer API, support for multiple
>> topics,
>> > > > > > minimize the number of requests to server". The id had to be
>> > changed
>> > > as
>> > > > > > there was a collision, and the KIP also had to be renamed, as
>> some
>> > of
>> > > > its
>> > > > > > motivations were outdated.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Daniel
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-07-31 Thread Dániel Urbán
Hi David,

There is another PR linked on KAFKA-8507, which is still open:
https://github.com/apache/kafka/pull/8123
Wasn't sure if it will go in, and wanted to avoid conflicts. Do you think I
should do the switch to '--bootstrap-server' anyway?

Thanks,
Daniel

David Jacot  ezt írta (időpont: 2020. júl. 30., Cs,
17:52):

> Hi Daniel,
>
> Thanks for the KIP.
>
> It seems that we have forgotten to include this tool in KIP-499. KAFKA-8507
> is resolved
> by this tool still uses the deprecated "--broker-list". I suggest to
> include "--bootstrap-server"
> in your public interfaces as well and fix this omission during the
> implementation.
>
> +1 (non-binding)
>
> Thanks,
> David
>
> On Thu, Jul 30, 2020 at 1:52 PM Kamal Chandraprakash <
> kamal.chandraprak...@gmail.com> wrote:
>
> > +1 (non-binding), thanks for the KIP!
> >
> > On Thu, Jul 30, 2020 at 3:31 PM Manikumar 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks for the KIP!
> > >
> > >
> > >
> > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán 
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > If you are interested in this KIP, please do not forget to vote.
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > Viktor Somogyi-Vass  ezt írta (időpont:
> 2020.
> > > > júl.
> > > > 28., K, 16:06):
> > > >
> > > > > +1 from me (non-binding), thanks for the KIP.
> > > > >
> > > > > On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán <
> urb.dani...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > > I'd like to start a vote on KIP-635. The KIP enhances the
> > > > GetOffsetShell
> > > > > > tool by enabling querying multiple topic-partitions, adding new
> > > > filtering
> > > > > > options, and adding a config override option.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> > > > > >
> > > > > > The original discussion thread was named "[DISCUSS] KIP-308:
> > > > > > GetOffsetShell: new KafkaConsumer API, support for multiple
> topics,
> > > > > > minimize the number of requests to server". The id had to be
> > changed
> > > as
> > > > > > there was a collision, and the KIP also had to be renamed, as
> some
> > of
> > > > its
> > > > > > motivations were outdated.
> > > > > >
> > > > > > Thanks,
> > > > > > Daniel
> > > > > >
> > > > >
> > > >
> > >
> >
>


Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-07-30 Thread Dániel Urbán
Hi everyone,

If you are interested in this KIP, please do not forget to vote.

Thanks,
Daniel

Viktor Somogyi-Vass  ezt írta (időpont: 2020. júl.
28., K, 16:06):

> +1 from me (non-binding), thanks for the KIP.
>
> On Mon, Jul 27, 2020 at 10:02 AM Dániel Urbán 
> wrote:
>
> > Hello everyone,
> >
> > I'd like to start a vote on KIP-635. The KIP enhances the GetOffsetShell
> > tool by enabling querying multiple topic-partitions, adding new filtering
> > options, and adding a config override option.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> >
> > The original discussion thread was named "[DISCUSS] KIP-308:
> > GetOffsetShell: new KafkaConsumer API, support for multiple topics,
> > minimize the number of requests to server". The id had to be changed as
> > there was a collision, and the KIP also had to be renamed, as some of its
> > motivations were outdated.
> >
> > Thanks,
> > Daniel
> >
>


[VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

2020-07-27 Thread Dániel Urbán
Hello everyone,

I'd like to start a vote on KIP-635. The KIP enhances the GetOffsetShell
tool by enabling querying multiple topic-partitions, adding new filtering
options, and adding a config override option.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override

The original discussion thread was named "[DISCUSS] KIP-308:
GetOffsetShell: new KafkaConsumer API, support for multiple topics,
minimize the number of requests to server". The id had to be changed as
there was a collision, and the KIP also had to be renamed, as some of its
motivations were outdated.

Thanks,
Daniel


Re: 回复: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-07-21 Thread Dániel Urbán
Hi,

I've updated the PR based on the discussion and the comments on the PR.
If there are no more issues, I'll start a vote in a few days.

Thanks,
Daniel

wang120445...@sina.com  ezt írta (időpont: 2020.
júl. 1., Sze, 3:26):

> maybe it just likes RBAC’s  show tables;
>
>
>
> wang120445...@sina.com
>
> 发件人: Hu Xi
> 发送时间: 2020-06-30 23:04
> 收件人: dev@kafka.apache.org
> 主题: 回复: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
> for multiple topics, minimize the number of requests to server
> That's a great KIP for GetOffsetShell tool. I have a question about the
> multiple-topic lookup situation.
>
> In a secured environment, what does the tool output if it has DESCRIBE
> privileges for some topics but hasn't for others?
>
> 
> 发件人: Dániel Urbán 
> 发送时间: 2020年6月30日 22:15
> 收件人: dev@kafka.apache.org 
> 主题: Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
> for multiple topics, minimize the number of requests to server
>
> Hi Manikumar,
> Thanks, went ahead and assigned a new ID, it is KIP-635 now:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> Daniel
>
> Manikumar  ezt írta (időpont: 2020. jún. 30.,
> K,
> 16:03):
>
> > Hi,
> >
> > Yes, we can assign new id to this KIP.
> >
> > Thanks.
> >
> > On Tue, Jun 30, 2020 at 6:59 PM Dániel Urbán 
> > wrote:
> >
> > > Hi,
> > >
> > > To help with the discussion, I also have a PR for this KIP now.
> > reflecting
> > > the current state of the KIP:
> https://github.com/apache/kafka/pull/8957.
> > > I would like to ask a committer to start the test job on it.
> > >
> > > One thing I realised though is that there is a KIP id collision, there
> is
> > > another KIP with the same id:
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
> > > What is the protocol in this case? Should I acquire a new id for the
> > > GetOffsetShell KIP, and update it?
> > >
> > > Thanks in advance,
> > > Daniel
> > >
> > > Dániel Urbán  ezt írta (időpont: 2020.
> jún.
> > > 30., K, 9:23):
> > >
> > > > Hi Manikumar,
> > > >
> > > > Thanks for the comments.
> > > > 1. Will change this - thought that "command-config" is used for admin
> > > > clients.
> > > > 2. It's not necessary, just felt like a nice quality-of-life feature
> -
> > > will
> > > > remove it.
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > On Tue, Jun 30, 2020 at 4:16 AM Manikumar  >
> > > > wrote:
> > > >
> > > > > Hi Daniel,
> > > > >
> > > > > Thanks for working on this KIP.  Proposed changes looks good to me,
> > > > >
> > > > > minor comments:
> > > > > 1. We use "command-config" option name in most of the cmdline tools
> > to
> > > > pass
> > > > > config
> > > > > properties file. We can use the same name here.
> > > > >
> > > > > 2. Not sure, if we need a separate option to pass an consumer
> > property.
> > > > > fewer options are better.
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > > > On Wed, Jun 24, 2020 at 8:53 PM Dániel Urbán <
> urb.dani...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I see that this KIP turned somewhat inactive - I'd like to pick
> it
> > up
> > > > and
> > > > > > work on it if it is okay.
> > > > > > Part of the work is done, as switching to the Consumer API is
> > already
> > > > in
> > > > > > trunk, but some functionality is still missing.
> > > > > >
> > > > > > I've seen the current PR and the discussion so far, only have a
> few
> > > > > things
> > > > > > to add:
> > > > > > - I like the idea of the topic-partition argument, it would be
> > useful
> > > > to
> > > > > > filter down to specific partitions.
> > > > > > - Instead of a topic list arg, a pattern would be more powerful,
> > and
> > > > also
> > > > > > fit better with the other tools (e.g. how the kafka-topics tool
> > > works).
> > > > > >
> > > > > > Regards,
> > > > > > Daniel
> > > > > >
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-30 Thread Dániel Urbán
That's a good question. In the PR I submitted, it would result in a list of
partitions contained by a topic for which the user has DESCRIBE privilege.
The tool utilizes Consumer.listTopics, so unauthorized topics are not
present in the response at all. The current version in trunk simply reports
that the topic does not exist if the user has no DESCRIBE privilege for it.

It would be hard (impossible?) to support detailed information about
unauthorized topics when using a pattern as a filter. It could be
manageable if the tool only supported a list of topics.

Maybe the only improvement needed is to explicitly document that the tool
only scans the authorized topics?

Hu Xi  ezt írta (időpont: 2020. jún. 30., K, 17:04):

> That's a great KIP for GetOffsetShell tool. I have a question about the
> multiple-topic lookup situation.
>
> In a secured environment, what does the tool output if it has DESCRIBE
> privileges for some topics but hasn't for others?
>
> ________
> 发件人: Dániel Urbán 
> 发送时间: 2020年6月30日 22:15
> 收件人: dev@kafka.apache.org 
> 主题: Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support
> for multiple topics, minimize the number of requests to server
>
> Hi Manikumar,
> Thanks, went ahead and assigned a new ID, it is KIP-635 now:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
> Daniel
>
> Manikumar  ezt írta (időpont: 2020. jún. 30.,
> K,
> 16:03):
>
> > Hi,
> >
> > Yes, we can assign new id to this KIP.
> >
> > Thanks.
> >
> > On Tue, Jun 30, 2020 at 6:59 PM Dániel Urbán 
> > wrote:
> >
> > > Hi,
> > >
> > > To help with the discussion, I also have a PR for this KIP now.
> > reflecting
> > > the current state of the KIP:
> https://github.com/apache/kafka/pull/8957.
> > > I would like to ask a committer to start the test job on it.
> > >
> > > One thing I realised though is that there is a KIP id collision, there
> is
> > > another KIP with the same id:
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
> > > What is the protocol in this case? Should I acquire a new id for the
> > > GetOffsetShell KIP, and update it?
> > >
> > > Thanks in advance,
> > > Daniel
> > >
> > > Dániel Urbán  ezt írta (időpont: 2020.
> jún.
> > > 30., K, 9:23):
> > >
> > > > Hi Manikumar,
> > > >
> > > > Thanks for the comments.
> > > > 1. Will change this - thought that "command-config" is used for admin
> > > > clients.
> > > > 2. It's not necessary, just felt like a nice quality-of-life feature
> -
> > > will
> > > > remove it.
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > On Tue, Jun 30, 2020 at 4:16 AM Manikumar  >
> > > > wrote:
> > > >
> > > > > Hi Daniel,
> > > > >
> > > > > Thanks for working on this KIP.  Proposed changes looks good to me,
> > > > >
> > > > > minor comments:
> > > > > 1. We use "command-config" option name in most of the cmdline tools
> > to
> > > > pass
> > > > > config
> > > > > properties file. We can use the same name here.
> > > > >
> > > > > 2. Not sure, if we need a separate option to pass an consumer
> > property.
> > > > > fewer options are better.
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > > > On Wed, Jun 24, 2020 at 8:53 PM Dániel Urbán <
> urb.dani...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I see that this KIP turned somewhat inactive - I'd like to pick
> it
> > up
> > > > and
> > > > > > work on it if it is okay.
> > > > > > Part of the work is done, as switching to the Consumer API is
> > already
> > > > in
> > > > > > trunk, but some functionality is still missing.
> > > > > >
> > > > > > I've seen the current PR and the discussion so far, only have a
> few
> > > > > things
> > > > > > to add:
> > > > > > - I like the idea of the topic-partition argument, it would be
> > useful
> > > > to
> > > > > > filter down to specific partitions.
> > > > > > - Instead of a topic list arg, a pattern would be more powerful,
> > and
> > > > also
> > > > > > fit better with the other tools (e.g. how the kafka-topics tool
> > > works).
> > > > > >
> > > > > > Regards,
> > > > > > Daniel
> > > > > >
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-30 Thread Dániel Urbán
Hi Manikumar,
Thanks, went ahead and assigned a new ID, it is KIP-635 now:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-635%3A+GetOffsetShell%3A+support+for+multiple+topics+and+consumer+configuration+override
Daniel

Manikumar  ezt írta (időpont: 2020. jún. 30., K,
16:03):

> Hi,
>
> Yes, we can assign new id to this KIP.
>
> Thanks.
>
> On Tue, Jun 30, 2020 at 6:59 PM Dániel Urbán 
> wrote:
>
> > Hi,
> >
> > To help with the discussion, I also have a PR for this KIP now.
> reflecting
> > the current state of the KIP: https://github.com/apache/kafka/pull/8957.
> > I would like to ask a committer to start the test job on it.
> >
> > One thing I realised though is that there is a KIP id collision, there is
> > another KIP with the same id:
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
> > What is the protocol in this case? Should I acquire a new id for the
> > GetOffsetShell KIP, and update it?
> >
> > Thanks in advance,
> > Daniel
> >
> > Dániel Urbán  ezt írta (időpont: 2020. jún.
> > 30., K, 9:23):
> >
> > > Hi Manikumar,
> > >
> > > Thanks for the comments.
> > > 1. Will change this - thought that "command-config" is used for admin
> > > clients.
> > > 2. It's not necessary, just felt like a nice quality-of-life feature -
> > will
> > > remove it.
> > >
> > > Thanks,
> > > Daniel
> > >
> > > On Tue, Jun 30, 2020 at 4:16 AM Manikumar 
> > > wrote:
> > >
> > > > Hi Daniel,
> > > >
> > > > Thanks for working on this KIP.  Proposed changes looks good to me,
> > > >
> > > > minor comments:
> > > > 1. We use "command-config" option name in most of the cmdline tools
> to
> > > pass
> > > > config
> > > > properties file. We can use the same name here.
> > > >
> > > > 2. Not sure, if we need a separate option to pass an consumer
> property.
> > > > fewer options are better.
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > > > On Wed, Jun 24, 2020 at 8:53 PM Dániel Urbán 
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I see that this KIP turned somewhat inactive - I'd like to pick it
> up
> > > and
> > > > > work on it if it is okay.
> > > > > Part of the work is done, as switching to the Consumer API is
> already
> > > in
> > > > > trunk, but some functionality is still missing.
> > > > >
> > > > > I've seen the current PR and the discussion so far, only have a few
> > > > things
> > > > > to add:
> > > > > - I like the idea of the topic-partition argument, it would be
> useful
> > > to
> > > > > filter down to specific partitions.
> > > > > - Instead of a topic list arg, a pattern would be more powerful,
> and
> > > also
> > > > > fit better with the other tools (e.g. how the kafka-topics tool
> > works).
> > > > >
> > > > > Regards,
> > > > > Daniel
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-30 Thread Dániel Urbán
Hi,

To help with the discussion, I also have a PR for this KIP now. reflecting
the current state of the KIP: https://github.com/apache/kafka/pull/8957.
I would like to ask a committer to start the test job on it.

One thing I realised though is that there is a KIP id collision, there is
another KIP with the same id:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85474993
What is the protocol in this case? Should I acquire a new id for the
GetOffsetShell KIP, and update it?

Thanks in advance,
Daniel

Dániel Urbán  ezt írta (időpont: 2020. jún.
30., K, 9:23):

> Hi Manikumar,
>
> Thanks for the comments.
> 1. Will change this - thought that "command-config" is used for admin
> clients.
> 2. It's not necessary, just felt like a nice quality-of-life feature - will
> remove it.
>
> Thanks,
> Daniel
>
> On Tue, Jun 30, 2020 at 4:16 AM Manikumar 
> wrote:
>
> > Hi Daniel,
> >
> > Thanks for working on this KIP.  Proposed changes looks good to me,
> >
> > minor comments:
> > 1. We use "command-config" option name in most of the cmdline tools to
> pass
> > config
> > properties file. We can use the same name here.
> >
> > 2. Not sure, if we need a separate option to pass an consumer property.
> > fewer options are better.
> >
> > Thanks,
> > Manikumar
> >
> > On Wed, Jun 24, 2020 at 8:53 PM Dániel Urbán 
> > wrote:
> >
> > > Hi,
> > >
> > > I see that this KIP turned somewhat inactive - I'd like to pick it up
> and
> > > work on it if it is okay.
> > > Part of the work is done, as switching to the Consumer API is already
> in
> > > trunk, but some functionality is still missing.
> > >
> > > I've seen the current PR and the discussion so far, only have a few
> > things
> > > to add:
> > > - I like the idea of the topic-partition argument, it would be useful
> to
> > > filter down to specific partitions.
> > > - Instead of a topic list arg, a pattern would be more powerful, and
> also
> > > fit better with the other tools (e.g. how the kafka-topics tool works).
> > >
> > > Regards,
> > > Daniel
> > >
> >
>


Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-30 Thread Dániel Urbán
Hi Manikumar,

Thanks for the comments.
1. Will change this - thought that "command-config" is used for admin
clients.
2. It's not necessary, just felt like a nice quality-of-life feature - will
remove it.

Thanks,
Daniel

On Tue, Jun 30, 2020 at 4:16 AM Manikumar  wrote:

> Hi Daniel,
>
> Thanks for working on this KIP.  Proposed changes looks good to me,
>
> minor comments:
> 1. We use "command-config" option name in most of the cmdline tools to pass
> config
> properties file. We can use the same name here.
>
> 2. Not sure, if we need a separate option to pass an consumer property.
> fewer options are better.
>
> Thanks,
> Manikumar
>
> On Wed, Jun 24, 2020 at 8:53 PM Dániel Urbán 
> wrote:
>
> > Hi,
> >
> > I see that this KIP turned somewhat inactive - I'd like to pick it up and
> > work on it if it is okay.
> > Part of the work is done, as switching to the Consumer API is already in
> > trunk, but some functionality is still missing.
> >
> > I've seen the current PR and the discussion so far, only have a few
> things
> > to add:
> > - I like the idea of the topic-partition argument, it would be useful to
> > filter down to specific partitions.
> > - Instead of a topic list arg, a pattern would be more powerful, and also
> > fit better with the other tools (e.g. how the kafka-topics tool works).
> >
> > Regards,
> > Daniel
> >
>


Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-24 Thread Dániel Urbán
Hi,

I see that this KIP turned somewhat inactive - I'd like to pick it up and
work on it if it is okay.
Part of the work is done, as switching to the Consumer API is already in
trunk, but some functionality is still missing.

I've seen the current PR and the discussion so far, only have a few things
to add:
- I like the idea of the topic-partition argument, it would be useful to
filter down to specific partitions.
- Instead of a topic list arg, a pattern would be more powerful, and also
fit better with the other tools (e.g. how the kafka-topics tool works).

Regards,
Daniel


contributor request

2020-06-24 Thread Dániel Urbán
Hi,
I'd like to work on some KIPs. Can you please add me to the contributors?
My username is durban on both JIRA and Confluence.
Thanks in advance,
Daniel