Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-17 Thread Randall Hauch
Looks great. Thanks again, Arjun! On Tue, Sep 17, 2019 at 1:17 AM Arjun Satish wrote: > Answers inline > > On Mon, Sep 16, 2019 at 5:06 PM Randall Hauch wrote: > > > Thanks for the updates, Arjun. If possible, it'd be great to have the KIP > > clarify a few things: > > > > 1) IIUC, the loggers

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-17 Thread Arjun Satish
Answers inline On Mon, Sep 16, 2019 at 5:06 PM Randall Hauch wrote: > Thanks for the updates, Arjun. If possible, it'd be great to have the KIP > clarify a few things: > > 1) IIUC, the loggers returned by the GET methods are only those loggers > that have been activated/used/set in the JVM. If

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-16 Thread Randall Hauch
Thanks for the updates, Arjun. If possible, it'd be great to have the KIP clarify a few things: 1) IIUC, the loggers returned by the GET methods are only those loggers that have been activated/used/set in the JVM. If this is the case, this should be specified. 2) It's possible to set a log level

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-16 Thread Arjun Satish
Good catch, Randall. Yes, we should be able to set the level of any logger given its name. If this is an ancestor, then the levels of all child classes are updated. I updated the KIP to be more explicit about what loggers we can set, and how they affect child classes, if any. Let me know what you

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-12 Thread Randall Hauch
Thanks for the KIP, Arjun. It's going to be really nice to be able to set the log levels dynamically, especially through the REST API. However, I think it's not clear what behavior the KIP is actually proposing with respect to which loggers are exposed (just those that are used, or common

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-12 Thread Jason Gustafson
Thanks Arjun. +1 On Thu, Sep 12, 2019 at 9:58 AM Gwen Shapira wrote: > The new REST API for logger management looks great to me. > > > On Thu, Sep 12, 2019 at 8:36 AM Arjun Satish > wrote: > > > > Bumping this thread. > > > > If there are no further comments, please add your votes here: > >

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-12 Thread Gwen Shapira
The new REST API for logger management looks great to me. On Thu, Sep 12, 2019 at 8:36 AM Arjun Satish wrote: > > Bumping this thread. > > If there are no further comments, please add your votes here: > https://www.mail-archive.com/dev@kafka.apache.org/msg100313.html > > Thanks in advance, >

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-12 Thread Arjun Satish
Bumping this thread. If there are no further comments, please add your votes here: https://www.mail-archive.com/dev@kafka.apache.org/msg100313.html Thanks in advance, Arjun On Fri, Sep 6, 2019 at 4:22 PM Arjun Satish wrote: > Thanks a lot, Jason! Answers inline. I'll also modify the kip to

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-06 Thread Arjun Satish
Thanks a lot, Jason! Answers inline. I'll also modify the kip to make these clear. On Fri, Sep 6, 2019 at 4:01 PM Jason Gustafson wrote: > Hi Arjun, > > The updated KIP looks good. Just a couple questions: > > 1. Is the /admin endpoint on the normal listener by default? If not, is > there a way

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-09-06 Thread Jason Gustafson
Hi Arjun, The updated KIP looks good. Just a couple questions: 1. Is the /admin endpoint on the normal listener by default? If not, is there a way to have it use the same listener? 2. Changes to logging configuration are not intended to be persistent, is that right? Also, I assume changes only

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-30 Thread Arjun Satish
OK. I didn't realize the plan was to deprecate and remove the JMX endpoint. KIP-412 says brokers will continue to expose the JMX API. JMX was selected so all components could follow the brokers. In light of this, I think we should simply aim for semantic equivalency across the different API for

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-29 Thread Colin McCabe
On Mon, Aug 26, 2019, at 14:03, Jason Gustafson wrote: > Hi Arjun, > > From a high level, I feel like we are making light of the JMX api because > it's convenient and the broker already has it. Personally I would take the > broker out of the picture. The JMX endpoint is not something we were

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-26 Thread Jason Gustafson
Hi Arjun, >From a high level, I feel like we are making light of the JMX api because it's convenient and the broker already has it. Personally I would take the broker out of the picture. The JMX endpoint is not something we were happy with, hence KIP-412. Ultimately I think we will deprecate and

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Arjun Satish
Jason, Thanks for your comments! I understand the usability issues with JMX that you mention. But it was chosen for the following reasons: 1. Cross-cutting functionality across different components (Kafka brokers, Connect workers and even with Streams jobs). If we go down the REST route, then

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Jason Gustafson
Let me elaborate a little bit. We made the decision early on for Connect to use HTTP instead of Kafka's custom RPC protocol. In exchange for losing some hygienic consistency with Kafka, we took easier integration with management tools. The scope of the connect REST APIs is really managing the

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Jason Gustafson
Hi Arjun, Thanks for the KIP. Do we really need a JMX-based API? Is there literally anyone in the world that wants to use JMX if they don't have to? I thought one of the major motivations of KIP-412 was how much of a pain JMX is. Thanks, Jason On Mon, Aug 19, 2019 at 5:28 PM Arjun Satish

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-19 Thread Arjun Satish
Thanks, Konstantine. Updated the KIP with the restrictions around log4j and added references to similar KIPs. Best, On Mon, Aug 19, 2019 at 3:20 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks Arjun, the example is useful! > > My point when I mentioned the restrictions

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-19 Thread Konstantine Karantasis
Thanks Arjun, the example is useful! My point when I mentioned the restrictions around log4j is that this is information is significant and IMO needs to be included in the KIP. Speaking of its relevance to KIP-412, I think a reference would be nice too. Konstantine On Thu, Aug 15, 2019 at

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-15 Thread Arjun Satish
Hey Konstantine, Thanks for the feedback. re: the use of log4j, yes, the proposed changes will only work if log4j is available in runtime. We will not add the mBean if log4j is not available in classpath. If we change from log4j 1 to 2, that would involve another KIP, and it would need to update

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-14 Thread Konstantine Karantasis
And one thing I forgot is also related to Chris's comment above. I agree that an example on how a user is expected to set the log level (for instance to DEBUG) would be nice, even if it's showing only one out of the many possible ways to achieve that. - Konstantine On Wed, Aug 14, 2019 at 4:38

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-14 Thread Konstantine Karantasis
Thanks Arjun for tackling the need to support this very useful feature. One thing I noticed while reading the KIP is that I would have loved to see more info regarding how this proposal depends on the underlying logging APIs and implementations. For instance, my understanding is that slf4j can

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-06 Thread Cyrus Vafadari
This looks like a useful feature, the strategy makes sense, and the KIP is thorough and nicely written. Thanks! Cyrus On Thu, Aug 1, 2019, 12:40 PM Chris Egerton wrote: > Thanks Arjun! Looks good to me. > > On Thu, Aug 1, 2019 at 12:33 PM Arjun Satish > wrote: > > > Thanks for the feedback,

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-01 Thread Chris Egerton
Thanks Arjun! Looks good to me. On Thu, Aug 1, 2019 at 12:33 PM Arjun Satish wrote: > Thanks for the feedback, Chris! > > Yes, the example is pretty much how Connect will use the new feature. > Tweaked the section to make this more clear. > > Best, > > On Fri, Jul 26, 2019 at 11:52 AM Chris

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-01 Thread Arjun Satish
Thanks for the feedback, Chris! Yes, the example is pretty much how Connect will use the new feature. Tweaked the section to make this more clear. Best, On Fri, Jul 26, 2019 at 11:52 AM Chris Egerton wrote: > Hi Arjun, > > This looks great. The changes to public interface are pretty small and

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-07-26 Thread Chris Egerton
Hi Arjun, This looks great. The changes to public interface are pretty small and moving the Log4jController class into the clients package seems like the right way to go. One question I have--it looks like the purpose of this KIP is to enable dynamic setting of log levels in the Connect

[DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-07-23 Thread Arjun Satish
Hi everyone. I'd like to propose the following KIP to implement changing log levels on the fly in Connect workers: https://cwiki.apache.org/confluence/display/KAFKA/KIP-495%3A+Dynamically+Adjust+Log+Levels+in+Connect Would like to hear your thoughts on this. Thanks very much, Arjun