Hi, Given that 5.0 is likely 2 years away and we won't have a branch to make these changes for a while, is it premature to be discussing KIPs such as this one? Perhaps we should wait and see how the logging landscape evolves to decide if we want to go with slf4j 2 or something else.
Ismael On Thu, Apr 10, 2025 at 3:05 AM TengYao Chi <kiting...@gmail.com> wrote: > Hello guys, > > I want to bump this vote thread manually. > Thanks for your attention. > > Best, > TengYao > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月24日 週一 上午3:58寫道： > > > Thanks TengYao, > > > > I note that the KIP's scope has shrunk a bit, and the goal now is only to > > upgrade slf4j to make it easy for users to replace log4j2 with another > > slf4j binding, but Kafka will only ship with log4j2 and not alternatives, > > so people who want to use something else will have to bring their own > jars. > > > > +1 (non-binding, not a committer) for that plan > > > > Den søn. 23. mar. 2025 kl. 12.13 skrev TengYao Chi <kiting...@gmail.com > >: > > > > > Hi everyone > > > > > > I have updated the content of KIP. > > > I want to highlight that although we set log4j2 as the default > > server-side > > > logging framework we couldn't change its scope to implementation in > > > `build.gradle`. > > > This will lead the downstream projects to encounter dependency > conflicts. > > > > > > Please take a look and share your feedback. > > > > > > Best, > > > TengYao > > > > > > TengYao Chi <kiting...@gmail.com> 於 2025年3月23日 週日 下午2:23寫道： > > > > > > > Hi Stig > > > > > > > > Sorry for the late reply, and thanks for your question. 😀 > > > > > > > > The title "Upgrade slf4j to 2.x" represents the core change enabling > > > > improved logging backend selection. > > > > The fundamental change is the SLF4J upgrade, which brings the new > > > provider > > > > selection mechanism (`-Dslf4j.provider`) as a key feature. > > > > Given that, I think the current title reflects the primary technical > > > > change, while the KIP details explain the resulting benefits and > > > > implementation approach. > > > > > > > > Best, > > > > TengYao > > > > > > > > > > > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月21日 週五 > 下午11:46寫道： > > > > > > > >> The title of the KIP seems a little odd, because if I understand > > > >> correctly, > > > >> the main change you want to make is to bundle multiple logging > > backends > > > >> with Kafka and make them selectable via a system property, and > > upgrading > > > >> sfl4j is a means to achieve that, not the goal itself? > > > >> > > > >> Den fre. 21. mar. 2025 kl. 12.18 skrev Chia-Ping Tsai < > > > chia7...@gmail.com > > > >> >: > > > >> > > > >> > hi Teng > > > >> > > > > >> > > The KIP will document that log4j2 is the only officially > supported > > > >> > server-side logging framework, and we will expose its > > configuration > > > >> file > > > >> > to > > > >> > users. > > > >> > > > > >> > on the server-side, we should keep current scope - compileOnly and > > > >> > releaseOnly - Otherwise, downstream projects could encounter > > > dependency > > > >> > conflicts [0] > > > >> > > > > >> > > I will revise the motivation section of the KIP to emphasize > that > > > the > > > >> > > > > >> > Please include the following description. > > > >> > > > > >> > "The rationale for this KIP is that upgrading SLF4J necessitates > > > >> > corresponding provider upgrades, which constitutes a breaking > > change." > > > >> > > > > >> > Also, we must upgrade the Log4j2 dependency based on SLF4J 2 > (i.e., > > > >> > log4j-slf4j-impl to log4j-slf4j2-impl) in 5.0 if we upgrade to > > slf4j2 > > > >> > > > > >> > [0] > > > https://github.com/apache/kafka/pull/17373#issuecomment-2577813317 > > > >> > > > > >> > Best, > > > >> > > > > >> > Chia-Ping > > > >> > > > > >> > > > > >> > TengYao Chi <kiting...@gmail.com> 於 2025年3月21日 週五 下午6:52寫道： > > > >> > > > > >> > > Hello everyone, > > > >> > > > > > >> > > Sorry for the late reply. > > > >> > > Based on the previous discussion, I would like to conclude with > > the > > > >> > > following points: > > > >> > > > > > >> > > 1. The upgrade to slf4j2 should be postponed to Kafka 5.0 due > > to > > > >> > > compatibility issues. > > > >> > > 2. I will revise the motivation section of the KIP to > emphasize > > > >> that > > > >> > the > > > >> > > key benefit is allowing users to select logging backends > > through > > > >> > > configuration rather than modifying JAR files. > > > >> > > 3. After the slf4j upgrade, users will be able to use the > > > >> > > `-Dslf4j.provider` system property to configure their > preferred > > > >> > logging > > > >> > > backend. > > > >> > > 4. The KIP will document that log4j2 is the only officially > > > >> supported > > > >> > > server-side logging framework, and we will expose its > > > configuration > > > >> > > file to > > > >> > > users. > > > >> > > To avoid breaking downstream compatibility, we will not bind > > the > > > >> > client > > > >> > > side to any specific logging framework. Users will need to > > manage > > > >> > their > > > >> > > own > > > >> > > logging libraries, but they can utilize the > `-Dslf4j.provider` > > > >> > property > > > >> > > once slf4j is upgraded. > > > >> > > 5. We have rejected alternatives that involve warnings and > > > >> classpath > > > >> > > ordering as they do not provide a solid solution to > > compatibility > > > >> > > issues. > > > >> > > > > > >> > > Does this summary make sense? > > > >> > > > > > >> > > Best, > > > >> > > TengYao > > > >> > > > > > >> > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月21日 週五 上午9:32寫道： > > > >> > > > > > >> > > > A solution that involves a warning and classpath ordering > > doesn't > > > >> meet > > > >> > > the > > > >> > > > bar for me. Good clarification though. > > > >> > > > > > > >> > > > Ismael > > > >> > > > > > > >> > > > On Thu, Mar 20, 2025 at 8:37 AM Farid Zakaria > > > >> > > > <fzaka...@confluent.io.invalid> > > > >> > > > wrote: > > > >> > > > > > > >> > > > > AFAIR SLF4J you don't have to remove the other backends; > > merely > > > >> make > > > >> > > > > sure yours is first on the CLASSPATH list :P > > > >> > > > > (SLF4J pre 2.0 would always emit a warning that it found 2+ > > > >> > > > StaticBinders) > > > >> > > > > > > > >> > > > > Interestingly, you could still whatever backend (i.e. Log4J) > > and > > > >> pipe > > > >> > > > > it through to another backend via another appender. > > > >> > > > > This is what SLF4J refers to bridges -- although you have to > > be > > > >> sure > > > >> > > > > not to create a circular loop. > > > >> > > > > > > > >> > > > > Then there is something also general like syslog. > > > >> > > > > > > > >> > > > > On Thu, Mar 20, 2025 at 1:15 AM Chia-Ping Tsai < > > > >> chia7...@gmail.com> > > > >> > > > wrote: > > > >> > > > > > > > > >> > > > > > hi Ismael > > > >> > > > > > > > > >> > > > > > thanks for all your response. > > > >> > > > > > > > > >> > > > > > > All that said, I am not actually sure we can do what I > > > >> described > > > >> > > > above > > > >> > > > > > while maintaining the compatibility required by a minor > > > release. > > > >> > > > > > > > > >> > > > > > Excuse me, are your concerns related to version matching, > as > > > >> > > discussed > > > >> > > > in > > > >> > > > > > [0]? If so, I concur that this represents a compatibility > > > issue, > > > >> > and > > > >> > > > the > > > >> > > > > > target version for this change should be 5.0. > Additionally, > > > >> there > > > >> > > was a > > > >> > > > > > related discussion previously documented in [1]. While we > > have > > > >> not > > > >> > > > > strictly > > > >> > > > > > enforced version matching during prior SLF4J updates, this > > KIP > > > >> > > provides > > > >> > > > > an > > > >> > > > > > opportunity to establish guidelines for upgrading sl4fj > that > > > >> have > > > >> > > > direct > > > >> > > > > > compatibility implications. > > > >> > > > > > > > > >> > > > > > [0] https://www.slf4j.org/faq.html#compatibility > > > >> > > > > > [1] > > > >> > > > https://github.com/apache/kafka/pull/16324#discussion_r1644671854 > > > >> > > > > > > > > >> > > > > > To Teng > > > >> > > > > > > > > >> > > > > > Could you please revise the KIP according to following > > > benefit? > > > >> > > > > > > > > >> > > > > > > The key benefit of this KIP is that you can add new > > logging > > > >> > > backends > > > >> > > > > and > > > >> > > > > > select it via a config. This is how most pluggable things > > > work. > > > >> But > > > >> > > it > > > >> > > > is > > > >> > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to > > *remove* > > > >> the > > > >> > > > default > > > >> > > > > > logging library picked by the project as well. That's much > > > more > > > >> > > > > intrusive. > > > >> > > > > > > > > >> > > > > > Best, > > > >> > > > > > Chia-Ping > > > >> > > > > > > > > >> > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月20日 週四 上午8:10寫道： > > > >> > > > > > > > > >> > > > > > > Hi Chia-Ping, > > > >> > > > > > > > > > >> > > > > > > I don't think we're in the business of shipping multiple > > > >> logging > > > >> > > > > libraries. > > > >> > > > > > > Here's my take: > > > >> > > > > > > > > > >> > > > > > > 1. We should pick one logging library for > > services/servers, > > > >> ship > > > >> > it > > > >> > > > and > > > >> > > > > > > include a configuration file for it. > > > >> > > > > > > 2. For clients, we leave it to the users to select the > > > logging > > > >> > > > library > > > >> > > > > - > > > >> > > > > > > clients run alongside applications and it's desirable to > > use > > > >> the > > > >> > > same > > > >> > > > > > > logging library for both (if we were starting from > > scratch, > > > we > > > >> > may > > > >> > > > have > > > >> > > > > > > decided to also include our default logging library for > > > >> clients > > > >> > as > > > >> > > > > well, > > > >> > > > > > > but it's hard to make that change now). > > > >> > > > > > > 3. For the cases where users want to use a different > > logging > > > >> > > library > > > >> > > > > for > > > >> > > > > > > services/servers (perhaps because they have standardized > > on > > > a > > > >> > > > different > > > >> > > > > > > logging library), they would have to add the additional > > jar > > > to > > > >> > the > > > >> > > > > > > classpath and change the relevant logging config. This > is > > no > > > >> > > > different > > > >> > > > > than > > > >> > > > > > > adding a different authorizer or any other pluggable > > > >> component. > > > >> > > > > > > > > > >> > > > > > > The key benefit of this KIP is that you can add new > > logging > > > >> > > backends > > > >> > > > > and > > > >> > > > > > > select it via a config. This is how most pluggable > things > > > >> work. > > > >> > But > > > >> > > > it > > > >> > > > > is > > > >> > > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to > > > *remove* > > > >> the > > > >> > > > > default > > > >> > > > > > > logging library picked by the project as well. That's > much > > > >> more > > > >> > > > > intrusive. > > > >> > > > > > > > > > >> > > > > > > All that said, I am not actually sure we can do what I > > > >> described > > > >> > > > above > > > >> > > > > > > while maintaining the compatibility required by a minor > > > >> release. > > > >> > > > > > > > > > >> > > > > > > Ismael > > > >> > > > > > > > > > >> > > > > > > On Wed, Mar 19, 2025, 2:03 PM Chia-Ping Tsai < > > > >> chia7...@gmail.com > > > >> > > > > > >> > > > > wrote: > > > >> > > > > > > > > > >> > > > > > > > hi Ismael > > > >> > > > > > > > > > > >> > > > > > > > > but they must also add whichever logging library > they > > > >> want to > > > >> > > > use. > > > >> > > > > > > > > > > >> > > > > > > > If users are required to modify JAR files to alter the > > > SLF4J > > > >> > > > > provider, > > > >> > > > > > > the > > > >> > > > > > > > value of this KIP is significantly diminished. I > believe > > > the > > > >> > > > primary > > > >> > > > > > > > benefit of this KIP lies in enabling users to > configure > > a > > > >> > system > > > >> > > > > property > > > >> > > > > > > > for SLF4J provider changes without JAR modifications. > > > >> > > Furthermore, > > > >> > > > by > > > >> > > > > > > > managing all SLF4J and provider JARs within Kafka, we > > can > > > >> > ensure > > > >> > > > > SLF4J > > > >> > > > > > > > version updates without compatibility concerns, as we > > can > > > >> > > guarantee > > > >> > > > > > > > provider JAR consistency with the SLF4J version in the > > > >> > > > distribution. > > > >> > > > > > > > > > > >> > > > > > > > Best, > > > >> > > > > > > > Chia-Ping > > > >> > > > > > > > > > > >> > > > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月19日 週三 > > > 下午12:00寫道： > > > >> > > > > > > > > > > >> > > > > > > > > Hi TengYao, > > > >> > > > > > > > > > > > >> > > > > > > > > It's a bit difficult to review the KIP. I don't > follow > > > >> most > > > >> > of > > > >> > > > the > > > >> > > > > > > > > motivation. The only one that I follow is: > > > >> > > > > > > > > > > > >> > > > > > > > > "Our current build configuration employs fragile > > > >> dependency > > > >> > > > > management > > > >> > > > > > > > > tricks to handle SLF4J backends. We can eliminate > > these > > > >> > brittle > > > >> > > > > build > > > >> > > > > > > > > mechanisms by transitioning to explicit provider > > > >> dependencies > > > >> > > > after > > > >> > > > > > > > > upgrading to 2.0." > > > >> > > > > > > > > > > > >> > > > > > > > > Additionally, I don't think we should do the > > following: > > > >> > > > > > > > > > > > >> > > > > > > > > "Add other popular slf4j backend binding provider > > > >> > > dependencies." > > > >> > > > > > > > > > > > >> > > > > > > > > A reasonable approach, in my opinion, would be: > > > >> > > > > > > > > > > > >> > > > > > > > > 1. Include the log4j2 dependency with the server > > modules > > > >> and > > > >> > > not > > > >> > > > > > > include > > > >> > > > > > > > > them with the client modules. > > > >> > > > > > > > > 2. Automatically configure the log4j2 dependency for > > the > > > >> > server > > > >> > > > > > > modules. > > > >> > > > > > > > > Users can override them via the system property, but > > > they > > > >> > must > > > >> > > > > also add > > > >> > > > > > > > > whichever logging library they want to use. > > > >> > > > > > > > > 3. Somehow configure slf4j 2.x to work like 1.x out > of > > > the > > > >> > box > > > >> > > > for > > > >> > > > > the > > > >> > > > > > > > > client module (for compatibility reasons). > > > >> > > > > > > > > > > > >> > > > > > > > > But I don't know if `3` is possible. If `3` is not > > > >> possible, > > > >> > I > > > >> > > > > don't > > > >> > > > > > > see > > > >> > > > > > > > > how we can make this a compatible change. > > > >> > > > > > > > > > > > >> > > > > > > > > Ismael > > > >> > > > > > > > > > > > >> > > > > > > > > On Tue, Mar 18, 2025 at 7:41 PM TengYao Chi < > > > >> > > kiting...@gmail.com > > > >> > > > > > > > >> > > > > > > wrote: > > > >> > > > > > > > > > > > >> > > > > > > > > > Hello everyone, > > > >> > > > > > > > > > > > > >> > > > > > > > > > I want to bump this thread manually. > > > >> > > > > > > > > > Any feedback or vote would be appreciated. > > > >> > > > > > > > > > > > > >> > > > > > > > > > Best, > > > >> > > > > > > > > > TengYao > > > >> > > > > > > > > > > > > >> > > > > > > > > > TengYao Chi <kiting...@gmail.com> 於 2025年3月10日 週一 > > > >> > 上午11:47寫道： > > > >> > > > > > > > > > > > > >> > > > > > > > > > > Hello guys, > > > >> > > > > > > > > > > I would like to remind you in the vote thread > that > > > the > > > >> > KIP > > > >> > > > has > > > >> > > > > been > > > >> > > > > > > > > > > updated, and I apologize for repeating it. > > > >> > > > > > > > > > > I have taken over this KIP from Muralidhar. > > > >> > > > > > > > > > > Since the original content is outdated as the > > > logging > > > >> > > > > framework has > > > >> > > > > > > > > been > > > >> > > > > > > > > > > widely changed, I have updated the content of > the > > > KIP. > > > >> > > > > > > > > > > Please take a look and share your thoughts. > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > Best Regards, > > > >> > > > > > > > > > > TengYao > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > Muralidhar Basani <muralidhar.bas...@aiven.io > > > .invalid> > > > >> 於 > > > >> > > > > > > 2024年9月24日 > > > >> > > > > > > > 週二 > > > >> > > > > > > > > > > 上午5:09寫道： > > > >> > > > > > > > > > > > > > >> > > > > > > > > > >> Hi, > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > >> I wanted to gently follow up on this thread in > > case > > > >> > anyone > > > >> > > > > has any > > > >> > > > > > > > > > >> thoughts > > > >> > > > > > > > > > >> or would like to take a look. > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > >> Thanks, > > > >> > > > > > > > > > >> Murali > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > >> On Mon, Sep 16, 2024 at 10:23 AM Muralidhar > > Basani > > > < > > > >> > > > > > > > > > >> muralidhar.bas...@aiven.io> wrote: > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > >> > Thanks Chia. > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > I have updated KIP with this quote, in the > > > >> migration > > > >> > > plan > > > >> > > > > > > section. > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > Thanks, > > > >> > > > > > > > > > >> > Murali > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > On Sun, Sep 15, 2024 at 3:30 PM Chia-Ping > Tsai > > < > > > >> > > > > > > > chia7...@gmail.com> > > > >> > > > > > > > > > >> wrote: > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> >> > > > >> > > > > > > > > > >> >> > Muralidhar Basani < > > muralidhar.bas...@aiven.io > > > >> > > .invalid> > > > >> > > > 於 > > > >> > > > > > > > > > 2024年9月15日 > > > >> > > > > > > > > > >> >> 晚上9:02 寫道： > > > >> > > > > > > > > > >> >> > > > > >> > > > > > > > > > >> >> > With this, I think, users don't have to > make > > > any > > > >> > > > explicit > > > >> > > > > > > > changes > > > >> > > > > > > > > > in > > > >> > > > > > > > > > >> >> their > > > >> > > > > > > > > > >> >> > code, if their provider is reload4j. And > if > > > >> it's a > > > >> > > > > different > > > >> > > > > > > > > > provider > > > >> > > > > > > > > > >> >> (like > > > >> > > > > > > > > > >> >> > logback, log4j), they would have to > upgrade > > > >> that to > > > >> > > > > match it > > > >> > > > > > > > with > > > >> > > > > > > > > > >> slf4j. > > > >> > > > > > > > > > >> >> > > > >> > > > > > > > > > >> >> If upgrading the matched provider is the > only > > > >> > explicit > > > >> > > > > change > > > >> > > > > > > and > > > >> > > > > > > > > we > > > >> > > > > > > > > > >> >> expect users have responsibility to keep > > > >> consistent > > > >> > > > version > > > >> > > > > > > when > > > >> > > > > > > > > > using > > > >> > > > > > > > > > >> >> other providers , could we write it down to > > the > > > >> KIP? > > > >> > > > > > > > > > >> >> > > > >> > > > > > > > > > >> >> That means we will update slf4j without KIP > in > > > the > > > >> > > future > > > >> > > > > > > except > > > >> > > > > > > > > for > > > >> > > > > > > > > > >> >> specific reason. > > > >> > > > > > > > > > >> >> > > > >> > > > > > > > > > >> >> Best, > > > >> > > > > > > > > > >> >> Chia-Ping > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > >
