[jira] [Created] (KAFKA-13960) ERROR Shutdown broker because all log dirs in c:\kafka\kafka-logs have failed (kafka.log.LogManager)

rohit k created KAFKA-13960: --- Summary: ERROR Shutdown broker because all log dirs in c:\kafka\kafka-logs have failed (kafka.log.LogManager) Key: KAFKA-13960 URL: https://issues.apache.org/jira/browse/KAFKA-13960

[jira] [Resolved] (KAFKA-13926) Proposal to have "HasField" predicate for kafka connect

[ https://issues.apache.org/jira/browse/KAFKA-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumud Kumar Srivatsava Tirupati resolved KAFKA-13926. - Resolution: Won't Fix Dropping in favor of improving

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

Hi Chris, Thanks for your comment. I might have misunderstood the filter SMT. Makes sense. Dropping this KIP for now. I will look at improving the existing SMTs separately. *---* *Thanks and Regards,* *Kumud Kumar Srivatsava Tirupati* On Sat, 4 Jun 2022 at 03:57, Chris Egerton wrote: > Hi

Re: Re-run Jenkins build

Hi José, I think you should have the same permissions as I have, and I can login jenkins. Anyway, I can help you re-run the jenkins build. Just point me the jenkins build link. Or you can actually merge with the latest trunk to trigger the build. Thank you. Luke On Sat, Jun 4, 2022 at 5:52 AM

Re: [DISCUSS] KIP-831: Add metric for log recovery progress

Hi Jun, Thanks for the comment. I've updated the KIP as: 1. remainingLogsToRecover*y* -> remainingLogsToRecover 2. remainingSegmentsToRecover*y* -> remainingSegmentsToRecover 3. The description of remainingSegmentsToRecover: The remaining segments for the current log assigned to the recovery

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #973

See

Re-run Jenkins build

Hey all, I am trying to re-run a Jenkins build. It looks like my Apache login doesn't work with Jenkins. Do I need to ask a Kafka PMC to add me to the Jenkins infrastructure? I see the following from an Apache wiki page. How do I get an account? Jenkins uses the Apache LDAP servers for

Re: [DISCUSS] Apache Kafka 3.3.0 Release

Thanks for the updates everyone. I added KIP-618, KIP-841, KIP-827 and KIP-834 to the planned KIPs for 3.3.0. David Jacot, it looks like KIP-841 is not linked from the "Kafka Improvement Proposals" page: https://cwiki.apache.org/confluence/x/4QwIAw -- -José

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #972

See

Re: [DISCUSS] Apache Kafka 3.3.0 Release

Hi José, Can we add KIP-834 as well? The vote has passed and I have PR in progress. Cheers, Jim On Fri, Jun 3, 2022 at 1:05 PM Mickael Maison wrote: > Hi José, > > Can we also add KIP-827? The vote has passed and I opened a PR today. > > Thanks, > Mickael > > On Fri, Jun 3, 2022 at 5:43 PM

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Thank you, Mickael. One more question: are you imaging these tooling/automation to call this API at a very low frequency? since high-frequency calls to this API are prohibitively expensive. Can you give some examples of low-frequency call use cases? I can think of some high-frequency call use

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #971

See Changes: -- [...truncated 632902 lines...] [2022-06-03T13:19:03.009Z] [2022-06-03T13:19:03.009Z]

[jira] [Created] (KAFKA-13959) Controller should unfence Broker with busy metadata log

Jose Armando Garcia Sancio created KAFKA-13959: -- Summary: Controller should unfence Broker with busy metadata log Key: KAFKA-13959 URL: https://issues.apache.org/jira/browse/KAFKA-13959

Re: [DISCUSS] Apache Kafka 3.3.0 Release

Hi José, Can we also add KIP-827? The vote has passed and I opened a PR today. Thanks, Mickael On Fri, Jun 3, 2022 at 5:43 PM David Jacot wrote: > > Hi José, > > KIP-841 has been accepted. Could we add it to the release plan? > > Thanks, > David > > On Wed, May 11, 2022 at 11:04 PM Chris

Re: [DISCUSS] KIP-831: Add metric for log recovery progress

Hi, Luke, Thanks for the explanation. 10. It makes sense to me now. Instead of using a longer name, perhaps we could keep the current name, but make the description clear that it's the remaining segments for the current log assigned to a thread. Also, would it be better to use ToRecover instead

[jira] [Resolved] (KAFKA-13803) Refactor Leader API Access

[ https://issues.apache.org/jira/browse/KAFKA-13803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-13803. - Fix Version/s: 3.3.0 Resolution: Fixed merged the PR to trunk > Refactor Leader API Access >

Kafka producer fails with expiring 2 record(s) for myTopic-1:120004 ms has passed since batch creation

0 I have an issue with kafka producer in production where I see below error. *"Publish failed, Expiring 2 record(s) for myTopic-1:120004 ms has passed since batch creation[[ org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s)

Re: [DISCUSS] Apache Kafka 3.3.0 Release

Hi José, KIP-841 has been accepted. Could we add it to the release plan? Thanks, David On Wed, May 11, 2022 at 11:04 PM Chris Egerton wrote: > > Hi José, > > Could we add KIP-618 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors) > to

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Thanks all! The vote passes with binding +1 votes from Colin, David, José and myself. David On Fri, Jun 3, 2022 at 2:14 AM José Armando García Sancio wrote: > > Thanks for proposing this improvement David Jacot. I think it is going > to make the graceful shutdown process much more efficient. >

Re: Log4j 1.2 update timeline?

Any word on this timeline by chance? Michael Cleveland (He/Him/His) Senior Mac Support Engineer, Engineering Working from Syracuse, New York Office: (516) 600-0927 BreadFinancial.com Alliance Data is now Bread Financial

[jira] [Resolved] (KAFKA-13883) KIP-835: Monitor Quorum

[ https://issues.apache.org/jira/browse/KAFKA-13883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-13883. Resolution: Fixed > KIP-835: Monitor Quorum >

[jira] [Resolved] (KAFKA-13918) Schedule or cancel nooprecord write on metadata version change

[ https://issues.apache.org/jira/browse/KAFKA-13918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-13918. Resolution: Duplicate > Schedule or cancel nooprecord write on

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

Hi Chris, Thanks for the explanation. This clears my thoughts. I can now agree that the concerns are totally different for SMTs and predicates. I also do agree with you that this might encourage SMTs to be poorly designed. Do you see this worth considering just for the filter use case?

Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

Hi Kumud, Responses inline. > But, I still believe this logic of predicate checking shouldn't be made a part of each individual SMT. After all, that is what the predicates are for right? I don't quite agree. I think the benefit of predicates is that they can allow you to selectively apply a

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi Cong, Maybe some people can do without this KIP. But in many cases, especially around tooling and automation, it's useful to be able to retrieve disk utilization values via the Kafka API rather than interfacing with a metrics system. Does that clarify the motivation? Thanks, Mickael On Wed,

[jira] [Created] (KAFKA-13958) Expose logdirs total and usable space via Kafka API

Mickael Maison created KAFKA-13958: -- Summary: Expose logdirs total and usable space via Kafka API Key: KAFKA-13958 URL: https://issues.apache.org/jira/browse/KAFKA-13958 Project: Kafka

Re: Log4j 1.2 update timeline?

Hi Michael, KIP-653 seeks to update the log4jv1 dependency to log4jv2. Following a discussion a month or so ago [1] I think the plan is to do this in Kafka 4.0. The discussion for KIP-833 [2] suggests we'll see Kafka 3.5 followed by Kafka 4.0, which should be around August 2023 (assuming we stick

Re: [DISCUSS] KIP-831: Add metric for log recovery progress

Hi Jun, > how do we implement kafka.log :type=LogManager,name=remainingSegmentsToRecovery,dir=([-._\/\w\d\s]+),threadNum=([0-9]+), which tracks at the segment level? It looks like the name is misleading. Suppose we have 2 log recovery threads (num.recovery.threads.per.data.dir=2), and 10 logs to

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

That's correct. David On Fri, Jun 3, 2022 at 2:11 AM José Armando García Sancio wrote: > > David Jacot wrote: > > At the moment, the KIP stipulates that the broker remains in > > InControlledShutdown state until it is re-registered with a new > > incarnation id. This implies that a broker can