Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #421

See Changes: [Ismael Juma] MINOR: Fix MirrorMakerConfigTest build failures -- [...truncated 7.13 MB...] LogValidatorTest > testUncompressedBatchWithoutRecord

Re: [VOTE] KIP-691: Transactional Producer Exception Handling

Thanks Boyang for the thorough discussion. The current KIP LGTM. On Thu, Jan 28, 2021 at 1:36 PM Boyang Chen wrote: > Hey everyone, > > I would like to start the voting for KIP-691 to improve transactional > producer exception handling: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-6

[jira] [Created] (KAFKA-12250) Add metadata record serde logic for KIP-631

Jason Gustafson created KAFKA-12250: --- Summary: Add metadata record serde logic for KIP-631 Key: KAFKA-12250 URL: https://issues.apache.org/jira/browse/KAFKA-12250 Project: Kafka Issue Type:

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #453

See Changes: [Ismael Juma] MINOR: Fix MirrorMakerConfigTest build failures -- [...truncated 3.56 MB...] ControllerEventManagerTest > testMetricsCleanedOnClo

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #470

See Changes: [Ismael Juma] MINOR: Fix MirrorMakerConfigTest build failures -- [...truncated 3.58 MB...] ZooKeeperClientTest > testSetDataNonExistentZNode() P

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #420

See Changes: [github] KAFKA-12237; Support lazy initialization of quorum voter addresses (#9985) -- [...truncated 3.60 MB...] LogValidatorTest > testOffsetAs

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #452

See Changes: [github] KAFKA-12237; Support lazy initialization of quorum voter addresses (#9985) -- [...truncated 3.62 MB...] AssignmentStateTest > [7] isr

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #469

See Changes: [github] KAFKA-12237; Support lazy initialization of quorum voter addresses (#9985) -- [...truncated 3.62 MB...] PASSED DefaultMessageFormatt

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Hey Guozhang, the CommitFailedException wrapping would cover all the non-fatal exceptions as we listed out in the KIP, generally speaking any exception that could recover safely by calling abortTxn should be wrapped. Best, Boyang On Thu, Jan 28, 2021 at 5:22 PM Guozhang Wang wrote: > Hey Boyan

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #451

See Changes: [github] KAFKA-10710; MM2 - Create herders only if source->target.enabled=true and heartbeats are disabled (#9589) -- [...truncated 3.62 MB...]

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #419

See Changes: [github] KAFKA-10710; MM2 - Create herders only if source->target.enabled=true and heartbeats are disabled (#9589) -- [...truncated 3.60 MB...]

[jira] [Created] (KAFKA-12249) KIP-500: Add client-side Decommission Broker RPC

Alok Nikhil created KAFKA-12249: --- Summary: KIP-500: Add client-side Decommission Broker RPC Key: KAFKA-12249 URL: https://issues.apache.org/jira/browse/KAFKA-12249 Project: Kafka Issue Type: Ta

[jira] [Created] (KAFKA-12248) KIP-500: Add broker heartbeat and registration RPCs

Alok Nikhil created KAFKA-12248: --- Summary: KIP-500: Add broker heartbeat and registration RPCs Key: KAFKA-12248 URL: https://issues.apache.org/jira/browse/KAFKA-12248 Project: Kafka Issue Type:

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #468

See Changes: [github] KAFKA-10710; MM2 - Create herders only if source->target.enabled=true and heartbeats are disabled (#9589) -- [...truncated 3.62 MB...]

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Hey Boyang, I think maybe there's something cracking here :) I'm just asking for clarifications that as of today, which non-fatal exceptions the newly introduced CommitFailedException would cover, and it seems to be only 1) unknown pid, 2) invalid pid mapping, and 3) concurrent transactions. Is th

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Hey Guozhang, I think TimeoutException would not be covered here as it potentially has a risk of hitting an illegal state on the broker side when the previous commit was actually successful. Users should try to increase their max.block.ms to avoid hitting the timeout as a base suggestion, which is

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Thanks Jason for the suggestion, that looks good to me too. Regarding the non-fatal exceptions wrapped as CommitFailed, I would like to clarify if we would cover all the following cases: 1) timeout, 2) unknown pid, 3) invalid pid mapping, 4) concurrent transactions? BTW I think it still makes sen

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #450

See Changes: [github] KAFKA-8744: Update Scala API to give names to processors (#9738) [github] MINOR: Build modules in parallel (#9975) -- [...truncated 3.

[VOTE] KIP-691: Transactional Producer Exception Handling

Hey everyone, I would like to start the voting for KIP-691 to improve transactional producer exception handling: https://cwiki.apache.org/confluence/display/KAFKA/KIP-691%3A+Enhance+Transactional+Producer+Exception+Handling let me know your thoughts, thanks! Boyang

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Thanks Jason, I agree with the proposed solution here, will update the KIP. On Thu, Jan 28, 2021 at 10:52 AM Jason Gustafson wrote: > Hi Boyang, > > It seems like a reasonable suggestion. I wonder if a flag is sufficient > though. The current `Callback` documentation treats "fatal" errors from t

Re: Impact of setting max.inflight.requests.per.connection > 1 in Kafka connect

I guess you wpuld get duplicates of you crash after data was written into the topics but before offsets were committed. So there is no data-loss nor re-ordering for this case, but duplication. -Matthias On 1/28/21 11:20 AM, nitin agarwal wrote: > Hi, > > By committing the offsets, I meant trac

Re: Impact of setting max.inflight.requests.per.connection > 1 in Kafka connect

Hi, By committing the offsets, I meant tracking the progress of how much data is read from the upstream system. In Kafka Connect this is being referred as committing the offsets. This is the method I was talking about https://github.com/a0x8o/kafka/blob/master/connect/runtime/src/main/java/org/apa

Re: [DISCUSS] KIP-691: Transactional Producer Exception Handling

Hi Boyang, It seems like a reasonable suggestion. I wonder if a flag is sufficient though. The current `Callback` documentation treats "fatal" errors from the perspective of the individual message that was sent. ``` * Non-Retriable exceptions (fatal, the message will never be

Re: [DISCUSS] Apache Kafka 2.6.2 release

Thanks so much for stepping up, Sophie! I'm +1 -John On Wed, 2021-01-27 at 17:59 -0500, Bill Bejeck wrote: > Thanks for taking this on Sophie. +1 > > Bill > > On Wed, Jan 27, 2021 at 5:59 PM Ismael Juma wrote: > > > Thanks Sophie! +1 > > > > Ismael > > > > On Wed, Jan 27, 2021 at 2:45 PM S

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #467

See Changes: [github] KAFKA-8744: Update Scala API to give names to processors (#9738) [github] MINOR: Build modules in parallel (#9975) -- [...truncated 3.

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Thanks, Levani! I was reflecting more on your KIP last night. One thing I should mention is that I have previously used the rack awareness feature of Elasticsearch, and found it to be pretty intuitive and also capable of what we needed in our AWS clusters. As you look at related work, you might t

Re: [VOTE] KIP-698: Add Explicit User Initialization of Broker-side State to Kafka Streams

+1 again from me. On Thu, Jan 28, 2021 at 8:06 AM John Roesler wrote: > Thank you for the KIP, Bruno! > > I'm +1 (binding) > > -John > > On Wed, 2021-01-27 at 14:18 -0600, Leah Thomas wrote: > > Hi Bruno, > > I'm still +1, non-binding. Thanks for the updates! > > > > Leah > > > > On Wed, Jan 27,

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #418

See Changes: [github] KAFKA-8744: Update Scala API to give names to processors (#9738) [github] MINOR: Build modules in parallel (#9975) -- [...truncated 3.5

[jira] [Created] (KAFKA-12247) Make removeStreamThread work better with static membership

Walker Carlson created KAFKA-12247: -- Summary: Make removeStreamThread work better with static membership Key: KAFKA-12247 URL: https://issues.apache.org/jira/browse/KAFKA-12247 Project: Kafka

Re: Impact of setting max.inflight.requests.per.connection > 1 in Kafka connect

I don't know all details of Connect... However, not sure what you mean by "committing offsets"? A source connector takes data from an external data source and writes it into a Kafka topic. Thus, there should not be any offsets to be committed. (Committing offsets only applies if you read from a t

Re: [VOTE] KIP-698: Add Explicit User Initialization of Broker-side State to Kafka Streams

Thank you for the KIP, Bruno! I'm +1 (binding) -John On Wed, 2021-01-27 at 14:18 -0600, Leah Thomas wrote: > Hi Bruno, > I'm still +1, non-binding. Thanks for the updates! > > Leah > > On Wed, Jan 27, 2021 at 1:56 PM Matthias J. Sax wrote: > > > Thanks for updating the KIP. > > > > +1 (bind

Jenkins build is back to normal : Kafka » kafka-trunk-jdk15 #466

See

Please help | KIP-500 | KAFKA-9119

Hi Team, Can you please help on the timelines of the below? Is it confirmed for Apache Kafka version 2.8 release. In case of any delays when do you anticipate the release. Please let me know. It would be really helpful. KIP-500: Replace ZooKe

[jira] [Resolved] (KAFKA-9974) Flaky Test OptimizedKTableIntegrationTest#shouldApplyUpdatesToStandbyStore

[ https://issues.apache.org/jira/browse/KAFKA-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña resolved KAFKA-9974. Resolution: Cannot Reproduce > Flaky Test OptimizedKTableIntegrationTest#shouldApplyUpdatesToStandb

[GitHub] [kafka-site] mimaison merged pull request #325: MINOR: Remove redundant apostrophe in doc

mimaison merged pull request #325: URL: https://github.com/apache/kafka-site/pull/325 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[jira] [Resolved] (KAFKA-6223) Please delete old releases from mirroring system

[ https://issues.apache.org/jira/browse/KAFKA-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-6223. --- Resolution: Fixed > Please delete old releases from mirroring system > --

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #417

See Changes: [github] KAFKA-10658 ErrantRecordReporter.report always return completed futur… (#9525) [github] MINOR: remove duplicate code of serializing auto-generated data (#9964) --

Jenkins build is back to normal : Kafka » kafka-2.7-jdk8 #109

See

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi John Thanks for the feedback (and for the great work on KIP441 :) ). Makes sense, will add a section in the KIP explaining rack awarenesses on high level and how it’s implemented in the different distributed systems. Thanks, Levani > On 27. Jan 2021, at 16:07, John Roesler wrote: > > Hi L

[GitHub] [kafka-site] mimaison opened a new pull request #327: Use downloads.apache.org for current releases

mimaison opened a new pull request #327: URL: https://github.com/apache/kafka-site/pull/327 As per https://www.apache.org/legal/release-policy.html, projects should use downloads.apache.org for current release artifacts instead of www.apache.org./dist. ---

[DISCUSS] KIP-699: Update FindCoordinator to resolve multiple Coordinators at a time

Hi all, I would like to start a discussion on KIP-699: https://cwiki.apache.org/confluence/display/KAFKA/KIP-699%3A+Update+FindCoordinator+to+resolve+multiple+Coordinators+at+a+time This will allow finding coordinators for multiple resources in a single FindCoordinator request. Let me know if yo

Build failed in Jenkins: Kafka » kafka-2.6-jdk8 #78

See Changes: [Chia-Ping Tsai] KAFKA-10658 ErrantRecordReporter.report always return completed futur… (#9525) -- [...truncated 6.34 MB...] org.apache.kafka.strea

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #416

See Changes: [github] MINOR: Upgrade to Scala 2.12.13 (#9981) [github] KAFKA-10867: Improved task idling (#9840) -- [...truncated 7.10 MB...] OutputVerifierT

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #465

See Changes: [github] KAFKA-10867: Improved task idling (#9840) [github] KAFKA-10658 ErrantRecordReporter.report always return completed futur… (#9525) --

[jira] [Created] (KAFKA-12246) Remove redundant comments in KafkaAdminClient

YI-CHEN WANG created KAFKA-12246: Summary: Remove redundant comments in KafkaAdminClient Key: KAFKA-12246 URL: https://issues.apache.org/jira/browse/KAFKA-12246 Project: Kafka Issue Type: Imp

[jira] [Created] (KAFKA-12245) fix flay org.apache.kafka.clients.consumer.internals.FetcherTest.testEarlierOffsetResetArrivesLate

Chia-Ping Tsai created KAFKA-12245: -- Summary: fix flay org.apache.kafka.clients.consumer.internals.FetcherTest.testEarlierOffsetResetArrivesLate Key: KAFKA-12245 URL: https://issues.apache.org/jira/browse/KAFKA-1

Jenkins build is back to normal : Kafka » kafka-trunk-jdk11 #447

See

Re: Impact of setting max.inflight.requests.per.connection > 1 in Kafka connect

Assuming the configurations are as follows: max.inflight.requests.per.connection=1 enable.idempotence=false Thanks, Nitin On Thu, Jan 28, 2021 at 1:53 PM nitin agarwal wrote: > Thanks for quick reply, I have understood this behaviour now. > I have another follow up question. > > Can the Source

Re: Impact of setting max.inflight.requests.per.connection > 1 in Kafka connect

Thanks for quick reply, I have understood this behaviour now. I have another follow up question. Can the Source connector write out of order messages in a case where there is a failure in committing the offset and the connector is restarted at the same time? Thanks, Nitin On Thu, Jan 28, 2021 at