Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #532

See Changes: [github] KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024) [github] MINOR: Move `RequestChannel.Response` creation logic into `RequestChannel` (#9912)

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #31

See Changes: [Chia-Ping Tsai] HOTFIX: Fix build error caused by ControllerApisTest.scala (#10146) -- [...truncated 3.61 MB...] LogTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #557

See Changes: [github] KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024) [github] MINOR: Move `RequestChannel.Response` creation logic into `RequestChannel` (#9912)

Re: [DISCUSS] KIP-715: Expose Committed offset in streams

Hello Walker, thanks for the KIP. A few thoughts: 1) Have you considered just relying on the `KafkaStreams#metrics()` that includes embedded consumer metrics that have the committed offsets instead of adding a new API? Not advocating that this is a better approach but want to make sure we

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Hello John, Thanks for the new PR, and sorry for zig-zagging this KIP design. I made a pass on the PR and it looks good to me overall. Left some minor comments. Guozhang On Mon, Feb 22, 2021 at 9:06 AM John Roesler wrote: > Hello all, > > I have updated the KIP now. The diff is visible here:

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #501

See Changes: [github] KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024) -- [...truncated 3.65 MB...] KafkaZkClientTest >

Jenkins build is back to normal : Kafka » kafka-trunk-jdk11 #531

See

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #30

See Changes: [Chia-Ping Tsai] KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024) -- [...truncated 1.80 MB...]

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #556

See Changes: [github] MINOR: fix some ducktape test issues (#10181) [github] KAFKA-12160: Remove max.poll.interval.ms from config docs (#10182) --

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #500

See Changes: [github] KAFKA-12160: Remove max.poll.interval.ms from config docs (#10182) -- [...truncated 3.65 MB...] SimpleAclAuthorizerTest >

Re: [DISCUSS] Kafka 3.0

+1 on getting to 3.0 for the June release this year too. Guozhang On Mon, Feb 22, 2021 at 6:54 PM Matthias J. Sax wrote: > To move this forward, I took the liberty to create a PR to bump the > version to 3.0.0-SNAPSHOT > > https://github.com/apache/kafka/pull/10186 > > Please let us know if

[jira] [Resolved] (KAFKA-12273) InterBrokerSendThread#pollOnce throws FatalExitError even though it is shutdown correctly

[ https://issues.apache.org/jira/browse/KAFKA-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12273. Resolution: Fixed > InterBrokerSendThread#pollOnce throws FatalExitError even though it

Re: [DISCUSS] Kafka 3.0

To move this forward, I took the liberty to create a PR to bump the version to 3.0.0-SNAPSHOT https://github.com/apache/kafka/pull/10186 Please let us know if there are any concerns. -Matthias On 2/16/21 5:18 PM, Ismael Juma wrote: > I'm +1 on 3.0 for the mid year release. > > On Tue, Feb

[jira] [Resolved] (KAFKA-10665) Flaky Test StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumberOfPartitions[Optimization = all]

[ https://issues.apache.org/jira/browse/KAFKA-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-10665. --- Resolution: Fixed > Flaky Test >

Re: Create KIP Permission

Marco, thanks for your interest. I could not find a user `marcolotz` -- note, that the wiki and Jira are two separate accounts. Thus, you need to create a wiki account in addition to your Jira account. -Matthias On 2/11/21 1:31 AM, Marco Aurélio Lotz wrote: > Dear Kafka Confluence Admin, > >

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #29

See Changes: [Matthias J. Sax] KAFKA-12160: Remove max.poll.interval.ms from config docs (#10182) -- [...truncated 1.80 MB...] QuorumStateTest >

Re: [DISCUSS] KIP-715: Expose Committed offset in streams

Thanks for the KIP! I personally think, that it might be sufficient to just report offsets of assigned tasks. Similar to metrics what are also reported only locally, users can roll-up/aggregate the information across instances manually. What I also don't understand is, what "idling" means?

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #555

See Changes: [github] MINOR: Test the new KIP-500 quorum mode in ducktape (#10105) -- [...truncated 7.32 MB...] UserQuotaTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #530

See Changes: [github] MINOR: fix some ducktape test issues (#10181) -- [...truncated 3.66 MB...] KafkaZkClientTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #499

See Changes: [github] MINOR: fix some ducktape test issues (#10181) -- [...truncated 3.64 MB...] TransactionsTest >

[jira] [Resolved] (KAFKA-12160) KafkaStreams configs are documented incorrectly

[ https://issues.apache.org/jira/browse/KAFKA-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12160. - Fix Version/s: 2.8.0 Resolution: Fixed > KafkaStreams configs are documented

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

This is a great explanation, thank you! On Mon, Feb 22, 2021 at 2:44 PM Chris Egerton wrote: > Hi Guozhang, > > Your understanding should be correct in most cases, but there are two finer > points that may interest you: > > 1. It's technically dependent on the implementation of the connector

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #529

See Changes: [github] MINOR: Test the new KIP-500 quorum mode in ducktape (#10105) -- [...truncated 3.67 MB...] KafkaZkClientTest >

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #28

See Changes: [Colin McCabe] MINOR: fix some ducktape test issues (#10181) -- [...truncated 1.80 MB...] KafkaConsumerTest >

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #498

See Changes: [github] MINOR: Test the new KIP-500 quorum mode in ducktape (#10105) -- [...truncated 3.64 MB...] TransactionsTest >

[jira] [Created] (KAFKA-12363) Simplify static group memberId update logic

Jason Gustafson created KAFKA-12363: --- Summary: Simplify static group memberId update logic Key: KAFKA-12363 URL: https://issues.apache.org/jira/browse/KAFKA-12363 Project: Kafka Issue

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #27

See Changes: [Colin McCabe] MINOR: Test the new KIP-500 quorum mode in ducktape (#10105) -- [...truncated 1.80 MB...] QuorumStateTest >

Re: [DISCUSS] KIP-715: Expose Committed offset in streams

Thanks Walker for the proposed KIP! This should definitely empower KStream users with better visibility. Meanwhile I got a couple of questions/suggestions: 1. typo "repost/report" in the motivation section. 2. What offsets do we report when the task is under restoration or rebalancing? 3.

[DISCUSS] KIP-715: Expose Committed offset in streams

Hello all, I would like to start discussion on KIP-715. This kip aims to make it easier to monitor Kafka Streams progress by exposing the committed offset in a similar way as the consumer client does. Here is the KIP: https://cwiki.apache.org/confluence/x/aRRRCg Best, Walker

[jira] [Created] (KAFKA-12362) Determine if a Task is idling

Walker Carlson created KAFKA-12362: -- Summary: Determine if a Task is idling Key: KAFKA-12362 URL: https://issues.apache.org/jira/browse/KAFKA-12362 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

Hi Guozhang, Your understanding should be correct in most cases, but there are two finer points that may interest you: 1. It's technically dependent on the implementation of the connector and how it chooses to allocate source partitions across its tasks; even if the number of tasks and source

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #554

See Changes: [github] MINOR: Raft request thread should discover api versions (#10157) -- [...truncated 7.32 MB...] UserQuotaTest >

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

Thanks Chris, yeah I think I agree with you that this does not necessarily have to be in the scope of this KIP. My understanding was that the source partitions -> tasks are not static but dynamic, but they are only changed when either the number of partitions changed or "tasks.max" config changed

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #528

See Changes: [Manikumar Reddy] KAFKA-12324: Upgrade jetty to fix CVE-2020-27218 [github] MINOR: Raft request thread should discover api versions (#10157)

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #497

See Changes: [Manikumar Reddy] KAFKA-12324: Upgrade jetty to fix CVE-2020-27218 [github] MINOR: Raft request thread should discover api versions (#10157) --

Re: PHP Kafka Client longyan/phpkafka

Feel free to create a wiki account and edit the page yourself. On 2/6/21 6:40 PM, Yurun wrote: > Hello! > > > Our team is maintaining a php kafka client. > > > We hope we can add this item to this > page:https://cwiki.apache.org/confluence/display/KAFKA/Clients > > > Introduction: > > >

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

Hi Guozhang, Thanks for taking a look, and for your suggestion! I think there is room for more intelligent fencing strategies, but I think that it'd have to be more nuanced than one based on task->worker assignments. Connectors can arbitrarily reassign source partitions (such as database tables,

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #553

See Changes: [Manikumar Reddy] KAFKA-12324: Upgrade jetty to fix CVE-2020-27218 -- [...truncated 3.64 MB...] UserQuotaTest >

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #26

See Changes: [Jason Gustafson] MINOR: Raft request thread should discover api versions (#10157) -- [...truncated 1.80 MB...] QuorumStateTest >

[jira] [Created] (KAFKA-12361) Change default connect producer request timeout

Jason Gustafson created KAFKA-12361: --- Summary: Change default connect producer request timeout Key: KAFKA-12361 URL: https://issues.apache.org/jira/browse/KAFKA-12361 Project: Kafka Issue

Re: [DISCUSS] KIP-618: Atomic commit of source connector records and offsets

Hi Gwen, Thanks for the feedback! 0. That's a great point; I've updated the motivation section with that rationale. 1. This enables safe "hard downgrades" of clusters where, instead of just disabling exactly-once support on each worker, each worker is rolled back to an earlier version of the

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #25

See Changes: [Manikumar Reddy] KAFKA-12324: Upgrade jetty to fix CVE-2020-27218 -- [...truncated 1.80 MB...] QuorumStateTest > testFollowerToLeaderOrResigned()

[jira] [Resolved] (KAFKA-12324) Upgrade jetty to fix CVE-2020-27218

[ https://issues.apache.org/jira/browse/KAFKA-12324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-12324. --- Fix Version/s: 2.8.0 2.6.2 2.7.1 Resolution: Fixed

Re: [DISCUSS] KIP-712: Shallow Mirroring

As Henry mentions in the KIP, we are seeing a great deal of improvements and efficiency by using the mirroring enhancement proposed in this KIP, and believe it would be equally beneficial to everyone that runs Kafka and Kafka Mirror at scale. I'm bumping up this thread in case there are

Build failed in Jenkins: Kafka » kafka-2.8-jdk8 #24

See Changes: [Randall Hauch] KAFKA-12340: Fix potential resource leak in Kafka*BackingStore (#10153) [Randall Hauch] KAFKA-12339: Add retry to admin client's listOffsets (#10152)

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Hello all, I have updated the KIP now. The diff is visible here: https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=165225765=13=12 The PR https://github.com/apache/kafka/pull/10137 is now available for reviews. Thanks so much to you all for your help on this! John On

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi Bruno, Thanks for the quick reply 5.Sorry, maybe I am not making it clear. What you have described is how it should work, yes. As it is stated in KIP, with the importance semantics in standby.replicas.awareness, if we have an active task on Node-1 and the first standby task on Node-5, the

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Thanks, Bill, I was waiting for feedback on the "currentLag" proposal before updating the KIP. Since there haven't been any objections to my new proposal, I'm in the process of updating the KIP document right now. Personally, I still like the original proposal to expose the metadata from the

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Thanks for the KIP update John, The KIP as it stands LGTM. One question, in your previous comment you stated that the KIP would introduce the `currentLag()` method, but the KIP seems to show the `metadata()` implementation. FWIW I like the metadata approach as it seems to provide information

[jira] [Created] (KAFKA-12360) Improve documentation of max.task.idle.ms (kafka-streams)

Domenico Delle Side created KAFKA-12360: --- Summary: Improve documentation of max.task.idle.ms (kafka-streams) Key: KAFKA-12360 URL: https://issues.apache.org/jira/browse/KAFKA-12360 Project:

[jira] [Resolved] (KAFKA-12339) Add retry to admin client's listOffsets

[ https://issues.apache.org/jira/browse/KAFKA-12339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-12339. --- Reviewer: Randall Hauch Resolution: Fixed Merged to `trunk`, and cherry-picked to: *

[jira] [Resolved] (KAFKA-12340) Recent change to use SharedTopicAdmin results in potential resource leak in deprecated backing store constructors

[ https://issues.apache.org/jira/browse/KAFKA-12340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-12340. --- Resolution: Fixed Merged to `trunk`, and cherry-picked to: * `2.8` for inclusion in 2.8.0

Re: Client/Broker Compatibility Matrix

I don't have sufficient knowledge to do that. From: Matthias J. Sax Sent: Thursday, February 18, 2021 6:57 PM To: dev@kafka.apache.org Subject: Re: Client/Broker Compatibility Matrix Feel free to edit the wiki page directly. On 2/18/21 11:19 AM, Gary Russell

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Hello all, Since there haven't been any big objections to my proposed design fix, I'm updating the KIP now and opening the PR for reviews. Hopefully, we can get this merged quickly and unblock the system tests. Thanks, John On Mon, 2021-02-22 at 09:49 -0600, John Roesler wrote: > Thanks for

RE: Re: [kafka-clients] Re: [VOTE] 2.6.1 RC3

On 2021/01/05 16:26:04 Ismael Juma wrote: > Hi Mickael, > > Does that CVE even affect Kafka? Not sure if we gain much by delaying the > release even longer. People who really care about the CVE can also use > 2.7.0. > > Ismael > > On Tue, Jan 5, 2021 at 8:12 AM Mickael Maison wrote: > > > Hi, > >

Re: [DISCUSS] Apache Kafka 2.8.0 release

Thanks for the heads-up, Chia-Ping, I agree it would be good to include that fix. Thanks, John On Mon, 2021-02-22 at 09:48 +, Chia-Ping Tsai wrote: > hi John, > > There is a PR (https://github.com/apache/kafka/pull/10024) fixing following > test error. > > 14:00:28 Execution failed for

Re: Re: [VOTE] KIP-695: Improve Streams Time Synchronization

Thanks for that idea, Chia-Ping, I agree that it would be nice for users to have a single API to get all kinds of metadata, but I'd rather introduce it in a KIP that would expose more than one piece of metadata. At the moment, I am motivated to keep this as simple as possible, which means only

[jira] [Created] (KAFKA-12359) Update Jetty to 11

Dongjin Lee created KAFKA-12359: --- Summary: Update Jetty to 11 Key: KAFKA-12359 URL: https://issues.apache.org/jira/browse/KAFKA-12359 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-12358) Migrate to Java 11

Dongjin Lee created KAFKA-12358: --- Summary: Migrate to Java 11 Key: KAFKA-12358 URL: https://issues.apache.org/jira/browse/KAFKA-12358 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi Levani, Thanks for the modifications! I have some follow up questions/comments: 5. Something is not clear to me. If the active is on Node-1 and the first replica is on Node-5 (different cluster, different zone), why would the second replica go to Node-4 that has a different cluster than

[jira] [Created] (KAFKA-12357) Do not inline methods from the scala package by default

Ismael Juma created KAFKA-12357: --- Summary: Do not inline methods from the scala package by default Key: KAFKA-12357 URL: https://issues.apache.org/jira/browse/KAFKA-12357 Project: Kafka Issue

Re: [DISCUSS] KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor

Hi Bruno, Thanks for the feedback. Please check my answers below: 1. No objections; sounds good. Updated KIP 2. No objections; sounds good. Updated KIP 3. Thanks for the information; I can change KIP only to expose prefix method instead of a constant if it’s the way forward. 4. Done. Updated

[jira] [Created] (KAFKA-12356) Drain the existing requests when shutdowning InterBrokerSendThread

Chia-Ping Tsai created KAFKA-12356: -- Summary: Drain the existing requests when shutdowning InterBrokerSendThread Key: KAFKA-12356 URL: https://issues.apache.org/jira/browse/KAFKA-12356 Project:

Re: [DISCUSS] Apache Kafka 2.8.0 release

hi John, There is a PR (https://github.com/apache/kafka/pull/10024) fixing following test error. 14:00:28 Execution failed for task ':core:test'. 14:00:28 > Process 'Gradle Test Executor 24' finished with non-zero exit value 1 14:00:28 This problem might be caused by incorrect test process