[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] dengziming opened a new pull request, #12076: MINOR: Fix SubscriptionInfoData name in exception message

2022-04-20 Thread GitBox
dengziming opened a new pull request, #12076: URL: https://github.com/apache/kafka/pull/12076 *More detailed description of your change* A small fix because of SubscriptionInfo.json has been renamed to SubscriptionInfoData.json *Summary of testing strategy (including rationale)*

[GitHub] [kafka] Aegeaner closed pull request #2753: KAFKA-4966:Producer throw a NullPointerException under a network environment where packet loss and error packets exist.

2022-04-20 Thread GitBox
Aegeaner closed pull request #2753: KAFKA-4966:Producer throw a NullPointerException under a network environment where packet loss and error packets exist. URL: https://github.com/apache/kafka/pull/2753 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854763466 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List

[GitHub] [kafka] showuon commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
showuon commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854747258 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -566,6 +565,8 @@ private List drainBatchesForOneNode(Cluster

[GitHub] [kafka] showuon commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
showuon commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854746909 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +559,14 @@ private List drainBatchesForOneNode(Cluster

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854746580 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -566,6 +565,8 @@ private List drainBatchesForOneNode(Cluster

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854745279 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +560,12 @@ private List

[GitHub] [kafka] showuon commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
showuon commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854734845 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -559,13 +560,12 @@ private List drainBatchesForOneNode(Cluster

[GitHub] [kafka] showuon commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
showuon commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r854733717 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -98,6 +98,54 @@ public void teardown() {

[GitHub] [kafka] RivenSun2 commented on pull request #12069: MINOR: Improve postProcessAndValidateIdempotenceConfigs method

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12069: URL: https://github.com/apache/kafka/pull/12069#issuecomment-1104655972 Hi @showuon Thanks for your review. I just retriggered jenkins build. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] sayantanu-dey commented on a diff in pull request #12035: KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so

2022-04-20 Thread GitBox
sayantanu-dey commented on code in PR #12035: URL: https://github.com/apache/kafka/pull/12035#discussion_r854734261 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1498,6 +1516,37 @@ public synchronized boolean close(final Duration timeout) throws

[GitHub] [kafka] showuon commented on pull request #12069: MINOR: Improve postProcessAndValidateIdempotenceConfigs method

2022-04-20 Thread GitBox
showuon commented on PR #12069: URL: https://github.com/apache/kafka/pull/12069#issuecomment-1104641591 @RivenSun2 , I don't know why the jenkins doesn't run the test. Could you rebase to the trunk branch to trigger the test again? Thanks. -- This is an automated message from the Apache

[GitHub] [kafka] showuon commented on a diff in pull request #12069: MINOR: Improve postProcessAndValidateIdempotenceConfigs method

2022-04-20 Thread GitBox
showuon commented on code in PR #12069: URL: https://github.com/apache/kafka/pull/12069#discussion_r854722883 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -490,11 +490,11 @@ private void postProcessAndValidateIdempotenceConfigs(final

[GitHub] [kafka] RivenSun2 commented on pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12010: URL: https://github.com/apache/kafka/pull/12010#issuecomment-1104577944 Hi @showuon @C0urante @divijvaidya Recently resubmitted the changes, if you have time, please help to review it again. Thanks a lot. -- This is an automated message from the

[GitHub] [kafka] RivenSun2 commented on pull request #12069: MINOR: Improve postProcessAndValidateIdempotenceConfigs method

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12069: URL: https://github.com/apache/kafka/pull/12069#issuecomment-1104572473 Hi @showuon could you help to review this PR? Thanks a lot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (KAFKA-13838) Improve the poll method of ConsumerNetworkClient

2022-04-20 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525352#comment-17525352 ] RivenSun commented on KAFKA-13838: -- Hi [~guozhang] Thanks for your review. cc [~hachikuji] , could you

[jira] [Updated] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13842: Description: Kafka Streams follows a continuous refinement model for aggregation. For

[jira] [Created] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13842: --- Summary: Add per-aggregation step before repartitioning Key: KAFKA-13842 URL: https://issues.apache.org/jira/browse/KAFKA-13842 Project: Kafka Issue

[GitHub] [kafka] mjsax merged pull request #12074: HOTFIX: fix broken trunk due to conflicting and overlapping commits

2022-04-20 Thread GitBox
mjsax merged PR #12074: URL: https://github.com/apache/kafka/pull/12074 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe opened a new pull request, #12075: KAFKA-13841: Fix a case where we were unable to place on fenced brokers in KRaft mode

2022-04-20 Thread GitBox
cmccabe opened a new pull request, #12075: URL: https://github.com/apache/kafka/pull/12075 This PR fixes a case where we were unable to place on fenced brokers In KRaft mode. Specifically, if we had a broker registration in the metadata log, but no associated heartbeat, previously the

[jira] [Created] (KAFKA-13841) Fix a case where we were unable to place on fenced brokers In KRaft mode

2022-04-20 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-13841: Summary: Fix a case where we were unable to place on fenced brokers In KRaft mode Key: KAFKA-13841 URL: https://issues.apache.org/jira/browse/KAFKA-13841 Project:

[GitHub] [kafka] mjsax opened a new pull request, #12074: HOTFIX: fix broken trunk due to conflicting and overlapping commits

2022-04-20 Thread GitBox
mjsax opened a new pull request, #12074: URL: https://github.com/apache/kafka/pull/12074 https://github.com/apache/kafka/pull/11896 has a green Jenkins build, but before it was merged https://github.com/apache/kafka/pull/11993 was merged and both PRs conflict breaking the build. -- This

[GitHub] [kafka] clayburn commented on a diff in pull request #12003: MINOR: Improve Gradle Caching and Fix Deprecations

2022-04-20 Thread GitBox
clayburn commented on code in PR #12003: URL: https://github.com/apache/kafka/pull/12003#discussion_r854483866 ## build.gradle: ## @@ -1216,12 +1237,16 @@ project(':clients') { testRuntimeOnly libs.jacksonJDK8Datatypes testImplementation libs.jose4j

[GitHub] [kafka] jsancio opened a new pull request, #12073: MINOR; Fix change record noop check

2022-04-20 Thread GitBox
jsancio opened a new pull request, #12073: URL: https://github.com/apache/kafka/pull/12073 When LeaderRecoveryState was added to the PartitionChangeRecord, the check for being a noop was not updated. This commit fixes that and improves the associated test avoid this oversight in the

[jira] [Updated] (KAFKA-13840) KafkaConsumer is unable to recover connection to group coordinator after commitOffsetsAsync exception

2022-04-20 Thread Kyle R Stehbens (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle R Stehbens updated KAFKA-13840: Description: Hi, I've discovered an issue with the java Kafka client (consumer) whereby a

[jira] [Created] (KAFKA-13840) KafkaConsumer is unable to recover connection to group coordinator after commitOffsetsAsync exception

2022-04-20 Thread Kyle R Stehbens (Jira)
Kyle R Stehbens created KAFKA-13840: --- Summary: KafkaConsumer is unable to recover connection to group coordinator after commitOffsetsAsync exception Key: KAFKA-13840 URL:

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12035: KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so

2022-04-20 Thread GitBox
guozhangwang commented on code in PR #12035: URL: https://github.com/apache/kafka/pull/12035#discussion_r854451233 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1498,6 +1516,37 @@ public synchronized boolean close(final Duration timeout) throws

[jira] [Assigned] (KAFKA-13588) We should consolidate `changelogFor` methods to simplify the generation of internal topic names

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-13588: - Assignee: Guozhang Wang (was: Sayantanu Dey) > We should consolidate `changelogFor`

[jira] [Commented] (KAFKA-13588) We should consolidate `changelogFor` methods to simplify the generation of internal topic names

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525203#comment-17525203 ] Guozhang Wang commented on KAFKA-13588: --- Thanks [~dsayan] I've merged your PR. We will keep this

[GitHub] [kafka] guozhangwang merged pull request #11703: KAFKA-13588: consolidate `changelogFor` methods to simplify the generation of internal topic names

2022-04-20 Thread GitBox
guozhangwang merged PR #11703: URL: https://github.com/apache/kafka/pull/11703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] junrao commented on pull request #12064: KAFKA-12841: Remove an additional call of onAcknowledgement

2022-04-20 Thread GitBox
junrao commented on PR #12064: URL: https://github.com/apache/kafka/pull/12064#issuecomment-1104294263 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] junrao commented on pull request #12064: KAFKA-12841: Remove an additional call of onAcknowledgement

2022-04-20 Thread GitBox
junrao commented on PR #12064: URL: https://github.com/apache/kafka/pull/12064#issuecomment-1104299988 test this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] junrao commented on pull request #12064: KAFKA-12841: Remove an additional call of onAcknowledgement

2022-04-20 Thread GitBox
junrao commented on PR #12064: URL: https://github.com/apache/kafka/pull/12064#issuecomment-1104298263 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Updated] (KAFKA-13834) batch drain for nodes might have starving issue

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-13834: -- Reviewer: Luke Chen (was: Guozhang Wang) > batch drain for nodes might have starving issue >

[GitHub] [kafka] guozhangwang commented on pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
guozhangwang commented on PR #12066: URL: https://github.com/apache/kafka/pull/12066#issuecomment-1104293963 @ruanliang-hualun Thanks for the PR, and thanks to @showuon for the detailed reviews!! I took a look at the edge test and I think without the actual fix it would indeed cause

[jira] [Commented] (KAFKA-13834) batch drain for nodes might have starving issue

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525181#comment-17525181 ] Guozhang Wang commented on KAFKA-13834: --- [~ruanliang] I've added you to the contributor list and

[jira] [Assigned] (KAFKA-13834) batch drain for nodes might have starving issue

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-13834: - Assignee: ruanliang > batch drain for nodes might have starving issue >

[GitHub] [kafka] C0urante commented on pull request #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-20 Thread GitBox
C0urante commented on PR #12041: URL: https://github.com/apache/kafka/pull/12041#issuecomment-1104271387 @guozhangwang thanks, but it seems like the disagreement here is on what actually constitutes an unnecessary warning message. If someone believes their consumer is auto-committing when

[jira] [Commented] (KAFKA-13838) Improve the poll method of ConsumerNetworkClient

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525174#comment-17525174 ] Guozhang Wang commented on KAFKA-13838: --- I did a quick look, and I think I agree with [~RivenSun]

[GitHub] [kafka] ahuang98 opened a new pull request, #12072: [WIP] KRaft upgrade - MetadataVersion

2022-04-20 Thread GitBox
ahuang98 opened a new pull request, #12072: URL: https://github.com/apache/kafka/pull/12072 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[jira] [Commented] (KAFKA-13838) Improve the poll method of ConsumerNetworkClient

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525168#comment-17525168 ] Guozhang Wang commented on KAFKA-13838: --- cc [~hachikuji] as you are working on the threading

[jira] [Updated] (KAFKA-13811) Investigate sliding windows performance

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-13811: -- Labels: perfomance (was: ) > Investigate sliding windows performance >

[GitHub] [kafka] guozhangwang merged pull request #12030: KAFKA-13785: [5/N][emit final] cache for time ordered window store

2022-04-20 Thread GitBox
guozhangwang merged PR #12030: URL: https://github.com/apache/kafka/pull/12030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-13768) Transactional producer exits because of expiration in RecordAccumulator

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525161#comment-17525161 ] Guozhang Wang commented on KAFKA-13768: --- Hello [~ddrid], I think what you're describing here is

[GitHub] [kafka] guozhangwang commented on pull request #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-20 Thread GitBox
guozhangwang commented on PR #12041: URL: https://github.com/apache/kafka/pull/12041#issuecomment-1104256988 I agree with @RivenSun2 's rationale here. More specifically I think 1) we are not really trying to just ignore all configs that are not used; instead we just want to 2) not print

[jira] [Commented] (KAFKA-13765) Describe-consumer admin should not return unstable membership information

2022-04-20 Thread Ryan Leslie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525151#comment-17525151 ] Ryan Leslie commented on KAFKA-13765: - Thanks for confirming, makes sense. > Describe-consumer

[jira] [Updated] (KAFKA-13839) Example connectors Maven artifact should use provided dependencies

2022-04-20 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-13839: --- Component/s: KafkaConnect > Example connectors Maven artifact should use provided

[jira] [Created] (KAFKA-13839) Example connectors Maven artifact should use provided dependencies

2022-04-20 Thread Jakub Scholz (Jira)
Jakub Scholz created KAFKA-13839: Summary: Example connectors Maven artifact should use provided dependencies Key: KAFKA-13839 URL: https://issues.apache.org/jira/browse/KAFKA-13839 Project: Kafka

[jira] [Commented] (KAFKA-13765) Describe-consumer admin should not return unstable membership information

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525150#comment-17525150 ] Guozhang Wang commented on KAFKA-13765: --- Hello [~rleslie] thanks for the careful thoughts. And I

[jira] [Resolved] (KAFKA-13799) Improve documentation for Kafka zero-copy

2022-04-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-13799. --- Fix Version/s: 3.3.0 Assignee: RivenSun Resolution: Fixed > Improve

[GitHub] [kafka] guozhangwang merged pull request #12052: KAFKA-13799: Improve documentation for Kafka zero-copy

2022-04-20 Thread GitBox
guozhangwang merged PR #12052: URL: https://github.com/apache/kafka/pull/12052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] tombentley commented on pull request #12006: KAFKA-13794: Follow up to fix comparator

2022-04-20 Thread GitBox
tombentley commented on PR #12006: URL: https://github.com/apache/kafka/pull/12006#issuecomment-1104214543 @ijuma did you have any ideas about how to fix this without the problem with hash collisions? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] junrao commented on a diff in pull request #12049: KAFKA-10888: Sticky partition leads to uneven produce msg

2022-04-20 Thread GitBox
junrao commented on code in PR #12049: URL: https://github.com/apache/kafka/pull/12049#discussion_r852454712 ## clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java: ## @@ -1321,21 +1339,30 @@ private ClusterResourceListeners

[jira] [Commented] (KAFKA-13538) Unexpected TopicExistsException related to Admin#createTopics after broker crash

2022-04-20 Thread Haoze Wu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525114#comment-17525114 ] Haoze Wu commented on KAFKA-13538: -- [~joecqupt] I see. But I guess the retry mechanism in the client

[GitHub] [kafka] RivenSun2 commented on pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12010: URL: https://github.com/apache/kafka/pull/12010#issuecomment-1104065097 Hi @C0urante @divijvaidya After some thought, we can remove the validator added to the (de)serializer configuration. Add validation in `append(De)serializerToConfig` method. It

[GitHub] [kafka] RivenSun2 commented on pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12010: URL: https://github.com/apache/kafka/pull/12010#issuecomment-1104026547 > I'm wondering if people might be constructing ProducerConfig and ConsumerConfig instances in order to do some kind of validation before storing a configuration somewhere, that's then

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854230524 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] RivenSun2 commented on pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12010: URL: https://github.com/apache/kafka/pull/12010#issuecomment-1103997882 Hi @C0urante Thank you for your review, and resubmitted the code changes again. > What are your thoughts on retaining the validation for the sasl.mechanism property, but only

[GitHub] [kafka] dajac commented on a diff in pull request #12071: KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (1)

2022-04-20 Thread GitBox
dajac commented on code in PR #12071: URL: https://github.com/apache/kafka/pull/12071#discussion_r854191977 ## clients/src/main/resources/common/message/LeaderAndIsrRequest.json: ## @@ -78,8 +78,8 @@ "about": "The leader epoch." }, { "name": "Isr", "type":

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
divijvaidya commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854178031 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
divijvaidya commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854161094 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854134099 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854134099 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854122954 ## clients/src/main/java/org/apache/kafka/common/config/internals/BrokerSecurityConfigs.java: ## @@ -89,4 +89,5 @@ public class BrokerSecurityConfigs { +

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
RivenSun2 commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854122673 ## clients/src/test/java/org/apache/kafka/common/config/SaslConfigsTest.java: ## @@ -122,4 +122,5 @@ public void

[GitHub] [kafka] mimaison merged pull request #12058: MINOR: Scala cleanups in core

2022-04-20 Thread GitBox
mimaison merged PR #12058: URL: https://github.com/apache/kafka/pull/12058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] C0urante commented on pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
C0urante commented on PR #12010: URL: https://github.com/apache/kafka/pull/12010#issuecomment-1103904025 Thanks @RivenSun2, this is looking pretty good. What are your thoughts on retaining the validation for the `sasl.mechanism` property, but only performing it when the property will

[GitHub] [kafka] C0urante commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
C0urante commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854094269 ## clients/src/test/java/org/apache/kafka/common/config/SaslConfigsTest.java: ## @@ -122,4 +122,5 @@ public void

[GitHub] [kafka] C0urante commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
C0urante commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854093483 ## clients/src/main/java/org/apache/kafka/common/config/internals/BrokerSecurityConfigs.java: ## @@ -89,4 +89,5 @@ public class BrokerSecurityConfigs { +

[GitHub] [kafka] C0urante commented on a diff in pull request #12010: KAFKA-13793: Add validators for configs that lack validators

2022-04-20 Thread GitBox
C0urante commented on code in PR #12010: URL: https://github.com/apache/kafka/pull/12010#discussion_r854092830 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -393,10 +394,14 @@ public class ProducerConfig extends AbstractConfig {

[GitHub] [kafka] showuon commented on a diff in pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2022-04-20 Thread GitBox
showuon commented on code in PR #11211: URL: https://github.com/apache/kafka/pull/11211#discussion_r854092062 ## streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBWindowStoreTest.java: ## @@ -181,15 +189,16 @@ public void testRolling() { ),

[GitHub] [kafka] dengziming commented on a diff in pull request #12071: KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (1)

2022-04-20 Thread GitBox
dengziming commented on code in PR #12071: URL: https://github.com/apache/kafka/pull/12071#discussion_r854082813 ## clients/src/main/resources/common/message/LeaderAndIsrRequest.json: ## @@ -78,8 +78,8 @@ "about": "The leader epoch." }, { "name": "Isr", "type":

[GitHub] [kafka] mimaison commented on pull request #12004: KAFKA-10095: Add stricter assertion in LogCleanerManagerTest

2022-04-20 Thread GitBox
mimaison commented on PR #12004: URL: https://github.com/apache/kafka/pull/12004#issuecomment-1103875583 Congratulations @divijvaidya ! Welcome to the community, I'm looking forward to seeing your next contributions -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] dajac opened a new pull request, #12071: KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (1)

2022-04-20 Thread GitBox
dajac opened a new pull request, #12071: URL: https://github.com/apache/kafka/pull/12071 This patch does some initial cleanups in the context of KAFKA-13790. Mainly, it renames `ZkVersion` field to `PartitionEpoch` in the `LeaderAndIsrRequest`, the `LeaderAndIsr` and the `Partition`.

[GitHub] [kafka] divijvaidya commented on pull request #12004: KAFKA-10095: Add stricter assertion in LogCleanerManagerTest

2022-04-20 Thread GitBox
divijvaidya commented on PR #12004: URL: https://github.com/apache/kafka/pull/12004#issuecomment-1103836282 Thank you for supporting this PR with the review @mimaison. Appreciate it! This was my first contribution to Apache Kafka 拾 -- This is an automated message from the Apache Git

[GitHub] [kafka] mimaison commented on a diff in pull request #12003: MINOR: Improve Gradle Caching and Fix Deprecations

2022-04-20 Thread GitBox
mimaison commented on code in PR #12003: URL: https://github.com/apache/kafka/pull/12003#discussion_r854023696 ## build.gradle: ## @@ -1216,12 +1237,16 @@ project(':clients') { testRuntimeOnly libs.jacksonJDK8Datatypes testImplementation libs.jose4j

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2022-04-20 Thread GitBox
vamossagar12 commented on code in PR #11211: URL: https://github.com/apache/kafka/pull/11211#discussion_r854022242 ## streams/src/test/java/org/apache/kafka/streams/state/internals/AbstractDualSchemaRocksDBSegmentedBytesStoreTest.java: ## @@ -157,43 +157,34 @@ public void

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2022-04-20 Thread GitBox
vamossagar12 commented on code in PR #11211: URL: https://github.com/apache/kafka/pull/11211#discussion_r854022038 ## streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBWindowStoreTest.java: ## @@ -181,15 +189,16 @@ public void testRolling() { ),

[GitHub] [kafka] showuon commented on pull request #12058: MINOR: Scala cleanups in core

2022-04-20 Thread GitBox
showuon commented on PR #12058: URL: https://github.com/apache/kafka/pull/12058#issuecomment-1103814774 > @showuon This PR adds Unit wherever needed. Ah, you're right! I look at the wrong way. LGTM, then! Thanks. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] RivenSun2 commented on pull request #12052: KAFKA-13799: Improve documentation for Kafka zero-copy

2022-04-20 Thread GitBox
RivenSun2 commented on PR #12052: URL: https://github.com/apache/kafka/pull/12052#issuecomment-1103807665 Hi @guozhangwang @showuon could you help to review the PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] mimaison commented on pull request #12058: MINOR: Scala cleanups in core

2022-04-20 Thread GitBox
mimaison commented on PR #12058: URL: https://github.com/apache/kafka/pull/12058#issuecomment-1103799754 I've merged https://github.com/apache/kafka/pull/12004 and rebased this on top. @showuon This PR adds `Unit` wherever needed. -- This is an automated message from the Apache

[jira] [Resolved] (KAFKA-10095) In LogCleanerManagerTest replace get().nonEmpty call with contains

2022-04-20 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-10095. Fix Version/s: 3.3.0 Resolution: Fixed > In LogCleanerManagerTest replace

[jira] [Assigned] (KAFKA-10095) In LogCleanerManagerTest replace get().nonEmpty call with contains

2022-04-20 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-10095: -- Assignee: Divij Vaidya (was: Sarah Gonsalves) > In LogCleanerManagerTest replace

[GitHub] [kafka] mimaison merged pull request #12004: KAFKA-10095: Add stricter assertion in LogCleanerManagerTest

2022-04-20 Thread GitBox
mimaison merged PR #12004: URL: https://github.com/apache/kafka/pull/12004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mimaison commented on pull request #12004: KAFKA-10095: Add stricter assertion in LogCleanerManagerTest

2022-04-20 Thread GitBox
mimaison commented on PR #12004: URL: https://github.com/apache/kafka/pull/12004#issuecomment-1103788476 Tests passed locally, merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mimaison commented on pull request #12068: MINOR: Fix warnings in Gradle 7

2022-04-20 Thread GitBox
mimaison commented on PR #12068: URL: https://github.com/apache/kafka/pull/12068#issuecomment-1103786188 @runningcode Sorry, I had not seen your PR. Since yours is fixing all the issues (mine ignored the warnings about generators), I'll close mine -- This is an automated message from the

[GitHub] [kafka] mimaison closed pull request #12068: MINOR: Fix warnings in Gradle 7

2022-04-20 Thread GitBox
mimaison closed pull request #12068: MINOR: Fix warnings in Gradle 7 URL: https://github.com/apache/kafka/pull/12068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] mimaison commented on a diff in pull request #12004: KAFKA-10095: Add stricter assertion in LogCleanerManagerTest

2022-04-20 Thread GitBox
mimaison commented on code in PR #12004: URL: https://github.com/apache/kafka/pull/12004#discussion_r853891196 ## core/src/test/scala/unit/kafka/log/LogCleanerManagerTest.scala: ## @@ -413,7 +413,7 @@ class LogCleanerManagerTest extends Logging { // updateCheckpoints

[GitHub] [kafka] ruanliang-hualun commented on a diff in pull request #12066: KAFKA-13834: fix drain batch starving issue

2022-04-20 Thread GitBox
ruanliang-hualun commented on code in PR #12066: URL: https://github.com/apache/kafka/pull/12066#discussion_r853824668 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -98,6 +99,54 @@ public void teardown() {

[GitHub] [kafka] runningcode commented on pull request #12068: MINOR: Fix warnings in Gradle 7

2022-04-20 Thread GitBox
runningcode commented on PR #12068: URL: https://github.com/apache/kafka/pull/12068#issuecomment-1103567288 @showuon would you also mind taking a look at https://github.com/apache/kafka/pull/12003 which addresses the same deprecation warnings and more? -- This is an automated message

[GitHub] [kafka] showuon commented on a diff in pull request #12058: MINOR: Scala cleanups in core

2022-04-20 Thread GitBox
showuon commented on code in PR #12058: URL: https://github.com/apache/kafka/pull/12058#discussion_r853770995 ## core/src/main/scala/kafka/server/PartitionMetadataFile.scala: ## @@ -44,8 +44,7 @@ object PartitionMetadataFile { } class