[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r591162851 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public

[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r591162851 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public

[GitHub] [kafka] dengziming opened a new pull request #10289: KAFKA-12440: ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot

dengziming opened a new pull request #10289: URL: https://github.com/apache/kafka/pull/10289 *More detailed description of your change* This pr follows up #10129 which add clusterId validation to FetchRequest. *Summary of testing strategy (including rationale)* Unit test.

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r591121500 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -92,4 +102,24 @@ public static Path createTempFile(Path logDir,

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r591116450 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -32,13 +32,13 @@ import org.apache.kafka.common.utils.Utils import

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r591084650 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -16,16 +16,21 @@ */ package org.apache.kafka.snapshot; +import

[GitHub] [kafka] jsancio commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

jsancio commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r591079811 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -16,16 +16,21 @@ */ package org.apache.kafka.snapshot; +import

[jira] [Commented] (KAFKA-6582) Partitions get underreplicated, with a single ISR, and doesn't recover. Other brokers do not take over and we need to manually restart the broker.

[ https://issues.apache.org/jira/browse/KAFKA-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298537#comment-17298537 ] SeaAndHill commented on KAFKA-6582: --- 我们遇到了相同问题 在 kafka_2.11-0.10.2.0 版本中 > Partitions get

[GitHub] [kafka] jsancio commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

jsancio commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590657332 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -32,13 +32,13 @@ import org.apache.kafka.common.utils.Utils import

[GitHub] [kafka] leigh-perry closed pull request #7244: Improve connect exception reporting

leigh-perry closed pull request #7244: URL: https://github.com/apache/kafka/pull/7244 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-12445) Improve the display of ConsumerPerformance indicators

[ https://issues.apache.org/jira/browse/KAFKA-12445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298532#comment-17298532 ] Wenbing Shen commented on KAFKA-12445: -- In my own environment, I do this. I don’t know if the

[jira] [Updated] (KAFKA-12445) Improve the display of ConsumerPerformance indicators

[ https://issues.apache.org/jira/browse/KAFKA-12445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenbing Shen updated KAFKA-12445: - Attachment: image-2021-03-10-13-30-27-734.png > Improve the display of ConsumerPerformance

[GitHub] [kafka] chia7712 commented on pull request #10202: KAFKA-12392: Deprecate the batch-size option in console producer

chia7712 commented on pull request #10202: URL: https://github.com/apache/kafka/pull/10202#issuecomment-794906679 @kamalcph thanks for filing KIP. noted that KIP must be accepted first :) see https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals

[GitHub] [kafka] chia7712 merged pull request #10116: MINOR: Add entityType for metadata record definitions

chia7712 merged pull request #10116: URL: https://github.com/apache/kafka/pull/10116 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Created] (KAFKA-12445) Improve the display of ConsumerPerformance indicators

Wenbing Shen created KAFKA-12445: Summary: Improve the display of ConsumerPerformance indicators Key: KAFKA-12445 URL: https://issues.apache.org/jira/browse/KAFKA-12445 Project: Kafka Issue

[GitHub] [kafka] chia7712 merged pull request #9457: MINOR: Trivial Cleanups

chia7712 merged pull request #9457: URL: https://github.com/apache/kafka/pull/9457 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] chia7712 commented on pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

chia7712 commented on pull request #9944: URL: https://github.com/apache/kafka/pull/9944#issuecomment-794897185 > can you take a look when I have the commit ready? sure :) This is an automated message from the Apache

[GitHub] [kafka] chia7712 commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

chia7712 commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r591051268 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/PartitionLeaderStrategy.java ## @@ -0,0 +1,185 @@ +/* + * Licensed to the

[GitHub] [kafka] chia7712 commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

chia7712 commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r591036839 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/DescribeProducersHandler.java ## @@ -0,0 +1,213 @@ +/* + * Licensed to the

[GitHub] [kafka] chia7712 commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

chia7712 commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r591036030 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] chia7712 commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

chia7712 commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r591031440 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] junrao commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

junrao commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r590742564 ## File path: clients/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,152 @@ +/* + * Licensed to the

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590923649 ## File path: raft/src/test/java/org/apache/kafka/snapshot/SnapshotsTest.java ## @@ -74,5 +79,44 @@ public void testInvalidSnapshotFilenames() {

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590942924 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590941485 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/DescribeProducersHandler.java ## @@ -0,0 +1,213 @@ +/* + * Licensed to

[GitHub] [kafka] jolshan commented on pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

jolshan commented on pull request #9944: URL: https://github.com/apache/kafka/pull/9944#issuecomment-79476 Currently working on merge conflicts. Should have a first pass out in the next day or so. There are a few changes that don't work with the previous refactor. @chia7712, can you

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590938521 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/PartitionLeaderStrategy.java ## @@ -0,0 +1,190 @@ +/* + * Licensed to the

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590934574 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590928976 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r590928976 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiDriver.java ## @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590923649 ## File path: raft/src/test/java/org/apache/kafka/snapshot/SnapshotsTest.java ## @@ -74,5 +79,44 @@ public void testInvalidSnapshotFilenames() {

[GitHub] [kafka] hachikuji commented on pull request #10281: KAFKA-12432: Fix AdminClient timeout handling in the presence of bad brokers

hachikuji commented on pull request #10281: URL: https://github.com/apache/kafka/pull/10281#issuecomment-794717132 > Previously, we relied solely on the broker to enforce request.timeout.ms. If the broker failed to enforce this timeout, the admin client would hang until the call's API

[GitHub] [kafka] kowshik commented on pull request #10280: KIP-405: Log layer refactor

kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794673540 @ijuma I've opened up the doc for comments. I've also updated it to use the name `UnifiedLog` instead of `GlobalLog`. Hopefully the intent is better communicated now in the

[GitHub] [kafka] ableegoldman commented on pull request #10284: KAFKA-12441: remove deprecated method StreamsBuilder#addGlobalStore

ableegoldman commented on pull request #10284: URL: https://github.com/apache/kafka/pull/10284#issuecomment-794594154 Unrelated test failed: `kafka.server.ListOffsetsRequestTest.testResponseIncludesLeaderEpoch()` This is an

[GitHub] [kafka] jolshan commented on a change in pull request #10286: KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

jolshan commented on a change in pull request #10286: URL: https://github.com/apache/kafka/pull/10286#discussion_r590800123 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -1120,7 +1121,7 @@ def run_cli_tool(self, node, cmd): def parse_describe_topic(self,

[GitHub] [kafka] jolshan commented on a change in pull request #10286: KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

jolshan commented on a change in pull request #10286: URL: https://github.com/apache/kafka/pull/10286#discussion_r590798792 ## File path: tests/kafkatest/version.py ## @@ -101,7 +101,7 @@ def kafka_configs_command_uses_bootstrap_server_scram(self): # User SCRAM

[GitHub] [kafka] jeqo commented on pull request #10288: [KAFKA-10434] KIP-666 implementation

jeqo commented on pull request #10288: URL: https://github.com/apache/kafka/pull/10288#issuecomment-794555226 Would be a good chance to also remove: ``` public interface WindowStore ... { @Deprecated void put(K key, V value); ```

[GitHub] [kafka] rondagostino commented on a change in pull request #10286: KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

rondagostino commented on a change in pull request #10286: URL: https://github.com/apache/kafka/pull/10286#discussion_r590754821 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -1120,7 +1121,7 @@ def run_cli_tool(self, node, cmd): def

[GitHub] [kafka] jeqo opened a new pull request #10288: [KAFKA-10434] KIP-666 implementation

jeqo opened a new pull request #10288: URL: https://github.com/apache/kafka/pull/10288 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] mjsax commented on a change in pull request #9744: KAFKA-10062: Add a method to retrieve the current timestamp as known by the Streams app

mjsax commented on a change in pull request #9744: URL: https://github.com/apache/kafka/pull/9744#discussion_r590766321 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/ProcessorContext.java ## @@ -289,4 +291,33 @@ Cancellable schedule(final Duration

[GitHub] [kafka] hachikuji commented on a change in pull request #10227: KAFKA-12382: add a README for KIP-500

hachikuji commented on a change in pull request #10227: URL: https://github.com/apache/kafka/pull/10227#discussion_r590765355 ## File path: config/nozk-broker.properties ## @@ -68,7 +71,7 @@ socket.request.max.bytes=104857600 # Log Basics

[GitHub] [kafka] hachikuji commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

hachikuji commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590758128 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -258,30 +251,62 @@ final class KafkaMetadataLog private ( } } -

[jira] [Updated] (KAFKA-10434) Remove deprecated methods on WindowStore

[ https://issues.apache.org/jira/browse/KAFKA-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10434: Labels: (was: needs-kip) > Remove deprecated methods on WindowStore >

[GitHub] [kafka] cmccabe merged pull request #10249: MINOR: disable round_trip_fault_test system tests for Raft quorums

cmccabe merged pull request #10249: URL: https://github.com/apache/kafka/pull/10249 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] cmccabe merged pull request #10237: MINOR: fix failing system test delegation_token_test

cmccabe merged pull request #10237: URL: https://github.com/apache/kafka/pull/10237 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] cmccabe commented on a change in pull request #10227: KAFKA-12382: add a README for KIP-500

cmccabe commented on a change in pull request #10227: URL: https://github.com/apache/kafka/pull/10227#discussion_r590746266 ## File path: KIP-500.md ## @@ -0,0 +1,131 @@ +KIP-500 Early Access Release + + +# Introduction +It is now possible to run

[GitHub] [kafka] jsancio commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

jsancio commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590663415 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -92,4 +102,24 @@ public static Path createTempFile(Path logDir,

[jira] [Commented] (KAFKA-6599) KTable KTable join semantics violated when caching enabled

[ https://issues.apache.org/jira/browse/KAFKA-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298306#comment-17298306 ] Matthias J. Sax commented on KAFKA-6599: Thanks for the detailed report! This is super helpful. –

[jira] [Created] (KAFKA-12444) Add TranslateSurrogates SMT to Kafka Connect

Siva Kunapuli created KAFKA-12444: - Summary: Add TranslateSurrogates SMT to Kafka Connect Key: KAFKA-12444 URL: https://issues.apache.org/jira/browse/KAFKA-12444 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-12444) Add TranslateSurrogates SMT to Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siva Kunapuli updated KAFKA-12444: -- Priority: Minor (was: Major) > Add TranslateSurrogates SMT to Kafka Connect >

[jira] [Created] (KAFKA-12443) Add TranslateSurrogates SMT to Kafka Connect

Siva Kunapuli created KAFKA-12443: - Summary: Add TranslateSurrogates SMT to Kafka Connect Key: KAFKA-12443 URL: https://issues.apache.org/jira/browse/KAFKA-12443 Project: Kafka Issue Type:

[GitHub] [kafka] sivakunapuli opened a new pull request #10287: TranslateSurrogates SMT

sivakunapuli opened a new pull request #10287: URL: https://github.com/apache/kafka/pull/10287 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* Kafka Connect

[GitHub] [kafka] jsancio commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

jsancio commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590647005 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -258,30 +251,62 @@ final class KafkaMetadataLog private ( } } -

[jira] [Updated] (KAFKA-10357) Handle accidental deletion of repartition-topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-10357: --- Fix Version/s: 3.0.0 > Handle accidental deletion of repartition-topics as

[jira] [Commented] (KAFKA-10357) Handle accidental deletion of repartition-topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298282#comment-17298282 ] Konstantine Karantasis commented on KAFKA-10357: [~cadonna] [~guozhang] the KIP has been

[jira] [Updated] (KAFKA-10091) Improve task idling

[ https://issues.apache.org/jira/browse/KAFKA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-10091: --- Fix Version/s: (was: 2.8.0) 3.0.0 > Improve task

[GitHub] [kafka] jolshan edited a comment on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

jolshan edited a comment on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794336035 @chia7712 I see. Thanks for the response. I'm not sure if https://github.com/apache/kafka/pull/9944 will be ready after fixing conflicts. There were still a few critical

[jira] [Commented] (KAFKA-10091) Improve task idling

[ https://issues.apache.org/jira/browse/KAFKA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298280#comment-17298280 ] Konstantine Karantasis commented on KAFKA-10091: I now also see that this was postponed

[GitHub] [kafka] jolshan commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

jolshan commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794336035 @chia7712 I see. Thanks for the response. I'm not sure if https://github.com/apache/kafka/pull/9944 will be ready after fixing conflicts. There were still a few critical tests

[jira] [Updated] (KAFKA-12439) When in KIP-500 mode, we should be able to assign new partitions to nodes that are fenced

[ https://issues.apache.org/jira/browse/KAFKA-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-12439: - Issue Type: Improvement (was: Bug) > When in KIP-500 mode, we should be able to assign new

[GitHub] [kafka] chia7712 commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

chia7712 commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794327880 > So should we go forward with #10269 first still? It needs some time to complete performance tests. It seems to me we can fix conflicting files in #9944 to get reviews

[jira] [Commented] (KAFKA-10091) Improve task idling

[ https://issues.apache.org/jira/browse/KAFKA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298273#comment-17298273 ] Konstantine Karantasis commented on KAFKA-10091: [~vvcephei] is this improvement now

[GitHub] [kafka] jolshan commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

jolshan commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794320479 So should we go forward with https://github.com/apache/kafka/pull/10269 first still? This is an automated

[GitHub] [kafka] C0urante commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

C0urante commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590637805 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -402,7 +402,7 @@ public void

[GitHub] [kafka] C0urante commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

C0urante commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590637805 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -402,7 +402,7 @@ public void

[GitHub] [kafka] hachikuji commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

hachikuji commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r590634783 ## File path: raft/src/test/java/org/apache/kafka/snapshot/SnapshotsTest.java ## @@ -74,5 +79,44 @@ public void testInvalidSnapshotFilenames() {

[GitHub] [kafka] rhauch commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

rhauch commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590633040 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -402,7 +402,7 @@ public void tick()

[GitHub] [kafka] jolshan opened a new pull request #10286: KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

jolshan opened a new pull request #10286: URL: https://github.com/apache/kafka/pull/10286 Changes system tests to support both ZK and raft topic IDs. Clarifies the IBP check is for zk code. Ran `upgrade_test.py` and `downgrade_test.py` as these both use the

[GitHub] [kafka] kowshik edited a comment on pull request #10280: KIP-405: Log layer refactor

kowshik edited a comment on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794226594 @ijuma The purpose of `GlobalLog` class is to serve as a higher layer, stitching together the unified view of both the local and remote portion of the log. Importantly,

[GitHub] [kafka] dongjoon-hyun opened a new pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1

dongjoon-hyun opened a new pull request #10285: URL: https://github.com/apache/kafka/pull/10285 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[jira] [Created] (KAFKA-12442) Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1

Dongjoon Hyun created KAFKA-12442: - Summary: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1 Key: KAFKA-12442 URL: https://issues.apache.org/jira/browse/KAFKA-12442 Project: Kafka Issue Type:

[GitHub] [kafka] chia7712 commented on pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

chia7712 commented on pull request #10269: URL: https://github.com/apache/kafka/pull/10269#issuecomment-794237954 > I was thinking about the jmh microbenchmarks that stress fetch, fetch session and so on. will copy that.

[GitHub] [kafka] ijuma commented on pull request #10280: KIP-405: Log layer refactor

ijuma commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794235118 @kowshik I can't comment on the doc, that's why I commented here. :) I didn't see any reason there for calling it `Global` btw. `Global` tends to imply something more than what

[GitHub] [kafka] ijuma commented on pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

ijuma commented on pull request #10269: URL: https://github.com/apache/kafka/pull/10269#issuecomment-794226627 @chia7712 I was thinking about the jmh microbenchmarks that stress fetch, fetch session and so on. This is an

[GitHub] [kafka] kowshik commented on pull request #10280: KIP-405: Log layer refactor

kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794226594 @ijuma The purpose of `GlobalLog` class is to serve as a higher layer, stitching together the unified view of both the local and remote portion of the log. Importantly, this

[GitHub] [kafka] chia7712 commented on pull request #10269: KAFKA-12410 KafkaAPis ought to group fetch data before generating fet…

chia7712 commented on pull request #10269: URL: https://github.com/apache/kafka/pull/10269#issuecomment-794221218 @ijuma The results of performance tests are attached. It does not show obvious performance regression. Will run more tests tomorrow.

[GitHub] [kafka] ijuma commented on a change in pull request #10173: KAFKA-9548 Added SPIs and public classes/interfaces introduced in KIP-405 for tiered storage feature in Kafka.

ijuma commented on a change in pull request #10173: URL: https://github.com/apache/kafka/pull/10173#discussion_r590586196 ## File path: clients/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] ijuma commented on pull request #10280: KIP-405: Log layer refactor

ijuma commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794218123 OK, thanks for the explanation. Btw, why do we call one of the logs `GlobalLog`? In what sense is it `Global`?

[GitHub] [kafka] kowshik edited a comment on pull request #10280: KIP-405: Log layer refactor

kowshik edited a comment on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794210868 @ijuma In this PR the intention was not to rename `Log` to `LocalLog`, but rather extract `LocalLog` class out of `Log`. My current plan is to do the alternative of what

[GitHub] [kafka] kowshik edited a comment on pull request #10280: KIP-405: Log layer refactor

kowshik edited a comment on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794210868 @ijuma In this PR the intention was not to rename `Log` to `LocalLog`, but rather extract `LocalLog` class out of `Log`. My current plan is to do the alternative of what

[GitHub] [kafka] kowshik commented on pull request #10280: KIP-405: Log layer refactor

kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-794210868 @ijuma In this PR the intention was not to rename `Log` to `LocalLog`, but rather extract `LocalLog` class out of `Log`. My current plan is to do the alternative of what you

[GitHub] [kafka] jolshan commented on a change in pull request #10282: KAFKA-12426: Missing logic to create partition.metadata files in RaftReplicaManager

jolshan commented on a change in pull request #10282: URL: https://github.com/apache/kafka/pull/10282#discussion_r590577901 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -425,36 +425,35 @@ class Partition(val topicPartition: TopicPartition, }

[GitHub] [kafka] jolshan commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

jolshan commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794195828 @ijuma on first glance, I'm not sure if it simplifies. But it will introduce changes. And these are likely changes we want to have eventually.

[GitHub] [kafka] kamalcph commented on pull request #10126: MINOR: Updated the Kafka JUnit extension document.

kamalcph commented on pull request #10126: URL: https://github.com/apache/kafka/pull/10126#issuecomment-794188424 @mumrah ping for review. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] kamalcph commented on pull request #10202: KAFKA-12392: Deprecate the batch-size option in console producer

kamalcph commented on pull request #10202: URL: https://github.com/apache/kafka/pull/10202#issuecomment-794187462 @chia7712 Please take a look. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] kamalcph commented on pull request #10259: MINOR: Provide valid examples in README page.

kamalcph commented on pull request #10259: URL: https://github.com/apache/kafka/pull/10259#issuecomment-794186203 @chia7712 ping for review This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] jolshan edited a comment on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

jolshan edited a comment on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-793061818 @hachikuji updated based on the conversations here + those we had offline. Quick summary is that we will only have one swap file upon crash in the log cleaning case, so we

[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r590561174 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public

[GitHub] [kafka] rhauch merged pull request #10118: KAFKA-10192: Increase max time to wait for worker to start in some integration tests

rhauch merged pull request #10118: URL: https://github.com/apache/kafka/pull/10118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] C0urante commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

C0urante commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590551492 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/DistributedHerderTest.java ## @@ -2071,6 +2073,84 @@ public

[GitHub] [kafka] rondagostino commented on a change in pull request #10282: KAFKA-12426: Missing logic to create partition.metadata files in RaftReplicaManager

rondagostino commented on a change in pull request #10282: URL: https://github.com/apache/kafka/pull/10282#discussion_r590548220 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -425,36 +425,35 @@ class Partition(val topicPartition: TopicPartition, }

[GitHub] [kafka] C0urante commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

C0urante commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590550843 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -402,7 +402,7 @@ public void

[GitHub] [kafka] rhauch commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

rhauch commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r590534609 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -402,7 +402,7 @@ public void tick()

[GitHub] [kafka] ijuma commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

ijuma commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794141685 Does the refactoring simplify #9944 or not? This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r590519538 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public

[GitHub] [kafka] cadonna commented on a change in pull request #10266: KAFKA-10357: Add validation method for internal topics

cadonna commented on a change in pull request #10266: URL: https://github.com/apache/kafka/pull/10266#discussion_r590511478 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -91,6 +104,225 @@ public

[jira] [Commented] (KAFKA-12313) Consider deprecating the default.windowed.serde.inner.class configs

[ https://issues.apache.org/jira/browse/KAFKA-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298130#comment-17298130 ] Sagar Rao commented on KAFKA-12313: --- Thanks [~ableegoldman] before convincing other devs, let me try

[jira] [Commented] (KAFKA-7641) Add `consumer.group.max.size` to cap consumer metadata size on broker

[ https://issues.apache.org/jira/browse/KAFKA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17298123#comment-17298123 ] Justin Jack commented on KAFKA-7641: This change will require a KIP as it is a public interface

[GitHub] [kafka] chia7712 commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

chia7712 commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794041782 > So is the plan to merge #10269 first or after #9944? yep. #10269 is in progress (tweak performance). I will take a look at #9944 :)

[GitHub] [kafka] jolshan commented on pull request #9758: MINOR: remove FetchResponse.AbortedTransaction and redundant construc…

jolshan commented on pull request #9758: URL: https://github.com/apache/kafka/pull/9758#issuecomment-794038027 @chia7712 sounds good. So is the plan to merge https://github.com/apache/kafka/pull/10269 first or after https://github.com/apache/kafka/pull/9944? Also thanks for making these

  1   2   >