[jira] [Commented] (KAFKA-12801) High CPU load after restarting brokers subsequent to quorum loss

2021-05-20 Thread Gunnar Morling (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348804#comment-17348804 ] Gunnar Morling commented on KAFKA-12801: Ok, so I've added the two JFR recordings to my example

[GitHub] [kafka] mjsax merged pull request #10737: KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs

2021-05-20 Thread GitBox
mjsax merged pull request #10737: URL: https://github.com/apache/kafka/pull/10737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] mjsax commented on pull request #10737: KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs

2021-05-20 Thread GitBox
mjsax commented on pull request #10737: URL: https://github.com/apache/kafka/pull/10737#issuecomment-845492098 Thanks for the PR @jlprat! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] cmccabe commented on a change in pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe commented on a change in pull request #10572: URL: https://github.com/apache/kafka/pull/10572#discussion_r636526690 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -387,6 +398,12 @@ public void

[GitHub] [kafka] showuon commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-20 Thread GitBox
showuon commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-845600016 Thank you, guys! :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] ableegoldman commented on a change in pull request #10690: MINOR: clarify message ordering with max in-flight requests and idempotent producer

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10690: URL: https://github.com/apache/kafka/pull/10690#discussion_r636617410 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java ## @@ -246,8 +246,8 @@ public static final String

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r636620946 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -77,6 +84,7 @@ import static

[GitHub] [kafka] kowshik commented on pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on pull request #10742: URL: https://github.com/apache/kafka/pull/10742#issuecomment-845641737 @ccding and @showuon: Thanks for the review! I've addressed your comment(s) and responded to your questions. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] showuon commented on a change in pull request #10690: MINOR: clarify message ordering with max in-flight requests and idempotent producer

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10690: URL: https://github.com/apache/kafka/pull/10690#discussion_r636611321 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java ## @@ -246,8 +246,8 @@ public static final String

[GitHub] [kafka] showuon commented on a change in pull request #10731: KAFKA-12815: Update JavaDocs of ValueTransformerWithKey

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10731: URL: https://github.com/apache/kafka/pull/10731#discussion_r636606646 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/ValueTransformerWithKey.java ## @@ -75,14 +75,20 @@ void init(final

[jira] [Comment Edited] (KAFKA-9296) Correlation id for response () does not match request ()

2021-05-20 Thread Lea (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348627#comment-17348627 ] Lea edited comment on KAFKA-9296 at 5/21/21, 4:02 AM: -- I had the same problem using

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r636619553 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/NamedTopologyIntegrationTest.java ## @@ -0,0 +1,32 @@ +/* + * Licensed to

[GitHub] [kafka] ableegoldman commented on pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on pull request #10609: URL: https://github.com/apache/kafka/pull/10609#issuecomment-845638871 Rebased after the TaskId changes in KIP-470, and responded to all comments. Not much has changed since the last review, just cleaning up here and there. It's pretty much

[GitHub] [kafka] kowshik commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636622730 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -2225,13 +2225,15 @@ object Log extends Logging { * @param dir The directory in which the

[GitHub] [kafka] kowshik commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636623193 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -172,7 +173,7 @@ object LogLoader extends Logging { private def

[jira] [Comment Edited] (KAFKA-9296) Correlation id for response () does not match request ()

2021-05-20 Thread Lea (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348627#comment-17348627 ] Lea edited comment on KAFKA-9296 at 5/21/21, 4:03 AM: -- I had the same problem using

[GitHub] [kafka] kowshik commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636623193 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -172,7 +173,7 @@ object LogLoader extends Logging { private def

[GitHub] [kafka] kowshik commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636623604 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -67,7 +67,7 @@ case class LoadLogParams(dir: File,

[GitHub] [kafka] kowshik commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636623604 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -67,7 +67,7 @@ case class LoadLogParams(dir: File,

[jira] [Updated] (KAFKA-12821) Kafka consumer skip reading from single partition

2021-05-20 Thread Prakash Patel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Patel updated KAFKA-12821: -- Description: I have logstash ( Java Kafka Consumer ) running on Kubernetes and consuming

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r636557299 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -462,39 +512,49 @@ private void

[jira] [Commented] (KAFKA-12801) High CPU load after restarting brokers subsequent to quorum loss

2021-05-20 Thread Gunnar Morling (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348802#comment-17348802 ] Gunnar Morling commented on KAFKA-12801: Hum, so even with zipping (which brings the size down

[GitHub] [kafka] ableegoldman commented on pull request #10740: Kafka 8613 kip 633 drop default grace period streams

2021-05-20 Thread GitBox
ableegoldman commented on pull request #10740: URL: https://github.com/apache/kafka/pull/10740#issuecomment-845517863 Hey @izzyacademy , I'm taking a look but just a quick heads up: the build failed to compile, looks like the problem is some of the demo classes use one of the

[GitHub] [kafka] cmccabe merged pull request #10705: KAFKA-12342: Reverse module dependency between Raft and Metadata

2021-05-20 Thread GitBox
cmccabe merged pull request #10705: URL: https://github.com/apache/kafka/pull/10705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] ableegoldman commented on a change in pull request #10740: Kafka 8613 kip 633 drop default grace period streams

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10740: URL: https://github.com/apache/kafka/pull/10740#discussion_r636513301 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/JoinWindows.java ## @@ -67,6 +68,7 @@ * @see KStream#outerJoin(KStream,

[GitHub] [kafka] showuon commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636598563 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -67,7 +67,7 @@ case class LoadLogParams(dir: File,

[GitHub] [kafka] jlprat commented on pull request #10737: KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs

2021-05-20 Thread GitBox
jlprat commented on pull request #10737: URL: https://github.com/apache/kafka/pull/10737#issuecomment-845501514 Thanks for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] izzyacademy opened a new pull request #10741: [KAFKA-12644] Add Missing Class-Level Javadoc to Exception Classes

2021-05-20 Thread GitBox
izzyacademy opened a new pull request #10741: URL: https://github.com/apache/kafka/pull/10741 …ache.kafka.common.errors.ApiException Added missing class-level javadocs to Exception classes *More detailed description of your change, if necessary. The PR title and PR message

[GitHub] [kafka] kowshik opened a new pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik opened a new pull request #10742: URL: https://github.com/apache/kafka/pull/10742 When https://github.com/apache/kafka/pull/10478 was merged, we accidentally lost the identifier/prefix that we used to previously log from some of the functions in the `Log` class using the `Logging`

[GitHub] [kafka] kowshik commented on pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
kowshik commented on pull request #10742: URL: https://github.com/apache/kafka/pull/10742#issuecomment-845503390 cc @junrao for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] ableegoldman commented on pull request #10735: KAFKA-12779: KIP-740, Clean up public API in TaskId and fix TaskMetadata#taskId()

2021-05-20 Thread GitBox
ableegoldman commented on pull request #10735: URL: https://github.com/apache/kafka/pull/10735#issuecomment-845505488 Just some unrelated failures in `MetricsDuringTopicCreationDeletionTest.testMetricsDuringTopicCreateDelete()`, merging to trunk -- This is an automated message from the

[GitHub] [kafka] izzyacademy commented on pull request #10740: Kafka 8613 kip 633 drop default grace period streams

2021-05-20 Thread GitBox
izzyacademy commented on pull request #10740: URL: https://github.com/apache/kafka/pull/10740#issuecomment-845505719 Thanks @mjsax and @ableegoldman for all your help and support on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] ableegoldman merged pull request #10735: KAFKA-12779: KIP-740, Clean up public API in TaskId and fix TaskMetadata#taskId()

2021-05-20 Thread GitBox
ableegoldman merged pull request #10735: URL: https://github.com/apache/kafka/pull/10735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Resolved] (KAFKA-12779) TaskMetadata should return actual TaskId rather than plain String

2021-05-20 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12779. Resolution: Fixed > TaskMetadata should return actual TaskId rather than

[GitHub] [kafka] ryannedolan commented on pull request #10648: KAFKA-9726: Add IdentityReplicationPolicy for MM2

2021-05-20 Thread GitBox
ryannedolan commented on pull request #10648: URL: https://github.com/apache/kafka/pull/10648#issuecomment-845517243 I still don't think an API change is required here. I was able to get your integration tests passing with this version of IdentityReplicationPolicy:

[jira] [Commented] (KAFKA-12743) [Kafka Streams] - cluster failover for stateful Kafka Streams applications

2021-05-20 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348830#comment-17348830 ] Guozhang Wang commented on KAFKA-12743: --- For example, there are a few open source projects like

[GitHub] [kafka] cmccabe commented on pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-20 Thread GitBox
cmccabe commented on pull request #10504: URL: https://github.com/apache/kafka/pull/10504#issuecomment-845532005 I'm OK with moving the condition variable refactor to a follow-on PR. Similarly, I see that you don't have any logger in ZkClusterInvocationContext.java, so we can hold off on

[GitHub] [kafka] cmccabe commented on a change in pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe commented on a change in pull request #10572: URL: https://github.com/apache/kafka/pull/10572#discussion_r636531195 ## File path: metadata/src/test/java/org/apache/kafka/controller/ReplicationControlManagerTest.java ## @@ -252,6 +252,73 @@ public void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r636557694 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -411,19 +442,37 @@ private void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-20 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r636561268 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/SubscriptionInfo.java ## @@ -125,6 +130,29 @@ public

[GitHub] [kafka] ccding commented on a change in pull request #10742: MINOR: Add log identifier/prefix printing in Log layer static functions

2021-05-20 Thread GitBox
ccding commented on a change in pull request #10742: URL: https://github.com/apache/kafka/pull/10742#discussion_r636574778 ## File path: core/src/main/scala/kafka/log/LogLoader.scala ## @@ -172,7 +173,7 @@ object LogLoader extends Logging { private def

[GitHub] [kafka] showuon commented on a change in pull request #10741: [KAFKA-12644] Add Missing Class-Level Javadoc to Exception Classes

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10741: URL: https://github.com/apache/kafka/pull/10741#discussion_r636602191 ## File path: clients/src/main/java/org/apache/kafka/common/errors/InvalidConfigurationException.java ## @@ -16,6 +16,9 @@ */ package

[GitHub] [kafka] cmccabe commented on a change in pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe commented on a change in pull request #10572: URL: https://github.com/apache/kafka/pull/10572#discussion_r636531647 ## File path: metadata/src/test/java/org/apache/kafka/controller/MockControllerMetrics.java ## @@ -48,7 +53,6 @@ public void

[GitHub] [kafka] cmccabe commented on a change in pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe commented on a change in pull request #10572: URL: https://github.com/apache/kafka/pull/10572#discussion_r636531647 ## File path: metadata/src/test/java/org/apache/kafka/controller/MockControllerMetrics.java ## @@ -48,7 +53,6 @@ public void

[GitHub] [kafka] vahidhashemian commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-20 Thread GitBox
vahidhashemian commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-84299 Thanks for tagging me @guozhangwang @ableegoldman. I'll try to review this within the next few days. -- This is an automated message from the Apache Git Service. To

[jira] [Updated] (KAFKA-12821) Kafka consumer skip reading from single partition

2021-05-20 Thread Prakash Patel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Patel updated KAFKA-12821: -- Description: I have logstash ( Java Kafka Consumer ) running on Kubernetes and consuming

[jira] [Updated] (KAFKA-12821) Kafka consumer skip reading from single partition

2021-05-20 Thread Prakash Patel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Patel updated KAFKA-12821: -- Description: I have logstash ( Java Kafka Consumer ) running on Kubernetes and consuming

[GitHub] [kafka] cmccabe commented on a change in pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe commented on a change in pull request #10572: URL: https://github.com/apache/kafka/pull/10572#discussion_r636538620 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -387,6 +398,12 @@ public void

[GitHub] [kafka] cmccabe merged pull request #10572: KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller

2021-05-20 Thread GitBox
cmccabe merged pull request #10572: URL: https://github.com/apache/kafka/pull/10572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] ableegoldman commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-20 Thread GitBox
ableegoldman commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-845549567 It's on my list to review in the next couple of weeks, if not sooner. Sorry I have not had time to get to this one yet, but I will  (and I agree we should also get

[jira] [Updated] (KAFKA-12816) Add tier storage configs.

2021-05-20 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-12816: --- Description: Add all the tier storage related configurations including remote log manager,

[jira] [Created] (KAFKA-12821) Kafka consumer skip reading from single partition

2021-05-20 Thread Prakash Patel (Jira)
Prakash Patel created KAFKA-12821: - Summary: Kafka consumer skip reading from single partition Key: KAFKA-12821 URL: https://issues.apache.org/jira/browse/KAFKA-12821 Project: Kafka Issue

[jira] [Updated] (KAFKA-12821) Kafka consumer skip reading from single partition

2021-05-20 Thread Prakash Patel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Patel updated KAFKA-12821: -- Description: I have logstash ( Java Kafka Consumer ) running on Kubernetes and consuming

[GitHub] [kafka] cmccabe commented on pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-20 Thread GitBox
cmccabe commented on pull request #10504: URL: https://github.com/apache/kafka/pull/10504#issuecomment-845535131 The bug (which could be a test bug?) seems intermittent and didn't re-occur when I ran the test locally. In case Jenkins purges the information before you get a chance to look,

[GitHub] [kafka] showuon commented on pull request #10457: KAFKA-12596: remove --zookeeper option from topic command

2021-05-20 Thread GitBox
showuon commented on pull request #10457: URL: https://github.com/apache/kafka/pull/10457#issuecomment-844854328 Failed tests are unrelated and flaky. Thanks. ``` Build / JDK 8 and Scala 2.12 /

[GitHub] [kafka] jlprat opened a new pull request #10737: KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs

2021-05-20 Thread GitBox
jlprat opened a new pull request #10737: URL: https://github.com/apache/kafka/pull/10737 Removes previously deprecated method getConsumerConfigs under StreamsConfig, deprecated since 2.0 ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12596: remove --zookeeper option from topic command

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r635788721 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandIntegrationTest.scala ## @@ -83,8 +85,9 @@ class TopicCommandWithAdminClientTest extends

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-20 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348078#comment-17348078 ] NEERAJ VAIDYA commented on KAFKA-12776: --- [~ableegoldman], [~guozhang] I am closing this as it

[GitHub] [kafka] showuon commented on pull request #10457: KAFKA-12596: remove --zookeeper option from topic command

2021-05-20 Thread GitBox
showuon commented on pull request #10457: URL: https://github.com/apache/kafka/pull/10457#issuecomment-844742672 @ijuma , Thanks for the comments. I've updated. Please take a look again. Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12596: remove --zookeeper option from topic command

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r635797848 ## File path: core/src/main/scala/kafka/admin/TopicCommand.scala ## @@ -739,18 +563,15 @@ object TopicCommand extends Logging {

[GitHub] [kafka] guozhangwang commented on pull request #10686: KAFKA-4083: prevent changing replication factor via partition reassignment

2021-05-20 Thread GitBox
guozhangwang commented on pull request #10686: URL: https://github.com/apache/kafka/pull/10686#issuecomment-844749454 The fix seems still relevant to me, @chia7712 wdyt? Could you review it? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] showuon commented on pull request #10338: KAFKA-10251: wait for consumer rebalance completed before consuming records

2021-05-20 Thread GitBox
showuon commented on pull request #10338: URL: https://github.com/apache/kafka/pull/10338#issuecomment-844819847 failed tests are unrelated and flaky (and no `TransactionsBounceTest` failed): ``` Build / JDK 15 and Scala 2.13 /

[GitHub] [kafka] showuon opened a new pull request #10736: KAFKA-9295: increase session timeout to 30 seconds

2021-05-20 Thread GitBox
showuon opened a new pull request #10736: URL: https://github.com/apache/kafka/pull/10736 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

[GitHub] [kafka] tombentley commented on pull request #10686: KAFKA-4083: prevent changing replication factor via partition reassignment

2021-05-20 Thread GitBox
tombentley commented on pull request #10686: URL: https://github.com/apache/kafka/pull/10686#issuecomment-844838547 While I agree that all partitions ought to have the same number of replicas, I think this fix is enforcing something different: That you can't change the RF for a partition.

[GitHub] [kafka] showuon commented on a change in pull request #10471: KAFKA-12597: remove deprecated zookeeper option in ReassignPartitionsCommand

2021-05-20 Thread GitBox
showuon commented on a change in pull request #10471: URL: https://github.com/apache/kafka/pull/10471#discussion_r635846586 ## File path: core/src/test/scala/unit/kafka/admin/ReassignPartitionsCommandArgsTest.scala ## @@ -230,15 +214,9 @@ class

[GitHub] [kafka] showuon commented on pull request #10471: KAFKA-12597: remove deprecated zookeeper option in ReassignPartitionsCommand

2021-05-20 Thread GitBox
showuon commented on pull request #10471: URL: https://github.com/apache/kafka/pull/10471#issuecomment-844828808 @ijuma , thanks for the comments. I've updated. Please take a look again. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] wanwenli edited a comment on pull request #10686: KAFKA-4083: prevent changing replication factor via partition reassignment

2021-05-20 Thread GitBox
wanwenli edited a comment on pull request #10686: URL: https://github.com/apache/kafka/pull/10686#issuecomment-844963635 > While I agree that all partitions ought to have the same number of replicas, I think this fix is enforcing something different: That you can't change the RF for a

[GitHub] [kafka] mimaison commented on pull request #10660: MINOR: Updating files with release 2.7.1

2021-05-20 Thread GitBox
mimaison commented on pull request #10660: URL: https://github.com/apache/kafka/pull/10660#issuecomment-844905617 @rajinisivaram @omkreddy Can you take a look? We need this PR and https://github.com/apache/kafka-site/pull/354 in order to release 2.7.1. Thanks -- This is an automated

[jira] [Commented] (KAFKA-12801) High CPU load after restarting brokers subsequent to quorum loss

2021-05-20 Thread Gunnar Morling (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348204#comment-17348204 ] Gunnar Morling commented on KAFKA-12801: Hey [~lucasbradstreet], ok, so I got two JFR recordings

[GitHub] [kafka] wanwenli commented on pull request #10686: KAFKA-4083: prevent changing replication factor via partition reassignment

2021-05-20 Thread GitBox
wanwenli commented on pull request #10686: URL: https://github.com/apache/kafka/pull/10686#issuecomment-844963635 > While I agree that all partitions ought to have the same number of replicas, I think this fix is enforcing something different: That you can't change the RF for a partition.

[jira] [Created] (KAFKA-12817) Make Task ID an Implementation Detail

2021-05-20 Thread Bruno Cadonna (Jira)
Bruno Cadonna created KAFKA-12817: - Summary: Make Task ID an Implementation Detail Key: KAFKA-12817 URL: https://issues.apache.org/jira/browse/KAFKA-12817 Project: Kafka Issue Type:

[GitHub] [kafka] mimaison merged pull request #10660: MINOR: Updating files with release 2.7.1

2021-05-20 Thread GitBox
mimaison merged pull request #10660: URL: https://github.com/apache/kafka/pull/10660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Resolved] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-20 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NEERAJ VAIDYA resolved KAFKA-12776. --- Resolution: Information Provided KIP-739 has considerable of overlap with this issue. >

[GitHub] [kafka] jlprat commented on pull request #10737: KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs

2021-05-20 Thread GitBox
jlprat commented on pull request #10737: URL: https://github.com/apache/kafka/pull/10737#issuecomment-845004387 cc @mjsax feel free to review this one if you have time. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Updated] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Utkin updated KAFKA-12818: - Attachment: Screenshot 2021-05-20 at 14.53.05.png > Memory leakage when kafka connect 2.7 uses

[jira] [Commented] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348316#comment-17348316 ] Viktor Utkin commented on KAFKA-12818: -- https://github.com/apache/kafka/pull/9136/files > Memory

[jira] [Created] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
Viktor Utkin created KAFKA-12818: Summary: Memory leakage when kafka connect 2.7 uses directory config provider Key: KAFKA-12818 URL: https://issues.apache.org/jira/browse/KAFKA-12818 Project: Kafka

[jira] [Updated] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Utkin updated KAFKA-12818: - Description: Hi, we noticed a Memory leakage problem when kafka connect 2.7 uses directory

[jira] [Updated] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Utkin updated KAFKA-12818: - Description: Hi, we noticed a Memory leakage problem when kafka connect 2.7 uses directory

[jira] [Created] (KAFKA-12819) Quality of life improvements for tests

2021-05-20 Thread Matthew de Detrich (Jira)
Matthew de Detrich created KAFKA-12819: -- Summary: Quality of life improvements for tests Key: KAFKA-12819 URL: https://issues.apache.org/jira/browse/KAFKA-12819 Project: Kafka Issue

[jira] [Commented] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348517#comment-17348517 ] Tom Bentley commented on KAFKA-12818: - As you can see from the code, DirectoryConfigProvider doesn't

[GitHub] [kafka] mumrah commented on a change in pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-20 Thread GitBox
mumrah commented on a change in pull request #10504: URL: https://github.com/apache/kafka/pull/10504#discussion_r636104630 ## File path: core/src/main/scala/kafka/coordinator/transaction/ProducerIdManager.scala ## @@ -75,87 +61,179 @@ trait ProducerIdGenerator { def

[jira] [Comment Edited] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348563#comment-17348563 ] Viktor Utkin edited comment on KAFKA-12818 at 5/20/21, 2:58 PM: Hi

[jira] [Commented] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2021-05-20 Thread Viktor Utkin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348595#comment-17348595 ] Viktor Utkin commented on KAFKA-12818: -- Hi [~tombentley], what memory usage diagnostics data do you

[GitHub] [kafka] jlprat commented on pull request #10724: KAFKA-12808: Remove Deprecated Methods under StreamsMetrics

2021-05-20 Thread GitBox
jlprat commented on pull request #10724: URL: https://github.com/apache/kafka/pull/10724#issuecomment-845226161 Fixed pushed, `@SuppressWarnings("deprecation")` is now removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] jlprat edited a comment on pull request #10724: KAFKA-12808: Remove Deprecated Methods under StreamsMetrics

2021-05-20 Thread GitBox
jlprat edited a comment on pull request #10724: URL: https://github.com/apache/kafka/pull/10724#issuecomment-845226161 Fixed pushed, `@SuppressWarnings("deprecation")` is now removed, Feel free to review @cadonna -- This is an automated message from the Apache Git Service. To respond

[jira] [Commented] (KAFKA-12815) KTable.transformValue might have incorrect record metadata

2021-05-20 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348646#comment-17348646 ] A. Sophie Blee-Goldman commented on KAFKA-12815: Ack, sorry I made a bad assumption

[jira] [Commented] (KAFKA-12743) [Kafka Streams] - cluster failover for stateful Kafka Streams applications

2021-05-20 Thread Sergey Zyrianov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348780#comment-17348780 ] Sergey Zyrianov commented on KAFKA-12743: - [~guozhang] could you expand on which "other

[GitHub] [kafka] abbccdda commented on pull request #10482: KAFKA-12499: add transaction timeout verification

2021-05-20 Thread GitBox
abbccdda commented on pull request #10482: URL: https://github.com/apache/kafka/pull/10482#issuecomment-845296716 Only unrelated test fails, merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] satishd commented on a change in pull request #10733: KAFKA-12816 Added tiered storage related configs including remote log manager configs.

2021-05-20 Thread GitBox
satishd commented on a change in pull request #10733: URL: https://github.com/apache/kafka/pull/10733#discussion_r636298299 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfig.java ## @@ -0,0 +1,323 @@ +/* + * Licensed to the

[jira] [Commented] (KAFKA-12817) Make Task ID an Implementation Detail

2021-05-20 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348619#comment-17348619 ] A. Sophie Blee-Goldman commented on KAFKA-12817: Thanks for filing this Bruno. I still

[jira] [Commented] (KAFKA-9296) Correlation id for response () does not match request ()

2021-05-20 Thread Lea (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348627#comment-17348627 ] Lea commented on KAFKA-9296: I had the same problem using kafka producer again. Environment: K8S POD At the

[jira] [Comment Edited] (KAFKA-9296) Correlation id for response () does not match request ()

2021-05-20 Thread Lea (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348627#comment-17348627 ] Lea edited comment on KAFKA-9296 at 5/20/21, 4:56 PM: -- I had the same problem using

[jira] [Comment Edited] (KAFKA-9296) Correlation id for response () does not match request ()

2021-05-20 Thread Lea (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348627#comment-17348627 ] Lea edited comment on KAFKA-9296 at 5/20/21, 4:57 PM: -- I had the same problem using

[jira] [Commented] (KAFKA-12801) High CPU load after restarting brokers subsequent to quorum loss

2021-05-20 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348632#comment-17348632 ] Colin McCabe commented on KAFKA-12801: -- [~gunnar.morling]: thanks for the bug report. Maybe try

[GitHub] [kafka] jsancio commented on a change in pull request #10705: KAFKA-12342: Reverse module dependency between Raft and Metadata

2021-05-20 Thread GitBox
jsancio commented on a change in pull request #10705: URL: https://github.com/apache/kafka/pull/10705#discussion_r636347212 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -56,24 +57,17 @@ void handleSnapshot(SnapshotReader reader);

[jira] [Commented] (KAFKA-9071) transactional.id.expiration.ms config value should be implemented as a Long

2021-05-20 Thread Joost van de Wijgerd (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348710#comment-17348710 ] Joost van de Wijgerd commented on KAFKA-9071: - nudge > transactional.id.expiration.ms config

[jira] [Commented] (KAFKA-9953) support multiple consumerGroupCoordinators in TransactionManager

2021-05-20 Thread Joost van de Wijgerd (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348711#comment-17348711 ] Joost van de Wijgerd commented on KAFKA-9953: - nudge > support multiple

[GitHub] [kafka] jsancio commented on a change in pull request #10705: KAFKA-12342: Reverse module dependency between Raft and Metadata

2021-05-20 Thread GitBox
jsancio commented on a change in pull request #10705: URL: https://github.com/apache/kafka/pull/10705#discussion_r636340373 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -624,50 +617,71 @@ public String toString() {

[GitHub] [kafka] mumrah commented on a change in pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-20 Thread GitBox
mumrah commented on a change in pull request #10504: URL: https://github.com/apache/kafka/pull/10504#discussion_r636104630 ## File path: core/src/main/scala/kafka/coordinator/transaction/ProducerIdManager.scala ## @@ -75,87 +61,179 @@ trait ProducerIdGenerator { def

  1   2   >