[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r784750507 ## File path: core/src/main/scala/kafka/log/UnifiedLog.scala ## @@ -288,13 +294,24 @@ class UnifiedLog(@volatile var logStartOffset: Long, @volatile

[GitHub] [kafka] dengziming opened a new pull request #11726: KAFKA-13637: Use default.api.timeout.ms as default timeout value for KafkaConsumer.endOffsets

2022-02-01 Thread GitBox
dengziming opened a new pull request #11726: URL: https://github.com/apache/kafka/pull/11726 *More detailed description of your change* In KafkaConsumer, we use `request.timeout.ms` in `endOffsets` and `default.api.timeout.ms` when in `beginningOffsets`, we should use

[jira] [Created] (KAFKA-13637) User default.api.timeout.ms config as default timeout for KafkaConsumer.endOffsets

2022-02-01 Thread dengziming (Jira)
dengziming created KAFKA-13637: -- Summary: User default.api.timeout.ms config as default timeout for KafkaConsumer.endOffsets Key: KAFKA-13637 URL: https://issues.apache.org/jira/browse/KAFKA-13637

[GitHub] [kafka] jasonk000 commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
jasonk000 commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797211462 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double value,

[GitHub] [kafka] jasonk000 commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
jasonk000 commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797211462 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double value,

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r797197548 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition:

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r797197548 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition:

[GitHub] [kafka] artemlivshits commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
artemlivshits commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797190190 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double

[GitHub] [kafka] philipnee commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
philipnee commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027466090 > @kirktrue Given how it works for the producer interceptor, it seems reasonable to do something similar for the consumer. @cmccabe do you agree? let's move the

[GitHub] [kafka] kirktrue commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
kirktrue commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027464655 Closing this PR in favor of #11689. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] kirktrue closed pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
kirktrue closed pull request #10951: URL: https://github.com/apache/kafka/pull/10951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mjsax commented on pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread GitBox
mjsax commented on pull request #11424: URL: https://github.com/apache/kafka/pull/11424#issuecomment-1027357002 Btw: this PR does not update the docs -- we should docs updates, too. (Also ok to do in a follow up PR.) To ensure backward compatibility, it might also be good to split

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485514#comment-17485514 ] Ismael Juma commented on KAFKA-12879: - [~cmccabe] It's straightforward to do `describe topics`

[jira] [Reopened] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13152: - The PR broke backward compatibility. Needed to revert the commit. Sorry about that. Lets

[GitHub] [kafka] mjsax commented on pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread GitBox
mjsax commented on pull request #11424: URL: https://github.com/apache/kafka/pull/11424#issuecomment-1027334009 Seems this PR breaks backward compatibility. Will need to revert it for now. Sorry. @vamossagar12 -- Can you do a new PR containing the commit plus the required fixes?

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485498#comment-17485498 ] Colin McCabe commented on KAFKA-12879: -- bq. My suggestion is to do 3 and to make sure the cause of

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485496#comment-17485496 ] Colin McCabe commented on KAFKA-12879: -- Let me give a little context here on the behavior. The

[GitHub] [kafka] ijuma commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
ijuma commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027303692 @kirktrue Given how it works for the producer interceptor, it seems reasonable to do something similar for the consumer. @cmccabe do you agree? -- This is an automated message

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485491#comment-17485491 ] Ismael Juma commented on KAFKA-12879: - My suggestion is to do 3 and to make sure the cause of

[jira] [Assigned] (KAFKA-13636) Committed offsets could be deleted during a rebalance if a group did not commit for a while

2022-02-01 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reassigned KAFKA-13636: --- Assignee: David Jacot > Committed offsets could be deleted during a rebalance if a group

[GitHub] [kafka] mimaison commented on pull request #11672: KAFKA-13577: Replace easymock with mockito in kafka:core - part 1

2022-02-01 Thread GitBox
mimaison commented on pull request #11672: URL: https://github.com/apache/kafka/pull/11672#issuecomment-1027218635 @tombentley @dajac Can you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Created] (KAFKA-13636) Committed offsets could be deleted during a rebalance if a group did not commit for a while

2022-02-01 Thread Damien Gasparina (Jira)
Damien Gasparina created KAFKA-13636: Summary: Committed offsets could be deleted during a rebalance if a group did not commit for a while Key: KAFKA-13636 URL:

[GitHub] [kafka] dajac commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796878885 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition:

[jira] [Resolved] (KAFKA-13595) Allow producing records with null values in Kafka Console Producer

2022-02-01 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-13595. Fix Version/s: 3.2.0 Resolution: Fixed > Allow producing records with null values

[jira] [Resolved] (KAFKA-13527) Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-13527. Fix Version/s: 3.2.0 Resolution: Fixed > Add top-level error code field to

[GitHub] [kafka] mimaison merged pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
mimaison merged pull request #11695: URL: https://github.com/apache/kafka/pull/11695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796852603 ## File path: core/src/main/scala/kafka/server/metadata/MetadataVersionManager.scala ## @@ -0,0 +1,104 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] mimaison merged pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread GitBox
mimaison merged pull request #11599: URL: https://github.com/apache/kafka/pull/11599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796852603 ## File path: core/src/main/scala/kafka/server/metadata/MetadataVersionManager.scala ## @@ -0,0 +1,104 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796836599 ## File path: core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala ## @@ -73,7 +74,7 @@ class BrokerMetadataListener( /** * The

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796836599 ## File path: core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala ## @@ -73,7 +74,7 @@ class BrokerMetadataListener( /** * The

[GitHub] [kafka] Indupa edited a comment on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-02-01 Thread GitBox
Indupa edited a comment on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1026731878 Hi @dongjinleekr , Can you please help me to build this latest patch you have released on top of Kafka_2.8,1. I tried using this following command to compile kafka

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796828017 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition:

[GitHub] [kafka] mumrah commented on pull request #11550: KRaft to KRaft upgrades

2022-02-01 Thread GitBox
mumrah commented on pull request #11550: URL: https://github.com/apache/kafka/pull/11550#issuecomment-1027091285 Closed in favor of #11677 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mumrah closed pull request #11550: KRaft to KRaft upgrades

2022-02-01 Thread GitBox
mumrah closed pull request #11550: URL: https://github.com/apache/kafka/pull/11550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-13600) Rebalances while streams is in degraded state can cause stores to be reassigned and restore from scratch

2022-02-01 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485343#comment-17485343 ] Bruno Cadonna commented on KAFKA-13600: --- [~tim.patterson] To come up with a perfect solution is

[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3

2022-02-01 Thread GitBox
cadonna commented on pull request #11690: URL: https://github.com/apache/kafka/pull/11690#issuecomment-1026894087 I re-triggered the builds. Once the builds are acceptable, I will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Updated] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhilesh Dubey updated KAFKA-13635: --- Description: While working with 2.7.1, we experienced offset reset on some consumer groups

[jira] [Updated] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhilesh Dubey updated KAFKA-13635: --- Affects Version/s: 2.7.1 > Make Consumer Group Protocol resilient to disk issues with

[jira] [Created] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
Akhilesh Dubey created KAFKA-13635: -- Summary: Make Consumer Group Protocol resilient to disk issues with __consumer_offsets Key: KAFKA-13635 URL: https://issues.apache.org/jira/browse/KAFKA-13635

[GitHub] [kafka] mimaison commented on pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
mimaison commented on pull request #11695: URL: https://github.com/apache/kafka/pull/11695#issuecomment-1026794372 Thanks @dajac for the review! I've pushed an update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] Indupa commented on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-02-01 Thread GitBox
Indupa commented on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1026731878 Hi @dongjinleekr , Can you please help me to build this latest patch you have released on top of Kafka_2.8,1. I tried using this following command to compile kafka source

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r796479255 ## File path: clients/src/main/java/org/apache/kafka/common/record/RemoteLogInputStream.java ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r796465473 ## File path: clients/src/main/java/org/apache/kafka/common/record/RemoteLogInputStream.java ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-13633) Merging multiple KStreams in one operation

2022-02-01 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated KAFKA-13633: - Labels: (was: needs-kip) > Merging multiple KStreams in one operation >

[jira] [Commented] (KAFKA-13633) Merging multiple KStreams in one operation

2022-02-01 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485163#comment-17485163 ] Nicholas Telford commented on KAFKA-13633: -- I actually did create a KIP, I just forgot to add

[GitHub] [kafka] dajac merged pull request #11547: fixed typo "Exsiting" -> "Existing"

2022-02-01 Thread GitBox
dajac merged pull request #11547: URL: https://github.com/apache/kafka/pull/11547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] dajac merged pull request #11723: MINOR: replace statement lambda with expression lambda

2022-02-01 Thread GitBox
dajac merged pull request #11723: URL: https://github.com/apache/kafka/pull/11723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] dajac commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796391527 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition:

[jira] [Commented] (KAFKA-9542) ZSTD Compression Not Working

2022-02-01 Thread kaushik srinivas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485114#comment-17485114 ] kaushik srinivas commented on KAFKA-9542: - [~lucasbradstreet]  We see this issue even with

[GitHub] [kafka] showuon commented on pull request #11691: KAFKA-13598: enable idempotence producer by default and validate the configs

2022-02-01 Thread GitBox
showuon commented on pull request #11691: URL: https://github.com/apache/kafka/pull/11691#issuecomment-1026588225 @hachikuji , thanks for your comments. I'll update it after Chinese new year holidays. And I agree with @ijuma to fix in v3.0.1, v3.1.1, because this feature was

[GitHub] [kafka] dajac commented on a change in pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11695: URL: https://github.com/apache/kafka/pull/11695#discussion_r796350195 ## File path: core/src/main/scala/kafka/tools/ConsoleProducer.scala ## @@ -362,11 +374,22 @@ object ConsoleProducer { } private def

[GitHub] [kafka] dajac commented on a change in pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11599: URL: https://github.com/apache/kafka/pull/11599#discussion_r796347687 ## File path: core/src/test/scala/integration/kafka/api/AuthorizerIntegrationTest.scala ## @@ -223,7 +223,7 @@ class AuthorizerIntegrationTest extends