[jira] [Commented] (KAFKA-6020) Broker side filtering

2021-09-23 Thread King Jin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419571#comment-17419571 ] King Jin commented on KAFKA-6020: - How about broker side filtering by Kafka Headers? The consumer sent

[GitHub] [kafka] ccding commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-09-23 Thread GitBox
ccding commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-926250962 Failed tests are irrelevant and passed on my local run. ``` Build / JDK 8 and Scala 2.12 /

[GitHub] [kafka] junrao commented on a change in pull request #11327: KAFKA-13305: fix NullPointerException in LogCleanerManager "uncleanable-bytes" gauge

2021-09-23 Thread GitBox
junrao commented on a change in pull request #11327: URL: https://github.com/apache/kafka/pull/11327#discussion_r715227056 ## File path: core/src/main/scala/kafka/log/LogCleanerManager.scala ## @@ -512,6 +514,27 @@ private[log] class LogCleanerManager(val logDirs: Seq[File],

[jira] [Resolved] (KAFKA-13070) LogManager shutdown races with periodic work scheduled by the instance

2021-09-23 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-13070. - Resolution: Duplicate > LogManager shutdown races with periodic work scheduled by the instance >

[jira] [Resolved] (KAFKA-13315) log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-13315. - Fix Version/s: 3.0.1 3.1.0 Resolution: Fixed merged the PR to trunk and 3.0.

[GitHub] [kafka] junrao commented on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread GitBox
junrao commented on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-926239908 cherry-picked to 3.0 branch too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-13272) KStream offset stuck after brokers outage

2021-09-23 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419498#comment-17419498 ] A. Sophie Blee-Goldman commented on KAFKA-13272: [~guozhang] any updates here? I'm

[GitHub] [kafka] junrao merged pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread GitBox
junrao merged pull request #11351: URL: https://github.com/apache/kafka/pull/11351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] vincent81jiang commented on a change in pull request #11327: KAFKA-13305: fix NullPointerException in LogCleanerManager "uncleanable-bytes" gauge

2021-09-23 Thread GitBox
vincent81jiang commented on a change in pull request #11327: URL: https://github.com/apache/kafka/pull/11327#discussion_r715169127 ## File path: core/src/main/scala/kafka/log/LogCleanerManager.scala ## @@ -512,6 +514,27 @@ private[log] class LogCleanerManager(val logDirs:

[GitHub] [kafka] vincent81jiang commented on a change in pull request #11327: KAFKA-13305: fix NullPointerException in LogCleanerManager "uncleanable-bytes" gauge

2021-09-23 Thread GitBox
vincent81jiang commented on a change in pull request #11327: URL: https://github.com/apache/kafka/pull/11327#discussion_r715168688 ## File path: core/src/main/scala/kafka/log/LogCleanerManager.scala ## @@ -512,6 +514,27 @@ private[log] class LogCleanerManager(val logDirs:

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r71507 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -141,9 +145,14 @@ class KafkaScheduler(val threads: Int, executor != null

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r71507 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -141,9 +145,14 @@ class KafkaScheduler(val threads: Int, executor != null

[GitHub] [kafka] junrao commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-23 Thread GitBox
junrao commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r715068733 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -141,9 +145,14 @@ class KafkaScheduler(val threads: Int, executor != null

[GitHub] [kafka] jeqo commented on a change in pull request #11356: [KAFKA-10539] Convert KStreamImpl joins to new PAPI

2021-09-23 Thread GitBox
jeqo commented on a change in pull request #11356: URL: https://github.com/apache/kafka/pull/11356#discussion_r715015262 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -200,11 +217,11 @@ private void

[GitHub] [kafka] jeqo opened a new pull request #11356: [KAFKA-10539] Convert KStreamImpl joins to new PAPI

2021-09-23 Thread GitBox
jeqo opened a new pull request #11356: URL: https://github.com/apache/kafka/pull/11356 As part of the migration to new Processor API, this PR converts KStream to KStream joins. Depends #11315 ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[jira] [Created] (KAFKA-13321) Notify listener of leader change on registration

2021-09-23 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-13321: -- Summary: Notify listener of leader change on registration Key: KAFKA-13321 URL: https://issues.apache.org/jira/browse/KAFKA-13321 Project:

[GitHub] [kafka] hachikuji merged pull request #11354: MINOR: Print lastTimestamp when dumping producer snapshots

2021-09-23 Thread GitBox
hachikuji merged pull request #11354: URL: https://github.com/apache/kafka/pull/11354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] wcarlson5 commented on a change in pull request #11347: KAFKA-13296: warn if previous assignment has duplicate partitions

2021-09-23 Thread GitBox
wcarlson5 commented on a change in pull request #11347: URL: https://github.com/apache/kafka/pull/11347#discussion_r714990115 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -351,6 +351,17 @@ public

[GitHub] [kafka] wcarlson5 commented on a change in pull request #11347: KAFKA-13296: warn if previous assignment has duplicate partitions

2021-09-23 Thread GitBox
wcarlson5 commented on a change in pull request #11347: URL: https://github.com/apache/kafka/pull/11347#discussion_r714987990 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -351,6 +351,17 @@ public

[GitHub] [kafka] satishd commented on pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker restarts.

2021-09-23 Thread GitBox
satishd commented on pull request #11058: URL: https://github.com/apache/kafka/pull/11058#issuecomment-925983449 Thanks @ccding for the review, addressed with inline replies. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] satishd commented on pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker restarts.

2021-09-23 Thread GitBox
satishd commented on pull request #11058: URL: https://github.com/apache/kafka/pull/11058#issuecomment-925982858 Thanks @junrao for the review. Addressed them with the latest commit and comments. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714976848 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataSnapshotFile.java ## @@ -0,0 +1,239 @@ +/* + *

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714976377 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataCache.java ## @@ -161,53 +161,46 @@ public void

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714975955 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataSnapshotFile.java ## @@ -0,0 +1,252 @@ +/* + *

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714975590 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogMetadataSerde.java ## @@ -39,6 +41,7 @@

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714975169 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java ## @@ -120,6 +174,35 @@ public void run() {

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714974207 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java ## @@ -120,6 +174,35 @@ public void run() {

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714973902 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java ## @@ -85,21 +90,68 @@ // Map of remote

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714972637 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java ## @@ -85,21 +90,68 @@ // Map of remote

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714971600 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerManager.java ## @@ -49,21 +52,23 @@ public

[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke

2021-09-23 Thread GitBox
satishd commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r714971397 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/CommittedOffsetsFile.java ## @@ -0,0 +1,80 @@ +/* + * Licensed

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714947277 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala ## @@ -929,6 +929,29 @@ class AbstractFetcherThreadTest {

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714936472 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala ## @@ -929,6 +929,29 @@ class AbstractFetcherThreadTest {

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714923995 ## File path: core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala ## @@ -3469,4 +3472,88 @@ class ReplicaManagerTest {

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714910220 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala ## @@ -929,6 +929,29 @@ class AbstractFetcherThreadTest {

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714901210 ## File path: core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala ## @@ -3469,4 +3472,88 @@ class ReplicaManagerTest {

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714899137 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -491,6 +491,21 @@ abstract class AbstractFetcherThread(name: String,

[jira] [Updated] (KAFKA-13320) Suggestion: SMT support for null key/value should be documented

2021-09-23 Thread Ben Ellis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Ellis updated KAFKA-13320: -- Description: While working with a JDBC Sink Connector, I noticed that some SMT choke on a tombstone

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714815080 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -862,6 +877,10 @@ case class PartitionFetchState(fetchOffset: Long,

[jira] [Updated] (KAFKA-13320) Suggestion: SMT support for null key/value should be documented

2021-09-23 Thread Ben Ellis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Ellis updated KAFKA-13320: -- Description: While working with a JDBC Sink Connector, I noticed that some SMT choke on a tombstone

[jira] [Created] (KAFKA-13320) Suggestion: SMT support for null key/value should be documented

2021-09-23 Thread Ben Ellis (Jira)
Ben Ellis created KAFKA-13320: - Summary: Suggestion: SMT support for null key/value should be documented Key: KAFKA-13320 URL: https://issues.apache.org/jira/browse/KAFKA-13320 Project: Kafka

[jira] [Commented] (KAFKA-13298) Improve documentation on EOS KStream requirements

2021-09-23 Thread Andy Chambers (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419166#comment-17419166 ] Andy Chambers commented on KAFKA-13298: --- I made a PR for this but initially did not use the

[GitHub] [kafka] cddr opened a new pull request #11355: Improve documentation on EOS KStream requirements

2021-09-23 Thread GitBox
cddr opened a new pull request #11355: URL: https://github.com/apache/kafka/pull/11355 Minor documentation fix to address: https://issues.apache.org/jira/browse/KAFKA-13298 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ]

[GitHub] [kafka] ijuma merged pull request #11321: MINOR: Replace EasyMock with Mockito in connect:basic-auth-extension

2021-09-23 Thread GitBox
ijuma merged pull request #11321: URL: https://github.com/apache/kafka/pull/11321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] lkokhreidze commented on pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-09-23 Thread GitBox
lkokhreidze commented on pull request #10851: URL: https://github.com/apache/kafka/pull/10851#issuecomment-925659295 Hi @cadonna is it possible to continue pushing this PR forward? I'm back from my holidays. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Resolved] (KAFKA-10544) Convert KTable aggregations to new PAPI

2021-09-23 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya resolved KAFKA-10544. -- Resolution: Fixed https://github.com/apache/kafka/pull/11316

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714557835 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1396,6 +1398,8 @@ class ReplicaManager(val config: KafkaConfig,

[jira] [Closed] (KAFKA-13302) [IEP-59] Support not default page size

2021-09-23 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov closed KAFKA-13302. --- > [IEP-59] Support not default page size > -- > >

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714549045 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherManagerTest.scala ## @@ -133,4 +137,75 @@ class AbstractFetcherManagerTest {

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-23 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714547943 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -491,6 +491,21 @@ abstract class AbstractFetcherThread(name: String,

[jira] [Assigned] (KAFKA-13319) Do not send AddOffsetsToTxn/TxnOffsetCommit if offsets map is empty

2021-09-23 Thread Ryan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan reassigned KAFKA-13319: Assignee: Ryan > Do not send AddOffsetsToTxn/TxnOffsetCommit if offsets map is empty >