[jira] [Commented] (KAFKA-10431) ProducerPerformance with payloadFile arg: add support for sequential or random outputs

[ https://issues.apache.org/jira/browse/KAFKA-10431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185615#comment-17185615 ] Zaahir Laher commented on KAFKA-10431: -- Yes, so basically the payloads from the file will be sent

[GitHub] [kafka] LMnet commented on a change in pull request #8955: KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616)

LMnet commented on a change in pull request #8955: URL: https://github.com/apache/kafka/pull/8955#discussion_r478127423 ## File path: streams/streams-scala/src/main/scala/org/apache/kafka/streams/scala/serialization/Serdes.scala ## @@ -0,0 +1,93 @@ +/* + * Copyright (C) 2018

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-681375395 > This issue will be resolved if we complete those delayed operations due to groupManager.storeGroup() elsewhere without holding any locks. So ```DelayedJoin``` DOES NOT

[GitHub] [kafka] junrao commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

junrao commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-681361030 The following is my understand. The current PR introduces a new deadlock through the following path. path 1 hold group lock ->

[GitHub] [kafka] mumrah commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

mumrah commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r478069145 ## File path: core/src/main/scala/kafka/server/AlterIsrManager.scala ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] mumrah commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

mumrah commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r478067221 ## File path: core/src/main/scala/kafka/server/AlterIsrManager.scala ## @@ -0,0 +1,145 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-681318790 > We keep DelayedJoin as it is, still passing in the group lock to DelayedOperation to avoid deadlocks due to two levels of locking. Just double check. We use a separate

[GitHub] [kafka] showuon commented on a change in pull request #9179: KAFKA-10390: Remove ignore case option when grep process info to be more specific

showuon commented on a change in pull request #9179: URL: https://github.com/apache/kafka/pull/9179#discussion_r477984582 ## File path: bin/kafka-server-stop.sh ## @@ -21,7 +21,7 @@ if [[ $(uname -s) == "OS/390" ]]; then fi PIDS=$(ps -A -o pid,jobname,comm | grep -i

[GitHub] [kafka] hachikuji commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

hachikuji commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r477618083 ## File path: clients/src/main/resources/common/message/AlterIsrResponse.json ## @@ -0,0 +1,38 @@ +// Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] ableegoldman commented on a change in pull request #9157: Update for KIP-450 to handle early records

ableegoldman commented on a change in pull request #9157: URL: https://github.com/apache/kafka/pull/9157#discussion_r477784084 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregate.java ## @@ -160,11 +160,18 @@ public

[jira] [Commented] (KAFKA-10431) ProducerPerformance with payloadFile arg: add support for sequential or random outputs

[ https://issues.apache.org/jira/browse/KAFKA-10431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185531#comment-17185531 ] huxihx commented on KAFKA-10431: Seems adding a new option to ProducerPerformance requires a KIP. I am

[GitHub] [kafka] huxihx commented on pull request #9218: MINOR: Fix shouldNotResetEpochHistoryHeadIfUndefinedPassed

huxihx commented on pull request #9218: URL: https://github.com/apache/kafka/pull/9218#issuecomment-681198703 @ijuma Please review this patch. Thanks. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] ableegoldman commented on a change in pull request #8834: KAFKA-10134: Enable heartbeat during PrepareRebalance and Depend On State For Poll Timeout

ableegoldman commented on a change in pull request #8834: URL: https://github.com/apache/kafka/pull/8834#discussion_r477663328 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java ## @@ -528,7 +528,6 @@ public void

[GitHub] [kafka] guozhangwang commented on a change in pull request #9222: KAFKA-10437: Implement test-utils and StateStore changes for KIP-478

guozhangwang commented on a change in pull request #9222: URL: https://github.com/apache/kafka/pull/9222#discussion_r477676262 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/StateStoreContext.java ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] jthompson6 commented on pull request #9216: KAFKA-10428: Fix schema for header conversion in MirrorSourceTask.

jthompson6 commented on pull request #9216: URL: https://github.com/apache/kafka/pull/9216#issuecomment-681165609 I see the header converter class is configurable in

[jira] [Commented] (KAFKA-10410) OnRestoreStart disappeared from StateRestoreCallback in 2.6.0 and reappeared in a useless place

[ https://issues.apache.org/jira/browse/KAFKA-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185511#comment-17185511 ] Sophie Blee-Goldman commented on KAFKA-10410: - It's hard to say without knowing specifically

[GitHub] [kafka] mjsax commented on a change in pull request #9156: KAFKA-10077: Filter downstream of state-store results in spurious tombstones

mjsax commented on a change in pull request #9156: URL: https://github.com/apache/kafka/pull/9156#discussion_r477622476 ## File path: streams/src/test/java/org/apache/kafka/streams/StreamsBuilderTest.java ## @@ -123,7 +123,7 @@ public void

[GitHub] [kafka] mjsax commented on pull request #9064: KAFKA-10205: Documentation and handling of non deterministic Topologies

mjsax commented on pull request #9064: URL: https://github.com/apache/kafka/pull/9064#issuecomment-681153031 Compile error: ``` /home/jenkins/jenkins-slave/workspace/kafka-pr-jdk11-scala2.13/streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamTaskTest.java:1989:

[GitHub] [kafka] mjsax commented on a change in pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

mjsax commented on a change in pull request #9000: URL: https://github.com/apache/kafka/pull/9000#discussion_r477620408 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamUtil.java ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe merged pull request #9194: KAFKA-10384: Separate converters from generated messages

cmccabe merged pull request #9194: URL: https://github.com/apache/kafka/pull/9194 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] mjsax commented on pull request #9121: KAFKA-10351: add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint

mjsax commented on pull request #9121: URL: https://github.com/apache/kafka/pull/9121#issuecomment-681148623 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] mjsax commented on a change in pull request #9121: KAFKA-10351: add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint

mjsax commented on a change in pull request #9121: URL: https://github.com/apache/kafka/pull/9121#discussion_r477616241 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImplTest.java ## @@ -180,6 +183,27 @@ public void

[GitHub] [kafka] mjsax commented on a change in pull request #9121: KAFKA-10351: add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint

mjsax commented on a change in pull request #9121: URL: https://github.com/apache/kafka/pull/9121#discussion_r477606207 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImplTest.java ## @@ -180,6 +183,27 @@ public void

[GitHub] [kafka] mjsax commented on pull request #9186: KAFKA-10277: Allow null keys with non-null mappedKey in KStreamKGlobalTable join

mjsax commented on pull request #9186: URL: https://github.com/apache/kafka/pull/9186#issuecomment-681138143 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Updated] (KAFKA-10395) TopologyTestDriver does not work with dynamic topic routing

[ https://issues.apache.org/jira/browse/KAFKA-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10395: Fix Version/s: 2.6.1 > TopologyTestDriver does not work with dynamic topic routing >

[GitHub] [kafka] mjsax commented on pull request #9174: KAFKA-10395: relax output topic check in TTD to work with dynamic routing

mjsax commented on pull request #9174: URL: https://github.com/apache/kafka/pull/9174#issuecomment-681137931 Cherry-picked to `2.6` branch. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] mjsax commented on pull request #9146: KAFKA-10316 Updated Kafka Streams upgrade-guide.html

mjsax commented on pull request #9146: URL: https://github.com/apache/kafka/pull/9146#issuecomment-681131093 My pleasure :) This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (KAFKA-10186) Aborting transaction with pending data should throw non-fatal exception

[ https://issues.apache.org/jira/browse/KAFKA-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185482#comment-17185482 ] Gokul Srinivas commented on KAFKA-10186: [~mjsax]- I've cleaned up the KIP. Please let me know

[jira] [Commented] (KAFKA-10410) OnRestoreStart disappeared from StateRestoreCallback in 2.6.0 and reappeared in a useless place

[ https://issues.apache.org/jira/browse/KAFKA-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185481#comment-17185481 ] Mark Shelton commented on KAFKA-10410: -- My implementation of StateRestoreCallback needs to know

[GitHub] [kafka] junrao commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

junrao commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-681109748 @chia7712 : Thanks for the reply. I like your overall idea and I think it can be used to solve the problem completely in a simpler way. 1. Instead of at `Partition`, we

[GitHub] [kafka] ryannedolan commented on pull request #9216: KAFKA-10428: Fix schema for header conversion in MirrorSourceTask.

ryannedolan commented on pull request #9216: URL: https://github.com/apache/kafka/pull/9216#issuecomment-681062141 Thanks for finding this. Would it make sense to implement a new nop converter for this purpose? Not sure whether a KIP would be required -- maybe it can be private to the

[GitHub] [kafka] jthompson6 commented on pull request #9216: KAFKA-10428: Fix schema for header conversion in MirrorSourceTask.

jthompson6 commented on pull request #9216: URL: https://github.com/apache/kafka/pull/9216#issuecomment-681056654 @ryannedolan are you the right person to tag here? This is an automated message from the Apache Git Service.

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r477485916 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -128,10 +136,23 @@ public void

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r477485106 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -367,14 +406,37 @@ public void

[GitHub] [kafka] vvcephei commented on a change in pull request #9222: KAFKA-10437: Implement test-utils and StateStore changes for KIP-478

vvcephei commented on a change in pull request #9222: URL: https://github.com/apache/kafka/pull/9222#discussion_r477477190 ## File path: streams/examples/src/test/java/org/apache/kafka/streams/examples/wordcount/WordCountProcessorTest.java ## @@ -36,19 +36,19 @@ public class

[GitHub] [kafka] mjsax merged pull request #9217: MINOR: fix JavaDoc

mjsax merged pull request #9217: URL: https://github.com/apache/kafka/pull/9217 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-10410) OnRestoreStart disappeared from StateRestoreCallback in 2.6.0 and reappeared in a useless place

[ https://issues.apache.org/jira/browse/KAFKA-10410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185367#comment-17185367 ] Guozhang Wang commented on KAFKA-10410: --- Not sure if I fully understand what do you mean by

[GitHub] [kafka] ning2008wisc opened a new pull request #9224: improve MM2 unit tests

ning2008wisc opened a new pull request #9224: URL: https://github.com/apache/kafka/pull/9224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] bbejeck commented on pull request #9207: Minor remove semicolon

bbejeck commented on pull request #9207: URL: https://github.com/apache/kafka/pull/9207#issuecomment-681005444 Tests seem to be hung, retesting retest this please This is an automated message from the Apache Git

[jira] [Commented] (KAFKA-10434) Remove deprecated methods on WindowStore

[ https://issues.apache.org/jira/browse/KAFKA-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185325#comment-17185325 ] Jorge Esteban Quilcate Otoya commented on KAFKA-10434: -- Also kind of weird to have

[GitHub] [kafka] jeqo commented on a change in pull request #9138: KAFKA-9929: Support backward iterator on WindowStore

jeqo commented on a change in pull request #9138: URL: https://github.com/apache/kafka/pull/9138#discussion_r477436882 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/CachingWindowStore.java ## @@ -416,26 +552,43 @@ private long

[GitHub] [kafka] vvcephei commented on a change in pull request #9177: KAFKA-9924: Add RocksDB metric num-entries-active-mem-table

vvcephei commented on a change in pull request #9177: URL: https://github.com/apache/kafka/pull/9177#discussion_r477424578 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/metrics/StreamsMetricsImpl.java ## @@ -396,34 +398,65 @@ private String

[jira] [Created] (KAFKA-10439) Connect's Values class loses precision for integers, larger than 64 bits

Oleksandr Diachenko created KAFKA-10439: --- Summary: Connect's Values class loses precision for integers, larger than 64 bits Key: KAFKA-10439 URL: https://issues.apache.org/jira/browse/KAFKA-10439

[GitHub] [kafka] satishd commented on a change in pull request #7561: [WIP] KAFKA-7739: Tiered storage

satishd commented on a change in pull request #7561: URL: https://github.com/apache/kafka/pull/7561#discussion_r477403016 ## File path: core/src/main/scala/kafka/log/remote/RemoteIndexCache.scala ## @@ -0,0 +1,218 @@ +/** + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lct45 commented on a change in pull request #9157: Update for KIP-450 to handle early records

lct45 commented on a change in pull request #9157: URL: https://github.com/apache/kafka/pull/9157#discussion_r477380346 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregate.java ## @@ -148,7 +153,7 @@ public void

[GitHub] [kafka] vvcephei commented on a change in pull request #8955: KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616)

vvcephei commented on a change in pull request #8955: URL: https://github.com/apache/kafka/pull/8955#discussion_r477358755 ## File path: streams/streams-scala/src/main/scala/org/apache/kafka/streams/scala/serialization/Serdes.scala ## @@ -0,0 +1,93 @@ +/* + * Copyright (C)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9179: KAFKA-10390: Remove ignore case option when grep process info to be more specific

lbradstreet commented on a change in pull request #9179: URL: https://github.com/apache/kafka/pull/9179#discussion_r477352942 ## File path: bin/kafka-server-stop.sh ## @@ -21,7 +21,7 @@ if [[ $(uname -s) == "OS/390" ]]; then fi PIDS=$(ps -A -o pid,jobname,comm | grep

[GitHub] [kafka] vvcephei commented on pull request #8955: KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616)

vvcephei commented on pull request #8955: URL: https://github.com/apache/kafka/pull/8955#issuecomment-680917350 Thanks, @LMnet ! The commit policy is just that the committer will squash the whole PR into one commit when merging, so you don't have to worry about it.

[GitHub] [kafka] showuon commented on pull request #9179: KAFKA-10390: Remove ignore case option when grep process info to be more specific

showuon commented on pull request #9179: URL: https://github.com/apache/kafka/pull/9179#issuecomment-680858304 @cmccabe @lbradstreet , could you please review this PR? Thanks. This is an automated message from the Apache Git

[GitHub] [kafka] showuon commented on pull request #9178: KAFKA-8362: fix the old checkpoint won't be removed after alter log dir

showuon commented on pull request #9178: URL: https://github.com/apache/kafka/pull/9178#issuecomment-680857804 @mingaliu @hachikuji @omkreddy , could you please review this PR? Thanks. This is an automated message from the

[GitHub] [kafka] showuon commented on pull request #9149: KAFKA-10340: improve the logging to help user know what is going on

showuon commented on pull request #9149: URL: https://github.com/apache/kafka/pull/9149#issuecomment-680857267 @kkonstantine , could you please review this small PR? Thanks. This is an automated message from the Apache Git

[GitHub] [kafka] showuon commented on pull request #9062: KAFKA-8098: fix the flaky test by disabling the auto commit to avoid member rejoining

showuon commented on pull request #9062: URL: https://github.com/apache/kafka/pull/9062#issuecomment-680856775 @abbccdda @omkreddy @feyman2016 @huxihx , could you help review this PR? Thanks. This is an automated message

[GitHub] [kafka] johnthotekat edited a comment on pull request #9146: KAFKA-10316 Updated Kafka Streams upgrade-guide.html

johnthotekat edited a comment on pull request #9146: URL: https://github.com/apache/kafka/pull/9146#issuecomment-680817137 Thanks to you for guiding me through!! And it was so nice to see you on the Kafka summit :) @mjsax

[GitHub] [kafka] johnthotekat commented on pull request #9146: KAFKA-10316 Updated Kafka Streams upgrade-guide.html

johnthotekat commented on pull request #9146: URL: https://github.com/apache/kafka/pull/9146#issuecomment-680817137 Thanks to you for guiding me through!! And it was so nice to see you on the Kafka summit :) This is an

[GitHub] [kafka] viktorsomogyi commented on pull request #9150: KAFKA-9839; Broker should accept control requests with newer broker epoch

viktorsomogyi commented on pull request #9150: URL: https://github.com/apache/kafka/pull/9150#issuecomment-680800344 @dajac pinging you as well, would you please review this and the 2.3 and 2.4 backports? :) This is an

[GitHub] [kafka] chia7712 opened a new pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

chia7712 opened a new pull request #9223: URL: https://github.com/apache/kafka/pull/9223 issue: https://issues.apache.org/jira/browse/KAFKA-10438 There is no checks for header key so instantiating key (bytes to string) is unnecessary. The risk of this PR is the exception of

[jira] [Created] (KAFKA-10438) Lazy initialization of record header to reduce memory usage in validating records

Chia-Ping Tsai created KAFKA-10438: -- Summary: Lazy initialization of record header to reduce memory usage in validating records Key: KAFKA-10438 URL: https://issues.apache.org/jira/browse/KAFKA-10438

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185062#comment-17185062 ] Jerry Wei commented on KAFKA-10134: --- [~guozhang] cool, I'll try it later, thanks > High CPU issue

[jira] [Resolved] (KAFKA-9444) Cannot connect to zookeeper after updating the kafka config

[ https://issues.apache.org/jira/browse/KAFKA-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rishabh Bohra resolved KAFKA-9444. -- Resolution: Invalid Please refer to [KAFKA-7335] - Store clusterId locally to ensure broker

[GitHub] [kafka] chia7712 commented on pull request #9206: MINOR: rewrite zipWithIndex by normal foreach to refrain unnecessary …

chia7712 commented on pull request #9206: URL: https://github.com/apache/kafka/pull/9206#issuecomment-680715966 > Do you have any data on the improvement this achieves? @ijuma the JMH result is attached. please take a look.