[GitHub] [kafka] vinothchandar commented on issue #8462: KAFKA-9846: Filter active tasks for running state in KafkaStreams#allLocalStorePartitionLags()

2020-04-21 Thread GitBox
vinothchandar commented on issue #8462: URL: https://github.com/apache/kafka/pull/8462#issuecomment-617567564 @guozhangwang So, I still need to add a test case around this specific scenario, tasks stuck in created state.. Seems it needs some engineering to create that scenario. (sophie

[GitHub] [kafka] vinothchandar commented on a change in pull request #8462: KAFKA-9846: Filter active tasks for running state in KafkaStreams#allLocalStorePartitionLags()

2020-04-21 Thread GitBox
vinothchandar commented on a change in pull request #8462: URL: https://github.com/apache/kafka/pull/8462#discussion_r412690139 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/LagFetchIntegrationTest.java ## @@ -295,9 +295,10 @@ public void

[jira] [Created] (KAFKA-9901) TimeoutError: Never saw message indicating StreamsTest finished startup on ducker@ducker07

2020-04-21 Thread jiamei xie (Jira)
jiamei xie created KAFKA-9901: - Summary: TimeoutError: Never saw message indicating StreamsTest finished startup on ducker@ducker07 Key: KAFKA-9901 URL: https://issues.apache.org/jira/browse/KAFKA-9901

[GitHub] [kafka] ijuma commented on a change in pull request #8517: MINOR: use monotonic clock for replica fetcher DelayedItem

2020-04-21 Thread GitBox
ijuma commented on a change in pull request #8517: URL: https://github.com/apache/kafka/pull/8517#discussion_r412680581 ## File path: core/src/main/scala/kafka/utils/DelayedItem.scala ## @@ -21,24 +21,19 @@ import java.util.concurrent._ import

[GitHub] [kafka] d8tltanc opened a new pull request #8528: Changed the system tests for --zookeeper flag removal

2020-04-21 Thread GitBox
d8tltanc opened a new pull request #8528: URL: https://github.com/apache/kafka/pull/8528 * Remove the --zookeeper flags for node versions supporting --bootstrap-server for TopicCommand. For the scram credential related code piece, switched to using --bootstrap-server even we don't

[GitHub] [kafka] ConcurrencyPractitioner commented on issue #7884: [KAFKA-8522] Streamline tombstone and transaction marker removal

2020-04-21 Thread GitBox
ConcurrencyPractitioner commented on issue #7884: URL: https://github.com/apache/kafka/pull/7884#issuecomment-617537261 test this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ConcurrencyPractitioner commented on issue #7884: [KAFKA-8522] Streamline tombstone and transaction marker removal

2020-04-21 Thread GitBox
ConcurrencyPractitioner commented on issue #7884: URL: https://github.com/apache/kafka/pull/7884#issuecomment-617537213 ok to test This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] senthilm-ms commented on issue #8103: KAFKA-7061: KIP-280 Enhanced log compaction

2020-04-21 Thread GitBox
senthilm-ms commented on issue #8103: URL: https://github.com/apache/kafka/pull/8103#issuecomment-617535197 @junrao @guozhangwang can you please review and let me know if you have any more comments. would like to close and move on.

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Attachment: leak3.jpg > Expired Sensor can not be remove,and it fills all the old generation,can't be > GC.

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Description: I tried kafka-client 2.1.0 and 2.1.1, but it exists both. I found that ,lots of Sensor and

[GitHub] [kafka] d8tltanc opened a new pull request #8527: Splitted unit tests for --zookeeper flag removal

2020-04-21 Thread GitBox
d8tltanc opened a new pull request #8527: URL: https://github.com/apache/kafka/pull/8527 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] showuon commented on issue #8482: KAFKA-9863: update the deprecated --zookeeper option in the documentation into --bootstrap-server

2020-04-21 Thread GitBox
showuon commented on issue #8482: URL: https://github.com/apache/kafka/pull/8482#issuecomment-617529478 @junrao @omkreddy , could you please help review this PR? This is a small and straightforward change to update the documentation only, but I think it's important to users. Thank you

[GitHub] [kafka] apovzner commented on issue #8509: KAFKA-9839: Broker should accept control requests with newer broker epoch

2020-04-21 Thread GitBox
apovzner commented on issue #8509: URL: https://github.com/apache/kafka/pull/8509#issuecomment-617525619 @dajac Sounds good! I added unit tests to KafkaApisTest as you suggested. This is an automated message from the Apache

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Description: I tried kafka-client 2.1.0 and 2.1.1, but it exists both. I found that ,lots of Sensor and

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Attachment: leak2.jpg > Expired Sensor can not be remove,and it fills all the old generation,can't be > GC.

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Description:   !leak2.jpg|width=612,height=227! !Leak1.jpg|width=354,height=286! > Expired Sensor can not

[jira] [Updated] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-9900: --- Attachment: Leak1.jpg > Expired Sensor can not be remove,and it fills all the old generation,can't be > GC.

[jira] [Created] (KAFKA-9900) Expired Sensor can not be remove,and it fills all the old generation,can't be GC.

2020-04-21 Thread Chenxin (Jira)
Chenxin created KAFKA-9900: -- Summary: Expired Sensor can not be remove,and it fills all the old generation,can't be GC. Key: KAFKA-9900 URL: https://issues.apache.org/jira/browse/KAFKA-9900 Project: Kafka

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412633489 ## File path: clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java ## @@ -547,17 +547,15 @@ public void

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412633349 ## File path: clients/src/test/java/org/apache/kafka/clients/MetadataTest.java ## @@ -199,19 +200,21 @@ public void

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412632269 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsForLeaderEpochClient.java ## @@ -85,10 +85,6 @@ protected

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412632493 ## File path: clients/src/main/java/org/apache/kafka/common/requests/MetadataResponse.java ## @@ -464,22 +466,41 @@ public static MetadataResponse

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412632092 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java ## @@ -498,6 +498,104 @@ public void

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412631698 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java ## @@ -498,6 +498,104 @@ public void

[GitHub] [kafka] abbccdda commented on a change in pull request #8486: KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable

2020-04-21 Thread GitBox
abbccdda commented on a change in pull request #8486: URL: https://github.com/apache/kafka/pull/8486#discussion_r412631879 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java ## @@ -498,6 +498,104 @@ public void

[jira] [Updated] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-2419: --- Attachment: leak2.jpg > Allow certain Sensors to be garbage collected after inactivity >

[jira] [Commented] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089231#comment-17089231 ] Chenxin commented on KAFKA-2419: Execuse me. Is this bug fix now? I just see a ExpireSensorTask in

[jira] [Updated] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-2419: --- Attachment: Leak1.jpg > Allow certain Sensors to be garbage collected after inactivity >

[jira] [Updated] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2020-04-21 Thread Chenxin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxin updated KAFKA-2419: --- Attachment: Leak1 > Allow certain Sensors to be garbage collected after inactivity >

[GitHub] [kafka] iceChen8123 commented on issue #233: KAFKA-2419; Garbage collect unused sensors

2020-04-21 Thread GitBox
iceChen8123 commented on issue #233: URL: https://github.com/apache/kafka/pull/233#issuecomment-617512892 Execuse me. Is this bug fix now? I just see a ExpireSensorTask in Metrics, but nowhere can use. Then I see beyond 800,000 sensors existed in my application, and it is in Old

[jira] [Commented] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to emit events after 24h

2020-04-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089214#comment-17089214 ] Matthias J. Sax commented on KAFKA-8924: {quote}there is no sensible default grace period {quote}

[jira] [Updated] (KAFKA-9898) Flaky Test StoreQueryIntegrationTest.shouldQueryAllStalePartitionStores

2020-04-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-9898: --- Component/s: unit tests > Flaky Test

[GitHub] [kafka] guozhangwang commented on issue #8462: KAFKA-9846: Filter active tasks for running state in KafkaStreams#allLocalStorePartitionLags()

2020-04-21 Thread GitBox
guozhangwang commented on issue #8462: URL: https://github.com/apache/kafka/pull/8462#issuecomment-617488326 test this please This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] andrewchoi5 commented on issue #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-04-21 Thread GitBox
andrewchoi5 commented on issue #8479: URL: https://github.com/apache/kafka/pull/8479#issuecomment-617479063 Thanks for referring Matthias. Would appreciate your review @hachikuji @cmccabe This is an automated message from

[jira] [Created] (KAFKA-9899) LogCleaner Tries To Clean Single Partition Over 1000x/Minute

2020-04-21 Thread Jeff Nadler (Jira)
Jeff Nadler created KAFKA-9899: -- Summary: LogCleaner Tries To Clean Single Partition Over 1000x/Minute Key: KAFKA-9899 URL: https://issues.apache.org/jira/browse/KAFKA-9899 Project: Kafka

[jira] [Commented] (KAFKA-7965) Flaky Test ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

2020-04-21 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089107#comment-17089107 ] Sophie Blee-Goldman commented on KAFKA-7965:

[jira] [Created] (KAFKA-9898) Flaky Test StoreQueryIntegrationTest.shouldQueryAllStalePartitionStores

2020-04-21 Thread Sophie Blee-Goldman (Jira)
Sophie Blee-Goldman created KAFKA-9898: -- Summary: Flaky Test StoreQueryIntegrationTest.shouldQueryAllStalePartitionStores Key: KAFKA-9898 URL: https://issues.apache.org/jira/browse/KAFKA-9898

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617439325 Unrelated java 11 failures: kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.

2020-04-21 Thread Yuexi Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089096#comment-17089096 ] Yuexi Liu commented on KAFKA-9335: -- [~vveeramani] I test it, the 2.4.1 fixed it >

[jira] [Commented] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to emit events after 24h

2020-04-21 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089090#comment-17089090 ] John Roesler commented on KAFKA-8924: - Thanks for the comment, [~cadonna] , I agree that it's not as

[GitHub] [kafka] vvcephei commented on issue #8254: KIP-557: Add Emit On Change Support

2020-04-21 Thread GitBox
vvcephei commented on issue #8254: URL: https://github.com/apache/kafka/pull/8254#issuecomment-617423885 I hope I didn't step on your toes, @ConcurrencyPractitioner , but I just wanted to make sure that you're unblocked to finish up this PR. Figuring out what's exactly wrong with those

[GitHub] [kafka] vvcephei commented on issue #8254: KIP-557: Add Emit On Change Support

2020-04-21 Thread GitBox
vvcephei commented on issue #8254: URL: https://github.com/apache/kafka/pull/8254#issuecomment-617423139 Ok, last one. The TransformValuesTest is just another case where the test input data is now considered idempotent, which is fine: ```diff ---

[GitHub] [kafka] vvcephei commented on issue #8254: KIP-557: Add Emit On Change Support

2020-04-21 Thread GitBox
vvcephei commented on issue #8254: URL: https://github.com/apache/kafka/pull/8254#issuecomment-617418517 The source topic restart integration test was actually just failing because the tests were polluting each others' topics. This is one way to fix it: ```diff diff --git

[jira] [Commented] (KAFKA-9897) Flaky Test StoreQueryIntegrationTest#shouldQuerySpecificActivePartitionStores

2020-04-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089052#comment-17089052 ] Matthias J. Sax commented on KAFKA-9897: [~NaviBrar] – would you be interested to look into this

[GitHub] [kafka] vvcephei commented on issue #8254: KIP-557: Add Emit On Change Support

2020-04-21 Thread GitBox
vvcephei commented on issue #8254: URL: https://github.com/apache/kafka/pull/8254#issuecomment-617392150 I also took a look at the foreign-key join test, which is actually telling us something awesome: your feature allows us to drop _unnecessary_ tombstones that we'd otherwise send under

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Jesse Anderson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089016#comment-17089016 ] Jesse Anderson commented on KAFKA-9882: --- Seeking each time was just an example. Adding a config

[GitHub] [kafka] vvcephei commented on issue #8520: Add explicit grace period to tumbling window example

2020-04-21 Thread GitBox
vvcephei commented on issue #8520: URL: https://github.com/apache/kafka/pull/8520#issuecomment-617376010 Gah, I forgot to add the reviewers to the merge commit. This is an automated message from the Apache Git Service. To

[GitHub] [kafka] surabhidixit opened a new pull request #8526: KAFKA-6867: corrected the typos in upgrade.html

2020-04-21 Thread GitBox
surabhidixit opened a new pull request #8526: URL: https://github.com/apache/kafka/pull/8526 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] vvcephei commented on issue #8254: KIP-557: Add Emit On Change Support

2020-04-21 Thread GitBox
vvcephei commented on issue #8254: URL: https://github.com/apache/kafka/pull/8254#issuecomment-617372772 Hey @ConcurrencyPractitioner , sorry it took so long. It's just again because the test happened to expect idempotent updates to flow through regularly, but not for anything important.

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089006#comment-17089006 ] Boyang Chen commented on KAFKA-9882: So you are trying to reset offset every time when you restart

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617364803 test this please This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617364180 test this please This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] ableegoldman commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
ableegoldman commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412424861 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ValidClientsByTaskLoadQueue.java ## @@ -0,0 +1,108 @@ +/*

[GitHub] [kafka] vvcephei commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412424445 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ValidClientsByTaskLoadQueue.java ## @@ -0,0 +1,108 @@ +/* + *

[GitHub] [kafka] vvcephei commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412423011 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ValidClientsByTaskLoadQueue.java ## @@ -0,0 +1,110 @@ +/* + *

[GitHub] [kafka] vvcephei commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412422082 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/TaskMovement.java ## @@ -16,128 +16,103 @@ */ package

[GitHub] [kafka] vvcephei commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412419973 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/TaskMovement.java ## @@ -16,128 +16,94 @@ */ package

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Jesse Anderson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088922#comment-17088922 ] Jesse Anderson commented on KAFKA-9882: --- {code:java} // Create KafkaConsumer and subscribe // Call

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088917#comment-17088917 ] Boyang Chen commented on KAFKA-9882: I see, to make the discussion more effective, could you draft a

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Jesse Anderson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088908#comment-17088908 ] Jesse Anderson commented on KAFKA-9882: --- subscribe() doesn't get a partition assignment until

[jira] [Commented] (KAFKA-9882) Add Block getAssignments()

2020-04-21 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1708#comment-1708 ] Boyang Chen commented on KAFKA-9882: I checked back on the original KIP 

[GitHub] [kafka] lbradstreet commented on a change in pull request #8517: MINOR: use monotonic clock for replica fetcher DelayedItem

2020-04-21 Thread GitBox
lbradstreet commented on a change in pull request #8517: URL: https://github.com/apache/kafka/pull/8517#discussion_r412313524 ## File path: core/src/main/scala/kafka/utils/DelayedItem.scala ## @@ -21,24 +21,23 @@ import java.util.concurrent._ import

[GitHub] [kafka] ijuma commented on a change in pull request #8417: KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy

2020-04-21 Thread GitBox
ijuma commented on a change in pull request #8417: URL: https://github.com/apache/kafka/pull/8417#discussion_r412309101 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java ## @@ -57,6 +59,10 @@ public ByteBuffer serialize(ApiKeys apiKey,

[GitHub] [kafka] lbradstreet commented on a change in pull request #8517: MINOR: use monotonic clock for replica fetcher DelayedItem

2020-04-21 Thread GitBox
lbradstreet commented on a change in pull request #8517: URL: https://github.com/apache/kafka/pull/8517#discussion_r412305902 ## File path: core/src/main/scala/kafka/utils/DelayedItem.scala ## @@ -21,24 +21,23 @@ import java.util.concurrent._ import

[GitHub] [kafka] mjsax commented on issue #8508: MINOR: Improve usage of LogCaptureAppender

2020-04-21 Thread GitBox
mjsax commented on issue #8508: URL: https://github.com/apache/kafka/pull/8508#issuecomment-617266065 Java 8 and Java 11 passed. Java 14: failed with unknown error... This is an automated message from the Apache Git

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617266153 Test this, please. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617265836 Test this, please. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] vvcephei commented on issue #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
vvcephei commented on issue #8497: URL: https://github.com/apache/kafka/pull/8497#issuecomment-617264248 Test this, please. This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Assigned] (KAFKA-9850) Move KStream#repartition operator validation during Topology build process

2020-04-21 Thread HaiyuanZhao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HaiyuanZhao reassigned KAFKA-9850: -- Assignee: HaiyuanZhao > Move KStream#repartition operator validation during Topology build

[GitHub] [kafka] tombentley commented on a change in pull request #8417: KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy

2020-04-21 Thread GitBox
tombentley commented on a change in pull request #8417: URL: https://github.com/apache/kafka/pull/8417#discussion_r412287986 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java ## @@ -57,6 +59,10 @@ public ByteBuffer serialize(ApiKeys

[GitHub] [kafka] chia7712 commented on a change in pull request #8517: MINOR: use monotonic clock for replica fetcher DelayedItem

2020-04-21 Thread GitBox
chia7712 commented on a change in pull request #8517: URL: https://github.com/apache/kafka/pull/8517#discussion_r412284404 ## File path: core/src/main/scala/kafka/utils/DelayedItem.scala ## @@ -21,24 +21,23 @@ import java.util.concurrent._ import

[GitHub] [kafka] chia7712 commented on a change in pull request #8517: MINOR: use monotonic clock for replica fetcher DelayedItem

2020-04-21 Thread GitBox
chia7712 commented on a change in pull request #8517: URL: https://github.com/apache/kafka/pull/8517#discussion_r412279368 ## File path: core/src/main/scala/kafka/utils/DelayedItem.scala ## @@ -21,24 +21,24 @@ import java.util.concurrent._ import

[GitHub] [kafka] avalsa commented on issue #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on issue #8221: URL: https://github.com/apache/kafka/pull/8221#issuecomment-61720 > I saw that the stream time update codepath is missing, could we add it back? I was thinking about a simpler approach here: since we could not decrement the stream-time, no matter we

[GitHub] [kafka] dajac commented on a change in pull request #8238: KAFKA-9130: KIP-518 Allow listing consumer groups per state

2020-04-21 Thread GitBox
dajac commented on a change in pull request #8238: URL: https://github.com/apache/kafka/pull/8238#discussion_r412182105 ## File path: core/src/test/scala/unit/kafka/server/KafkaApisTest.scala ## @@ -1553,6 +1554,50 @@ class KafkaApisTest {

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412178952 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/RegexSourceIntegrationTest.java ## @@ -179,6 +179,44 @@ public void

[jira] [Commented] (KAFKA-9617) Replica Fetcher can mark partition as failed when max.message.bytes is changed

2020-04-21 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088692#comment-17088692 ] Stanislav Kozlovski commented on KAFKA-9617: Feel free to assign yourself [~showuon]! >

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412178952 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/RegexSourceIntegrationTest.java ## @@ -179,6 +179,44 @@ public void

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412178952 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/RegexSourceIntegrationTest.java ## @@ -179,6 +179,44 @@ public void

[jira] [Commented] (KAFKA-9850) Move KStream#repartition operator validation during Topology build process

2020-04-21 Thread HaiyuanZhao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088647#comment-17088647 ] HaiyuanZhao commented on KAFKA-9850: [~bchen225242] I want to pick this up. Can u assign this to me?

[GitHub] [kafka] dajac commented on issue #8509: KAFKA-9839: Broker should accept control requests with newer broker epoch

2020-04-21 Thread GitBox
dajac commented on issue #8509: URL: https://github.com/apache/kafka/pull/8509#issuecomment-617137684 @apovzner Yeah, you're right. It does not increase the overall coverage. I spent quite some time in the `KafkaApis` recently and I noticed that its unit test coverage is quite low.

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412099165 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StandbyTask.java ## @@ -115,7 +115,7 @@ public void suspend() { }

[GitHub] [kafka] cadonna commented on a change in pull request #8497: KAFKA-6145: KIP-441 Build state constrained assignment from balanced one

2020-04-21 Thread GitBox
cadonna commented on a change in pull request #8497: URL: https://github.com/apache/kafka/pull/8497#discussion_r412047332 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ValidClientsByTaskLoadQueue.java ## @@ -0,0 +1,106 @@ +/* + *

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412055008 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -305,6 +297,12 @@ public void resume() {

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412049922 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamTaskTest.java ## @@ -1127,7 +1127,7 @@ public void

[GitHub] [kafka] tombentley commented on issue #8311: KAFKA-9434: automated protocol for alterReplicaLogDirs

2020-04-21 Thread GitBox
tombentley commented on issue #8311: URL: https://github.com/apache/kafka/pull/8311#issuecomment-617069226 @dajac thanks for the review, I've addressed all your comments. This is an automated message from the Apache Git

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412020510 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/PartitionGroup.java ## @@ -93,6 +96,29 @@ long partitionTimestamp(final

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412020510 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/PartitionGroup.java ## @@ -93,6 +96,29 @@ long partitionTimestamp(final

[GitHub] [kafka] avalsa commented on a change in pull request #8221: KAFKA-9561: update task input partitions after rebalance

2020-04-21 Thread GitBox
avalsa commented on a change in pull request #8221: URL: https://github.com/apache/kafka/pull/8221#discussion_r412015236 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/PartitionGroup.java ## @@ -50,7 +53,7 @@ */ public class PartitionGroup

[GitHub] [kafka] stanislavkozlovski commented on a change in pull request #8524: KAFKA-9866: Avoid election for topics where preferred leader is not in ISR

2020-04-21 Thread GitBox
stanislavkozlovski commented on a change in pull request #8524: URL: https://github.com/apache/kafka/pull/8524#discussion_r412014026 ## File path: core/src/main/scala/kafka/controller/KafkaController.scala ## @@ -1066,6 +1066,7 @@ class KafkaController(val config: KafkaConfig,

[GitHub] [kafka] dajac opened a new pull request #8525: [WIP] KAFKA-9885; Evict last members of a group when the maximum allowed is reached

2020-04-21 Thread GitBox
dajac opened a new pull request #8525: URL: https://github.com/apache/kafka/pull/8525 WIP ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[jira] [Assigned] (KAFKA-9617) Replica Fetcher can mark partition as failed when max.message.bytes is changed

2020-04-21 Thread Wang Ge (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Ge reassigned KAFKA-9617: -- Assignee: (was: Wang Ge) > Replica Fetcher can mark partition as failed when max.message.bytes is

[jira] [Assigned] (KAFKA-9617) Replica Fetcher can mark partition as failed when max.message.bytes is changed

2020-04-21 Thread Wang Ge (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Ge reassigned KAFKA-9617: -- Assignee: Wang Ge > Replica Fetcher can mark partition as failed when max.message.bytes is changed >

[GitHub] [kafka] LiamClarkeNZ commented on a change in pull request #8520: Add explicit grace period to tumbling window example

2020-04-21 Thread GitBox
LiamClarkeNZ commented on a change in pull request #8520: URL: https://github.com/apache/kafka/pull/8520#discussion_r412007274 ## File path: docs/streams/developer-guide/dsl-api.html ## @@ -3262,12 +3262,15 @@ KTable-KTable Foreign-Key import

[jira] [Commented] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to emit events after 24h

2020-04-21 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088442#comment-17088442 ] Bruno Cadonna commented on KAFKA-8924: -- [~vvcephei] I like your proposal {code:java} public static

[GitHub] [kafka] leonardge opened a new pull request #8524: Avoid starting election for topics where preferred leader is not in s…

2020-04-21 Thread GitBox
leonardge opened a new pull request #8524: URL: https://github.com/apache/kafka/pull/8524 …ync. *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.*

[jira] [Updated] (KAFKA-9781) TimestampConverter / Allow to specify a time zone when converting unix epoch to string

2020-04-21 Thread fml2 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fml2 updated KAFKA-9781: Description: TimestampConverter can convert a unix epoch value (long; number of milliseconds since Jan 01 1970

[jira] [Assigned] (KAFKA-9866) Do not attempt to elect preferred leader replicas which are outside ISR

2020-04-21 Thread Wang Ge (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Ge reassigned KAFKA-9866: -- Assignee: Wang Ge > Do not attempt to elect preferred leader replicas which are outside ISR >

[jira] [Commented] (KAFKA-9895) Truncation request on broker start up may cause OffsetOutOfRangeException

2020-04-21 Thread Boquan Tang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088354#comment-17088354 ] Boquan Tang commented on KAFKA-9895: [~ijuma] We've only recently upgraded from 2.2.1 to 2.4.0, we

[jira] [Commented] (KAFKA-9895) Truncation request on broker start up may cause OffsetOutOfRangeException

2020-04-21 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088352#comment-17088352 ] Ismael Juma commented on KAFKA-9895: Thanks for the report. Is there a reason you are using 2.4.0