[GitHub] [kafka] guozhangwang commented on pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-14 Thread GitBox
guozhangwang commented on pull request #11057: URL: https://github.com/apache/kafka/pull/11057#issuecomment-880417140 Also cc @cmccabe @hachikuji . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] guozhangwang commented on pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-14 Thread GitBox
guozhangwang commented on pull request #11057: URL: https://github.com/apache/kafka/pull/11057#issuecomment-880416855 @vvcephei @ableegoldman @showuon LMK what do you think about this approach. I have not do the due diligence of test coverage yet, will do that if people are +1 on this

[GitHub] [kafka] guozhangwang opened a new pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-14 Thread GitBox
guozhangwang opened a new pull request #11057: URL: https://github.com/apache/kafka/pull/11057 This is an idea I had for attacking this on the consumer client level: 1. When listOffset result is retrieved inside Fetcher, check if the partitions are part of the subscriptions of the

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r670134315 ## File path: metadata/src/main/java/org/apache/kafka/controller/PartitionReassignmentRevert.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r670132470 ## File path: metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java ## @@ -0,0 +1,243 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] guozhangwang commented on a change in pull request #10941: KAFKA-10847: Remove internal config for enabling the fix

2021-07-14 Thread GitBox
guozhangwang commented on a change in pull request #10941: URL: https://github.com/apache/kafka/pull/10941#discussion_r670119642 ## File path: docs/streams/upgrade-guide.html ## @@ -117,6 +117,21 @@ Streams API We removed the default implementation of

[GitHub] [kafka] guozhangwang commented on pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-14 Thread GitBox
guozhangwang commented on pull request #10953: URL: https://github.com/apache/kafka/pull/10953#issuecomment-880375143 @cadonna and everyone: I've addressed the comments and add `JoinWindows`. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] guozhangwang commented on a change in pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-14 Thread GitBox
guozhangwang commented on a change in pull request #10953: URL: https://github.com/apache/kafka/pull/10953#discussion_r670103326 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/TimeWindows.java ## @@ -146,7 +146,7 @@ public static TimeWindows of(final

[GitHub] [kafka] showuon commented on a change in pull request #11019: KAFKA-13059: refactor DeleteConsumerGroupOffsetsHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11019: URL: https://github.com/apache/kafka/pull/11019#discussion_r670111886 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/internals/DeleteConsumerGroupOffsetsHandlerTest.java ## @@ -67,48 +69,88 @@ public

[GitHub] [kafka] jolshan opened a new pull request #11056: KAFKA-13092: Perf regression in LISR requests

2021-07-14 Thread GitBox
jolshan opened a new pull request #11056: URL: https://github.com/apache/kafka/pull/11056 After noticing increased LISR times, we discovered a lot of time was spent synchronously flushing the partition metadata file. This PR changes the code so we asynchronously flush the files. We

[GitHub] [kafka] showuon commented on a change in pull request #11035: KAFKA-13072: refactor RemoveMembersFromConsumerGroupHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11035: URL: https://github.com/apache/kafka/pull/11035#discussion_r670091468 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java ## @@ -3680,6 +3685,72 @@ public void

[GitHub] [kafka] yanspirit commented on pull request #11020: KAFKA-12937; mm2 can start from the ending of a topic

2021-07-14 Thread GitBox
yanspirit commented on pull request #11020: URL: https://github.com/apache/kafka/pull/11020#issuecomment-880345058 > Not sure why I was tagged — did you mean somebody else? hello, the github recommend. there's no Assignees, I don't know how to trigger the flow -- This is

[GitHub] [kafka] showuon commented on a change in pull request #11035: KAFKA-13072: refactor RemoveMembersFromConsumerGroupHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11035: URL: https://github.com/apache/kafka/pull/11035#discussion_r670079198 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/RemoveMembersFromConsumerGroupHandler.java ## @@ -79,55 +90,82 @@ public

[jira] [Commented] (KAFKA-9366) Upgrade log4j to log4j2

2021-07-14 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380951#comment-17380951 ] Dongjin Lee commented on KAFKA-9366: [~kkonstantine] Got it. > Upgrade log4j to log4j2 >

[GitHub] [kafka] showuon commented on a change in pull request #11022: KAFKA-13063: refactor DescribeConsumerGroupsHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11022: URL: https://github.com/apache/kafka/pull/11022#discussion_r670075392 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java ## @@ -2688,8 +2688,7 @@ public void

[GitHub] [kafka] ableegoldman commented on pull request #10877: KAFKA-12925: adding presfixScan operation for missed implementations

2021-07-14 Thread GitBox
ableegoldman commented on pull request #10877: URL: https://github.com/apache/kafka/pull/10877#issuecomment-880336482 Merged to trunk and cherrypicked to 3.0 and 2.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] showuon commented on a change in pull request #11022: KAFKA-13063: refactor DescribeConsumerGroupsHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11022: URL: https://github.com/apache/kafka/pull/11022#discussion_r670073886 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/DescribeConsumerGroupsHandler.java ## @@ -151,38 +154,45 @@ public String

[GitHub] [kafka] showuon commented on a change in pull request #11022: KAFKA-13063: refactor DescribeConsumerGroupsHandler and tests

2021-07-14 Thread GitBox
showuon commented on a change in pull request #11022: URL: https://github.com/apache/kafka/pull/11022#discussion_r670073415 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/DescribeConsumerGroupsHandler.java ## @@ -109,16 +109,19 @@ public String

[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()

2021-07-14 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380939#comment-17380939 ] A. Sophie Blee-Goldman commented on KAFKA-13010: Not the exact same test, but I did

[jira] [Assigned] (KAFKA-13092) Perf regression in LISR requests

2021-07-14 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-13092: -- Assignee: Justine Olshan > Perf regression in LISR requests >

[jira] [Updated] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()

2021-07-14 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-13010: --- Attachment:

[jira] [Created] (KAFKA-13092) Perf regression in LISR requests

2021-07-14 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-13092: -- Summary: Perf regression in LISR requests Key: KAFKA-13092 URL: https://issues.apache.org/jira/browse/KAFKA-13092 Project: Kafka Issue Type: Bug

[GitHub] [kafka] showuon closed pull request #10973: KAFKA-13033: COORDINATOR_NOT_AVAILABLE should be unmapped

2021-07-14 Thread GitBox
showuon closed pull request #10973: URL: https://github.com/apache/kafka/pull/10973 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Resolved] (KAFKA-12970) Make tiered storage related schemas adopt flexible versions feature.

2021-07-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-12970. Resolution: Fixed This is already addressed as mentioned in the comment: 

[jira] [Commented] (KAFKA-13089) Revisit the usage of BufferSuppliers in Kraft

2021-07-14 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380914#comment-17380914 ] Jose Armando Garcia Sancio commented on KAFKA-13089: {quote}Can each listener hold a

[jira] [Created] (KAFKA-13091) Increment HW after shrinking ISR through AlterIsr

2021-07-14 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-13091: --- Summary: Increment HW after shrinking ISR through AlterIsr Key: KAFKA-13091 URL: https://issues.apache.org/jira/browse/KAFKA-13091 Project: Kafka

[GitHub] [kafka] mjsax commented on pull request #11055: HOTFIX: Init stream-stream left/outer join emit interval correctly

2021-07-14 Thread GitBox
mjsax commented on pull request #11055: URL: https://github.com/apache/kafka/pull/11055#issuecomment-880286834 @kkonstantine I think we should try to get this into 3.0 release. Thoughts? Call for review @guozhangwang @spena -- This is an automated message from the Apache Git

[GitHub] [kafka] mjsax commented on a change in pull request #11055: HOTFIX: Init stream-stream left/outer join emit interval correctly

2021-07-14 Thread GitBox
mjsax commented on a change in pull request #11055: URL: https://github.com/apache/kafka/pull/11055#discussion_r670030416 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -103,7 +103,6 @@ public void init(final

[GitHub] [kafka] mjsax opened a new pull request #11055: HOTFIX: Init stream-stream left/outer join emit interval correctly

2021-07-14 Thread GitBox
mjsax opened a new pull request #11055: URL: https://github.com/apache/kafka/pull/11055 Follow up to KAFKA-10847 (https://github.com/apache/kafka/pull/10917). The above fix intended to reduce the emit frequency to save the creation cost of RocksDB iterators. However, we incorrectly

[jira] [Commented] (KAFKA-13089) Revisit the usage of BufferSuppliers in Kraft

2021-07-14 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380892#comment-17380892 ] Ismael Juma commented on KAFKA-13089: - The BufferSupplier implementations we have are not

[GitHub] [kafka] junrao commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
junrao commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669998610 ## File path: metadata/src/main/java/org/apache/kafka/controller/PartitionReassignmentRevert.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] ableegoldman merged pull request #10877: KAFKA-12925: adding presfixScan operation for missed implementations

2021-07-14 Thread GitBox
ableegoldman merged pull request #10877: URL: https://github.com/apache/kafka/pull/10877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] ableegoldman commented on a change in pull request #10877: KAFKA-12925: adding presfixScan operation for missed implementations

2021-07-14 Thread GitBox
ableegoldman commented on a change in pull request #10877: URL: https://github.com/apache/kafka/pull/10877#discussion_r670001461 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java ## @@ -845,6 +845,64 @@ public void

[GitHub] [kafka] cmccabe commented on a change in pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #11054: URL: https://github.com/apache/kafka/pull/11054#discussion_r670005690 ## File path: raft/src/test/java/org/apache/kafka/raft/MockLog.java ## @@ -489,6 +489,11 @@ public void initializeLeaderEpoch(int epoch) {

[GitHub] [kafka] cmccabe commented on a change in pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #11054: URL: https://github.com/apache/kafka/pull/11054#discussion_r670005192 ## File path: raft/src/main/java/org/apache/kafka/snapshot/SnapshotReader.java ## @@ -84,7 +84,7 @@ public int lastContainedLogEpoch() { */

[GitHub] [kafka] cmccabe commented on a change in pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #11054: URL: https://github.com/apache/kafka/pull/11054#discussion_r670004898 ## File path: raft/src/main/java/org/apache/kafka/raft/ReplicatedLog.java ## @@ -280,11 +280,19 @@ default long truncateToEndOffset(OffsetAndEpoch

[jira] [Resolved] (KAFKA-13080) Fetch snapshot request are not directed to kraft in controller

2021-07-14 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-13080. Resolution: Fixed > Fetch snapshot request are not directed to

[jira] [Resolved] (KAFKA-13078) Closing FileRawSnapshotWriter too early

2021-07-14 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-13078. Resolution: Fixed > Closing FileRawSnapshotWriter too early >

[GitHub] [kafka] ableegoldman commented on pull request #10877: KAFKA-12925: adding presfixScan operation for missed implementations

2021-07-14 Thread GitBox
ableegoldman commented on pull request #10877: URL: https://github.com/apache/kafka/pull/10877#issuecomment-880255698 > I have added a test for IQ. Some reason, I wasn't able to run the test on my local as I was getting a build failure due to scala test classes. I would watch out for the

[GitHub] [kafka] jsancio opened a new pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-14 Thread GitBox
jsancio opened a new pull request #11054: URL: https://github.com/apache/kafka/pull/11054 Check and verify generated snapshots for the controllers and the brokers. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify

[jira] [Created] (KAFKA-13090) Improve cluster snapshot integration test

2021-07-14 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-13090: -- Summary: Improve cluster snapshot integration test Key: KAFKA-13090 URL: https://issues.apache.org/jira/browse/KAFKA-13090 Project: Kafka

[jira] [Resolved] (KAFKA-13073) Simulation test fails due to inconsistency in MockLog's implementation

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-13073. Resolution: Fixed > Simulation test fails due to inconsistency in

[jira] [Comment Edited] (KAFKA-7632) Support Compression Level

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380866#comment-17380866 ] Konstantine Karantasis edited comment on KAFKA-7632 at 7/14/21, 9:50 PM:

[jira] [Commented] (KAFKA-9366) Upgrade log4j to log4j2

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380874#comment-17380874 ] Konstantine Karantasis commented on KAFKA-9366: --- This feature was not approved on time for

[jira] [Updated] (KAFKA-9366) Upgrade log4j to log4j2

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-9366: -- Fix Version/s: (was: 3.0.0) 3.1.0 > Upgrade log4j to

[GitHub] [kafka] niket-goel opened a new pull request #11053: [KAFKA-13015] Ducktape System Tests to for Metadata Snapshots

2021-07-14 Thread GitBox
niket-goel opened a new pull request #11053: URL: https://github.com/apache/kafka/pull/11053 [WIP] This PR implements system tests in ducktape to test the ability of brokers and controllers to generate and consume snapshots and catch up with the metadata log. ### Committer

[jira] [Commented] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380872#comment-17380872 ] Konstantine Karantasis commented on KAFKA-12495: This issue corresponds to a corner case

[jira] [Updated] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-12495: --- Fix Version/s: 3.1.0 > Unbalanced connectors/tasks distribution will happen

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380871#comment-17380871 ] Konstantine Karantasis commented on KAFKA-12283: As mentioned above, this failure

[jira] [Updated] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-12283: --- Fix Version/s: (was: 3.0.0) 3.1.0 > Flaky Test >

[jira] [Commented] (KAFKA-7632) Support Compression Level

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380866#comment-17380866 ] Konstantine Karantasis commented on KAFKA-7632: --- This feature was not approved in time for

[jira] [Updated] (KAFKA-7632) Support Compression Level

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-7632: -- Fix Version/s: (was: 3.0.0) 3.1.0 > Support Compression

[jira] [Created] (KAFKA-13089) Revisit the usage of BufferSuppliers in Kraft

2021-07-14 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-13089: -- Summary: Revisit the usage of BufferSuppliers in Kraft Key: KAFKA-13089 URL: https://issues.apache.org/jira/browse/KAFKA-13089 Project:

[GitHub] [kafka] vvcephei commented on a change in pull request #10994: KAFKA-8410: Update the docs to reference the new PAPI

2021-07-14 Thread GitBox
vvcephei commented on a change in pull request #10994: URL: https://github.com/apache/kafka/pull/10994#discussion_r669947367 ## File path: docs/streams/developer-guide/processor-api.html ## @@ -86,12 +86,48 @@ Overviewclose() method. Note that Kafka Streams may re-use a

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669944074 ## File path: core/src/test/scala/integration/kafka/server/RaftClusterTest.scala ## @@ -422,4 +351,75 @@ class RaftClusterTest { listenerName =

[GitHub] [kafka] xvrl opened a new pull request #11052: Use ByteBuffers for LZ4 OutputStream

2021-07-14 Thread GitBox
xvrl opened a new pull request #11052: URL: https://github.com/apache/kafka/pull/11052 Our current LZ4 OutputStream implementation allocates compression buffers internally and relies on intermediate byte arrays for input and output buffers. With this change we now use ByteBuffers

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669926069 ## File path: metadata/src/test/java/org/apache/kafka/metadata/PartitionRegistrationTest.java ## @@ -105,4 +105,42 @@ public void

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669925103 ## File path: metadata/src/test/java/org/apache/kafka/controller/PartitionChangeBuilderTest.java ## @@ -0,0 +1,246 @@ +/* + * Licensed to the Apache

[jira] [Commented] (KAFKA-13010) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()

2021-07-14 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380838#comment-17380838 ] A. Sophie Blee-Goldman commented on KAFKA-13010: [~wcarl...@confluent.io] I'm guessing

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669911056 ## File path: metadata/src/main/java/org/apache/kafka/image/TopicDelta.java ## @@ -103,7 +103,8 @@ public TopicImage apply() { for (Entry entry

[GitHub] [kafka] dajac commented on a change in pull request #11035: KAFKA-13072: refactor RemoveMembersFromConsumerGroupHandler and tests

2021-07-14 Thread GitBox
dajac commented on a change in pull request #11035: URL: https://github.com/apache/kafka/pull/11035#discussion_r669905082 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/RemoveMembersFromConsumerGroupHandler.java ## @@ -79,55 +90,82 @@ public

[GitHub] [kafka] jsancio closed pull request #6777: MINOR: Run the Java and Scala documentation in Jenkins

2021-07-14 Thread GitBox
jsancio closed pull request #6777: URL: https://github.com/apache/kafka/pull/6777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mjsax commented on pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-14 Thread GitBox
mjsax commented on pull request #10953: URL: https://github.com/apache/kafka/pull/10953#issuecomment-880159699 > but when users upgrade the Streams apps to 3.0 we guarantee retention time T+X I don't agree to this statement. It's a bug in the implementation (that this PR fixes)

[GitHub] [kafka] jolshan commented on a change in pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-14 Thread GitBox
jolshan commented on a change in pull request #10952: URL: https://github.com/apache/kafka/pull/10952#discussion_r669899334 ## File path: clients/src/main/java/org/apache/kafka/clients/MetadataCache.java ## @@ -130,13 +150,30 @@ MetadataCache mergeWith(String newClusterId,

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669896829 ## File path: metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java ## @@ -180,6 +187,44 @@ public LeaderAndIsrPartitionState

[GitHub] [kafka] dajac commented on a change in pull request #11022: KAFKA-13063: refactor DescribeConsumerGroupsHandler and tests

2021-07-14 Thread GitBox
dajac commented on a change in pull request #11022: URL: https://github.com/apache/kafka/pull/11022#discussion_r669797588 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/DescribeConsumerGroupsHandler.java ## @@ -109,16 +109,19 @@ public String

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669891898 ## File path: metadata/src/main/java/org/apache/kafka/controller/PartitionReassignmentRevert.java ## @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cmccabe commented on a change in pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #10753: URL: https://github.com/apache/kafka/pull/10753#discussion_r669215608 ## File path: metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java ## @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-14 Thread GitBox
hachikuji commented on a change in pull request #10952: URL: https://github.com/apache/kafka/pull/10952#discussion_r669881047 ## File path: clients/src/main/java/org/apache/kafka/clients/MetadataCache.java ## @@ -130,13 +150,30 @@ MetadataCache mergeWith(String newClusterId,

[GitHub] [kafka] lkokhreidze commented on pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-07-14 Thread GitBox
lkokhreidze commented on pull request #10851: URL: https://github.com/apache/kafka/pull/10851#issuecomment-880127165 Thanks @cadonna for the feedback. I've replied/addressed all of your comments. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] lkokhreidze commented on a change in pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-07-14 Thread GitBox
lkokhreidze commented on a change in pull request #10851: URL: https://github.com/apache/kafka/pull/10851#discussion_r669865915 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/HighAvailabilityTaskAssignor.java ## @@ -51,12 +57,12 @@

[jira] [Updated] (KAFKA-13033) coordinator not available error should cause add into unmap list to do a new lookup

2021-07-14 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-13033: Priority: Blocker (was: Major) > coordinator not available error should cause add into unmap

[GitHub] [kafka] lkokhreidze commented on a change in pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-07-14 Thread GitBox
lkokhreidze commented on a change in pull request #10851: URL: https://github.com/apache/kafka/pull/10851#discussion_r669853519 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/StandbyTaskAssignor.java ## @@ -0,0 +1,39 @@ +/* + *

[GitHub] [kafka] lkokhreidze commented on a change in pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-07-14 Thread GitBox
lkokhreidze commented on a change in pull request #10851: URL: https://github.com/apache/kafka/pull/10851#discussion_r669851397 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/HighAvailabilityTaskAssignor.java ## @@ -51,12 +57,12 @@

[GitHub] [kafka] lkokhreidze commented on a change in pull request #10851: KAFKA-6718 / Rack aware standby task assignor

2021-07-14 Thread GitBox
lkokhreidze commented on a change in pull request #10851: URL: https://github.com/apache/kafka/pull/10851#discussion_r669849371 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ClientTagAwareStandbyTaskAssignor.java ## @@ -0,0 +1,212

[jira] [Updated] (KAFKA-12308) ConfigDef.parseType deadlock

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-12308: --- Fix Version/s: 3.0.0 > ConfigDef.parseType deadlock >

[jira] [Resolved] (KAFKA-12308) ConfigDef.parseType deadlock

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-12308. Resolution: Fixed > ConfigDef.parseType deadlock >

[jira] [Commented] (KAFKA-7421) Deadlock in Kafka Connect during class loading

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380773#comment-17380773 ] Konstantine Karantasis commented on KAFKA-7421: --- The fix has now been merged. Let's keep

[jira] [Resolved] (KAFKA-7421) Deadlock in Kafka Connect during class loading

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-7421. --- Resolution: Fixed > Deadlock in Kafka Connect during class loading >

[jira] [Updated] (KAFKA-7421) Deadlock in Kafka Connect during class loading

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-7421: -- Fix Version/s: 3.0.0 > Deadlock in Kafka Connect during class loading >

[jira] [Commented] (KAFKA-12308) ConfigDef.parseType deadlock

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380772#comment-17380772 ] Konstantine Karantasis commented on KAFKA-12308: Adding the comment that I added in the

[GitHub] [kafka] cmccabe merged pull request #11048: KAFKA-13083: In KRaft mode, fix issue with ISR in manual partition assignments

2021-07-14 Thread GitBox
cmccabe merged pull request #11048: URL: https://github.com/apache/kafka/pull/11048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (KAFKA-7421) Deadlock in Kafka Connect during class loading

2021-07-14 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-7421: -- Summary: Deadlock in Kafka Connect during class loading (was: Deadlock in

[GitHub] [kafka] kkonstantine merged pull request #8259: KAFKA-7421: Ensure Connect's PluginClassLoader is truly parallel capable and resolve deadlock occurrences

2021-07-14 Thread GitBox
kkonstantine merged pull request #8259: URL: https://github.com/apache/kafka/pull/8259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] kkonstantine commented on pull request #8259: KAFKA-7421: Ensure Connect's PluginClassLoader is truly parallel capable and resolve deadlock occurrences

2021-07-14 Thread GitBox
kkonstantine commented on pull request #8259: URL: https://github.com/apache/kafka/pull/8259#issuecomment-880082421 The test failures were not relevant. Merging to `trunk` and `3.0` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] jolshan commented on a change in pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-14 Thread GitBox
jolshan commented on a change in pull request #10952: URL: https://github.com/apache/kafka/pull/10952#discussion_r669815073 ## File path: clients/src/main/java/org/apache/kafka/clients/Metadata.java ## @@ -391,10 +393,15 @@ else if (metadata.error() ==

[GitHub] [kafka] hachikuji commented on a change in pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-14 Thread GitBox
hachikuji commented on a change in pull request #10952: URL: https://github.com/apache/kafka/pull/10952#discussion_r669812349 ## File path: clients/src/main/java/org/apache/kafka/clients/Metadata.java ## @@ -377,6 +397,12 @@ else if (metadata.error() ==

[GitHub] [kafka] hachikuji commented on a change in pull request #10952: KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name

2021-07-14 Thread GitBox
hachikuji commented on a change in pull request #10952: URL: https://github.com/apache/kafka/pull/10952#discussion_r669810537 ## File path: clients/src/main/java/org/apache/kafka/clients/Metadata.java ## @@ -391,10 +393,15 @@ else if (metadata.error() ==

[GitHub] [kafka] kkonstantine commented on a change in pull request #8259: KAFKA-7421: Ensure Connect's PluginClassLoader is truly parallel capable and resolve deadlock occurrences

2021-07-14 Thread GitBox
kkonstantine commented on a change in pull request #8259: URL: https://github.com/apache/kafka/pull/8259#discussion_r669806313 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/DelegatingClassLoader.java ## @@ -82,11 +93,19 @@

[GitHub] [kafka] cmccabe commented on pull request #11048: KAFKA-13083: In KRaft mode, fix issue with ISR in manual partition assignments

2021-07-14 Thread GitBox
cmccabe commented on pull request #11048: URL: https://github.com/apache/kafka/pull/11048#issuecomment-880053828 I updated this and fixed a case in createPartitions where the same issue existed (added a test for that as well) -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] cmccabe commented on a change in pull request #11048: KAFKA-13083: In KRaft mode, fix issue with ISR in manual partition assignments

2021-07-14 Thread GitBox
cmccabe commented on a change in pull request #11048: URL: https://github.com/apache/kafka/pull/11048#discussion_r669790479 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -377,8 +377,20 @@ private ApiError

[GitHub] [kafka] kowshik commented on pull request #10280: KAFKA-12554: Refactor Log layer

2021-07-14 Thread GitBox
kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-880044495 @ijuma Discussed with @satishd. We are not planning to include this in 3.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Updated] (KAFKA-13088) Replace EasyMock with Mockito for ForwardingDisabledProcessorContextTest

2021-07-14 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-13088: -- Component/s: streams > Replace EasyMock with Mockito for

[jira] [Commented] (KAFKA-3539) KafkaProducer.send() may block even though it returns the Future

2021-07-14 Thread Ewen Cheslack-Postava (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380730#comment-17380730 ] Ewen Cheslack-Postava commented on KAFKA-3539: -- [~moses.nakamura] I spent a bunch of time on

[GitHub] [kafka] cmccabe commented on pull request #11048: KAFKA-13083: In KRaft mode, fix issue with ISR in manual partition assignments

2021-07-14 Thread GitBox
cmccabe commented on pull request #11048: URL: https://github.com/apache/kafka/pull/11048#issuecomment-880035001 > I'm assuming we don't need a similar check for the automatic assignment path since the replica placer will only select unfenced replicas. Right. -- This is an

[GitHub] [kafka] tang7526 opened a new pull request #11051: KAFKA-13088: Replace EasyMock with Mockito for ForwardingDisabledProcessorContextTest

2021-07-14 Thread GitBox
tang7526 opened a new pull request #11051: URL: https://github.com/apache/kafka/pull/11051 https://issues.apache.org/jira/browse/KAFKA-13088 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[jira] [Created] (KAFKA-13088) Replace EasyMock with Mockito for ForwardingDisabledProcessorContextTest

2021-07-14 Thread Chun-Hao Tang (Jira)
Chun-Hao Tang created KAFKA-13088: - Summary: Replace EasyMock with Mockito for ForwardingDisabledProcessorContextTest Key: KAFKA-13088 URL: https://issues.apache.org/jira/browse/KAFKA-13088 Project:

[GitHub] [kafka] ijuma commented on pull request #10280: KAFKA-12554: Refactor Log layer

2021-07-14 Thread GitBox
ijuma commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-879991378 What is the reason for including a refactoring in 3.0 after the feature freeze? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] mumrah commented on a change in pull request #11048: KAFKA-13083: In KRaft mode, fix issue with ISR in manual partition assignments

2021-07-14 Thread GitBox
mumrah commented on a change in pull request #11048: URL: https://github.com/apache/kafka/pull/11048#discussion_r669693620 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -377,8 +377,20 @@ private ApiError

[jira] [Commented] (KAFKA-13052) Replace uses of SerDes in the docs with Serdes

2021-07-14 Thread James Galasyn (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380643#comment-17380643 ] James Galasyn commented on KAFKA-13052: --- [~lct45], fixed in

  1   2   >