[GitHub] [kafka] chia7712 commented on pull request #9162: MINOR: refactor Log to get rid of "return" in nested anonymous function

2020-09-09 Thread GitBox
chia7712 commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-689963179 @ijuma I have attached test result and ```NonLocalReturnException``` is gone. Please take a look. This is an

[GitHub] [kafka] chia7712 commented on pull request #9278: MINOR: remove DelayedOperations.checkAndCompleteFetch

2020-09-09 Thread GitBox
chia7712 commented on pull request #9278: URL: https://github.com/apache/kafka/pull/9278#issuecomment-689937879 @junrao FYI This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] chia7712 commented on a change in pull request #9278: MINOR: remove DelayedOperations.checkAndCompleteFetch

2020-09-09 Thread GitBox
chia7712 commented on a change in pull request #9278: URL: https://github.com/apache/kafka/pull/9278#discussion_r486027311 ## File path: core/src/test/scala/unit/kafka/cluster/PartitionTest.scala ## @@ -922,102 +918,6 @@ class PartitionTest extends AbstractPartitionTest {

[GitHub] [kafka] chia7712 opened a new pull request #9278: MINOR: remove DelayedOperations.checkAndCompleteFetch

2020-09-09 Thread GitBox
chia7712 opened a new pull request #9278: URL: https://github.com/apache/kafka/pull/9278 inspired by https://github.com/apache/kafka/pull/8657#discussion_r484092945 ``` With this change, DelayedOperations.checkAndCompleteFetch() is only used in tests. I am wondering if it can be

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-09-09 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-689921735 > Thanks a lot for staying on this tricky issue and finding a simpler solution! thanks for all suggestions. I benefit a lot from it.

[jira] [Commented] (KAFKA-10170) ReplicaManager should be responsible for checking delayed operations after appending to the log.

2020-09-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193293#comment-17193293 ] Chia-Ping Tsai commented on KAFKA-10170: This is addressed by

[jira] [Resolved] (KAFKA-10170) ReplicaManager should be responsible for checking delayed operations after appending to the log.

2020-09-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10170. Resolution: Duplicate > ReplicaManager should be responsible for checking delayed

[jira] [Commented] (KAFKA-10467) kafka-topic --describe fails for topic created by "produce"

2020-09-09 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193292#comment-17193292 ] Luke Chen commented on KAFKA-10467: --- > Any idea what causes this? -> Sorry, I cannot reproduce it in

[GitHub] [kafka] cmccabe commented on pull request #9277: MINOR: Fix JSON generation of nested structs with non-matching type/name

2020-09-09 Thread GitBox
cmccabe commented on pull request #9277: URL: https://github.com/apache/kafka/pull/9277#issuecomment-689896549 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] cmccabe merged pull request #9269: MINOR: add ImplicitLinkedHashCollection#moveToEnd

2020-09-09 Thread GitBox
cmccabe merged pull request #9269: URL: https://github.com/apache/kafka/pull/9269 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji opened a new pull request #9277: MINOR: Fix JSON generation of nested structs with non-matching type/name

2020-09-09 Thread GitBox
hachikuji opened a new pull request #9277: URL: https://github.com/apache/kafka/pull/9277 The schema specification allows a struct type name to differ from the field name. This works with the generated `Message` classes, but not with the generated JSON converter. The problem seems to be

[GitHub] [kafka] junrao commented on a change in pull request #9178: KAFKA-8362: fix the old checkpoint won't be removed after alter log dir

2020-09-09 Thread GitBox
junrao commented on a change in pull request #9178: URL: https://github.com/apache/kafka/pull/9178#discussion_r485958438 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1698,8 +1698,12 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] rondagostino commented on pull request #9274: KAFKA-10131: Remove use_zk_connection flag

2020-09-09 Thread GitBox
rondagostino commented on pull request #9274: URL: https://github.com/apache/kafka/pull/9274#issuecomment-689862257 @cmccabe Thanks for the review. I put the logic back into `kafka_acls.py`. Take a look and let me know if this is what you were thinking.

[GitHub] [kafka] wushujames commented on pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics

2020-09-09 Thread GitBox
wushujames commented on pull request #9276: URL: https://github.com/apache/kafka/pull/9276#issuecomment-689856879 @guozhangwang Can I get a review? This is an automated message from the Apache Git Service. To respond to

[jira] [Assigned] (KAFKA-10473) Website is missing docs on JMX metrics for partition size-on-disk (kafka.log:type=Log,name=*)

2020-09-09 Thread James Cheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng reassigned KAFKA-10473: --- Assignee: James Cheng > Website is missing docs on JMX metrics for partition size-on-disk

[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics

2020-09-09 Thread GitBox
wushujames commented on a change in pull request #9276: URL: https://github.com/apache/kafka/pull/9276#discussion_r485956062 ## File path: docs/ops.html ## @@ -1129,6 +1128,26 @@ Security Considerations for Remote Mon

[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics

2020-09-09 Thread GitBox
wushujames commented on a change in pull request #9276: URL: https://github.com/apache/kafka/pull/9276#discussion_r485955567 ## File path: docs/ops.html ## @@ -1129,6 +1128,26 @@ Security Considerations for Remote Mon

[GitHub] [kafka] wushujames commented on a change in pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics

2020-09-09 Thread GitBox
wushujames commented on a change in pull request #9276: URL: https://github.com/apache/kafka/pull/9276#discussion_r485955266 ## File path: docs/ops.html ## @@ -1129,6 +1128,26 @@ Security Considerations for Remote Mon

[GitHub] [kafka] wushujames opened a new pull request #9276: KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics

2020-09-09 Thread GitBox
wushujames opened a new pull request #9276: URL: https://github.com/apache/kafka/pull/9276 Add docs on the following JMX metrics kafka.log,type=Log,name=Size kafka.log,type=Log,name=NumLogSegments kafka.log,type=Log,name=LogStartOffset kafka.log,type=Log,name=LogEndOffset

[jira] [Updated] (KAFKA-10473) Website is missing docs on JMX metrics for partition size-on-disk (kafka.log:type=Log,name=*)

2020-09-09 Thread James Cheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng updated KAFKA-10473: Summary: Website is missing docs on JMX metrics for partition size-on-disk

[jira] [Created] (KAFKA-10473) Website is missing docs on JMX metrics for partition size-on-disk

2020-09-09 Thread James Cheng (Jira)
James Cheng created KAFKA-10473: --- Summary: Website is missing docs on JMX metrics for partition size-on-disk Key: KAFKA-10473 URL: https://issues.apache.org/jira/browse/KAFKA-10473 Project: Kafka

[GitHub] [kafka] guozhangwang commented on a change in pull request #9247: KAFKA-10362: When resuming Streams active task with EOS, the checkpoint file is deleted

2020-09-09 Thread GitBox
guozhangwang commented on a change in pull request #9247: URL: https://github.com/apache/kafka/pull/9247#discussion_r485941828 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorStateManager.java ## @@ -662,4 +662,10 @@ public

[GitHub] [kafka] guozhangwang commented on pull request #9247: KAFKA-10362: When resuming Streams active task with EOS, the checkpoint file is deleted

2020-09-09 Thread GitBox
guozhangwang commented on pull request #9247: URL: https://github.com/apache/kafka/pull/9247#issuecomment-689840075 test this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] junrao commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-09-09 Thread GitBox
junrao commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-689838971 @chia7712 : Thanks a lot for staying on this tricky issue and finding a simpler solution! This is an automated

[jira] [Assigned] (KAFKA-8334) Occasional OffsetCommit Timeout

2020-09-09 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reassigned KAFKA-8334: -- Fix Version/s: 2.7.0 Assignee: Chia-Ping Tsai Resolution: Fixed Merged the PR to

[GitHub] [kafka] junrao merged pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-09-09 Thread GitBox
junrao merged pull request #8657: URL: https://github.com/apache/kafka/pull/8657 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji opened a new pull request #9275: KAFKA-10435; Fetch protocol changes for KIP-595

2020-09-09 Thread GitBox
hachikuji opened a new pull request #9275: URL: https://github.com/apache/kafka/pull/9275 This patch bumps the `Fetch` protocol as specified by KIP-595: https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum. The main differences are the

[GitHub] [kafka] ableegoldman commented on a change in pull request #9239: Adding reverse iterator usage for sliding windows processing (extending KIP-450)

2020-09-09 Thread GitBox
ableegoldman commented on a change in pull request #9239: URL: https://github.com/apache/kafka/pull/9239#discussion_r485927114 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregate.java ## @@ -150,23 +152,46 @@ public

[GitHub] [kafka] guozhangwang commented on pull request #9237: KAFKA-10454 / Update copartitionSourceGroups when optimization algorithm is triggered

2020-09-09 Thread GitBox
guozhangwang commented on pull request #9237: URL: https://github.com/apache/kafka/pull/9237#issuecomment-689825230 test this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] cmccabe commented on pull request #9274: KAFKA-10131: Remove use_zk_connection flag

2020-09-09 Thread GitBox
cmccabe commented on pull request #9274: URL: https://github.com/apache/kafka/pull/9274#issuecomment-689824902 Thanks, @rondagostino . Is there any way to avoid moving the acls stuff out of `kafka_acls.py`? It seems nice to have some separation rather than having everything in

[jira] [Commented] (KAFKA-10258) Get rid of use_zk_connection flag in kafka.py public methods

2020-09-09 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193178#comment-17193178 ] Ron Dagostino commented on KAFKA-10258: --- Fixes included in PR for

[jira] [Commented] (KAFKA-10213) Prefer --bootstrap-server in ducktape tests for Kafka clients

2020-09-09 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193175#comment-17193175 ] Ron Dagostino commented on KAFKA-10213: --- Fixes included in PR for

[GitHub] [kafka] rondagostino opened a new pull request #9274: KAFKA-10131: Remove use_zk_connection flag

2020-09-09 Thread GitBox
rondagostino opened a new pull request #9274: URL: https://github.com/apache/kafka/pull/9274 Eliminate "use_zk_connection" and similarly-named flags in public system test methods. System tests should always use admin client when the broker supports it. We do need to bootstrap ACLs prior

[GitHub] [kafka] lkokhreidze commented on pull request #9237: KAFKA-10454 / Update copartitionSourceGroups when optimization algorithm is triggered

2020-09-09 Thread GitBox
lkokhreidze commented on pull request #9237: URL: https://github.com/apache/kafka/pull/9237#issuecomment-689804031 Ran `./gradlew :streams:clean :streams:test` locally, everything passed. This is an automated message from

[GitHub] [kafka] lkokhreidze commented on a change in pull request #9237: KAFKA-10454 / Update copartitionSourceGroups when optimization algorithm is triggered

2020-09-09 Thread GitBox
lkokhreidze commented on a change in pull request #9237: URL: https://github.com/apache/kafka/pull/9237#discussion_r485885898 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -632,8 +632,18 @@ public final

[GitHub] [kafka] lkokhreidze commented on a change in pull request #9237: KAFKA-10454 / Update copartitionSourceGroups when optimization algorithm is triggered

2020-09-09 Thread GitBox
lkokhreidze commented on a change in pull request #9237: URL: https://github.com/apache/kafka/pull/9237#discussion_r485885898 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -632,8 +632,18 @@ public final

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9273: init changes for shutdownRequest

2020-09-09 Thread GitBox
wcarlson5 commented on a change in pull request #9273: URL: https://github.com/apache/kafka/pull/9273#discussion_r485857758 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -1436,6 +1447,16 @@ public void

[jira] [Commented] (KAFKA-10454) Kafka Streams Stuck in infinite REBALANCING loop when stream <> table join partitions don't match

2020-09-09 Thread Levani Kokhreidze (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193155#comment-17193155 ] Levani Kokhreidze commented on KAFKA-10454: --- Hey [~vvcephei] thanks for the feedback. Finally

[GitHub] [kafka] lkokhreidze commented on pull request #9237: KAFKA-10454 / integration test

2020-09-09 Thread GitBox
lkokhreidze commented on pull request #9237: URL: https://github.com/apache/kafka/pull/9237#issuecomment-689788570 call for review @mjsax @vvcephei @guozhangwang @bbejeck This is an automated message from the Apache Git

[GitHub] [kafka] lkokhreidze commented on a change in pull request #9237: KAFKA-10454 / integration test

2020-09-09 Thread GitBox
lkokhreidze commented on a change in pull request #9237: URL: https://github.com/apache/kafka/pull/9237#discussion_r485886423 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamTableJoinTopologyOptimizationIntegrationTest.java ## @@ -0,0 +1,242 @@

[GitHub] [kafka] lkokhreidze commented on a change in pull request #9237: KAFKA-10454 / integration test

2020-09-09 Thread GitBox
lkokhreidze commented on a change in pull request #9237: URL: https://github.com/apache/kafka/pull/9237#discussion_r485885898 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -632,8 +632,18 @@ public final

[GitHub] [kafka] mjsax commented on a change in pull request #9156: KAFKA-10077: Filter downstream of state-store results in spurious tombstones

2020-09-09 Thread GitBox
mjsax commented on a change in pull request #9156: URL: https://github.com/apache/kafka/pull/9156#discussion_r485846736 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/AuthorizableRequestContext.java ## @@ -17,11 +17,12 @@ package

[GitHub] [kafka] wcarlson5 opened a new pull request #9273: init changes for shutdownRequest

2020-09-09 Thread GitBox
wcarlson5 opened a new pull request #9273: URL: https://github.com/apache/kafka/pull/9273 Adding an option to shutdown an entire streams application for a Streams thread ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify

[GitHub] [kafka] jonhkr commented on pull request #9256: Fix some Gradle deprecation warnings

2020-09-09 Thread GitBox
jonhkr commented on pull request #9256: URL: https://github.com/apache/kafka/pull/9256#issuecomment-689757461 @ijuma Could you take a look? This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] mjsax merged pull request #9157: Update for KIP-450 to handle early records

2020-09-09 Thread GitBox
mjsax merged pull request #9157: URL: https://github.com/apache/kafka/pull/9157 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-10467) kafka-topic --describe fails for topic created by "produce"

2020-09-09 Thread Swayam Raina (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17193116#comment-17193116 ] Swayam Raina commented on KAFKA-10467: -- Any idea what causes this? Also, can anyone provide an

[GitHub] [kafka] vvcephei commented on pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
vvcephei commented on pull request #9267: URL: https://github.com/apache/kafka/pull/9267#issuecomment-689715414 Thanks, @guozhangwang ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] apovzner commented on pull request #9249: KAFKA-10458: Add update quota for TokenBucket registered with Sensor

2020-09-09 Thread GitBox
apovzner commented on pull request #9249: URL: https://github.com/apache/kafka/pull/9249#issuecomment-689715022 @dajac I like your approach better. I will close this PR. This is an automated message from the Apache Git

[GitHub] [kafka] apovzner closed pull request #9249: KAFKA-10458: Add update quota for TokenBucket registered with Sensor

2020-09-09 Thread GitBox
apovzner closed pull request #9249: URL: https://github.com/apache/kafka/pull/9249 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei merged pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei merged pull request #9221: URL: https://github.com/apache/kafka/pull/9221 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei commented on pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei commented on pull request #9221: URL: https://github.com/apache/kafka/pull/9221#issuecomment-689703973 Thanks, @bbejeck ! Just to be sure, I'm merging trunk into this branch and running all the tests before I squash-and-merge.

[GitHub] [kafka] vvcephei commented on pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
vvcephei commented on pull request #9267: URL: https://github.com/apache/kafka/pull/9267#issuecomment-689702908 Thanks for the review, @wcarlson5 and @guozhangwang ! I've re-worded the logs to avoid internal class names and added the state before calling restore. Aside from those

[GitHub] [kafka] vvcephei commented on a change in pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9267: URL: https://github.com/apache/kafka/pull/9267#discussion_r485771434 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -706,13 +662,17 @@ void runOnce() {

[GitHub] [kafka] vvcephei commented on a change in pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9267: URL: https://github.com/apache/kafka/pull/9267#discussion_r485767592 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -752,6 +712,77 @@ void runOnce() {

[GitHub] [kafka] guozhangwang commented on a change in pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
guozhangwang commented on a change in pull request #9267: URL: https://github.com/apache/kafka/pull/9267#discussion_r485763899 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -689,6 +644,7 @@ void runOnce() {

[GitHub] [kafka] chia7712 commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-09-09 Thread GitBox
chia7712 commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r485761001 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/record/RecordBatchIterationBenchmark.java ## @@ -142,15 +169,15 @@ private ByteBuffer

[GitHub] [kafka] vvcephei commented on a change in pull request #9267: MINOR: Add debug logs for StreamThread

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9267: URL: https://github.com/apache/kafka/pull/9267#discussion_r485759565 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -689,6 +644,7 @@ void runOnce() {

[GitHub] [kafka] bbejeck commented on a change in pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
bbejeck commented on a change in pull request #9221: URL: https://github.com/apache/kafka/pull/9221#discussion_r485754938 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StatefulProcessorNode.java ## @@ -99,5 +99,14 @@ public void

[GitHub] [kafka] vvcephei commented on pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei commented on pull request #9221: URL: https://github.com/apache/kafka/pull/9221#issuecomment-689669494 Thanks for the review @bbejeck ! I really appreciate your time. I've responded to your feedback. This is

[GitHub] [kafka] vvcephei commented on a change in pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9221: URL: https://github.com/apache/kafka/pull/9221#discussion_r485742765 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StatefulProcessorNode.java ## @@ -99,5 +99,14 @@ public void

[GitHub] [kafka] vvcephei commented on a change in pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9221: URL: https://github.com/apache/kafka/pull/9221#discussion_r485735152 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/ProcessorParameters.java ## @@ -26,22 +30,57 @@ * Used by the Join

[GitHub] [kafka] vvcephei commented on a change in pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
vvcephei commented on a change in pull request #9221: URL: https://github.com/apache/kafka/pull/9221#discussion_r485732127 ## File path: streams/src/main/java/org/apache/kafka/streams/Topology.java ## @@ -772,6 +806,94 @@ public synchronized Topology addStateStore(final

[jira] [Created] (KAFKA-10472) Consider migrating Topology methods to the Builder pattern

2020-09-09 Thread John Roesler (Jira)
John Roesler created KAFKA-10472: Summary: Consider migrating Topology methods to the Builder pattern Key: KAFKA-10472 URL: https://issues.apache.org/jira/browse/KAFKA-10472 Project: Kafka

[GitHub] [kafka] bbejeck commented on a change in pull request #9221: KAFKA-10436: Implement KIP-478 Topology changes

2020-09-09 Thread GitBox
bbejeck commented on a change in pull request #9221: URL: https://github.com/apache/kafka/pull/9221#discussion_r48569 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StatefulProcessorNode.java ## @@ -99,5 +99,14 @@ public void

[GitHub] [kafka] rajinisivaram merged pull request #9200: MINOR: mirror integration tests should not call System.exit

2020-09-09 Thread GitBox
rajinisivaram merged pull request #9200: URL: https://github.com/apache/kafka/pull/9200 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] rajinisivaram commented on pull request #9200: MINOR: mirror integration tests should not call System.exit

2020-09-09 Thread GitBox
rajinisivaram commented on pull request #9200: URL: https://github.com/apache/kafka/pull/9200#issuecomment-689624548 @lbradstreet Thanks for the PR, merging to trunk. This is an automated message from the Apache Git Service.

[GitHub] [kafka] ning2008wisc commented on pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-09-09 Thread GitBox
ning2008wisc commented on pull request #9224: URL: https://github.com/apache/kafka/pull/9224#issuecomment-689622169 very appreciated for any feedback on what to test additionally and how to get close to the real scenario. Some of the ideas come from

[jira] [Comment Edited] (KAFKA-10304) Revisit and improve the tests of MirrorMaker 2

2020-09-09 Thread Ning Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192664#comment-17192664 ] Ning Zhang edited comment on KAFKA-10304 at 9/9/20, 3:00 PM: - Hi [~mimaison]

[jira] [Updated] (KAFKA-10304) Revisit and improve the tests of MirrorMaker 2

2020-09-09 Thread Ning Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ning Zhang updated KAFKA-10304: --- Description: In a different MM2 change, [some

[GitHub] [kafka] ijuma commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-09-09 Thread GitBox
ijuma commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r485675758 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/record/RecordBatchIterationBenchmark.java ## @@ -142,15 +169,15 @@ private ByteBuffer

[GitHub] [kafka] dongjinleekr commented on a change in pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-09-09 Thread GitBox
dongjinleekr commented on a change in pull request #9266: URL: https://github.com/apache/kafka/pull/9266#discussion_r485673100 ## File path: core/src/test/scala/kafka/utils/LoggingTest.scala ## @@ -58,4 +60,20 @@ class LoggingTest extends Logging {

[GitHub] [kafka] tombentley commented on a change in pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-09-09 Thread GitBox
tombentley commented on a change in pull request #9266: URL: https://github.com/apache/kafka/pull/9266#discussion_r485671503 ## File path: core/src/test/scala/kafka/utils/LoggingTest.scala ## @@ -58,4 +60,20 @@ class LoggingTest extends Logging {

[jira] [Updated] (KAFKA-10470) zstd decompression with small batches is slow and causes excessive GC

2020-09-09 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-10470: Priority: Major (was: Minor) > zstd decompression with small batches is slow and causes

[jira] [Resolved] (KAFKA-10468) Log4jController.getLoggers serialization

2020-09-09 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-10468. - Resolution: Duplicate Duplicate of KAFKA-10403. > Log4jController.getLoggers serialization >

[jira] [Resolved] (KAFKA-10403) Replace scala collection by java collection in generating MBeans attributes

2020-09-09 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-10403. - Fix Version/s: 2.7.0 Resolution: Fixed > Replace scala collection by java collection in

[GitHub] [kafka] ijuma merged pull request #9182: KAFKA-10403 Replace scala collection by java collection in generating…

2020-09-09 Thread GitBox
ijuma merged pull request #9182: URL: https://github.com/apache/kafka/pull/9182 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] chia7712 commented on pull request #9257: KAFKA-10463 the necessary utilities in Dockerfile should include git

2020-09-09 Thread GitBox
chia7712 commented on pull request #9257: URL: https://github.com/apache/kafka/pull/9257#issuecomment-689608467 > Can you please add a comment explaining that git is included in openjdk:8, but not openjdk:11? done. Thanks for reviews!

[GitHub] [kafka] chia7712 commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-09-09 Thread GitBox
chia7712 commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r48574 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/record/RecordBatchIterationBenchmark.java ## @@ -142,15 +169,15 @@ private ByteBuffer

[GitHub] [kafka] ijuma commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-09-09 Thread GitBox
ijuma commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r485659670 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/record/RecordBatchIterationBenchmark.java ## @@ -142,15 +169,15 @@ private ByteBuffer

[GitHub] [kafka] ijuma commented on a change in pull request #9202: KAFKA-10401: Fix the currentStateTimeStamp doesn't get set correctly

2020-09-09 Thread GitBox
ijuma commented on a change in pull request #9202: URL: https://github.com/apache/kafka/pull/9202#discussion_r485652591 ## File path: core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala ## @@ -1401,24 +1401,20 @@ object GroupMetadataManager { val

[jira] [Updated] (KAFKA-10466) Add Regex option for replacement on MaskField SMT

2020-09-09 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjin Lee updated KAFKA-10466: Labels: Needs-kip newbie (was: ) > Add Regex option for replacement on MaskField SMT >

[GitHub] [kafka] ijuma commented on pull request #9178: KAFKA-8362: fix the old checkpoint won't be removed after alter log dir

2020-09-09 Thread GitBox
ijuma commented on pull request #9178: URL: https://github.com/apache/kafka/pull/9178#issuecomment-689590729 @junrao maybe you can review this. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ijuma commented on pull request #9257: KAFKA-10463 the necessary utilities in Dockerfile should include git

2020-09-09 Thread GitBox
ijuma commented on pull request #9257: URL: https://github.com/apache/kafka/pull/9257#issuecomment-689590183 Thanks for the PR. Can you please add a comment explaining that `git` is included in openjdk:8, but not openjdk:11?

[jira] [Commented] (KAFKA-10284) Group membership update due to static member rejoin should be persisted

2020-09-09 Thread feyman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192885#comment-17192885 ] feyman commented on KAFKA-10284: Just FYI, created a PR: https://github.com/apache/kafka/pull/9270 >

[GitHub] [kafka] lct45 commented on a change in pull request #9239: Adding reverse iterator usage for sliding windows processing (extending KIP-450)

2020-09-09 Thread GitBox
lct45 commented on a change in pull request #9239: URL: https://github.com/apache/kafka/pull/9239#discussion_r485623786 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KGroupedStreamImplTest.java ## @@ -239,52 +241,81 @@ private void

[GitHub] [kafka] lct45 commented on a change in pull request #9239: Adding reverse iterator usage for sliding windows processing (extending KIP-450)

2020-09-09 Thread GitBox
lct45 commented on a change in pull request #9239: URL: https://github.com/apache/kafka/pull/9239#discussion_r485621499 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SlidingWindowedCogroupedKStreamImplTest.java ## @@ -246,4 +238,31 @@ private

[GitHub] [kafka] lct45 commented on a change in pull request #9239: Adding reverse iterator usage for sliding windows processing (extending KIP-450)

2020-09-09 Thread GitBox
lct45 commented on a change in pull request #9239: URL: https://github.com/apache/kafka/pull/9239#discussion_r485620554 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SuppressScenarioTest.java ## @@ -493,52 +494,59 @@ public void

[GitHub] [kafka] dongjinleekr commented on a change in pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-09-09 Thread GitBox
t;TRACE") Review comment: @tombentley A small propsal: would `org.apache.log4j.Level.TRACE.toString` be better? It passes compilation checks. ![20200909-223825](https://user-images.githubusercontent.com/2375128/92605881-3fd02d00-f

[GitHub] [kafka] dajac commented on pull request #9249: KAFKA-10458: Add update quota for TokenBucket registered with Sensor

2020-09-09 Thread GitBox
dajac commented on pull request #9249: URL: https://github.com/apache/kafka/pull/9249#issuecomment-689564871 @apovzner I have looked at this further and I have thought about another approach. I have opened a PR for it: https://github.com/apache/kafka/pull/9272. I hope that you don't mind.

[GitHub] [kafka] dajac opened a new pull request #9272: KAFKA-10458; Updating controller quota does not work since Token Bucket

2020-09-09 Thread GitBox
dajac opened a new pull request #9272: URL: https://github.com/apache/kafka/pull/9272 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 1:00 PM: I have this issue too

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 1:00 PM: I have this issue too

[jira] [Updated] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BDeus updated KAFKA-10413: -- Attachment: connect_worker_balanced.png > rebalancing leads to unevenly balanced connectors >

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 12:58 PM: - I have this issue too

[jira] [Updated] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BDeus updated KAFKA-10413: -- Attachment: (was: connect_worker_balanced.png) > rebalancing leads to unevenly balanced connectors >

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 12:58 PM: - I have this issue too

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 12:57 PM: - I have this issue too

[jira] [Commented] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus commented on KAFKA-10413: --- I have this issue too when EC2 scale in (S3 sink connector). It seems

[jira] [Comment Edited] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192827#comment-17192827 ] BDeus edited comment on KAFKA-10413 at 9/9/20, 12:23 PM: - I have this issue too

[jira] [Updated] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

2020-09-09 Thread BDeus (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BDeus updated KAFKA-10413: -- Attachment: connect_worker_balanced.png > rebalancing leads to unevenly balanced connectors >

  1   2   >