[GitHub] [kafka] guozhangwang commented on a change in pull request #10613: KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied

2021-05-04 Thread GitBox
guozhangwang commented on a change in pull request #10613: URL: https://github.com/apache/kafka/pull/10613#discussion_r626253159 ## File path: streams/src/test/java/org/apache/kafka/streams/TopologyTest.java ## @@ -863,6 +920,60 @@ public void

[GitHub] [kafka] guozhangwang commented on a change in pull request #10613: KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied

2021-05-04 Thread GitBox
guozhangwang commented on a change in pull request #10613: URL: https://github.com/apache/kafka/pull/10613#discussion_r626250519 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -155,7 +156,7 @@ public long get() {

[GitHub] [kafka] guozhangwang commented on a change in pull request #9441: KAFKA-10614: Ensure group state (un)load is executed in the submitted order

2021-05-04 Thread GitBox
guozhangwang commented on a change in pull request #9441: URL: https://github.com/apache/kafka/pull/9441#discussion_r626250346 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -279,30 +279,33 @@ class KafkaApis(val requestChannel: RequestChannel,

[GitHub] [kafka] guozhangwang commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
guozhangwang commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626249438 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java ## @@

[jira] [Updated] (KAFKA-12748) Explore new RocksDB options to consider enabling by default

2021-05-04 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12748: -- Labels: rocksdb (was: ) > Explore new RocksDB options to consider enabling by default >

[jira] [Commented] (KAFKA-12748) Explore new RocksDB options to consider enabling by default

2021-05-04 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339399#comment-17339399 ] Guozhang Wang commented on KAFKA-12748: --- Thanks for making a pass on the new options! This is a

[GitHub] [kafka] guozhangwang commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
guozhangwang commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-832406252 LGTM. Leaving to @ableegoldman for a final look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] showuon commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-832388860 Failed test is flaky and un-related: ``` tests/Build/JDK 15 and Scala 2.13 passed tests/Build/JDK 8 and Scala 2.12 passed Build / JDK 11 and Scala 2.13 /

[GitHub] [kafka] showuon commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-832356675 Ah! thanks for the jenkins catching this! We should `verifyValidityAndBalance` before verifying assignment for cooperative protocol to have additional rebalance if necessary.

[jira] [Updated] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12749: --- Affects Version/s: (was: 2.6.2) (was: 2.6.1)

[jira] [Commented] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339355#comment-17339355 ] A. Sophie Blee-Goldman commented on KAFKA-12749: Thanks for filing a ticket – we should

[jira] [Updated] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12749: --- Fix Version/s: 3.0.0 > Changelog topic config on suppressed KTable lost >

[jira] [Updated] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12749: --- Fix Version/s: 2.8.1 > Changelog topic config on suppressed KTable lost >

[jira] [Updated] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12749: --- Labels: newbie newbie++ (was: ) > Changelog topic config on suppressed

[GitHub] [kafka] ableegoldman commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
ableegoldman commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626176038 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java ## @@ -396,6 +396,21 @@ public void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
ableegoldman commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626175996 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java ## @@ -396,6 +396,21 @@ public void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
ableegoldman commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626171667 ## File path: gradle/dependencies.gradle ## @@ -103,7 +103,7 @@ versions += [ netty: "4.1.62.Final", powermock: "2.0.9", reflections:

[GitHub] [kafka] ableegoldman commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
ableegoldman commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626056309 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java ## @@ -411,13 +408,25 @@ private

[GitHub] [kafka] lct45 commented on pull request #10631: MINOR: Stop using hamcrest in system tests

2021-05-04 Thread GitBox
lct45 commented on pull request #10631: URL: https://github.com/apache/kafka/pull/10631#issuecomment-832310245 @cadonna @vvcephei for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] lct45 opened a new pull request #10631: MINOR: Stop using hamcrest in system tests

2021-05-04 Thread GitBox
lct45 opened a new pull request #10631: URL: https://github.com/apache/kafka/pull/10631 We currently use `hamcrest` imports to check the outputs of the `RelationalSmokeTest`, but with the new gradle updates, the proper hamcrest imports are no longer included in the test jar. This

[jira] [Updated] (KAFKA-12748) Explore new RocksDB options to consider enabling by default

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12748: --- Description: With the rocksdb version bump comes a lot of new options, some

[GitHub] [kafka] guozhangwang commented on a change in pull request #10568: KAFKA-8897: Upgrade RocksDB to 6.16.4

2021-05-04 Thread GitBox
guozhangwang commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626160997 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java ## @@ -396,6 +396,21 @@ public void

[jira] [Commented] (KAFKA-9177) Pause completed partitions on restore consumer

2021-05-04 Thread Andrey Polyakov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339336#comment-17339336 ] Andrey Polyakov commented on KAFKA-9177: It's possible this got missed as part of KAFKA-9113? I'm

[jira] [Assigned] (KAFKA-12234) Extend OffsetFetch requests to accept multiple group ids.

2021-05-04 Thread Sanjana Kaundinya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjana Kaundinya reassigned KAFKA-12234: - Assignee: Sanjana Kaundinya > Extend OffsetFetch requests to accept multiple

[jira] [Comment Edited] (KAFKA-8147) Add changelog topic configuration to KTable suppress

2021-05-04 Thread Philip Bourke (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339321#comment-17339321 ] Philip Bourke edited comment on KAFKA-8147 at 5/4/21, 10:08 PM:

[jira] [Commented] (KAFKA-8147) Add changelog topic configuration to KTable suppress

2021-05-04 Thread Philip Bourke (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339321#comment-17339321 ] Philip Bourke commented on KAFKA-8147: -- [~ableegoldman]

[jira] [Created] (KAFKA-12749) Changelog topic config on suppressed KTable lost

2021-05-04 Thread Philip Bourke (Jira)
Philip Bourke created KAFKA-12749: - Summary: Changelog topic config on suppressed KTable lost Key: KAFKA-12749 URL: https://issues.apache.org/jira/browse/KAFKA-12749 Project: Kafka Issue

[GitHub] [kafka] jolshan commented on a change in pull request #10602: KAFKA-12724: Add 2.8.0 to system tests and streams upgrade tests.

2021-05-04 Thread GitBox
jolshan commented on a change in pull request #10602: URL: https://github.com/apache/kafka/pull/10602#discussion_r626115350 ## File path: tests/kafkatest/tests/streams/streams_upgrade_test.py ## @@ -25,12 +25,12 @@ from kafkatest.services.zookeeper import ZookeeperService

[jira] [Updated] (KAFKA-8326) Add Serde> support

2021-05-04 Thread Daniyar Yeralin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniyar Yeralin updated KAFKA-8326: --- Description: _This ticket proposes adding new {color:#4c9aff}ListSerializer{color} and

[jira] [Created] (KAFKA-12748) Explore new RocksDB options to consider enabling by default

2021-05-04 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12748: -- Summary: Explore new RocksDB options to consider enabling by default Key: KAFKA-12748 URL: https://issues.apache.org/jira/browse/KAFKA-12748

[GitHub] [kafka] scott-hendricks commented on a change in pull request #10621: MINOR: Fix a couple Trogdor issues.

2021-05-04 Thread GitBox
scott-hendricks commented on a change in pull request #10621: URL: https://github.com/apache/kafka/pull/10621#discussion_r626055041 ## File path: trogdor/src/main/java/org/apache/kafka/trogdor/workload/GaussianTimestampConstantPayloadGenerator.java ## @@ -0,0 +1,125 @@ +/* +

[GitHub] [kafka] scott-hendricks commented on a change in pull request #10621: MINOR: Fix a couple Trogdor issues.

2021-05-04 Thread GitBox
scott-hendricks commented on a change in pull request #10621: URL: https://github.com/apache/kafka/pull/10621#discussion_r626049787 ## File path: trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConstantThroughputGenerator.java ## @@ -26,17 +26,11 @@ * The lower the

[GitHub] [kafka] scott-hendricks commented on a change in pull request #10621: MINOR: Fix a couple Trogdor issues.

2021-05-04 Thread GitBox
scott-hendricks commented on a change in pull request #10621: URL: https://github.com/apache/kafka/pull/10621#discussion_r626049414 ## File path: trogdor/src/main/java/org/apache/kafka/trogdor/workload/TimestampRecordProcessor.java ## @@ -38,7 +38,7 @@ * * Example spec:

[GitHub] [kafka] cadonna closed pull request #10587: KAFKA-8897: Upgrade RocksDB to 6.8.1

2021-05-04 Thread GitBox
cadonna closed pull request #10587: URL: https://github.com/apache/kafka/pull/10587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] cadonna commented on pull request #10587: KAFKA-8897: Upgrade RocksDB to 6.8.1

2021-05-04 Thread GitBox
cadonna commented on pull request #10587: URL: https://github.com/apache/kafka/pull/10587#issuecomment-832173985 Superseeded by https://github.com/apache/kafka/pull/10568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] cadonna commented on a change in pull request #10568: [DO NOT MERGE] KAFKA-8897: Upgrade RocksDB

2021-05-04 Thread GitBox
cadonna commented on a change in pull request #10568: URL: https://github.com/apache/kafka/pull/10568#discussion_r626032529 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java ## @@ -396,6 +396,21 @@ public void close()

[GitHub] [kafka] ableegoldman commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
ableegoldman commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-832158493 @showuon it looks like there are some related test failures, can you look into those? -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2021-05-04 Thread Frank Yi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339143#comment-17339143 ] Frank Yi commented on KAFKA-12635: -- [~yangguo1220] I was able to repro using the steps below on 2.8.0

[GitHub] [kafka] guozhangwang commented on pull request #10294: KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore

2021-05-04 Thread GitBox
guozhangwang commented on pull request #10294: URL: https://github.com/apache/kafka/pull/10294#issuecomment-832072486 Merged to trunk, thanks @jeqo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] guozhangwang merged pull request #10294: KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore

2021-05-04 Thread GitBox
guozhangwang merged pull request #10294: URL: https://github.com/apache/kafka/pull/10294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] C0urante opened a new pull request #10630: MINOR: Stop logging raw record contents above TRACE level in WorkerSourceTask

2021-05-04 Thread GitBox
C0urante opened a new pull request #10630: URL: https://github.com/apache/kafka/pull/10630 Accidental logging of record contents is a security risk as they may contain sensitive information such as PII. This downgrades the level of log messages in the `WorkerSourceTask` class that contain

[GitHub] [kafka] C0urante commented on pull request #10630: MINOR: Stop logging raw record contents above TRACE level in WorkerSourceTask

2021-05-04 Thread GitBox
C0urante commented on pull request #10630: URL: https://github.com/apache/kafka/pull/10630#issuecomment-832062862 @rhauch @tombentley could you please review at your earliest convenience? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] spena commented on a change in pull request #10613: KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied

2021-05-04 Thread GitBox
spena commented on a change in pull request #10613: URL: https://github.com/apache/kafka/pull/10613#discussion_r625833095 ## File path: streams/src/test/java/org/apache/kafka/streams/TopologyTest.java ## @@ -863,6 +920,60 @@ public void

[GitHub] [kafka] tombentley commented on pull request #9878: KAFKA-6987: Add KafkaFuture.toCompletionStage()

2021-05-04 Thread GitBox
tombentley commented on pull request #9878: URL: https://github.com/apache/kafka/pull/9878#issuecomment-831972566 @dajac @ijuma please could you review this, it's the implementation for KIP-707. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] showuon commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-831972238 @ableegoldman @guozhangwang , thanks for your good comments. I've addressed them in this commit:

[GitHub] [kafka] showuon commented on a change in pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on a change in pull request #10509: URL: https://github.com/apache/kafka/pull/10509#discussion_r625814273 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -149,12 +149,8 @@ private boolean

[GitHub] [kafka] spena commented on a change in pull request #10613: KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied

2021-05-04 Thread GitBox
spena commented on a change in pull request #10613: URL: https://github.com/apache/kafka/pull/10613#discussion_r625811787 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -155,7 +156,7 @@ public long get() {

[GitHub] [kafka] tombentley commented on a change in pull request #9441: KAFKA-10614: Ensure group state (un)load is executed in the submitted order

2021-05-04 Thread GitBox
tombentley commented on a change in pull request #9441: URL: https://github.com/apache/kafka/pull/9441#discussion_r625802963 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -279,30 +279,33 @@ class KafkaApis(val requestChannel: RequestChannel, new

[GitHub] [kafka] kamalcph commented on a change in pull request #10259: MINOR: Provide valid examples in README page.

2021-05-04 Thread GitBox
kamalcph commented on a change in pull request #10259: URL: https://github.com/apache/kafka/pull/10259#discussion_r625788577 ## File path: tests/README.md ## @@ -97,26 +97,26 @@ Examining CI run * Set BUILD_ID is travis ci's build id. E.g. build id is

[jira] [Commented] (KAFKA-12743) [Kafka Streams] - regex name for state-store change-log topic

2021-05-04 Thread Sergey Zyrianov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338959#comment-17338959 ] Sergey Zyrianov commented on KAFKA-12743: - Hi [~guozhang], Looks like the workaround wont work

[GitHub] [kafka] ncliang commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-05-04 Thread GitBox
ncliang commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r625714846 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -482,6 +486,22 @@ private synchronized void

[GitHub] [kafka] jeqo commented on a change in pull request #10294: KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore

2021-05-04 Thread GitBox
jeqo commented on a change in pull request #10294: URL: https://github.com/apache/kafka/pull/10294#discussion_r625703299 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/CompositeReadOnlyWindowStore.java ## @@ -68,10 +65,9 @@ public V fetch(final K

[jira] [Commented] (KAFKA-10660) Poll time out logstash

2021-05-04 Thread Duong Pham (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1733#comment-1733 ] Duong Pham commented on KAFKA-10660: Hi [~showuon], have you fixed this problem ? > Poll time out

[GitHub] [kafka] showuon commented on a change in pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on a change in pull request #10509: URL: https://github.com/apache/kafka/pull/10509#discussion_r625598009 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -163,127 +159,180 @@ private boolean

[GitHub] [kafka] showuon commented on a change in pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-05-04 Thread GitBox
showuon commented on a change in pull request #10509: URL: https://github.com/apache/kafka/pull/10509#discussion_r625597504 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -163,127 +159,180 @@ private boolean