[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Sean Guo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146776#comment-17146776 ] Sean Guo commented on KAFKA-10134: -- We can try the patch next week to see whether this also addresses

[GitHub] [kafka] guozhangwang commented on pull request #8925: KAFKA-9974: Make produce-sync flush

2020-06-26 Thread GitBox
guozhangwang commented on pull request #8925: URL: https://github.com/apache/kafka/pull/8925#issuecomment-650487412 > Could we include the purpose of this PR in the title, such as "Integration test shouldApplyUpdatesToStandbyStore fix:... " SG

[GitHub] [kafka] abbccdda commented on pull request #8925: KAFKA-9974: Make produce-sync flush

2020-06-26 Thread GitBox
abbccdda commented on pull request #8925: URL: https://github.com/apache/kafka/pull/8925#issuecomment-650486850 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650483907 backported to 2.6, 2.5, and 2.4. I ran the streams and client tests each time, as well as systemTestLibs. This

[jira] [Commented] (KAFKA-10173) BufferUnderflowException during Kafka Streams Upgrade

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146758#comment-17146758 ] John Roesler commented on KAFKA-10173: -- Ok, the fix is now backported all the way back to 2.4. I'll

[jira] [Commented] (KAFKA-10207) Untrimmed Index files cause premature log segment deletions on startup

2020-06-26 Thread Johnny Malizia (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146757#comment-17146757 ] Johnny Malizia commented on KAFKA-10207: I didn't see where to assign this to me, but I went

[GitHub] [kafka] Johnny-Malizia opened a new pull request #8936: Fixed padded timeindex causing premature data deletion

2020-06-26 Thread GitBox
Johnny-Malizia opened a new pull request #8936: URL: https://github.com/apache/kafka/pull/8936 In some cases when a new log segment is rolled, the previous segment's timeindex and offset index do not have the excess 0 bytes trimmed. This results in a situation where the next load from

[jira] [Updated] (KAFKA-10207) Untrimmed Index files cause premature log segment deletions on startup

2020-06-26 Thread Johnny Malizia (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnny Malizia updated KAFKA-10207: --- Description:

[jira] [Commented] (KAFKA-10173) BufferUnderflowException during Kafka Streams Upgrade

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146749#comment-17146749 ] John Roesler commented on KAFKA-10173: -- Ok, I've just merged the fix

[jira] [Commented] (KAFKA-8630) Unit testing a streams processor with a WindowStore throws a ClassCastException

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146747#comment-17146747 ] John Roesler commented on KAFKA-8630: - Hey, all, I inadvertently started a new ticket for this

[jira] [Assigned] (KAFKA-8630) Unit testing a streams processor with a WindowStore throws a ClassCastException

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reassigned KAFKA-8630: --- Assignee: John Roesler > Unit testing a streams processor with a WindowStore throws a >

[GitHub] [kafka] vvcephei merged pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei merged pull request #8905: URL: https://github.com/apache/kafka/pull/8905 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650476597 Failures were unrelated: ``` kafka.api.PlaintextConsumerTest > testLowMaxFetchSizeForRequestAndPartition FAILED org.scalatest.exceptions.TestFailedException:

[jira] [Resolved] (KAFKA-10185) Streams should log summarized restoration information at info level

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10185. -- Resolution: Fixed > Streams should log summarized restoration information at info level >

[jira] [Commented] (KAFKA-10185) Streams should log summarized restoration information at info level

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146739#comment-17146739 ] John Roesler commented on KAFKA-10185: -- Hey [~rhauch] , Sorry, I should have emailed the list when

[jira] [Updated] (KAFKA-10185) Streams should log summarized restoration information at info level

2020-06-26 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10185: - Fix Version/s: (was: 2.7.0) 2.5.1 2.6.0 > Streams

[GitHub] [kafka] vvcephei commented on a change in pull request #8927: KAFKA-10200: Fix testability of PAPI with windowed stores

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8927: URL: https://github.com/apache/kafka/pull/8927#discussion_r446473287 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorContextUtils.java ## @@ -0,0 +1,39 @@ +/* + * Licensed to the

[GitHub] [kafka] vvcephei commented on pull request #8914: MINOR: Do not swallow exception when collecting PIDs

2020-06-26 Thread GitBox
vvcephei commented on pull request #8914: URL: https://github.com/apache/kafka/pull/8914#issuecomment-650474359 Thanks for running the tests. One test failed. I haven't looked to see if it's important:

[GitHub] [kafka] vvcephei commented on a change in pull request #8902: KAFKA-10179: Pass correct changelog topic to state serdes

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8902: URL: https://github.com/apache/kafka/pull/8902#discussion_r446472130 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/AbstractStateManager.java ## @@ -0,0 +1,33 @@ +/* + * Licensed to the

[GitHub] [kafka] vvcephei commented on a change in pull request #8902: KAFKA-10179: Pass correct changelog topic to state serdes

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8902: URL: https://github.com/apache/kafka/pull/8902#discussion_r446471839 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java ## @@ -103,8 +104,13 @@ public void init(final

[GitHub] [kafka] vvcephei commented on a change in pull request #8902: KAFKA-10179: Pass correct changelog topic to state serdes

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8902: URL: https://github.com/apache/kafka/pull/8902#discussion_r446471736 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStoreTest.java ## @@ -124,18 +141,19 @@ public void before()

[GitHub] [kafka] showuon commented on a change in pull request #8712: KAFKA-10006: Don't create internal topics when LeaderNotAvailableException

2020-06-26 Thread GitBox
showuon commented on a change in pull request #8712: URL: https://github.com/apache/kafka/pull/8712#discussion_r446126644 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java ## @@ -125,35 +131,37 @@ public

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Neo Wu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146724#comment-17146724 ] Neo Wu commented on KAFKA-10134: something like this fix my issues, but i am not sure whether this is

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Neo Wu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146723#comment-17146723 ] Neo Wu commented on KAFKA-10134: maybe besides fetchablePartitions, it also should check records =

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Neo Wu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146721#comment-17146721 ] Neo Wu commented on KAFKA-10134: Hi, [~guozhang] Thanks for quick update!  I reviewed and tested your

[jira] [Created] (KAFKA-10207) Untrimmed Index files cause premature log segment deletions on startup

2020-06-26 Thread Johnny Malizia (Jira)
Johnny Malizia created KAFKA-10207: -- Summary: Untrimmed Index files cause premature log segment deletions on startup Key: KAFKA-10207 URL: https://issues.apache.org/jira/browse/KAFKA-10207 Project:

[jira] [Updated] (KAFKA-10189) Reset metric EventQueueTimeMs

2020-06-26 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10189: Labels: (was: core) > Reset metric EventQueueTimeMs > -- > >

[jira] [Updated] (KAFKA-10189) Reset metric EventQueueTimeMs

2020-06-26 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10189: Labels: core (was: pull-request-available) > Reset metric EventQueueTimeMs >

[jira] [Updated] (KAFKA-10189) Reset metric EventQueueTimeMs

2020-06-26 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10189: Component/s: core > Reset metric EventQueueTimeMs > -- > >

[jira] [Commented] (KAFKA-10179) State Store Passes Wrong Changelog Topic to Serde for Optimized Source Tables

2020-06-26 Thread Almog Gavra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146707#comment-17146707 ] Almog Gavra commented on KAFKA-10179: - [~ableegoldman] I confirmed locally that nothing "breaks" if

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146706#comment-17146706 ] Guozhang Wang commented on KAFKA-10134: --- I've provided a PR for trunk, but it should apply cleanly

[GitHub] [kafka] jeffkbkim opened a new pull request #8935: reset event queue time histogram when queue is empty

2020-06-26 Thread GitBox
jeffkbkim opened a new pull request #8935: URL: https://github.com/apache/kafka/pull/8935 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *

[GitHub] [kafka] guozhangwang commented on pull request #8934: KAFKA-10134: Use long poll if we do not have fetchable partitions

2020-06-26 Thread GitBox
guozhangwang commented on pull request #8934: URL: https://github.com/apache/kafka/pull/8934#issuecomment-650464132 @hachikuji @ijuma This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] guozhangwang opened a new pull request #8934: KAFKA-10134: Use long poll if we do not have fetchable partitions

2020-06-26 Thread GitBox
guozhangwang opened a new pull request #8934: URL: https://github.com/apache/kafka/pull/8934 The intention of using poll(0) is to not block on rebalance but still return some data; however, if there's no fetchable partitions then there's no point of polling with 0ms. What's worse, with

[GitHub] [kafka] guozhangwang commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
guozhangwang commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446461812 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryTimeOrderedKeyValueBuffer.java ## @@ -258,34 +263,43 @@ private

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650462969 Ok, @guozhangwang , This is my "final" iteration. I pulled the system tests out, and I'll follow up with another PR later. This PR should be sufficient for the basic purpose,

[jira] [Resolved] (KAFKA-6453) Reconsider timestamp propagation semantics

2020-06-26 Thread Victoria Bialas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Bialas resolved KAFKA-6453. Resolution: Fixed Fixed by James Galasyn in https://github.com/apache/kafka/pull/8920 >

[GitHub] [kafka] ableegoldman closed pull request #6633: [MINOR] Better estimate size of cache

2020-06-26 Thread GitBox
ableegoldman closed pull request #6633: URL: https://github.com/apache/kafka/pull/6633 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] ableegoldman closed pull request #7413: KAFKA-8649: version probing with upgraded leader

2020-06-26 Thread GitBox
ableegoldman closed pull request #7413: URL: https://github.com/apache/kafka/pull/7413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-10177) Replace/improve Percentiles metrics

2020-06-26 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146681#comment-17146681 ] Sophie Blee-Goldman commented on KAFKA-10177: - I haven't personally looked into HrdHistogram

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146678#comment-17146678 ] Guozhang Wang commented on KAFKA-10134: --- [~seanguo] I think [~neowu0]'s brought up issue is a

[jira] [Resolved] (KAFKA-10166) Excessive TaskCorruptedException seen in testing

2020-06-26 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-10166. - Resolution: Fixed > Excessive TaskCorruptedException seen in testing >

[jira] [Assigned] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-26 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-10134: - Assignee: Guozhang Wang > High CPU issue during rebalance in Kafka consumer after

[GitHub] [kafka] guozhangwang commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
guozhangwang commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650426719 Cherry-picked to 2.6 since it is a blocker, cc @rhauch This is an automated message from the Apache Git

[GitHub] [kafka] guozhangwang merged pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
guozhangwang merged pull request #8926: URL: https://github.com/apache/kafka/pull/8926 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] ableegoldman commented on a change in pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
ableegoldman commented on a change in pull request #8929: URL: https://github.com/apache/kafka/pull/8929#discussion_r446431689 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java ## @@ -226,7 +227,7 @@ public

[GitHub] [kafka] satishbellapu commented on pull request #8921: KAFKA-10160: Kafka MM2 consumer configuration

2020-06-26 Thread GitBox
satishbellapu commented on pull request #8921: URL: https://github.com/apache/kafka/pull/8921#issuecomment-650419736 + @cmccabe @rajinisivaram for review. This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ableegoldman commented on pull request #8925: KAFKA-9974: Make produce-sync flush

2020-06-26 Thread GitBox
ableegoldman commented on pull request #8925: URL: https://github.com/apache/kafka/pull/8925#issuecomment-650414999 Is Jenkins linking to the wrong builds? When I try to see which tests failed here it seems to bring me to the results for two completely different PRs. Really

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650413094 I will follow up shortly to extract the system tests to a separate PR, since we're having trouble running the tests at all right now, and we wouldn't know if they are even more

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650412771 Hey @guozhangwang , you might want to take a look at that last fix. The duck-typing code was producing an OOME some times, when it would just interpret a random integer out of

[GitHub] [kafka] ableegoldman commented on a change in pull request #8927: KAFKA-10200: Fix testability of PAPI with windowed stores

2020-06-26 Thread GitBox
ableegoldman commented on a change in pull request #8927: URL: https://github.com/apache/kafka/pull/8927#discussion_r446418844 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorContextUtils.java ## @@ -0,0 +1,39 @@ +/* + * Licensed to

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446409619 ## File path: build.gradle ## @@ -97,7 +97,7 @@ ext { buildVersionFileName = "kafka-version.properties" defaultMaxHeapSize = "2g" -

[GitHub] [kafka] ableegoldman commented on a change in pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
ableegoldman commented on a change in pull request #8929: URL: https://github.com/apache/kafka/pull/8929#discussion_r446408964 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java ## @@ -226,7 +227,7 @@ public

[GitHub] [kafka] abbccdda commented on a change in pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
abbccdda commented on a change in pull request #8929: URL: https://github.com/apache/kafka/pull/8929#discussion_r446404393 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java ## @@ -50,7 +51,7 @@ private final

[GitHub] [kafka] ryannedolan commented on pull request #8930: Mirror Maker 2: offset-syncs variable

2020-06-26 Thread GitBox
ryannedolan commented on pull request #8930: URL: https://github.com/apache/kafka/pull/8930#issuecomment-650389551 @cgetzen the larger change would be worth it IMO. I think a KIP is the right way forward. Happy to help, but you should be able to get permission to create a KIP yourself.

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650387794 Ah, that heap space thing was legit. Fix coming... This is an automated message from the Apache Git Service. To

[GitHub] [kafka] abbccdda commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
abbccdda commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650385489 ```12:23:50 FAILURE: Build failed with an exception. 12:23:50 12:23:50 * What went wrong: 12:23:50 Execution failed for task ':streams:unitTest'. 12:23:50 > Process

[GitHub] [kafka] lct45 commented on pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
lct45 commented on pull request #8929: URL: https://github.com/apache/kafka/pull/8929#issuecomment-650384491 Failed connect tests on JDC 14 org.apache.kafka.connect.mirror.MirrorConnectorsIntegrationTest.testReplication

[GitHub] [kafka] ableegoldman commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
ableegoldman commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650382721 Java 8 and 14 builds passed, Java 11 build failed with...zero failures? This is an automated message from

[GitHub] [kafka] cgetzen edited a comment on pull request #8930: Mirror Maker 2: offset-syncs variable

2020-06-26 Thread GitBox
cgetzen edited a comment on pull request #8930: URL: https://github.com/apache/kafka/pull/8930#issuecomment-650381294 Hi @ryannedolan, wondering if you could spare a little guidance. I created `offset-syncs.topic.override` variable with a default of `null`. What I would have loved

[jira] [Commented] (KAFKA-9509) Fix flaky test MirrorConnectorsIntegrationTest.testReplication

2020-06-26 Thread Leah Thomas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146612#comment-17146612 ] Leah Thomas commented on KAFKA-9509: Failed again: h3. Stacktrace java.lang.RuntimeException: Could

[GitHub] [kafka] cgetzen commented on pull request #8930: Mirror Maker 2: offset-syncs variable

2020-06-26 Thread GitBox
cgetzen commented on pull request #8930: URL: https://github.com/apache/kafka/pull/8930#issuecomment-650381294 Hi @ryannedolan, wondering if you could spare a little guidance. I created `offset-syncs.topic.override` variable with a default of `null`. What I would have loved to do is

[GitHub] [kafka] gharris1727 commented on a change in pull request #8910: KAFKA-10188: Prevent SinkTask::preCommit from being called after SinkTask::stop

2020-06-26 Thread GitBox
gharris1727 commented on a change in pull request #8910: URL: https://github.com/apache/kafka/pull/8910#discussion_r446379953 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -689,6 +692,10 @@ else if

[jira] [Updated] (KAFKA-10181) AlterConfig/IncrementalAlterConfig should go to controller

2020-06-26 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10181: Fix Version/s: 2.7.0 > AlterConfig/IncrementalAlterConfig should go to controller >

[jira] [Updated] (KAFKA-10181) AlterConfig/IncrementalAlterConfig should go to controller

2020-06-26 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10181: Component/s: admin > AlterConfig/IncrementalAlterConfig should go to controller >

[jira] [Commented] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2020-06-26 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146596#comment-17146596 ] João Oliveirinha commented on KAFKA-9693: - [~junrao] thanks for the analysis and hypothesis. I

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650355203 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vvcephei commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650354756 Hmm. Still saw the OOME in https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/3134/ This is an automated

[GitHub] [kafka] guozhangwang commented on pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
guozhangwang commented on pull request #8905: URL: https://github.com/apache/kafka/pull/8905#issuecomment-650354464 test this This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446365992 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryTimeOrderedKeyValueBuffer.java ## @@ -258,34 +263,43 @@ private void

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446364196 ## File path: streams/upgrade-system-tests-25/src/test/java/org/apache/kafka/streams/tests/StreamsSmokeTest.java ## @@ -0,0 +1,99 @@ +/* + * Licensed to

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446364364 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/TimeOrderedKeyValueBufferChangelogDeserializationHelper.java ## @@ -0,0

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446363798 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryTimeOrderedKeyValueBuffer.java ## @@ -361,26 +366,20 @@ private void

[GitHub] [kafka] guozhangwang commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
guozhangwang commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446343541 ## File path: tests/kafkatest/tests/streams/streams_upgrade_test.py ## @@ -189,8 +192,8 @@ def test_upgrade_downgrade_brokers(self, from_version,

[GitHub] [kafka] ijuma commented on pull request #8579: KAFKA-9930: Prevent ReplicaFetcherThread from throwing UnknownTopicOrPartitionException upon topic creation and deletion.

2020-06-26 Thread GitBox
ijuma commented on pull request #8579: URL: https://github.com/apache/kafka/pull/8579#issuecomment-650319283 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] ableegoldman commented on pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest - call KafkaStreams#cleanU…

2020-06-26 Thread GitBox
ableegoldman commented on pull request #8913: URL: https://github.com/apache/kafka/pull/8913#issuecomment-650316087 The system tests I'm triggering keep failing, but not because of this PR. Not sure what's going on :/ But I'll try again:

[GitHub] [kafka] efeg commented on a change in pull request #8579: KAFKA-9930: Prevent ReplicaFetcherThread from throwing UnknownTopicOrPartitionException upon topic creation and deletion.

2020-06-26 Thread GitBox
efeg commented on a change in pull request #8579: URL: https://github.com/apache/kafka/pull/8579#discussion_r446327952 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -382,6 +382,12 @@ abstract class AbstractFetcherThread(name: String,

[GitHub] [kafka] omkreddy closed pull request #8932: MINOR: rename "NOT_INITALIZED" to "NOT_INITIALIZED"

2020-06-26 Thread GitBox
omkreddy closed pull request #8932: URL: https://github.com/apache/kafka/pull/8932 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] andrewchoi5 commented on a change in pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-06-26 Thread GitBox
andrewchoi5 commented on a change in pull request #8479: URL: https://github.com/apache/kafka/pull/8479#discussion_r446318540 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -499,7 +500,16 @@ class Partition(val topicPartition: TopicPartition,

[GitHub] [kafka] ableegoldman commented on a change in pull request #8902: KAFKA-10179: Pass correct changelog topic to state serdes

2020-06-26 Thread GitBox
ableegoldman commented on a change in pull request #8902: URL: https://github.com/apache/kafka/pull/8902#discussion_r446316588 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStoreTest.java ## @@ -124,18 +141,19 @@ public void

[GitHub] [kafka] junrao commented on a change in pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-06-26 Thread GitBox
junrao commented on a change in pull request #8479: URL: https://github.com/apache/kafka/pull/8479#discussion_r446316668 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -499,7 +500,16 @@ class Partition(val topicPartition: TopicPartition,

[jira] [Commented] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2020-06-26 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146527#comment-17146527 ] Jun Rao commented on KAFKA-9693: [~joliveirinha]: Thanks for the analysis. To trigger the flush during

[GitHub] [kafka] ijuma commented on pull request #8931: MINOR: Update Scala to 2.13.3

2020-06-26 Thread GitBox
ijuma commented on pull request #8931: URL: https://github.com/apache/kafka/pull/8931#issuecomment-650297987 One job passed, two had a single unrelated test failure. This is an automated message from the Apache Git Service.

[GitHub] [kafka] ijuma merged pull request #8931: MINOR: Update Scala to 2.13.3

2020-06-26 Thread GitBox
ijuma merged pull request #8931: URL: https://github.com/apache/kafka/pull/8931 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-10166) Excessive TaskCorruptedException seen in testing

2020-06-26 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146501#comment-17146501 ] Sophie Blee-Goldman commented on KAFKA-10166: - Thanks for the detailed analysis Bruno. In

[GitHub] [kafka] andrewchoi5 commented on a change in pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-06-26 Thread GitBox
andrewchoi5 commented on a change in pull request #8479: URL: https://github.com/apache/kafka/pull/8479#discussion_r446309532 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -499,7 +500,16 @@ class Partition(val topicPartition: TopicPartition,

[GitHub] [kafka] ableegoldman commented on a change in pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
ableegoldman commented on a change in pull request #8929: URL: https://github.com/apache/kafka/pull/8929#discussion_r446308395 ## File path: gradle/spotbugs-exclude.xml ## @@ -348,14 +348,11 @@ For a detailed description of spotbugs bug categories, see https://spotbugs.read

[GitHub] [kafka] junrao commented on a change in pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-06-26 Thread GitBox
junrao commented on a change in pull request #8479: URL: https://github.com/apache/kafka/pull/8479#discussion_r446306280 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -499,7 +500,16 @@ class Partition(val topicPartition: TopicPartition,

[GitHub] [kafka] vvcephei commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
vvcephei commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650288339 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
vvcephei commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650288465 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] abbccdda commented on pull request #8902: KAFKA-10179: Pass correct changelog topic to state serdes

2020-06-26 Thread GitBox
abbccdda commented on pull request #8902: URL: https://github.com/apache/kafka/pull/8902#issuecomment-650275504 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] abbccdda commented on pull request #8914: MINOR: Do not swallow exception when collecting PIDs

2020-06-26 Thread GitBox
abbccdda commented on pull request #8914: URL: https://github.com/apache/kafka/pull/8914#issuecomment-650275130 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vvcephei commented on pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
vvcephei commented on pull request #8929: URL: https://github.com/apache/kafka/pull/8929#issuecomment-650270782 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] vvcephei commented on pull request #8929: KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams

2020-06-26 Thread GitBox
vvcephei commented on pull request #8929: URL: https://github.com/apache/kafka/pull/8929#issuecomment-650271007 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vvcephei commented on a change in pull request #8905: KAFKA-10173: Fix suppress changelog binary schema compatibility

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8905: URL: https://github.com/apache/kafka/pull/8905#discussion_r446265559 ## File path: build.gradle ## @@ -97,7 +97,7 @@ ext { buildVersionFileName = "kafka-version.properties" defaultMaxHeapSize = "2g" -

[jira] [Commented] (KAFKA-10206) Admin can transiently return incorrect results about topics

2020-06-26 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146444#comment-17146444 ] Tom Bentley commented on KAFKA-10206: - I think the broker needs some alternative to replying with

[jira] [Created] (KAFKA-10206) Admin can transiently return incorrect results about topics

2020-06-26 Thread Tom Bentley (Jira)
Tom Bentley created KAFKA-10206: --- Summary: Admin can transiently return incorrect results about topics Key: KAFKA-10206 URL: https://issues.apache.org/jira/browse/KAFKA-10206 Project: Kafka

[GitHub] [kafka] guozhangwang commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
guozhangwang commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650261830 > Seems like all the Topology testDriver tests failed, but I got a green build running locally. Do they not run with `./gradlew streams:test`? They are included in

[GitHub] [kafka] ableegoldman commented on pull request #8926: KAFKA-10166: always write checkpoint before closing an (initialized) task

2020-06-26 Thread GitBox
ableegoldman commented on pull request #8926: URL: https://github.com/apache/kafka/pull/8926#issuecomment-650251400 Seems like all the Topology testDriver tests failed, but I got a green build running locally. Do they not run with `./gradlew streams:test`?

[GitHub] [kafka] vvcephei commented on a change in pull request #8927: KAFKA-10200: Fix testability of PAPI with windowed stores

2020-06-26 Thread GitBox
vvcephei commented on a change in pull request #8927: URL: https://github.com/apache/kafka/pull/8927#discussion_r446263222 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorContextUtils.java ## @@ -0,0 +1,39 @@ +/* + * Licensed to the

  1   2   >