[jira] [Created] (KAFKA-12732) Possible Kerberos configuration bug in mirrormaker

2021-04-29 Thread Wayland Goodliffe (Jira)
Wayland Goodliffe created KAFKA-12732: - Summary: Possible Kerberos configuration bug in mirrormaker Key: KAFKA-12732 URL: https://issues.apache.org/jira/browse/KAFKA-12732 Project: Kafka

[GitHub] [kafka] hachikuji merged pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-29 Thread GitBox
hachikuji merged pull request #10480: URL: https://github.com/apache/kafka/pull/10480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Resolved] (KAFKA-12730) A single Kerberos login failure fails all future connections from Java 9 onwards

2021-04-29 Thread Rajini Sivaram (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-12730. Reviewer: Manikumar Resolution: Fixed > A single Kerberos login failure fails all

[GitHub] [kafka] rajinisivaram merged pull request #10611: KAFKA-12730; Avoid duplicate logout if Kerberos login fails to prevent NPE

2021-04-29 Thread GitBox
rajinisivaram merged pull request #10611: URL: https://github.com/apache/kafka/pull/10611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] rajinisivaram commented on pull request #10611: KAFKA-12730; Avoid duplicate logout if Kerberos login fails to prevent NPE

2021-04-29 Thread GitBox
rajinisivaram commented on pull request #10611: URL: https://github.com/apache/kafka/pull/10611#issuecomment-829240637 @omkreddy Thanks for the review, merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Created] (KAFKA-12731) High number of rebalances lead to GC Overhead limit exceeded JVM crash

2021-04-29 Thread Filip (Jira)
Filip created KAFKA-12731: - Summary: High number of rebalances lead to GC Overhead limit exceeded JVM crash Key: KAFKA-12731 URL: https://issues.apache.org/jira/browse/KAFKA-12731 Project: Kafka

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623156186 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -631,13 +648,31 @@ private void rewind() {

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623156186 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -631,13 +648,31 @@ private void rewind() {

[GitHub] [kafka] vitojeng commented on pull request #10597: KAFKA-5876: Apply StreamsNotStartedException for Interactive Queries

2021-04-29 Thread GitBox
vitojeng commented on pull request #10597: URL: https://github.com/apache/kafka/pull/10597#issuecomment-829215736 @ableegoldman Please take a look. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (KAFKA-7260) Failures while attempting to move partitions with kafka-reassign-partitions.sh

2021-04-29 Thread Sagar Pathak (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1733#comment-1733 ] Sagar Pathak commented on KAFKA-7260: - Is this issue still open ? We are facing same problem in

[GitHub] [kafka] C0urante commented on pull request #10605: KAFKA-12726 prevent a stuck Task.stop() from blocking subsequent Task.stops()s

2021-04-29 Thread GitBox
C0urante commented on pull request #10605: URL: https://github.com/apache/kafka/pull/10605#issuecomment-829162686 Thanks for the ping @tombentley and thanks for the PR @ryannedolan! I've left a more thorough summary of my thoughts on the ticket as I'm not sure a PR is required at all at

[jira] [Commented] (KAFKA-12726) misbehaving Task.stop() can prevent other Tasks from stopping

2021-04-29 Thread Ryanne Dolan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335592#comment-17335592 ] Ryanne Dolan commented on KAFKA-12726: -- [~ChrisEgerton] ah, indeed I have conflated

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r623069794 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogMetadataSerde.java ## @@ -0,0 +1,105 @@

[jira] [Commented] (KAFKA-12726) misbehaving Task.stop() can prevent other Tasks from stopping

2021-04-29 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335638#comment-17335638 ] Chris Egerton commented on KAFKA-12726: --- Ahhh, that makes sense. We wrestled with this a bit in

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r622766875 ## File path: storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogSegmentMetadata.java ## @@ -270,9 +247,9 @@ public String

[jira] [Resolved] (KAFKA-12265) Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-29 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12265. - Resolution: Fixed > Move the BatchAccumulator in KafkaRaftClient to LeaderState >

[GitHub] [kafka] ableegoldman commented on pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman commented on pull request #10608: URL: https://github.com/apache/kafka/pull/10608#issuecomment-829526743 Handful of unrelated failures (as always, these days  ) -- left a comment on all their flaky test tickets -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] ryannedolan commented on pull request #10605: KAFKA-12726 prevent a stuck Task.stop() from blocking subsequent Task.stops()s

2021-04-29 Thread GitBox
ryannedolan commented on pull request #10605: URL: https://github.com/apache/kafka/pull/10605#issuecomment-829527496 @C0urante thanks for pointing out the root of our confusion here. You're right that task.stop() not being Task.stop() threw me. I believe the problem still exists

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623370233 ## File path: raft/src/test/java/org/apache/kafka/raft/MockLog.java ## @@ -363,33 +362,30 @@ public LogFetchInfo read(long startOffset, Isolation

[GitHub] [kafka] spena opened a new pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix

2021-04-29 Thread GitBox
spena opened a new pull request #10612: URL: https://github.com/apache/kafka/pull/10612 Adds an internal flag that can be used to disable the fixes in KAFKA-10847. It defaults to `true` if the flag is not set or has an invalid boolean value. The flag is named

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623321728 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -680,13 +717,13 @@ public void

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623366415 ## File path: core/src/main/scala/kafka/tools/TestRaftServer.scala ## @@ -226,7 +232,11 @@ class TestRaftServer( reader.close() }

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623370427 ## File path: raft/src/test/java/org/apache/kafka/raft/MockLog.java ## @@ -426,48 +422,55 @@ public RawSnapshotWriter createSnapshot(OffsetAndEpoch

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623370656 ## File path: raft/src/test/java/org/apache/kafka/raft/internals/RecordsIteratorTest.java ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software

[jira] [Created] (KAFKA-12733) KRaft: always bump leader epoch when changing the ISR for a controlled shutdown

2021-04-29 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-12733: Summary: KRaft: always bump leader epoch when changing the ISR for a controlled shutdown Key: KAFKA-12733 URL: https://issues.apache.org/jira/browse/KAFKA-12733

[GitHub] [kafka] cmccabe merged pull request #10564: MINOR: clean up some replication code

2021-04-29 Thread GitBox
cmccabe merged pull request #10564: URL: https://github.com/apache/kafka/pull/10564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] spena opened a new pull request #10613: KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied

2021-04-29 Thread GitBox
spena opened a new pull request #10613: URL: https://github.com/apache/kafka/pull/10613 When users supply in-memory stores for left/outer joins, then the internal shared outer store must be switch to in-memory store too. This will allow users who want to keep all stores in memory to

[GitHub] [kafka] ableegoldman merged pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman merged pull request #10608: URL: https://github.com/apache/kafka/pull/10608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] ableegoldman commented on pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman commented on pull request #10608: URL: https://github.com/apache/kafka/pull/10608#issuecomment-829528563 I'm going to merge this now since Walker approved it (thanks!) and I'm waiting to rebase the NamedTopology PR on top of these changes. If anyone comes across this and has

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623367035 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -60,19 +61,21 @@ @Override void close(); -class Batch { +

[jira] [Assigned] (KAFKA-12668) MockScheduler is not safe to use in concurrent code.

2021-04-29 Thread Maksim Iakunin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Iakunin reassigned KAFKA-12668: -- Assignee: Maksim Iakunin > MockScheduler is not safe to use in concurrent code. >

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
wcarlson5 commented on a change in pull request #10608: URL: https://github.com/apache/kafka/pull/10608#discussion_r623295241 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImplTest.java ## @@ -152,28 +150,6 @@ public void

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623409533 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -680,13 +717,13 @@ public void

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623368881 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftClientTestContext.java ## @@ -1081,24 +1103,25 @@ OptionalInt currentClaimedEpoch() {

[GitHub] [kafka] rondagostino commented on pull request #10604: MINOR: system test spelling/pydoc/dead code fixes

2021-04-29 Thread GitBox
rondagostino commented on pull request #10604: URL: https://github.com/apache/kafka/pull/10604#issuecomment-829572710 > Could you also update the "Co-located Raft quorum" comment in the init section with starting node id as 1? Nice catch, @kamalcph! I think I fixed what you were

[GitHub] [kafka] kowshik commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
kowshik commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r623276793 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogSegmentMetadataTransform.java ## @@ -0,0

[jira] [Commented] (KAFKA-7964) Flaky Test ConsumerBounceTest#testConsumerReceivesFatalExceptionWhenGroupPassesMaxSize

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335780#comment-17335780 ] A. Sophie Blee-Goldman commented on KAFKA-7964: --- Failed again, with a different error this

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623328064 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -680,13 +717,13 @@ public void

[jira] [Commented] (KAFKA-5492) LogRecoveryTest.testHWCheckpointWithFailuresSingleLogSegment transient failure

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335786#comment-17335786 ] A. Sophie Blee-Goldman commented on KAFKA-5492: --- Failed again: Stacktrace

[jira] [Resolved] (KAFKA-6409) LogRecoveryTest (testHWCheckpointWithFailuresSingleLogSegment) is flaky

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-6409. --- Resolution: Duplicate > LogRecoveryTest

[jira] [Commented] (KAFKA-12668) MockScheduler is not safe to use in concurrent code.

2021-04-29 Thread Maksim Iakunin (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335684#comment-17335684 ] Maksim Iakunin commented on KAFKA-12668: This is my first issue in the Kafka project and I'll

[GitHub] [kafka] junrao commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
junrao commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r623251580 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/BytesApiMessageSerde.java ## @@ -0,0 +1,68 @@ +/* +

[GitHub] [kafka] cmccabe commented on a change in pull request #10564: MINOR: clean up some replication code

2021-04-29 Thread GitBox
cmccabe commented on a change in pull request #10564: URL: https://github.com/apache/kafka/pull/10564#discussion_r623286027 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -356,7 +367,9 @@ public void

[jira] [Commented] (KAFKA-9295) KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335773#comment-17335773 ] A. Sophie Blee-Goldman commented on KAFKA-9295: --- Failed again, same stacktrace. I take back

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623369093 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftEventSimulationTest.java ## @@ -717,7 +720,8 @@ void start(int nodeId) {

[GitHub] [kafka] ableegoldman opened a new pull request #10615: KAFKA-12648: basic skeleton API for NamedTopology

2021-04-29 Thread GitBox
ableegoldman opened a new pull request #10615: URL: https://github.com/apache/kafka/pull/10615 Basically just the API of https://github.com/apache/kafka/pull/10609, should not contain any logical changes in Streams at this point (for example I also stripped out the protocol change for

[GitHub] [kafka] wenbingshen commented on pull request #10617: KAFKA-12734: LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow when skip activeSegment sanityCheck

2021-04-29 Thread GitBox
wenbingshen commented on pull request #10617: URL: https://github.com/apache/kafka/pull/10617#issuecomment-829766268 @junrao @ijuma @chia7712 Can you take a look at this PR? Thanks. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Assigned] (KAFKA-12734) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow when skip activeSegment sanityCheck

2021-04-29 Thread Wenbing Shen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenbing Shen reassigned KAFKA-12734: Assignee: Wenbing Shen > LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow

[GitHub] [kafka] feyman2016 commented on a change in pull request #10593: KAFKA-10800 Validate the snapshot id when the state machine creates a snapshot

2021-04-29 Thread GitBox
feyman2016 commented on a change in pull request #10593: URL: https://github.com/apache/kafka/pull/10593#discussion_r623574263 ## File path: raft/src/test/java/org/apache/kafka/snapshot/SnapshotWriterTest.java ## @@ -38,7 +38,7 @@ @Test public void

[GitHub] [kafka] ableegoldman edited a comment on pull request #10559: MINOR: diable RaftClusterTest first

2021-04-29 Thread GitBox
ableegoldman edited a comment on pull request #10559: URL: https://github.com/apache/kafka/pull/10559#issuecomment-829797388 Do we really need to disable them or is there a fix in the works? I'm fine with disabling so long as someone is actively looking into this, but given this is such a

[GitHub] [kafka] ableegoldman commented on pull request #10559: MINOR: diable RaftClusterTest first

2021-04-29 Thread GitBox
ableegoldman commented on pull request #10559: URL: https://github.com/apache/kafka/pull/10559#issuecomment-829797388 Do we really need to disable them or is there a fix in the works? I'm fine with disabling so long as someone is actively looking into this, but given this is such a

[jira] [Updated] (KAFKA-10555) Improve client state machine

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-10555: --- Fix Version/s: 2.8.0 > Improve client state machine >

[GitHub] [kafka] ableegoldman edited a comment on pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman edited a comment on pull request #10608: URL: https://github.com/apache/kafka/pull/10608#issuecomment-829528563 I'm going to merge this now since Walker approved it (thanks!) and I'm waiting to rebase the NamedTopology PR on top of these changes. If anyone comes across this

[GitHub] [kafka] wenbingshen opened a new pull request #10617: KAFKA-12734: LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow when skip activeSegment sanityCheck

2021-04-29 Thread GitBox
wenbingshen opened a new pull request #10617: URL: https://github.com/apache/kafka/pull/10617 The detailed stack information is linked to this jiraId [KAFKA-12734](https://issues.apache.org/jira/browse/KAFKA-12734) ### Committer Checklist (excluded from commit message) - [ ]

[GitHub] [kafka] feyman2016 commented on a change in pull request #10593: KAFKA-10800 Validate the snapshot id when the state machine creates a snapshot

2021-04-29 Thread GitBox
feyman2016 commented on a change in pull request #10593: URL: https://github.com/apache/kafka/pull/10593#discussion_r623569039 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2268,6 +2269,25 @@ private Long append(int epoch, List records,

[GitHub] [kafka] feyman2016 commented on a change in pull request #10593: KAFKA-10800 Validate the snapshot id when the state machine creates a snapshot

2021-04-29 Thread GitBox
feyman2016 commented on a change in pull request #10593: URL: https://github.com/apache/kafka/pull/10593#discussion_r623574263 ## File path: raft/src/test/java/org/apache/kafka/snapshot/SnapshotWriterTest.java ## @@ -38,7 +38,7 @@ @Test public void

[GitHub] [kafka] ableegoldman commented on pull request #10615: KAFKA-12648: basic skeleton API for NamedTopology

2021-04-29 Thread GitBox
ableegoldman commented on pull request #10615: URL: https://github.com/apache/kafka/pull/10615#issuecomment-829737080 @wcarlson5 @guozhangwang @rodesai -- Walker let me know if there's anything else you need, but I think this should be sufficient to unblock work on the POC while I

[GitHub] [kafka] ableegoldman commented on a change in pull request #10615: KAFKA-12648: basic skeleton API for NamedTopology

2021-04-29 Thread GitBox
ableegoldman commented on a change in pull request #10615: URL: https://github.com/apache/kafka/pull/10615#discussion_r623529336 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -64,50 +97,98 @@ public static TaskId parse(final String

[jira] [Created] (KAFKA-12734) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow when skip activeSegment sanityCheck

2021-04-29 Thread Wenbing Shen (Jira)
Wenbing Shen created KAFKA-12734: Summary: LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow when skip activeSegment sanityCheck Key: KAFKA-12734 URL: https://issues.apache.org/jira/browse/KAFKA-12734

[GitHub] [kafka] lwsbox commented on pull request #5997: KAFKA-7697: Avoid blocking for leaderIsrUpdateLock in DelayedFetch

2021-04-29 Thread GitBox
lwsbox commented on pull request #5997: URL: https://github.com/apache/kafka/pull/5997#issuecomment-829737677 > Discussed with Jason and will create a new PR that avoids acquiring `Partition#leaderIsrUpdateLock` while holding the lock. which PR? -- This is an automated message

[jira] [Commented] (KAFKA-12718) SessionWindows are closed too early

2021-04-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337019#comment-17337019 ] Matthias J. Sax commented on KAFKA-12718: - Well, `gracePeriod` is a generic concept that applies

[jira] [Commented] (KAFKA-12726) misbehaving Task.stop() can prevent other Tasks from stopping

2021-04-29 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337024#comment-17337024 ] Chris Egerton commented on KAFKA-12726: --- Okay, that makes the problem clearer (y). Maybe we can

[GitHub] [kafka] mjsax merged pull request #10548: KAFKA-12396 added a nullcheck before trying to retrieve a key

2021-04-29 Thread GitBox
mjsax merged pull request #10548: URL: https://github.com/apache/kafka/pull/10548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] mjsax commented on pull request #10548: KAFKA-12396 added a nullcheck before trying to retrieve a key

2021-04-29 Thread GitBox
mjsax commented on pull request #10548: URL: https://github.com/apache/kafka/pull/10548#issuecomment-829758387 Thanks for the PR @Nathan22177! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] ableegoldman edited a comment on pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman edited a comment on pull request #10608: URL: https://github.com/apache/kafka/pull/10608#issuecomment-829528563 I'm going to merge this now since Walker approved it (thanks!) and I'm waiting to rebase the NamedTopology PR on top of these changes. If anyone comes across this

[GitHub] [kafka] hachikuji opened a new pull request #10616: KAFKA-12709; Add Admin API for `ListTransactions`

2021-04-29 Thread GitBox
hachikuji opened a new pull request #10616: URL: https://github.com/apache/kafka/pull/10616 This patch adds `Admin` support for the `listTransactions` API, which was added by

[GitHub] [kafka] feyman2016 commented on a change in pull request #10593: KAFKA-10800 Validate the snapshot id when the state machine creates a snapshot

2021-04-29 Thread GitBox
feyman2016 commented on a change in pull request #10593: URL: https://github.com/apache/kafka/pull/10593#discussion_r623566646 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2268,6 +2269,25 @@ private Long append(int epoch, List records,

[jira] [Commented] (KAFKA-12629) Flaky Test RaftClusterTest

2021-04-29 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337048#comment-17337048 ] Luke Chen commented on KAFKA-12629: --- I have a PR to disable the frequently failed tests. Still waiting

[GitHub] [kafka] dengziming opened a new pull request #10618: MINOR: Fix wrong import control declaration

2021-04-29 Thread GitBox
dengziming opened a new pull request #10618: URL: https://github.com/apache/kafka/pull/10618 *More detailed description of your change* `IntegrationTestHelper` is not right. *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or

[GitHub] [kafka] dengziming commented on pull request #10618: MINOR: Fix wrong import control declaration

2021-04-29 Thread GitBox
dengziming commented on pull request #10618: URL: https://github.com/apache/kafka/pull/10618#issuecomment-829787194 @mumrah ,Hello, PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Updated] (KAFKA-12663) Update FindCoordinator to resolve multiple Coordinators at a time

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-12663: --- Fix Version/s: 3.0.0 > Update FindCoordinator to resolve multiple

[jira] [Commented] (KAFKA-12663) Update FindCoordinator to resolve multiple Coordinators at a time

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337059#comment-17337059 ] Konstantine Karantasis commented on KAFKA-12663: According

[jira] [Commented] (KAFKA-12234) Extend OffsetFetch requests to accept multiple group ids.

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337067#comment-17337067 ] Konstantine Karantasis commented on KAFKA-12234: According to 

[jira] [Updated] (KAFKA-12234) Extend OffsetFetch requests to accept multiple group ids.

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis updated KAFKA-12234: --- Fix Version/s: 3.0.0 > Extend OffsetFetch requests to accept multiple group

[GitHub] [kafka] xjin-Confluent opened a new pull request #10614: MINOR: Upgrade jersey to 2.34

2021-04-29 Thread GitBox
xjin-Confluent opened a new pull request #10614: URL: https://github.com/apache/kafka/pull/10614 This is to resolve CVE: CVE-2021-28168 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status

[jira] [Commented] (KAFKA-12718) SessionWindows are closed too early

2021-04-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337042#comment-17337042 ] John Roesler commented on KAFKA-12718: -- Thanks for reporting this, [~mjsax] ! What [~ableegoldman]

[GitHub] [kafka] ableegoldman commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-04-29 Thread GitBox
ableegoldman commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-829779733 Hey @yeralin one other thing, can you give the KIP document a quick pass and make sure everything in there is up to date with what we've discussed and anything else that's

[jira] [Comment Edited] (KAFKA-12663) Update FindCoordinator to resolve multiple Coordinators at a time

2021-04-29 Thread Konstantine Karantasis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337059#comment-17337059 ] Konstantine Karantasis edited comment on KAFKA-12663 at 4/30/21, 4:27 AM:

[GitHub] [kafka] Nathan22177 commented on pull request #10548: KAFKA-12396 added a nullcheck before trying to retrieve a key

2021-04-29 Thread GitBox
Nathan22177 commented on pull request #10548: URL: https://github.com/apache/kafka/pull/10548#issuecomment-829635490 > To avoid checkstyle and test issues, and reduce review turn-around time, I would recommend to run unit tests and checkstyle locally before pushing update: `./gradlew

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623442240 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -680,13 +717,13 @@ public void

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-04-29 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r623442240 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -680,13 +717,13 @@ public void

[jira] [Assigned] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-29 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-12682: -- Assignee: Justine Olshan > Kraft MetadataPartitionsBuilder _localChanged and

[jira] [Commented] (KAFKA-12718) SessionWindows are closed too early

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17336997#comment-17336997 ] A. Sophie Blee-Goldman commented on KAFKA-12718: {quote}I was just looking into

[GitHub] [kafka] vitojeng commented on pull request #10597: KAFKA-5876: Apply StreamsNotStartedException for Interactive Queries

2021-04-29 Thread GitBox
vitojeng commented on pull request #10597: URL: https://github.com/apache/kafka/pull/10597#issuecomment-829640637 Oh, I get your point. This make sense and totally agree. Thanks @ableegoldman for explanation. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ableegoldman commented on a change in pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-04-29 Thread GitBox
ableegoldman commented on a change in pull request #10509: URL: https://github.com/apache/kafka/pull/10509#discussion_r623436340 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -163,127 +159,180 @@ private

[jira] [Assigned] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-29 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-12682: -- Assignee: (was: Justine Olshan) > Kraft MetadataPartitionsBuilder _localChanged

[jira] [Commented] (KAFKA-12726) misbehaving Task.stop() can prevent other Tasks from stopping

2021-04-29 Thread Ryanne Dolan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17336550#comment-17336550 ] Ryanne Dolan commented on KAFKA-12726: -- [~ChrisEgerton] Yeah, the problem is that we were seeing,

[GitHub] [kafka] Nathan22177 commented on a change in pull request #10548: KAFKA-12396 added a nullcheck before trying to retrieve a key

2021-04-29 Thread GitBox
Nathan22177 commented on a change in pull request #10548: URL: https://github.com/apache/kafka/pull/10548#discussion_r621063125 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/MeteredSessionStoreTest.java ## @@ -472,11 +472,26 @@ public void

[GitHub] [kafka] jolshan commented on pull request #10602: KAFKA-12724: Add 2.8.0 to system tests and streams upgrade tests.

2021-04-29 Thread GitBox
jolshan commented on pull request #10602: URL: https://github.com/apache/kafka/pull/10602#issuecomment-829412493 Can we also add some of the newer versions to`downgrade_test.py`? We've fallen behind on some of the versions there. -- This is an automated message from the Apache Git

[GitHub] [kafka] junrao commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
junrao commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r623253824 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogSegmentMetadataTransform.java ## @@ -0,0

[GitHub] [kafka] ableegoldman commented on a change in pull request #10608: MINOR: clean up some remaining locking stuff in StateDirectory

2021-04-29 Thread GitBox
ableegoldman commented on a change in pull request #10608: URL: https://github.com/apache/kafka/pull/10608#discussion_r623325017 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImplTest.java ## @@ -152,28 +150,6 @@ public

[jira] [Commented] (KAFKA-12629) Flaky Test RaftClusterTest

2021-04-29 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335783#comment-17335783 ] A. Sophie Blee-Goldman commented on KAFKA-12629: This test has multiple failures on

[GitHub] [kafka] ableegoldman commented on pull request #10597: KAFKA-5876: Apply StreamsNotStartedException for Interactive Queries

2021-04-29 Thread GitBox
ableegoldman commented on pull request #10597: URL: https://github.com/apache/kafka/pull/10597#issuecomment-829564963 Thanks @vitojeng -- just to clarify, I was proposing that we do throw the StreamsNotStartedException for the `#allMetadata`, `#allMetadataForStore`, and

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623367352 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -311,8 +311,18 @@ private void updateListenersProgress(long

[GitHub] [kafka] jsancio commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-29 Thread GitBox
jsancio commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r623367561 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/RecordsIterator.java ## @@ -0,0 +1,238 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] showuon commented on pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-04-29 Thread GitBox
showuon commented on pull request #10509: URL: https://github.com/apache/kafka/pull/10509#issuecomment-829025045 @ableegoldman , I've addressed all your comments, and have some code refactor. What I did are: 1. remove `sortedAllPartitions` to save memory 2. put all logic related to

[GitHub] [kafka] tombentley commented on pull request #9441: KAFKA-10614: Ensure group state (un)load is executed in the submitted order

2021-04-29 Thread GitBox
tombentley commented on pull request #9441: URL: https://github.com/apache/kafka/pull/9441#issuecomment-829027535 @hachikuji that's much better, thank you. I added some logging where we ignore on the remove path, hope that's OK. -- This is an automated message from the Apache Git

[jira] [Created] (KAFKA-12730) A single Kerberos login failure fails all future connections from Java 9 onwards

2021-04-29 Thread Rajini Sivaram (Jira)
Rajini Sivaram created KAFKA-12730: -- Summary: A single Kerberos login failure fails all future connections from Java 9 onwards Key: KAFKA-12730 URL: https://issues.apache.org/jira/browse/KAFKA-12730

[GitHub] [kafka] showuon commented on a change in pull request #10509: KAFKA-12464: enhance constrained sticky Assign algorithm

2021-04-29 Thread GitBox
showuon commented on a change in pull request #10509: URL: https://github.com/apache/kafka/pull/10509#discussion_r622780715 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -163,127 +159,181 @@ private boolean

[GitHub] [kafka] kowshik commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-29 Thread GitBox
kowshik commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r622786594 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogMetadataSerde.java ## @@ -0,0 +1,105 @@

  1   2   >