[jira] [Updated] (KAFKA-12537) Single Threaded applications will not work with SHUTDOWN_APPLICATION

2021-03-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12537: --- Affects Version/s: (was: 3.0.0) > Single Threaded applications will not

[jira] [Updated] (KAFKA-12537) Single Threaded applications will not work with SHUTDOWN_APPLICATION

2021-03-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12537: --- Summary: Single Threaded applications will not work with

[jira] [Resolved] (KAFKA-12537) Single Threaded EOS applications will not work with SHUTDOWN_APPLICATION

2021-03-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12537. Fix Version/s: 2.8.0 Resolution: Fixed > Single Threaded EOS

[GitHub] [kafka] ableegoldman edited a comment on pull request #10387: KAFKA-12537: fix application shutdown corner case with only one thread

2021-03-26 Thread GitBox
ableegoldman edited a comment on pull request #10387: URL: https://github.com/apache/kafka/pull/10387#issuecomment-808632135 Merged to trunk, just running the tests before pushing to 2.8 @vvcephei edit: done -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ableegoldman commented on pull request #10387: KAFKA-12537: fix application shutdown corner case with only one thread

2021-03-26 Thread GitBox
ableegoldman commented on pull request #10387: URL: https://github.com/apache/kafka/pull/10387#issuecomment-808632135 Merged to trunk, just running the tests before pushing to 2.8 @vvcephei -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] ableegoldman merged pull request #10387: KAFKA-12537: fix application shutdown corner case with only one thread

2021-03-26 Thread GitBox
ableegoldman merged pull request #10387: URL: https://github.com/apache/kafka/pull/10387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602659052 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,38 +519,60 @@ void

[GitHub] [kafka] ableegoldman commented on pull request #10387: KAFKA-12537: fix application shutdown corner case with only one thread

2021-03-26 Thread GitBox
ableegoldman commented on pull request #10387: URL: https://github.com/apache/kafka/pull/10387#issuecomment-808619295 Looks like just one unrelated flaky test failure: `kafka.common.record.MemoryRecordsBuilderTest.[18] magic=1, bufferOffset=15, compressionType=GZIP` -- This is an

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602652039 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,38 +519,60 @@ void

[GitHub] [kafka] mjsax commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602643004 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,38 +519,60 @@ void handleRevocation(final

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602641612 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,38 +519,60 @@ void

[GitHub] [kafka] ableegoldman closed pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman closed pull request #10417: URL: https://github.com/apache/kafka/pull/10417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] ableegoldman commented on pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman commented on pull request #10417: URL: https://github.com/apache/kafka/pull/10417#issuecomment-808600449 Closing this PR since we've ported this fix over to https://github.com/apache/kafka/pull/10387 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] yeralin commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-03-26 Thread GitBox
yeralin commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-808566124 Bump -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [kafka] ableegoldman commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602616296 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamsUncaughtExceptionHandlerIntegrationTest.java ## @@ -177,6 +178,7 @@

[GitHub] [kafka] ableegoldman commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602616135 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamsUncaughtExceptionHandlerIntegrationTest.java ## @@ -177,6 +178,7 @@

[GitHub] [kafka] vvcephei commented on a change in pull request #10378: KAFKA-7106: remove deprecated Windows APIs

2021-03-26 Thread GitBox
vvcephei commented on a change in pull request #10378: URL: https://github.com/apache/kafka/pull/10378#discussion_r602602147 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/SessionWindows.java ## @@ -73,33 +72,16 @@ public final class SessionWindows {

[GitHub] [kafka] ableegoldman commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602606489 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] mjsax commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602605779 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,38 +519,60 @@ void handleRevocation(final

[GitHub] [kafka] mjsax commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602604314 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,41 +519,62 @@ void handleRevocation(final

[GitHub] [kafka] wcarlson5 commented on pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
wcarlson5 commented on pull request #10387: URL: https://github.com/apache/kafka/pull/10387#issuecomment-808533844 @ableegoldman I removed the global only case improvement and made https://issues.apache.org/jira/browse/KAFKA-12565. Now this just has a fix for if there is one thread

[jira] [Updated] (KAFKA-12565) Global thread only topologies should be able to shutdown applications via the uncaught exception handler

2021-03-26 Thread Walker Carlson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walker Carlson updated KAFKA-12565: --- Description: Global thread only topologies should be able to shutdown applications via the

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602602653 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,41 +519,62 @@ void

[jira] [Created] (KAFKA-12565) Global thread only topologies should be able to shutdown applications via the uncaught exception handler

2021-03-26 Thread Walker Carlson (Jira)
Walker Carlson created KAFKA-12565: -- Summary: Global thread only topologies should be able to shutdown applications via the uncaught exception handler Key: KAFKA-12565 URL:

[GitHub] [kafka] mjsax commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption during handleCorruption and handleRevocation

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602602353 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -509,41 +519,62 @@ void handleRevocation(final

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602601435 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamsUncaughtExceptionHandlerIntegrationTest.java ## @@ -177,6 +178,7 @@

[GitHub] [kafka] ableegoldman commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602595373 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -565,59 +565,57 @@ public void run() {

[GitHub] [kafka] kowshik commented on pull request #10280: KAFKA-12554: Refactor Log layer

2021-03-26 Thread GitBox
kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-808514958 @junrao Just a heads up on the following. I'm working on the changes for KAFKA-12552 (https://github.com/apache/kafka/pull/10401 to extract segments map) and KAFKA-12553

[GitHub] [kafka] mjsax commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602591990 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -2333,6 +2334,75 @@ void runOnce() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602591231 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] mjsax commented on a change in pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10417: URL: https://github.com/apache/kafka/pull/10417#discussion_r602590947 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -565,59 +565,57 @@ public void run() { *

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r601987024 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -589,11 +589,7 @@ boolean runLoop() {

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602587542 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -611,6 +607,9 @@ boolean runLoop() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602585671 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -611,6 +607,9 @@ boolean runLoop() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602584888 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602584404 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -167,18 +167,7 @@ void

[GitHub] [kafka] mjsax commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
mjsax commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602581215 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -167,18 +167,7 @@ void handleCorruption(final Set

[GitHub] [kafka] ableegoldman opened a new pull request #10417: HOTFIX: wrap StreamThread#runLoop in outer catch block

2021-03-26 Thread GitBox
ableegoldman opened a new pull request #10417: URL: https://github.com/apache/kafka/pull/10417 We've been amassing more and more `catch` blocks around the `runOnce` loop in the StreamThread's `runLoop`. Some of these have complex handling which is performed inside the catch block, such as

[GitHub] [kafka] ableegoldman commented on a change in pull request #10412: KAFKA-12562: Remove deprecated APIs in KafkaStreams and returned state classes

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10412: URL: https://github.com/apache/kafka/pull/10412#discussion_r602561350 ## File path: docs/streams/upgrade-guide.html ## @@ -98,6 +98,9 @@ Streams API WindowBytesStoreSupplier#segments: deprecated

[jira] [Resolved] (KAFKA-10396) Overall memory of container keep on growing due to kafka stream / rocksdb and OOM killed once limit reached

2021-03-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-10396. Resolution: Fixed > Overall memory of container keep on growing due to

[GitHub] [kafka] ableegoldman commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602528819 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -611,6 +607,9 @@ boolean runLoop() {

[GitHub] [kafka] guozhangwang commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
guozhangwang commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602527847 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -167,18 +167,7 @@ void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602525060 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602525060 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602525060 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[jira] [Commented] (KAFKA-10853) Replication protocol deficiencies with workloads requiring high durability guarantees

2021-03-26 Thread Kyle Ambroff-Kao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309621#comment-17309621 ] Kyle Ambroff-Kao commented on KAFKA-10853: -- We spent a lot of time discussing this internally

[GitHub] [kafka] ableegoldman commented on a change in pull request #10416: KAFKA-8784: remove default close for RocksDBConfigSetter

2021-03-26 Thread GitBox
ableegoldman commented on a change in pull request #10416: URL: https://github.com/apache/kafka/pull/10416#discussion_r602495343 ## File path: streams/src/test/java/org/apache/kafka/test/MockRocksDbConfigSetter.java ## @@ -32,4 +32,9 @@ public void setConfig(final String

[jira] [Updated] (KAFKA-8784) Remove default implementation of RocksDBConfigSetter#close

2021-03-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-8784: -- Description: In 2.3 we added a new method to RocksDBConfigSetter that will be

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10387: KAFKA-12537: get EOS corner case

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10387: URL: https://github.com/apache/kafka/pull/10387#discussion_r602472401 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -492,22 +492,18 @@ private void

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KAFKA-12554: Refactor Log layer

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602052452 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10407: KAFKA-12523: handle TaskCorruption for revoked tasks & remove commit in handleCorruption

2021-03-26 Thread GitBox
wcarlson5 commented on a change in pull request #10407: URL: https://github.com/apache/kafka/pull/10407#discussion_r602456496 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -611,6 +607,9 @@ boolean runLoop() {

[GitHub] [kafka] wcarlson5 commented on pull request #10416: KAFKA-8784: remove default close for RocksDBConfigSetter

2021-03-26 Thread GitBox
wcarlson5 commented on pull request #10416: URL: https://github.com/apache/kafka/pull/10416#issuecomment-808370533 @ableegoldman -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] wcarlson5 edited a comment on pull request #10416: KAFKA-8784: remove default close for RocksDBConfigSetter

2021-03-26 Thread GitBox
wcarlson5 edited a comment on pull request #10416: URL: https://github.com/apache/kafka/pull/10416#issuecomment-808370533 @ableegoldman for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] wcarlson5 opened a new pull request #10416: KAFKA-8784: remove default close for RocksDBConfigSetter

2021-03-26 Thread GitBox
wcarlson5 opened a new pull request #10416: URL: https://github.com/apache/kafka/pull/10416 remove default ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation

[GitHub] [kafka] junrao commented on a change in pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-03-26 Thread GitBox
junrao commented on a change in pull request #10405: URL: https://github.com/apache/kafka/pull/10405#discussion_r602428120 ## File path: clients/src/main/java/org/apache/kafka/common/record/FileRecords.java ## @@ -249,6 +266,7 @@ public void renameTo(File f) throws IOException

[GitHub] [kafka] highluck commented on pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-26 Thread GitBox
highluck commented on pull request #9851: URL: https://github.com/apache/kafka/pull/9851#issuecomment-808339074 @chia7712 thanks updated code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] chia7712 commented on a change in pull request #10389: MINOR: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch

2021-03-26 Thread GitBox
chia7712 commented on a change in pull request #10389: URL: https://github.com/apache/kafka/pull/10389#discussion_r602393980 ## File path: core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala ## @@ -154,7 +160,10 @@ class ListOffsetsRequestTest extends

[jira] [Commented] (KAFKA-12563) Something wrong with MM2 metrics

2021-03-26 Thread Bui Thanh MInh (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309493#comment-17309493 ] Bui Thanh MInh commented on KAFKA-12563: It seems helpful, I will try, thanks. > Something

[jira] [Resolved] (KAFKA-12563) Something wrong with MM2 metrics

2021-03-26 Thread Bui Thanh MInh (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bui Thanh MInh resolved KAFKA-12563. Resolution: Resolved > Something wrong with MM2 metrics >

[jira] [Commented] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read.

2021-03-26 Thread kai.tian (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309487#comment-17309487 ] kai.tian commented on KAFKA-7870: -  Hi Tyran, when our borker had this exception, we found that the value

[GitHub] [kafka] ijuma commented on pull request #10414: MINOR: Self-managed -> Raft

2021-03-26 Thread GitBox
ijuma commented on pull request #10414: URL: https://github.com/apache/kafka/pull/10414#issuecomment-808245571 Thanks @rondagostino. I addressed the review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (KAFKA-12563) Something wrong with MM2 metrics

2021-03-26 Thread Ryanne Dolan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309447#comment-17309447 ] Ryanne Dolan commented on KAFKA-12563: -- You can look at the lag of the internal consumers to get an

[GitHub] [kafka] ijuma commented on pull request #10415: JDK 16 test

2021-03-26 Thread GitBox
ijuma commented on pull request #10415: URL: https://github.com/apache/kafka/pull/10415#issuecomment-808230478 @rajinisivaram we have one test failure (`testUnsupportedTlsVersion`) in the clients module with JDK 16: > Failed authentication with localhost/127.0.0.1 (SSL handshake

[GitHub] [kafka] ijuma opened a new pull request #10415: JDK 16 test

2021-03-26 Thread GitBox
ijuma opened a new pull request #10415: URL: https://github.com/apache/kafka/pull/10415 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes)

[GitHub] [kafka] rondagostino commented on a change in pull request #10414: MINOR: Self-managed -> Raft

2021-03-26 Thread GitBox
rondagostino commented on a change in pull request #10414: URL: https://github.com/apache/kafka/pull/10414#discussion_r602270763 ## File path: config/raft/README.md ## @@ -65,16 +68,19 @@ controllers, you can tolerate 1 failure; with 5 controllers, you can tolerate 2 ##

[GitHub] [kafka] ijuma commented on a change in pull request #10389: MINOR: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch

2021-03-26 Thread GitBox
ijuma commented on a change in pull request #10389: URL: https://github.com/apache/kafka/pull/10389#discussion_r602257805 ## File path: core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala ## @@ -154,7 +160,10 @@ class ListOffsetsRequestTest extends

[GitHub] [kafka] chia7712 commented on a change in pull request #10389: MINOR: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch

2021-03-26 Thread GitBox
chia7712 commented on a change in pull request #10389: URL: https://github.com/apache/kafka/pull/10389#discussion_r602237567 ## File path: core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala ## @@ -154,7 +160,10 @@ class ListOffsetsRequestTest extends

[GitHub] [kafka] ijuma opened a new pull request #10414: MINOR: Self-managed -> Raft

2021-03-26 Thread GitBox
ijuma opened a new pull request #10414: URL: https://github.com/apache/kafka/pull/10414 Self-managed is also used in the context of Cloud vs on-prem and it can be confusing. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ]

[GitHub] [kafka] ijuma commented on a change in pull request #10389: MINOR: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch

2021-03-26 Thread GitBox
ijuma commented on a change in pull request #10389: URL: https://github.com/apache/kafka/pull/10389#discussion_r602220116 ## File path: core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala ## @@ -154,7 +160,10 @@ class ListOffsetsRequestTest extends

[jira] [Commented] (KAFKA-12373) Improve KafkaRaftClient handling of graceful shutdown

2021-03-26 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309384#comment-17309384 ] Sagar Rao commented on KAFKA-12373: --- hi [~jsancio], I have started looking at this and had a question:

[GitHub] [kafka] chia7712 commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-03-26 Thread GitBox
chia7712 commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r602163446 ## File path: clients/src/test/java/org/apache/kafka/common/requests/JoinGroupRequestTest.java ## @@ -64,7 +65,7 @@ public void

[GitHub] [kafka] chia7712 commented on pull request #10389: MINOR: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch

2021-03-26 Thread GitBox
chia7712 commented on pull request #10389: URL: https://github.com/apache/kafka/pull/10389#issuecomment-808075755 unrelated failure. will merge trunk to trigger QA again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (KAFKA-10201) Update codebase to use more inclusive terms

2021-03-26 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309285#comment-17309285 ] Tom Bentley commented on KAFKA-10201: - [~xvrl] are you intending to come back to this work? >

[jira] [Commented] (KAFKA-12563) Something wrong with MM2 metrics

2021-03-26 Thread Bui Thanh MInh (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309281#comment-17309281 ] Bui Thanh MInh commented on KAFKA-12563: Thanks, but as I understand the meaning of this metric,

[jira] [Commented] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read.

2021-03-26 Thread Tyran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309279#comment-17309279 ] Tyran commented on KAFKA-7870: -- Hi Kai, we have set our open file limit to 10 and our brokers do not

[jira] [Commented] (KAFKA-12478) Consumer group may lose data for newly expanded partitions when add partitions for topic if the group is set to consume from the latest

2021-03-26 Thread hudeqi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309276#comment-17309276 ] hudeqi commented on KAFKA-12478: Ok, If I submit a patch, do you prefer to modify the logic on the

[jira] [Commented] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read.

2021-03-26 Thread kai.tian (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309275#comment-17309275 ] kai.tian commented on KAFKA-7870: -  If the exception occurs in the broker section, this KAFKA-7757

[jira] [Updated] (KAFKA-12564) KTable#filter-method called twice after aggregation

2021-03-26 Thread Jessica Johann (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jessica Johann updated KAFKA-12564: --- Description: Libraries from build.sbt: {{"org.apache.kafka" % "kafka_2.13" % "2.7.0",}}

[jira] [Updated] (KAFKA-12564) KTable#filter-method called twice after aggregation

2021-03-26 Thread Jessica Johann (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jessica Johann updated KAFKA-12564: --- Description: Libraries from build.sbt: {{"org.apache.kafka" % "kafka_2.13" % "2.7.0",}}

[jira] [Updated] (KAFKA-12564) KTable#filter-method called twice after aggregation

2021-03-26 Thread Jessica Johann (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jessica Johann updated KAFKA-12564: --- Description: Libraries from build.sbt: {{"org.apache.kafka" % "kafka_2.13" % "2.7.0",}}

[jira] [Updated] (KAFKA-12564) KTable#filter-method called twice after aggregation

2021-03-26 Thread Jessica Johann (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jessica Johann updated KAFKA-12564: --- Description: Libraries from build.sbt: {{"org.apache.kafka" % "kafka_2.13" % "2.7.0",}}

[jira] [Created] (KAFKA-12564) KTable#filter-method called twice after aggregation

2021-03-26 Thread Jessica Johann (Jira)
Jessica Johann created KAFKA-12564: -- Summary: KTable#filter-method called twice after aggregation Key: KAFKA-12564 URL: https://issues.apache.org/jira/browse/KAFKA-12564 Project: Kafka

[jira] [Assigned] (KAFKA-12519) Consider Removing Streams Old Built-in Metrics Version

2021-03-26 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reassigned KAFKA-12519: - Assignee: Bruno Cadonna > Consider Removing Streams Old Built-in Metrics Version >

[jira] [Updated] (KAFKA-12519) Consider Removing Streams Old Built-in Metrics Version

2021-03-26 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-12519: -- Labels: needs-kip (was: ) > Consider Removing Streams Old Built-in Metrics Version >

[GitHub] [kafka] showuon commented on pull request #10413: MINOR: fix the wrong/missing anchor caused link error

2021-03-26 Thread GitBox
showuon commented on pull request #10413: URL: https://github.com/apache/kafka/pull/10413#issuecomment-808030560 @bbejeck @mjsax , could you help review this PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] showuon opened a new pull request #10413: MINOR: fix the wrong/missing anchor caused link error

2021-03-26 Thread GitBox
showuon opened a new pull request #10413: URL: https://github.com/apache/kafka/pull/10413 Finding this link issue when reading the streaming doc: https://kafka.apache.org/27/documentation/streams/developer-guide/config-streams.html#optional-configuration-parameters 1.

[GitHub] [kafka] kowshik commented on pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on pull request #10280: URL: https://github.com/apache/kafka/pull/10280#issuecomment-808012495 @junrao Thanks a lot for the review! I've addressed your comments in 63be325b8b3ff5514e22137bb02dfc70e10f7a72. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602062784 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1816,8 +1292,12 @@ class Log(@volatile private var _dir: File, */ private def

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602071574 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -836,178 +581,15 @@ class Log(@volatile private var _dir: File,

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602052452 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602062784 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1816,8 +1292,12 @@ class Log(@volatile private var _dir: File, */ private def

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602062784 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1816,8 +1292,12 @@ class Log(@volatile private var _dir: File, */ private def

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602056487 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1794,20 +1269,21 @@ class Log(@volatile private var _dir: File, ret.toSeq.sortBy(-_)

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602056301 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -213,32 +208,38 @@ case object SnapshotGenerated extends LogStartOffsetIncrementReason { }

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602056245 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -213,32 +208,38 @@ case object SnapshotGenerated extends LogStartOffsetIncrementReason { }

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602055675 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602055217 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602055217 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602052452 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] kowshik commented on a change in pull request #10280: KIP-405: Log layer refactor

2021-03-26 Thread GitBox
kowshik commented on a change in pull request #10280: URL: https://github.com/apache/kafka/pull/10280#discussion_r602052393 ## File path: core/src/main/scala/kafka/log/LocalLog.scala ## @@ -0,0 +1,1561 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

  1   2   >