[GitHub] [kafka] gitlw commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-12 Thread GitBox
gitlw commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r849092831 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2615,7 +2615,7 @@ class ReplicaManagerTest { @Test def

[GitHub] [kafka] gitlw commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-12 Thread GitBox
gitlw commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r849092616 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -687,8 +687,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,

[GitHub] [kafka] gitlw commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-12 Thread GitBox
gitlw commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r849092133 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -687,8 +687,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,

[GitHub] [kafka] gitlw commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-12 Thread GitBox
gitlw commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r849091999 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -678,7 +678,7 @@ class UnifiedLog(@volatile var logStartOffset: Long, * * @throws KafkaStorageException

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-12 Thread GitBox
RivenSun2 commented on code in PR #12041: URL: https://github.com/apache/kafka/pull/12041#discussion_r849062185 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -774,8 +774,12 @@ public KafkaConsumer(Map configs, );

[jira] [Created] (KAFKA-13827) Data Loss on moving to KRAFT mode from v3.1

2022-04-12 Thread Sharad Garg (Jira)
Sharad Garg created KAFKA-13827: --- Summary: Data Loss on moving to KRAFT mode from v3.1 Key: KAFKA-13827 URL: https://issues.apache.org/jira/browse/KAFKA-13827 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-13826) Unable to migrate to KRAFT mode from v2.8

2022-04-12 Thread Sharad Garg (Jira)
Sharad Garg created KAFKA-13826: --- Summary: Unable to migrate to KRAFT mode from v2.8 Key: KAFKA-13826 URL: https://issues.apache.org/jira/browse/KAFKA-13826 Project: Kafka Issue Type: Bug

[GitHub] [kafka] RivenSun2 commented on pull request #11985: MINOR: Supplement the description of `Valid Values` in the documentation of `compression.type`

2022-04-12 Thread GitBox
RivenSun2 commented on PR #11985: URL: https://github.com/apache/kafka/pull/11985#issuecomment-1097543104 Hi @guozhangwang Thank you very much. I would appreciate it if you had time to help review KIP-828. I have listed all configurations missing validators in this KIP. Thanks --

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-12 Thread GitBox
guozhangwang commented on code in PR #12041: URL: https://github.com/apache/kafka/pull/12041#discussion_r849060150 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -774,8 +774,12 @@ public KafkaConsumer(Map configs, );

[GitHub] [kafka] guozhangwang merged pull request #11985: MINOR: Supplement the description of `Valid Values` in the documentation of `compression.type`

2022-04-12 Thread GitBox
guozhangwang merged PR #11985: URL: https://github.com/apache/kafka/pull/11985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] RivenSun2 commented on pull request #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-12 Thread GitBox
RivenSun2 commented on PR #12041: URL: https://github.com/apache/kafka/pull/12041#issuecomment-1097540003 Hi @guozhangwang @showuon please help to review PR when available. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] RivenSun2 opened a new pull request, #12041: MINOR: ignore unused configuration when ConsumerCoordinator is not constructed

2022-04-12 Thread GitBox
RivenSun2 opened a new pull request, #12041: URL: https://github.com/apache/kafka/pull/12041 Following PR #11940, ignore unused config when ConsumerCoordinator is not constructed ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [

[GitHub] [kafka] C0urante commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
C0urante commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r849033605 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceConnector.java: ## @@ -28,4 +30,46 @@ public abstract class SourceConnector extends Connector {

[jira] [Commented] (KAFKA-13816) Downgrading Connect rebalancing protocol from incremental to eager causes duplicate task instances

2022-04-12 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521429#comment-17521429 ] Chris Egerton commented on KAFKA-13816: --- Rough draft of a fix (no unit testing):

[GitHub] [kafka] C0urante commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
C0urante commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r849027063 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceConnector.java: ## @@ -28,4 +30,46 @@ protected SourceConnectorContext context() {

[GitHub] [kafka] C0urante commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
C0urante commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r849027063 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceConnector.java: ## @@ -28,4 +30,46 @@ protected SourceConnectorContext context() {

[GitHub] [kafka] C0urante commented on pull request #11914: MINOR: Correct Connect docs on connector/task states

2022-04-12 Thread GitBox
C0urante commented on PR #11914: URL: https://github.com/apache/kafka/pull/11914#issuecomment-1097493961 Good point, I've removed mention of the `DESTROYED` state from that section as well. I've also updated both to include the newly-introduced `RESTARTING` state since it was

[GitHub] [kafka] C0urante commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-12 Thread GitBox
C0urante commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r849010637 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long

[GitHub] [kafka] jsancio commented on pull request #12040: MINOR: Verify stopReplica if broker epoch not stale

2022-04-12 Thread GitBox
jsancio commented on PR #12040: URL: https://github.com/apache/kafka/pull/12040#issuecomment-1097470092 Hi @mimaison are you able to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] jsancio opened a new pull request, #12040: MINOR: Verify stopReplica if broker epoch not stale

2022-04-12 Thread GitBox
jsancio opened a new pull request, #12040: URL: https://github.com/apache/kafka/pull/12040 Verify that ReplicaManager.stopReplica is called if the stop replica request doesn't result in a stale broker epoch error. ### Committer Checklist (excluded from commit message) - [ ]

[GitHub] [kafka] RivenSun2 commented on pull request #11985: MINOR: Supplement the description of `Valid Values` in the documentation of `compression.type`

2022-04-12 Thread GitBox
RivenSun2 commented on PR #11985: URL: https://github.com/apache/kafka/pull/11985#issuecomment-1097458348 Hi @mimaison Please help to review when you are free. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] RivenSun2 commented on a diff in pull request #11985: MINOR: Supplement the description of `Valid Values` in the documentation of `compression.type`

2022-04-12 Thread GitBox
RivenSun2 commented on code in PR #11985: URL: https://github.com/apache/kafka/pull/11985#discussion_r848984047 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -329,7 +330,7 @@ in("all", "-1", "0",

[jira] [Closed] (KAFKA-13825) Fix some topic metric should remove when from leader to follower(deprecated)

2022-04-12 Thread zhaobo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaobo closed KAFKA-13825. -- > Fix some topic metric should remove when from leader to follower(deprecated) >

[jira] [Resolved] (KAFKA-13825) Fix some topic metric should remove when from leader to follower(deprecated)

2022-04-12 Thread zhaobo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaobo resolved KAFKA-13825. Resolution: Not A Problem > Fix some topic metric should remove when from leader to follower(deprecated)

[jira] [Updated] (KAFKA-13825) Fix some topic metric should remove when from leader to follower(deprecated)

2022-04-12 Thread zhaobo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaobo updated KAFKA-13825: --- Summary: Fix some topic metric should remove when from leader to follower(deprecated) (was: Fix some topic

[jira] [Updated] (KAFKA-13825) Fix some topic metric should remove when from leader to follower

2022-04-12 Thread zhaobo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaobo updated KAFKA-13825: --- Issue Type: Test (was: Bug) > Fix some topic metric should remove when from leader to follower >

[jira] [Created] (KAFKA-13825) Fix some topic metric should remove when from leader to follower

2022-04-12 Thread zhaobo (Jira)
zhaobo created KAFKA-13825: -- Summary: Fix some topic metric should remove when from leader to follower Key: KAFKA-13825 URL: https://issues.apache.org/jira/browse/KAFKA-13825 Project: Kafka Issue

[GitHub] [kafka] kirktrue opened a new pull request, #12039: WIP

2022-04-12 Thread GitBox
kirktrue opened a new pull request, #12039: URL: https://github.com/apache/kafka/pull/12039 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] lihaosky commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
lihaosky commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848952472 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[GitHub] [kafka] philipnee opened a new pull request, #12038: [WIP] KAFKA-13421

2022-04-12 Thread GitBox
philipnee opened a new pull request, #12038: URL: https://github.com/apache/kafka/pull/12038 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[jira] [Created] (KAFKA-13824) Pass time object from constructor so that we can mock it if needed

2022-04-12 Thread Hao Li (Jira)
Hao Li created KAFKA-13824: -- Summary: Pass time object from constructor so that we can mock it if needed Key: KAFKA-13824 URL: https://issues.apache.org/jira/browse/KAFKA-13824 Project: Kafka

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848934177 ## streams/src/test/java/org/apache/kafka/streams/integration/TimeWindowedKStreamIntegrationTest.java: ## @@ -0,0 +1,497 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r845622207 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -184,6 +247,75 @@ public void process(final Record record) {

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848932512 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -184,6 +247,75 @@ public void process(final Record record) {

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848931827 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Labels: beginner newbie (was: ) > Schedule nextTimeToEmit to system time every time

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Component/s: streams > Schedule nextTimeToEmit to system time every time instead of just

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848930635 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848930028 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[GitHub] [kafka] lihaosky opened a new pull request, #12037: KAFKA-13785: [6/N][Emit final] emit final for sliding window

2022-04-12 Thread GitBox
lihaosky opened a new pull request, #12037: URL: https://github.com/apache/kafka/pull/12037 ### Description Implementation to emit final for sliding window agg. This PR is on top of https://github.com/apache/kafka/pull/11896 ### Testing pending unit test and integration test

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848930028 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[GitHub] [kafka] mjsax commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-12 Thread GitBox
mjsax commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r848929570 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/TimeWindowedKStreamImpl.java: ## @@ -232,11 +247,19 @@ );

[GitHub] [kafka] mjsax commented on a diff in pull request #12030: KAFKA-13785: [5/N][emit final] cache for time ordered window store

2022-04-12 Thread GitBox
mjsax commented on code in PR #12030: URL: https://github.com/apache/kafka/pull/12030#discussion_r848927151 ## streams/src/main/java/org/apache/kafka/streams/state/internals/TimeOrderedCachingWindowStore.java: ## @@ -0,0 +1,694 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848914833 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -104,6 +116,10 @@ private static Logger createLogger(int nodeId) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848914196 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -351,30 +429,55 @@ void checkSection(Action action, private

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848912193 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -104,6 +116,10 @@ private static Logger createLogger(int nodeId) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848912193 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -104,6 +116,10 @@ private static Logger createLogger(int nodeId) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848910920 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAcl.java: ## @@ -96,6 +97,13 @@ public String principal() { return principal; } +

[GitHub] [kafka] cmccabe commented on a diff in pull request #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12031: URL: https://github.com/apache/kafka/pull/12031#discussion_r848909874 ## clients/src/main/java/org/apache/kafka/common/resource/ResourcePattern.java: ## @@ -89,7 +89,7 @@ public ResourcePatternFilter toFilter() { @Override

[GitHub] [kafka] cmccabe commented on pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on PR #12036: URL: https://github.com/apache/kafka/pull/12036#issuecomment-1097274697 looks like there is a build error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848908269 ## core/src/main/scala/kafka/server/BrokerServer.scala: ## @@ -153,6 +150,8 @@ class BrokerServer( var metadataPublisher: BrokerMetadataPublisher = null + var

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848905774 ## clients/src/main/java/org/apache/kafka/clients/admin/FeatureUpdate.java: ## @@ -23,33 +23,92 @@ */ public class FeatureUpdate { private final short

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848905064 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -25,109 +25,138 @@ import java.util.Map.Entry; import java.util.Map; import

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848902316 ## metadata/src/main/java/org/apache/kafka/controller/ProducerIdControlManager.java: ## @@ -34,10 +35,15 @@ public class ProducerIdControlManager { private final

[GitHub] [kafka] kowshik commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access

2022-04-12 Thread GitBox
kowshik commented on code in PR #12005: URL: https://github.com/apache/kafka/pull/12005#discussion_r848807096 ## core/src/main/scala/kafka/server/LeaderEndPoint.scala: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848899851 ## clients/src/main/java/org/apache/kafka/clients/admin/UpdateFeaturesOptions.java: ## @@ -26,4 +26,14 @@ */ @InterfaceStability.Evolving public class

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848898186 ## clients/src/main/java/org/apache/kafka/clients/admin/FeatureUpdate.java: ## @@ -23,33 +23,92 @@ */ public class FeatureUpdate { private final short

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848895075 ## clients/src/main/java/org/apache/kafka/clients/admin/FeatureUpdate.java: ## @@ -23,33 +23,92 @@ */ public class FeatureUpdate { private final short

[GitHub] [kafka] cmccabe commented on a diff in pull request #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
cmccabe commented on code in PR #12036: URL: https://github.com/apache/kafka/pull/12036#discussion_r848894276 ## clients/src/main/java/org/apache/kafka/clients/admin/FeatureUpdate.java: ## @@ -23,33 +23,92 @@ */ public class FeatureUpdate { private final short

[GitHub] [kafka] junrao commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-12 Thread GitBox
junrao commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r848692773 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -687,8 +687,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,

[GitHub] [kafka] rittikaadhikari commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access

2022-04-12 Thread GitBox
rittikaadhikari commented on code in PR #12005: URL: https://github.com/apache/kafka/pull/12005#discussion_r848840665 ## core/src/main/scala/kafka/server/LeaderEndPoint.scala: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[GitHub] [kafka] mumrah commented on pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-04-12 Thread GitBox
mumrah commented on PR #11677: URL: https://github.com/apache/kafka/pull/11677#issuecomment-1097159701 The feature flags part of this was split out into https://github.com/apache/kafka/pull/12036/files -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] mumrah opened a new pull request, #12036: KAFKA-13823 Feature flag changes from KIP-778

2022-04-12 Thread GitBox
mumrah opened a new pull request, #12036: URL: https://github.com/apache/kafka/pull/12036 This PR includes the changes to feature flags that were outlined in KIP-778. ## Single finalized version level The UpdateFeatures RPC and the FeatureLevelRecord were updated to remove the

[jira] [Created] (KAFKA-13823) Remove "max" version level from finalized features

2022-04-12 Thread David Arthur (Jira)
David Arthur created KAFKA-13823: Summary: Remove "max" version level from finalized features Key: KAFKA-13823 URL: https://issues.apache.org/jira/browse/KAFKA-13823 Project: Kafka Issue

[GitHub] [kafka] mumrah commented on pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-04-12 Thread GitBox
mumrah commented on PR #11677: URL: https://github.com/apache/kafka/pull/11677#issuecomment-1097150125 I've split out the feature flags work into https://github.com/mumrah/kafka/pull/new/KAFKA-13410-feature-flag-changes -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] kowshik commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access

2022-04-12 Thread GitBox
kowshik commented on code in PR #12005: URL: https://github.com/apache/kafka/pull/12005#discussion_r848806569 ## core/src/main/scala/kafka/server/LeaderEndPoint.scala: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] [kafka] kowshik commented on a diff in pull request #12005: KAFKA-13803: Refactor Leader API Access

2022-04-12 Thread GitBox
kowshik commented on code in PR #12005: URL: https://github.com/apache/kafka/pull/12005#discussion_r848803367 ## core/src/main/scala/kafka/server/LeaderEndPoint.scala: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] [kafka] dajac commented on pull request #11916: KAFKA-12703; Allow unencrypted private keys when using PEM files

2022-04-12 Thread GitBox
dajac commented on PR #11916: URL: https://github.com/apache/kafka/pull/11916#issuecomment-1097108920 I will get to this one soon. Sorry for the delay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Assigned] (KAFKA-13421) Fix ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

2022-04-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee reassigned KAFKA-13421: -- Assignee: Philip Nee > Fix >

[GitHub] [kafka] jeqo commented on pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on PR #11993: URL: https://github.com/apache/kafka/pull/11993#issuecomment-1097081889 Thank you, @vvcephei !! I have applied most suggestions, let me know your thoughts  -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] lihaosky commented on a diff in pull request #12030: KAFKA-13785: [5/N][emit final] cache for time ordered window store

2022-04-12 Thread GitBox
lihaosky commented on code in PR #12030: URL: https://github.com/apache/kafka/pull/12030#discussion_r848733142 ## streams/src/main/java/org/apache/kafka/streams/state/internals/TimeOrderedCachingWindowStore.java: ## @@ -0,0 +1,694 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cadonna commented on pull request #12018: KAFKA-13542: add rebalance reason in Kafka Streams

2022-04-12 Thread GitBox
cadonna commented on PR #12018: URL: https://github.com/apache/kafka/pull/12018#issuecomment-1097030238 @lihaosky you might be right about the flakiness. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] cadonna commented on pull request #12018: KAFKA-13542: add rebalance reason in Kafka Streams

2022-04-12 Thread GitBox
cadonna commented on PR #12018: URL: https://github.com/apache/kafka/pull/12018#issuecomment-1097025545 I retriggered the build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] lihaosky commented on pull request #12018: KAFKA-13542: add rebalance reason in Kafka Streams

2022-04-12 Thread GitBox
lihaosky commented on PR #12018: URL: https://github.com/apache/kafka/pull/12018#issuecomment-1097024817 @dajac , can you re-trigger the test run? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] lihaosky commented on pull request #12018: KAFKA-13542: add rebalance reason in Kafka Streams

2022-04-12 Thread GitBox
lihaosky commented on PR #12018: URL: https://github.com/apache/kafka/pull/12018#issuecomment-1097023687 I ran it 3 times locally with JDK 17 and scala 2.13 and it didn't fail. Maybe flaky? Not sure if related to

[jira] [Created] (KAFKA-13822) Update Kafka Streams Adjust Thread Count tests to new Processor API

2022-04-12 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-13822: Summary: Update Kafka Streams Adjust Thread Count tests to new Processor API Key: KAFKA-13822 URL:

[GitHub] [kafka] jeqo commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r848698083 ## streams/src/test/java/org/apache/kafka/streams/integration/AdjustStreamThreadCountTest.java: ## @@ -421,6 +421,7 @@ public void

[jira] [Updated] (KAFKA-13821) Update Kafka Streams WordCount demo to new Processor API

2022-04-12 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-13821: - Priority: Minor (was: Major) > Update Kafka Streams WordCount

[jira] [Updated] (KAFKA-13821) Update Kafka Streams WordCount demo to new Processor API

2022-04-12 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-13821: - Summary: Update Kafka Streams WordCount demo to new Processor

[jira] [Updated] (KAFKA-13821) Update Kafka Streams demo to new Processor API

2022-04-12 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-13821: - Description: Once KIP-820 is merged and release, WordCount[1]

[jira] [Assigned] (KAFKA-13790) ReplicaManager should be robust to all partition updates from kraft metadata log

2022-04-12 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-13790: --- Assignee: David Jacot (was: Jason Gustafson) > ReplicaManager should be robust to

[GitHub] [kafka] jeqo commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r848669558 ## streams/test-utils/src/main/java/org/apache/kafka/streams/processor/api/MockProcessorContext.java: ## @@ -336,6 +344,14 @@ public void setRecordMetadata(final String

[jira] [Commented] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521243#comment-17521243 ] Matthias J. Sax commented on KAFKA-13813: - The change is intentionally, as users wanted to get

[GitHub] [kafka] jeqo commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r848653709 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/ProcessorParameters.java: ## @@ -37,30 +43,67 @@ @SuppressWarnings("deprecation") // Old

[GitHub] [kafka] jeqo commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r848644551 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java: ## @@ -1501,9 +1508,60 @@ public void process(final

[jira] [Created] (KAFKA-13821) Update Kafka Streams demo to new Processor API

2022-04-12 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-13821: Summary: Update Kafka Streams demo to new Processor API Key: KAFKA-13821 URL: https://issues.apache.org/jira/browse/KAFKA-13821

[GitHub] [kafka] jeqo commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-12 Thread GitBox
jeqo commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r848641109 ## streams/examples/src/main/java/org/apache/kafka/streams/examples/wordcount/WordCountTransformerDemo.java: ## @@ -119,6 +119,7 @@ public Set> stores() { }

[jira] [Commented] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521234#comment-17521234 ] Matthias J. Sax commented on KAFKA-13820: - Sounds related to

[GitHub] [kafka] mimaison commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
mimaison commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r848581552 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceConnector.java: ## @@ -28,4 +30,46 @@ public abstract class SourceConnector extends Connector {

[GitHub] [kafka] mimaison commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
mimaison commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r848580141 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceConnector.java: ## @@ -28,4 +30,46 @@ protected SourceConnectorContext context() {

[GitHub] [kafka] mimaison commented on a diff in pull request #11773: KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618)

2022-04-12 Thread GitBox
mimaison commented on code in PR #11773: URL: https://github.com/apache/kafka/pull/11773#discussion_r848576922 ## connect/api/src/main/java/org/apache/kafka/connect/source/SourceTask.java: ## @@ -16,17 +16,63 @@ */ package org.apache.kafka.connect.source; -import

[GitHub] [kafka] mimaison commented on a diff in pull request #11985: MINOR: Supplement the description of `Valid Values` in the documentation of `compression.type`

2022-04-12 Thread GitBox
mimaison commented on code in PR #11985: URL: https://github.com/apache/kafka/pull/11985#discussion_r848556316 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -329,7 +330,7 @@ in("all", "-1", "0",

[GitHub] [kafka] sayantanu-dey opened a new pull request, #12035: KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so

2022-04-12 Thread GitBox
sayantanu-dey opened a new pull request, #12035: URL: https://github.com/apache/kafka/pull/12035 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.*

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-12 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long

[GitHub] [kafka] mimaison commented on pull request #11914: MINOR: Correct Connect docs on connector/task states

2022-04-12 Thread GitBox
mimaison commented on PR #11914: URL: https://github.com/apache/kafka/pull/11914#issuecomment-1096782440 We should also update the [Connect metrics](https://kafka.apache.org/documentation/#connect_monitoring) section that mentions `DESTROYED` as a possible state. -- This is an automated

[GitHub] [kafka] divijvaidya opened a new pull request, #12034: Kafka-12319: Improve rate calculation for first sample window

2022-04-12 Thread GitBox
divijvaidya opened a new pull request, #12034: URL: https://github.com/apache/kafka/pull/12034 ## Background Apache Kafka's rate-limiting algorithm uses a variation of fixed window algorithm where the duration of time window is controlled by two configurations:

[GitHub] [kafka] github-olivier-abdesselam commented on pull request #10367: KAFKA-12495: allow consecutive revoke in incremental cooperative assignor in connector

2022-04-12 Thread GitBox
github-olivier-abdesselam commented on PR #10367: URL: https://github.com/apache/kafka/pull/10367#issuecomment-1096645368 > @showuon @C0urante @kkonstantine What is the status of this PR? As far as I understand this PR might resolve

[GitHub] [kafka] dajac commented on pull request #12018: KAFKA-13542: add rebalance reason in Kafka Streams

2022-04-12 Thread GitBox
dajac commented on PR #12018: URL: https://github.com/apache/kafka/pull/12018#issuecomment-1096632657 @lihaosky Could you verify if that failed test is a concern? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] dajac commented on pull request #12027: MINOR: Fix TestDowngrade.test_upgrade_and_downgrade

2022-04-12 Thread GitBox
dajac commented on PR #12027: URL: https://github.com/apache/kafka/pull/12027#issuecomment-1096608283 Successful run:

[jira] [Updated] (KAFKA-13804) Log broker shutdown reason during startup at the end of log output

2022-04-12 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-13804: -- Description: Currently, when the broker hit exception during startup, we will first log the

  1   2   >