[jira] [Created] (KAFKA-15893) Bump MetadataVersion for directory assignments

2023-11-24 Thread Igor Soarez (Jira)
Igor Soarez created KAFKA-15893: --- Summary: Bump MetadataVersion for directory assignments Key: KAFKA-15893 URL: https://issues.apache.org/jira/browse/KAFKA-15893 Project: Kafka Issue Type:

Re: [PR] KAFKA-15143: Adding in MockFixedKeyProcessorContext and Test [kafka]

2023-11-24 Thread via GitHub
s7pandey commented on code in PR #14605: URL: https://github.com/apache/kafka/pull/14605#discussion_r1404742800 ## streams/test-utils/src/test/java/org/apache/kafka/streams/test/MockFixedKeyProcessorContextTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software

Re: [PR] MINOR: Add retry for CI [kafka]

2023-11-24 Thread via GitHub
ex172000 commented on PR #14828: URL: https://github.com/apache/kafka/pull/14828#issuecomment-1826221078 > Thanks for the PR. Note that The CI configuration already sets retries, we don't need a CI server configuration. Is the CI retry a global setting? It might make more sense if we

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404635960 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -36,14 +36,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest

Re: [PR] KAFKA-15831: KIP-1000 protocol and admin client [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14811: URL: https://github.com/apache/kafka/pull/14811#discussion_r1404634302 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3759,6 +3760,21 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] KAFKA-15831: KIP-1000 protocol and admin client [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14811: URL: https://github.com/apache/kafka/pull/14811#discussion_r1404634074 ## clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java: ## @@ -7090,6 +7093,51 @@ private static MemberDescription

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-11-24 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1404613145 ## streams/src/main/java/org/apache/kafka/streams/state/internals/LogicalKeyValueSegment.java: ## @@ -170,7 +173,30 @@ public boolean isOpen() { @Override

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-11-24 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1404613145 ## streams/src/main/java/org/apache/kafka/streams/state/internals/LogicalKeyValueSegment.java: ## @@ -170,7 +173,30 @@ public boolean isOpen() { @Override

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-11-24 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1404610275 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -263,6 +266,78 @@ public VersionedRecord get(final Bytes key,

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-24 Thread via GitHub
apoorvmittal10 commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1404599341 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerTest.java: ## @@ -66,6 +69,28 @@ public

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-11-24 Thread via GitHub
kamalcph commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1404590243 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1754,6 +1754,7 @@ class UnifiedLog(@volatile var logStartOffset: Long,

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-11-24 Thread via GitHub
kamalcph commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1404591766 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1754,6 +1754,7 @@ class UnifiedLog(@volatile var logStartOffset: Long,

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-11-24 Thread via GitHub
kamalcph commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1404590243 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1754,6 +1754,7 @@ class UnifiedLog(@volatile var logStartOffset: Long,

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-11-24 Thread via GitHub
kamalcph commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1404590243 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1754,6 +1754,7 @@ class UnifiedLog(@volatile var logStartOffset: Long,

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-24 Thread via GitHub
scholzj commented on PR #14756: URL: https://github.com/apache/kafka/pull/14756#issuecomment-1825984878 Thanks a lot for your help with this @mimaison and @viktorsomogyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Resolved] (KAFKA-15464) Allow dynamic reloading of certificates with different DN / SANs

2023-11-24 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-15464. Fix Version/s: 3.7.0 Resolution: Fixed > Allow dynamic reloading of certificates

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-24 Thread via GitHub
kamalcph commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1404579784 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerTest.java: ## @@ -66,6 +69,28 @@ public

[PR] MINOR: Fix the flaky TBRLMM `testInternalTopicExists` test [kafka]

2023-11-24 Thread via GitHub
kamalcph opened a new pull request, #14840: URL: https://github.com/apache/kafka/pull/14840 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404560191 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -36,14 +36,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest

[jira] [Created] (KAFKA-15892) Flaky test: testAlterSinkConnectorOffsets – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest

2023-11-24 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15892: - Summary: Flaky test: testAlterSinkConnectorOffsets – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest Key: KAFKA-15892 URL:

[jira] [Created] (KAFKA-15891) Flaky test: testResetSinkConnectorOffsetsOverriddenConsumerGroupId – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest

2023-11-24 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15891: - Summary: Flaky test: testResetSinkConnectorOffsetsOverriddenConsumerGroupId – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest Key: KAFKA-15891 URL:

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-24 Thread via GitHub
apoorvmittal10 commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1404552551 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerTest.java: ## @@ -66,6 +69,28 @@ public

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-11-24 Thread via GitHub
apoorvmittal10 commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1825923669 New failing - 12 Build / JDK 11 and Scala 2.13 / testAlterSinkConnectorOffsets – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest 12s Build / JDK 11 and

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
dajac commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404537279 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -36,14 +36,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest extends

Re: [PR] KAFKA-14412: Add ProcessingThread tag interface [kafka]

2023-11-24 Thread via GitHub
nicktelford commented on PR #14839: URL: https://github.com/apache/kafka/pull/14839#issuecomment-1825885812 This is part of KIP-892, and has been broken out into a separate PR to reduce the review burden on the main KIP-892 implementation, since it can be merged independently. Note:

[PR] KAFKA-14412: Add ProcessingThread tag interface [kafka]

2023-11-24 Thread via GitHub
nicktelford opened a new pull request, #14839: URL: https://github.com/apache/kafka/pull/14839 This interface provides a common supertype for `StreamThread` and `DefaultTaskExecutor.TaskExecutorThread`, which will be used by KIP-892 to differentiate between "processing" threads and

Re: [PR] KAFKA-15361: Process and persist dir info with broker registration [kafka]

2023-11-24 Thread via GitHub
soarez commented on PR #14838: URL: https://github.com/apache/kafka/pull/14838#issuecomment-1825859013 @cmccabe @pprovenzano @rondagostino PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15361: Process and persist dir info with broker registration [kafka]

2023-11-24 Thread via GitHub
soarez commented on PR #14838: URL: https://github.com/apache/kafka/pull/14838#issuecomment-1825858721 This builds on #14820 - KAFKA-15886: Always specify directories for new partition registrations – so this is marked as draft until #14820 is merged. **Reviews**: please focus on

[PR] KAFKA-15361: Process and persist dir info with broker registration [kafka]

2023-11-24 Thread via GitHub
soarez opened a new pull request, #14838: URL: https://github.com/apache/kafka/pull/14838 Controllers should process and persist directory information from the broker registration request ### Committer Checklist (excluded from commit message) - [ ] Verify design and

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-24 Thread via GitHub
viktorsomogyi merged PR #14756: URL: https://github.com/apache/kafka/pull/14756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-24 Thread via GitHub
viktorsomogyi commented on PR #14756: URL: https://github.com/apache/kafka/pull/14756#issuecomment-1825833128 There were failing unit tests but rerunning them locally from the PR they all passed, therefore they're likely unrelated flakies. Merging the PR, thanks @scholzj for the

Re: [PR] KAFKA-15801: improve Kafka broker/NetworkClient logging for connectiv… [kafka]

2023-11-24 Thread via GitHub
ijuma commented on code in PR #14799: URL: https://github.com/apache/kafka/pull/14799#discussion_r1404471013 ## clients/src/main/java/org/apache/kafka/clients/NetworkClient.java: ## @@ -855,10 +855,11 @@ private void handleTimedOutConnections(List responses, long now)

Re: [PR] MINOR: Add retry for CI [kafka]

2023-11-24 Thread via GitHub
ijuma commented on PR #14828: URL: https://github.com/apache/kafka/pull/14828#issuecomment-1825825013 Thanks for the PR. Note that The CI configuration already sets retries, we don't need a CI server configuration. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-24 Thread via GitHub
apoorvmittal10 commented on code in PR #14770: URL: https://github.com/apache/kafka/pull/14770#discussion_r1404379306 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -207,34 +207,35 @@ class BrokerLifecycleManagerTest { } @Test - def

Re: [PR] KAFKA-15154: Acquire lock when reading checkQuotas [kafka]

2023-11-24 Thread via GitHub
mstepan commented on PR #13969: URL: https://github.com/apache/kafka/pull/13969#issuecomment-1825678364 I would suggest using StampedLock instead of just fully synchronized keyword here. Something like https://github.com/apache/kafka/pull/14837/commits -- This is an automated

[jira] [Commented] (KAFKA-15154) Potential bug: We don't acquire lock when reading checkQuotas

2023-11-24 Thread Maksym Stepanenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789452#comment-17789452 ] Maksym Stepanenko commented on KAFKA-15154: --- I believe instead of just adding a

[PR] KAFKA-15154: proper StampedLock synchronization added [kafka]

2023-11-24 Thread via GitHub
mstepan opened a new pull request, #14837: URL: https://github.com/apache/kafka/pull/14837 All read and write access to Sensor metrics field should be properly synchronized according to JMM. The best way to separate read and write is to use ReadWriteLock or more advanced version of

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-24 Thread via GitHub
soarez commented on code in PR #14770: URL: https://github.com/apache/kafka/pull/14770#discussion_r1404314260 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -207,34 +207,35 @@ class BrokerLifecycleManagerTest { } @Test - def

Re: [PR] MINOR: Fix flaky BrokerLifecycleManagerTest [kafka]

2023-11-24 Thread via GitHub
soarez commented on PR #14836: URL: https://github.com/apache/kafka/pull/14836#issuecomment-1825622434 Flakiness detected since #14770 cc @apoorvmittal10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] MINOR: Fix flaky BrokerLifecycleManagerTest [kafka]

2023-11-24 Thread via GitHub
soarez opened a new pull request, #14836: URL: https://github.com/apache/kafka/pull/14836 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-24 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1404286606 ## clients/src/main/java/org/apache/kafka/common/protocol/Errors.java: ## @@ -386,7 +388,9 @@ public enum Errors { STALE_MEMBER_EPOCH(113, "The member

[PR] KAFKA-15890: Consumer.poll with long timeout unaware of assigned partitions [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield opened a new pull request, #14835: URL: https://github.com/apache/kafka/pull/14835 In the new consumer, `Consumer.poll(Duration timeout)` blocks for the entire duration. If the consumer is joining a group and has not yet received its assignments, the poll begins before an

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1404253943 ## clients/src/main/java/org/apache/kafka/common/protocol/Errors.java: ## @@ -386,7 +388,9 @@ public enum Errors { STALE_MEMBER_EPOCH(113, "The member

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404233938 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -111,6 +115,40 @@ abstract class BaseConsumerTest extends AbstractConsumerTest {

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
dajac commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404226250 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -111,6 +115,40 @@ abstract class BaseConsumerTest extends AbstractConsumerTest { } object

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on PR #14758: URL: https://github.com/apache/kafka/pull/14758#issuecomment-1825499807 This PR now reflects the changes in KAFKA-14781 and also tests the new consumer. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404213789 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -1947,4 +2082,215 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404208505 ## core/src/test/scala/kafka/utils/TestInfoUtils.scala: ## @@ -39,6 +39,12 @@ object TestInfoUtils { } else { throw new

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1404207856 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -34,13 +35,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-11-24 Thread via GitHub
dajac commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1404037483 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -691,6 +708,10 @@ private void

Re: [PR] KAFKA-15803: Update leader epoch in commitAsync and committed [kafka]

2023-11-24 Thread via GitHub
lucasbru merged PR #14817: URL: https://github.com/apache/kafka/pull/14817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15803: Update leader epoch in commitAsync and committed [kafka]

2023-11-24 Thread via GitHub
lucasbru commented on PR #14817: URL: https://github.com/apache/kafka/pull/14817#issuecomment-1825403240 Test failures unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-13531: Disable flaky NamedTopologyIntegrationTest [kafka]

2023-11-24 Thread via GitHub
lucasbru commented on PR #14830: URL: https://github.com/apache/kafka/pull/14830#issuecomment-1825399087 Test failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-13531: Disable flaky NamedTopologyIntegrationTest [kafka]

2023-11-24 Thread via GitHub
lucasbru merged PR #14830: URL: https://github.com/apache/kafka/pull/14830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (KAFKA-15817) Avoid reconnecting to the same IP address if multiple addresses are available

2023-11-24 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-15817: --- Fix Version/s: (was: 3.6.1) > Avoid reconnecting to the same IP address if multiple