[PR] MINOR: Upgrade Zstd-jni to 1.5.5-9 [kafka]

2023-11-18 Thread via GitHub
divijvaidya opened a new pull request, #14798: URL: https://github.com/apache/kafka/pull/14798 ## Major diffs between versions 1. Upgrade minimum JDK from 6 to 8 2. Dependency (sbt) upgrade to work with JDK 17 3. Add new platforms such as win/aarch64 None of the changes are

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398336216 ## build.gradle: ## @@ -1643,19 +1700,6 @@ project(':server-common') { } } - sourceSets { -main { - java { -srcDirs = ["src/main/java"] -

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398336156 ## build.gradle: ## @@ -842,6 +842,62 @@ tasks.create(name: "jarConnect", dependsOn: connectPkgs.collect { it + ":jar" }) tasks.create(name: "testConnect",

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398336088 ## build.gradle: ## @@ -842,6 +842,62 @@ tasks.create(name: "jarConnect", dependsOn: connectPkgs.collect { it + ":jar" }) tasks.create(name: "testConnect",

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
divijvaidya commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398332350 ## build.gradle: ## @@ -842,6 +842,62 @@ tasks.create(name: "jarConnect", dependsOn: connectPkgs.collect { it + ":jar" }) tasks.create(name: "testConnect",

[jira] [Commented] (KAFKA-10873) Inaccurate "Ignoring stop request for unowned connector" log messages

2023-11-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787561#comment-17787561 ] Sagar Rao commented on KAFKA-10873: --- FWIW, we hit a scenario today where the same sequence of logs

Re: [PR] MINOR: Delete unused classes [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14797: URL: https://github.com/apache/kafka/pull/14797#discussion_r1398326335 ## core/src/test/scala/unit/kafka/server/ServerGenerateBrokerIdTest.scala: ## @@ -131,11 +131,7 @@ class ServerGenerateBrokerIdTest extends QuorumTestHarness {

[PR] MINOR: Delete unused classes [kafka]

2023-11-18 Thread via GitHub
ijuma opened a new pull request, #14797: URL: https://github.com/apache/kafka/pull/14797 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes)

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1398324375 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -1048,20 +1054,15 @@ class KafkaServer( * * @return The brokerId. */ - private def

Re: [PR] MINOR: Fix unstable sorting in AssignmentsManagerTest [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14794: URL: https://github.com/apache/kafka/pull/14794#discussion_r1398317507 ## core/src/test/java/kafka/server/AssignmentsManagerTest.java: ## @@ -72,6 +75,29 @@ void tearDown() throws InterruptedException { manager.close(); } +

Re: [PR] KAFKA-14585: 1st part : Java versions for metadata/broker and updated LogConfig [kafka]

2023-11-18 Thread via GitHub
github-actions[bot] commented on PR #14106: URL: https://github.com/apache/kafka/pull/14106#issuecomment-1817733757 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] KAFKA-15357: Aggregate and propagate assignments [kafka]

2023-11-18 Thread via GitHub
ijuma commented on PR #14369: URL: https://github.com/apache/kafka/pull/14369#issuecomment-1817732550 @cmccabe Please check all the Java/Scala versions before merging, this PR clearly broke the Java 8/Scala 2.12 build in multiple ways. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-14957: Update-Description-String [kafka]

2023-11-18 Thread via GitHub
Owen-CH-Leung commented on PR #13909: URL: https://github.com/apache/kafka/pull/13909#issuecomment-1817731594 Hi @mjsax , can I get some feedback from you ? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
guozhangwang commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1398306353 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/OuterStreamJoinStoreFactory.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software

Re: [PR] MINOR: Do not check whether updating tasks exist in the waiting loop [kafka]

2023-11-18 Thread via GitHub
guozhangwang commented on PR #14791: URL: https://github.com/apache/kafka/pull/14791#issuecomment-1817710719 LGTM! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-11-18 Thread via GitHub
guozhangwang commented on PR #14426: URL: https://github.com/apache/kafka/pull/14426#issuecomment-1817710512 Also made a very quick pass, and I think the fix is spot on. It would be great to get this merged sooner. -- This is an automated message from the Apache Git Service. To respond

[jira] [Commented] (KAFKA-15326) Decouple Processing Thread from Polling Thread

2023-11-18 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787544#comment-17787544 ] Guozhang Wang commented on KAFKA-15326: --- My bad not linking the first couple of PRs with this

[jira] [Updated] (KAFKA-14524) Modularize `core` monolith

2023-11-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-14524: Description: The `core` module has grown too large and it's time to split it into multiple

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398292530 ## server/src/main/java/org/apache/kafka/server/metrics/ClientMetricsConfigs.java: ## @@ -14,7 +14,7 @@ * See the License for the specific language governing

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398292085 ## build.gradle: ## @@ -1670,6 +1714,10 @@ project(':server-common') { checkstyle { configProperties =

Re: [PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma commented on code in PR #14796: URL: https://github.com/apache/kafka/pull/14796#discussion_r1398291786 ## build.gradle: ## @@ -1643,19 +1700,6 @@ project(':server-common') { } } - sourceSets { -main { - java { -srcDirs = ["src/main/java"] -

[PR] KAFKA-15854: Move Java classes from `kafka.server` to the `server` module [kafka]

2023-11-18 Thread via GitHub
ijuma opened a new pull request, #14796: URL: https://github.com/apache/kafka/pull/14796 We only move the classes that have no dependencies on Scala classes. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test

[jira] [Created] (KAFKA-15854) Move Java classes from kafka.server to the server module

2023-11-18 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-15854: --- Summary: Move Java classes from kafka.server to the server module Key: KAFKA-15854 URL: https://issues.apache.org/jira/browse/KAFKA-15854 Project: Kafka Issue

[jira] [Assigned] (KAFKA-15854) Move Java classes from kafka.server to the server module

2023-11-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-15854: --- Assignee: Ismael Juma > Move Java classes from kafka.server to the server module >

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r139828 ## streams/src/test/java/org/apache/kafka/streams/StreamsConfigTest.java: ## @@ -1038,13 +1038,15 @@ public void

[jira] [Created] (KAFKA-15853) Move KafkaConfig to server module

2023-11-18 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-15853: --- Summary: Move KafkaConfig to server module Key: KAFKA-15853 URL: https://issues.apache.org/jira/browse/KAFKA-15853 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-15852) Move broker code from `core` to `server` module

2023-11-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15852: Summary: Move broker code from `core` to `server` module (was: Move broker code from `core` to

[jira] [Updated] (KAFKA-15852) Move server code from `core` to `server` module

2023-11-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15852: Summary: Move server code from `core` to `server` module (was: Move broker code from `core` to

[jira] [Updated] (KAFKA-15852) Move broker code from `core` to `broker` module

2023-11-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15852: Issue Type: Improvement (was: Bug) > Move broker code from `core` to `broker` module >

Re: [PR] KAFKA-14483 Move/Rewrite of LocalLog to storage module. [kafka]

2023-11-18 Thread via GitHub
ijuma commented on PR #14034: URL: https://github.com/apache/kafka/pull/14034#issuecomment-1817663128 @satishd Let me know when this is ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Created] (KAFKA-15852) Move broker code from `core` to `broker` module

2023-11-18 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-15852: --- Summary: Move broker code from `core` to `broker` module Key: KAFKA-15852 URL: https://issues.apache.org/jira/browse/KAFKA-15852 Project: Kafka Issue Type:

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1398270264 ## streams/src/main/java/org/apache/kafka/streams/state/DslWindowParams.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1398270071 ## streams/src/main/java/org/apache/kafka/streams/state/DslSessionParams.java: ## @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1398269890 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreFactory.java: ## @@ -66,6 +66,10 @@ default void configure(final StreamsConfig config) {

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-18 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1398269442 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/StreamJoinedStoreFactory.java: ## @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-15836: KafkaConsumer subscribes to multiple topics does not respect max.poll.records [kafka]

2023-11-18 Thread via GitHub
AndrewJSchofield commented on code in PR #14789: URL: https://github.com/apache/kafka/pull/14789#discussion_r1398266495 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java: ## @@ -1124,6 +1126,62 @@ public void testFetchMaxPollRecords() {

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-11-18 Thread via GitHub
ocadaruma commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1398265953 ## storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java: ## @@ -308,7 +308,14 @@ public void truncateFromEnd(long endOffset) {

Re: [PR] MINOR: Do not check whether updating tasks exist in the waiting loop [kafka]

2023-11-18 Thread via GitHub
lucasbru merged PR #14791: URL: https://github.com/apache/kafka/pull/14791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-18 Thread via GitHub
CalvinConfluent commented on PR #14612: URL: https://github.com/apache/kafka/pull/14612#issuecomment-1817598315 - Updated the API schema with the Cursor. It is needed both in request and response. - Removed the RequestLimitReached error. -- This is an automated message from the Apache

Re: [PR] KAFKA-15776: Use the FETCH request timeout as the delay timeout for DelayedRemoteFetch [kafka]

2023-11-18 Thread via GitHub
divijvaidya commented on PR #14778: URL: https://github.com/apache/kafka/pull/14778#issuecomment-1817593722 Could you please help me understand how this change works with `fetch.max.wait.ms` from a user perspective i.e. what happens when we are retrieving data from both local & remote in a

Re: [PR] KAFKA-14509 [WIP] [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-11-18 Thread via GitHub
riedelmax commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1398242511 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -445,6 +446,42 @@ public List listGroups(List statesFi

Re: [PR] KAFKA-14509 [WIP] [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-11-18 Thread via GitHub
riedelmax commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1398242511 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -445,6 +446,42 @@ public List listGroups(List statesFi

Re: [PR] KAFKA-14509 [WIP] [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-11-18 Thread via GitHub
riedelmax commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1398242438 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupMember.java: ## @@ -545,6 +547,32 @@ public String

Re: [PR] KAFKA-14509 [WIP] [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-11-18 Thread via GitHub
riedelmax commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1398236262 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8660,6 +8665,68 @@ public void testListGroups() {

[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-18 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787499#comment-17787499 ] Phuc Hong Tran commented on KAFKA-15341: [~showuon] [~satish.duggana] I’m trying to add remote

Re: [PR] MINOR: Fix unstable sorting in AssignmentsManagerTest [kafka]

2023-11-18 Thread via GitHub
divijvaidya commented on code in PR #14794: URL: https://github.com/apache/kafka/pull/14794#discussion_r1398097515 ## core/src/test/java/kafka/server/AssignmentsManagerTest.java: ## @@ -72,6 +75,29 @@ void tearDown() throws InterruptedException { manager.close();

[jira] [Commented] (KAFKA-15736) KRaft support in PlaintextConsumerTest

2023-11-18 Thread Allen Kang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787464#comment-17787464 ] Allen Kang commented on KAFKA-15736: Can I pick up this? It looks great to me for a first-time