Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
OmniaGM commented on PR #15728: URL: https://github.com/apache/kafka/pull/15728#issuecomment-2066275194 Thanks for getting the KIP out there for discussion and for fixing the tests. Should this PR be a draft until we have the KIP voted in by the community? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
aaron-ai commented on PR #15728: URL: https://github.com/apache/kafka/pull/15728#issuecomment-2066276867 > Thanks for getting the KIP out there for discussion and for fixing the tests. Should this PR be a draft until we have the KIP voted in by the community? OK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
OmniaGM commented on code in PR #15728:
URL: https://github.com/apache/kafka/pull/15728#discussion_r1572155918
##
connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java:
##
@@ -169,6 +170,7 @@ static Map sourceConsumerConfig(Map props) {
result.putAll(Utils.entriesWithPrefix(props, CONSUMER_CLIENT_PREFIX));
result.putAll(Utils.entriesWithPrefix(props, SOURCE_PREFIX +
CONSUMER_CLIENT_PREFIX));
result.put(ENABLE_AUTO_COMMIT_CONFIG, "false");
+result.put(ALLOW_AUTO_CREATE_TOPICS_CONFIG, "false");
Review Comment:
@aaron-ai thanks for drafting a KIP that quickly. I left a couple of
comments there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
aaron-ai commented on PR #15728: URL: https://github.com/apache/kafka/pull/15728#issuecomment-2066221145 KIP has been created here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1039%3A+Disable+automatic+topic+creation+for+MirrorMaker2+consumers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
aaron-ai commented on code in PR #15728:
URL: https://github.com/apache/kafka/pull/15728#discussion_r1569813058
##
connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java:
##
@@ -169,6 +170,7 @@ static Map sourceConsumerConfig(Map props) {
result.putAll(Utils.entriesWithPrefix(props, CONSUMER_CLIENT_PREFIX));
result.putAll(Utils.entriesWithPrefix(props, SOURCE_PREFIX +
CONSUMER_CLIENT_PREFIX));
result.put(ENABLE_AUTO_COMMIT_CONFIG, "false");
+result.put(ALLOW_AUTO_CREATE_TOPICS_CONFIG, "false");
Review Comment:
@C0urante OK, I will draft a KIP later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
C0urante commented on code in PR #15728:
URL: https://github.com/apache/kafka/pull/15728#discussion_r1569197938
##
connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java:
##
@@ -169,6 +170,7 @@ static Map sourceConsumerConfig(Map props) {
result.putAll(Utils.entriesWithPrefix(props, CONSUMER_CLIENT_PREFIX));
result.putAll(Utils.entriesWithPrefix(props, SOURCE_PREFIX +
CONSUMER_CLIENT_PREFIX));
result.put(ENABLE_AUTO_COMMIT_CONFIG, "false");
+result.put(ALLOW_AUTO_CREATE_TOPICS_CONFIG, "false");
Review Comment:
While this may be more ideal for many use cases, it's a potentially-breaking
change that I don't believe we can make without waiting for a major release
(right now, that'd be 4.0.0), and an accepted KIP. If you're unfamiliar with
the KIP process, you can read more about it here:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Purpose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]
OmniaGM commented on PR #15728:
URL: https://github.com/apache/kafka/pull/15728#issuecomment-2061493764
Hi @aaron-ai thanks for the PR. Can you please have a look into the failing
tests
herehttps://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15728/1/#showFailuresLink
I believe they are related for example
[MirrorConnectorConfigTest](
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15728/1/testReport/junit/org.apache.kafka.connect.mirror/MirrorConnectorConfigTest/Build___JDK_21_and_Scala_2_13___testSourceConsumerConfigWithSourcePrefix__/)
is failing with
```
org.opentest4j.AssertionFailedError: source.consumer. source consumer config
not matching ==> expected: <{enable.auto.commit=false,
max.poll.interval.ms=100, auto.offset.reset=latest,
client.id=source1->target2|ConnectorName|test}> but was:
<{enable.auto.commit=false, max.poll.interval.ms=100,
allow.auto.create.topics=false, auto.offset.reset=latest,
client.id=source1->target2|ConnectorName|test}>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
