[jira] [Commented] (KAFKA-7723) Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
[
https://issues.apache.org/jira/browse/KAFKA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797260#comment-16797260
]
ASF GitHub Bot commented on KAFKA-7723:
---
sweat123 commented on pull request #6026: KAFKA-7723: Support override kafka
connect worker api configuration with rest api
URL: https://github.com/apache/kafka/pull/6026
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Connect support override worker kafka api configuration with connector
> configuration that post by rest api
>
>
> Key: KAFKA-7723
> URL: https://issues.apache.org/jira/browse/KAFKA-7723
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
>Reporter: laomei
>Priority: Minor
> Labels: needs-kip
>
> I'm using kafka sink connect; "auto.offset.reset" is set in
> connect-distributed*.properties;
> It works for all connector which in one worker; So the consumer will poll
> records from latest or earliest; I can not control the auto.offset.reset in
> connector configs post with rest api;
> So I think is necessary to override worker kafka api configs with connector
> configs;
> Like this
> {code:java}
> {
> "name": "test",
> "config": {
> "consumer.auto.offset.reset": "latest",
> "consumer.xxx"
> "connector.class": "com.laomei.sis.solr.SolrConnector",
> "tasks.max": "1",
> "poll.interval.ms": "100",
> "connect.timeout.ms": "6",
> "topics": "test"
> }
> }
> {code}
> We can override kafka consumer auto offset reset in sink connector;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7723) Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
[
https://issues.apache.org/jira/browse/KAFKA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797258#comment-16797258
]
laomei commented on KAFKA-7723:
---
[~rhauch] I'm sorry for the late response;
I have viewed KAFKA-6890 after you mentioned that; I think we can close this
issue and I will close the PR soon;
Thanks a lot!
> Kafka Connect support override worker kafka api configuration with connector
> configuration that post by rest api
>
>
> Key: KAFKA-7723
> URL: https://issues.apache.org/jira/browse/KAFKA-7723
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
>Reporter: laomei
>Priority: Minor
> Labels: needs-kip
>
> I'm using kafka sink connect; "auto.offset.reset" is set in
> connect-distributed*.properties;
> It works for all connector which in one worker; So the consumer will poll
> records from latest or earliest; I can not control the auto.offset.reset in
> connector configs post with rest api;
> So I think is necessary to override worker kafka api configs with connector
> configs;
> Like this
> {code:java}
> {
> "name": "test",
> "config": {
> "consumer.auto.offset.reset": "latest",
> "consumer.xxx"
> "connector.class": "com.laomei.sis.solr.SolrConnector",
> "tasks.max": "1",
> "poll.interval.ms": "100",
> "connect.timeout.ms": "6",
> "topics": "test"
> }
> }
> {code}
> We can override kafka consumer auto offset reset in sink connector;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7723) Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
[
https://issues.apache.org/jira/browse/KAFKA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793635#comment-16793635
]
Randall Hauch commented on KAFKA-7723:
--
[~laomei], first of all, thanks for logging this issue, creating
[KIP-407|https://cwiki.apache.org/confluence/display/KAFKA/KIP-407%3A+Kafka+Connect+support+override+worker+kafka+api+configuration+with+connector+configuration+that+post+by+rest+api],
and creating a pull request.
However, I think this is nearly identical to (or rather a subset of) KAFKA-6890
/
[KIP-296|https://cwiki.apache.org/confluence/display/KAFKA/KIP-296%3A+Connector+level+configurability+for+client+configs],
which IMO has the correct scope and where a discussion is taking place about
the requirements and user experience. This proposal seems to differ from
KAFKA-6890 / KIP-296 is that the approach proposed here only addresses
connector configurations specified via the REST API, and not via configuration
files passed to the standalone Connect worker. This would be a significant
departure from the current behavior, where the REST API and file configurations
are completely compatible.
Since KAFKA-6890 / KIP-296 are older, can we resolve this issue as DUPLICATE,
close the PR without merging, and withdraw KIP-407?
> Kafka Connect support override worker kafka api configuration with connector
> configuration that post by rest api
>
>
> Key: KAFKA-7723
> URL: https://issues.apache.org/jira/browse/KAFKA-7723
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
>Reporter: laomei
>Priority: Minor
> Labels: needs-kip
>
> I'm using kafka sink connect; "auto.offset.reset" is set in
> connect-distributed*.properties;
> It works for all connector which in one worker; So the consumer will poll
> records from latest or earliest; I can not control the auto.offset.reset in
> connector configs post with rest api;
> So I think is necessary to override worker kafka api configs with connector
> configs;
> Like this
> {code:java}
> {
> "name": "test",
> "config": {
> "consumer.auto.offset.reset": "latest",
> "consumer.xxx"
> "connector.class": "com.laomei.sis.solr.SolrConnector",
> "tasks.max": "1",
> "poll.interval.ms": "100",
> "connect.timeout.ms": "6",
> "topics": "test"
> }
> }
> {code}
> We can override kafka consumer auto offset reset in sink connector;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7723) Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
[
https://issues.apache.org/jira/browse/KAFKA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718427#comment-16718427
]
ASF GitHub Bot commented on KAFKA-7723:
---
sweat123 opened a new pull request #6026: KAFKA-7723: Support override kafka
connect worker api configuration with rest api
URL: https://github.com/apache/kafka/pull/6026
More details https://issues.apache.org/jira/browse/KAFKA-7723
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Connect support override worker kafka api configuration with connector
> configuration that post by rest api
>
>
> Key: KAFKA-7723
> URL: https://issues.apache.org/jira/browse/KAFKA-7723
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
>Reporter: laomei
>Priority: Minor
>
> I'm using kafka sink connect; "auto.offset.reset" is set in
> connect-distributed*.properties;
> It works for all connector which in one worker; So the consumer will poll
> records from latest or earliest; I can not control the auto.offset.reset in
> connector configs post with rest api;
> So I think is necessary to override worker kafka api configs with connector
> configs;
> Like this
> {code:java}
> {
> "name": "test",
> "config": {
> "consumer.auto.offset.reset": "latest",
> "consumer.xxx"
> "connector.class": "com.laomei.sis.solr.SolrConnector",
> "tasks.max": "1",
> "poll.interval.ms": "100",
> "connect.timeout.ms": "6",
> "topics": "test"
> }
> }
> {code}
> We can override kafka consumer auto offset reset in sink connector;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
