[
https://issues.apache.org/jira/browse/KAFKA-9845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080017#comment-17080017
]
ASF GitHub Bot commented on KAFKA-9845:
---------------------------------------
C0urante commented on pull request #8455: KAFKA-9845: Warn users about using
config providers with plugin.path property
URL: https://github.com/apache/kafka/pull/8455
[Jira](https://issues.apache.org/jira/browse/KAFKA-9845)
These changes cause the transformed worker configuration to be passed to the
`Plugins` instance which performs plugin path scanning, instead of the raw
(pre-transform) worker configuration. This has the added benefit that worker
configuration validation now takes place _before_ plugin path scanning, which
in some environments can take a while.
Unfortunately, they may interfere with the loading of properties of type
`CLASS` from the `WorkerConfig`. The changes will need to be revisited in order
to address this circular dependency issue: `Plugins` needs to be instantiated
and `compareAndSwapWithDelegatingLoader` invoked before creating the worker
config since it loads classes when instantiated, and the config provider logic
specified in the worker config needs to be performed on the `plugin.path`
property before instantiating the `Plugins` instance. This is especially
complicated by the fact that config providers are loaded as plugins.
No clear workaround is apparent at the moment that wouldn't involve a
substantial rewrite, and the scope of the problem is limited enough that simply
documenting it as a "won't fix" seems the best approach for now.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> plugin.path property does not work with config provider
> -------------------------------------------------------
>
> Key: KAFKA-9845
> URL: https://issues.apache.org/jira/browse/KAFKA-9845
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.3.0, 2.4.0, 2.3.1, 2.5.0, 2.4.1
> Reporter: Chris Egerton
> Priority: Major
>
> The config provider mechanism doesn't work if used for the {{plugin.path}}
> property of a standalone or distributed Connect worker. This is because the
> {{Plugins}} instance which performs plugin path scanning is created using the
> raw worker config, pre-transformation (see
> [ConnectStandalone|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectStandalone.java#L79]
> and
> [ConnectDistributed|https://github.com/apache/kafka/blob/371ad143a6bb973927c89c0788d048a17ebac91a/connect/runtime/src/main/java/org/apache/kafka/connect/cli/ConnectDistributed.java#L91]).
> Unfortunately, because config providers are loaded as plugins, there's a
> circular dependency issue here. The {{Plugins}} instance needs to be created
> _before_ the {{DistributedConfig}}/{{StandaloneConfig}} is created in order
> for the config providers to be loaded correctly, and the config providers
> need to be loaded in order to perform their logic on any properties
> (including the {{plugin.path}} property).
> There is no clear fix for this issue in the code base, and the only known
> workaround is to refrain from using config providers for the {{plugin.path}}
> property.
> A couple improvements could potentially be made to improve the UX when this
> issue arises:
> # Alter the config logging performed by the {{DistributedConfig}} and
> {{StandaloneConfig}} classes to _always_ log the raw value for the
> {{plugin.path}} property. Right now, the transformed value is logged even
> though it isn't used, which is likely to cause confusion.
> # Issue a {{WARN}}- or even {{ERROR}}-level log message when it's detected
> that the user is attempting to use config providers for the {{plugin.path}}
> property, which states that config providers cannot be used for that specific
> property, instructs them to change the value for the property accordingly,
> and/or informs them of the actual value that the framework will use for that
> property when performing plugin path scanning.
> We should _not_ throw an error on startup if this condition is detected, as
> this could cause previously-functioning, benignly-misconfigured Connect
> workers to fail to start after an upgrade.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
