[
https://issues.apache.org/jira/browse/KAFKA-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17904776#comment-17904776
]
Snehashis Pal commented on KAFKA-18211:
---------------------------------------
The fix I made was to use {{overrideClassLoaders(source.loaders()}} instead of
{{addClassLoader}}
[here|https://github.com/apache/kafka/blob/520681c38dbefe497181c4fd5dfc793d54233408/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java#L80].
Based on
[this|https://github.com/classgraph/classgraph/blob/6f9012f2a193ebfefe4a4384e7642820e7aab0f5/src/main/java/nonapi/io/github/classgraph/classpath/ClassLoaderFinder.java#L118]
code, {{addClassLoader}} by default includes the jdk classloaders first and
only then scans the provided loader. This looks to be the reason why the
classpath plugin is always getting loaded ahead of the isolated on in
plugin.path. Overriding the classloader which prevents this seems to be the
right fix.
> ClassGraph scanning does not correctly find isolated connect plugins
> --------------------------------------------------------------------
>
> Key: KAFKA-18211
> URL: https://issues.apache.org/jira/browse/KAFKA-18211
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Affects Versions: 4.0.0
> Reporter: Snehashis Pal
> Priority: Blocker
>
> Connect used to use reflections scanner for scanning and identifying connect
> plugins in its plugin.path. This would load said plugins in isolation via the
> use of a child first PluginClassloader, which is designed to load class from
> its set of URIs before delegating to parent, if not found. This effectively
> enforces that if a plugin and its dependencies are part of a plugin path it
> would not conflict with other plugins in the plugin path or plugins in
> classpath.
>
> GlassGraph was introduced as a replacement for the older reflections scanner
> in [KAFKA-15203 Use Classgraph since org.reflections is no longer under
> maintainence by PARADOXST · Pull Request #16604 ·
> apache/kafka|https://github.com/apache/kafka/pull/16604]. It is used in place
> of reflections scanner for finding plugins during plugin scanning. The issue
> here is that it is missing any plugins present in isolated plugin paths if
> its already present in classpath. We can repro this by adding the json
> converter under an isolated plugin path and starting connect with debug logs.
> We can see the logs from ReflectionsScanner and find that the ClassGraph
> loader is always fetching the plugin from the classpath even though the
> PluginClassLoader is provided. This is causing
> [kafka/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java
> at 520681c38dbefe497181c4fd5dfc793d54233408 ·
> apache/kafka|https://github.com/apache/kafka/blob/520681c38dbefe497181c4fd5dfc793d54233408/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java#L134]
> check to fail with the logs like.
> {code:java}
> [2024-12-11 07:29:28,968] DEBUG class
> org.apache.kafka.connect.json.JsonConverter from other classloader
> jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
> C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
> excluding to prevent isolated loading
> (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135)
> [2024-12-11 07:29:28,969] DEBUG class
> org.apache.kafka.connect.json.JsonConverter from other classloader
> jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
> C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
> excluding to prevent isolated loading
> (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
