[
https://issues.apache.org/jira/browse/KAFKA-18211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Snehashis Pal updated KAFKA-18211:
----------------------------------
Description:
Connect used to use reflections scanner for scanning and identifying connect
plugins in its plugin.path. This would load said plugins in isolation via the
use of a child first PluginClassloader, which is designed to load class from
its set of URIs before delegating to parent, if not found. This effectively
enforces that if a plugin and its dependencies are part of a plugin path it
would not conflict with other plugins in the plugin path or plugins in
classpath.
GlassGraph was introduced as a replacement for the older reflections scanner in
[KAFKA-15203 Use Classgraph since org.reflections is no longer under
maintainence by PARADOXST · Pull Request #16604 ·
apache/kafka|https://github.com/apache/kafka/pull/16604]. It is used in place
of reflections scanner for finding plugins during plugin scanning. The issue
here is that it is missing any plugins present in isolated plugin paths if its
already present in classpath. We can repro this by adding the json converter
under an isolated plugin path and starting connect with debug logs. We can see
the logs from ReflectionsScanner and find that the ClassGraph loader is always
fetching the plugin from the classpath even though the PluginClassLoader is
provided. This is causing
[kafka/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java
at 520681c38dbefe497181c4fd5dfc793d54233408 ·
apache/kafka|https://github.com/apache/kafka/blob/520681c38dbefe497181c4fd5dfc793d54233408/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java#L134]
check to fail with the logs like.
{code:java}
[2024-12-11 07:29:28,968] DEBUG class
org.apache.kafka.connect.json.JsonConverter from other classloader
jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
excluding to prevent isolated loading
(org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135)
[2024-12-11 07:29:28,969] DEBUG class
org.apache.kafka.connect.json.JsonConverter from other classloader
jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
excluding to prevent isolated loading
(org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135) {code}
was:
Connect used to use reflections scanner for scanning and identifying connect
plugins in its plugin.path. This would load said plugins in isolation via the
use of a child first PluginClassloader, which is designed to load class from
its set of URIs before delegating to parent, if not found. This effectively
enforces that if a plugin and its dependencies are part of a plugin path it
would not conflict with other plugins in the plugin path or plugins in
classpath.
GlassGraph was introduced as a replacement for the older reflections scanner in
[KAFKA-15203 Use Classgraph since org.reflections is no longer under
maintainence by PARADOXST · Pull Request #16604 ·
apache/kafka|https://github.com/apache/kafka/pull/16604]. It is used in place
of reflections scanner for finding plugins during plugin scanning. The issue
here is that it is missing any plugins present in isolated plugin paths if its
already present in classpath.
> ClassGraph scanning does not correctly find isolated connect plugins
> --------------------------------------------------------------------
>
> Key: KAFKA-18211
> URL: https://issues.apache.org/jira/browse/KAFKA-18211
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Affects Versions: 4.0.0
> Reporter: Snehashis Pal
> Priority: Blocker
>
> Connect used to use reflections scanner for scanning and identifying connect
> plugins in its plugin.path. This would load said plugins in isolation via the
> use of a child first PluginClassloader, which is designed to load class from
> its set of URIs before delegating to parent, if not found. This effectively
> enforces that if a plugin and its dependencies are part of a plugin path it
> would not conflict with other plugins in the plugin path or plugins in
> classpath.
>
> GlassGraph was introduced as a replacement for the older reflections scanner
> in [KAFKA-15203 Use Classgraph since org.reflections is no longer under
> maintainence by PARADOXST · Pull Request #16604 ·
> apache/kafka|https://github.com/apache/kafka/pull/16604]. It is used in place
> of reflections scanner for finding plugins during plugin scanning. The issue
> here is that it is missing any plugins present in isolated plugin paths if
> its already present in classpath. We can repro this by adding the json
> converter under an isolated plugin path and starting connect with debug logs.
> We can see the logs from ReflectionsScanner and find that the ClassGraph
> loader is always fetching the plugin from the classpath even though the
> PluginClassLoader is provided. This is causing
> [kafka/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java
> at 520681c38dbefe497181c4fd5dfc793d54233408 ·
> apache/kafka|https://github.com/apache/kafka/blob/520681c38dbefe497181c4fd5dfc793d54233408/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/ReflectionScanner.java#L134]
> check to fail with the logs like.
> {code:java}
> [2024-12-11 07:29:28,968] DEBUG class
> org.apache.kafka.connect.json.JsonConverter from other classloader
> jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
> C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
> excluding to prevent isolated loading
> (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135)
> [2024-12-11 07:29:28,969] DEBUG class
> org.apache.kafka.connect.json.JsonConverter from other classloader
> jdk.internal.loader.ClassLoaders$AppClassLoader@c387f44 is visible from
> C:\Users\user\Desktop\confluent\testing\plugins3\connect-file-1.2.1-T-0.9.0-P-3.1,
> excluding to prevent isolated loading
> (org.apache.kafka.connect.runtime.isolation.ReflectionScanner:135) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
