[
https://issues.apache.org/jira/browse/KAFKA-6139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Burd resolved KAFKA-6139.
--------------------------------
Resolution: Information Provided
Error was due to having the plugin directory set a level too low (pointing
inside the plugin).
> error when loading plugins
> --------------------------
>
> Key: KAFKA-6139
> URL: https://issues.apache.org/jira/browse/KAFKA-6139
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.11.0.0
> Reporter: Eugene Burd
>
> I am trying to run the Big Query connector for KafkaConnect
> (https://github.com/wepay/kafka-connect-bigquery). I have configured it
> using a docker container, dropped the jar files for the connector in the
> container and configured the plugin.path in the kafka connect config to point
> to the directory with the jars.
> Upon startup, Kafka is scanning the plugin folder and evaluating all the jar
> files found. It encounters the connector jar file, tests to see if it
> extends a connector (in this case kcbq-connector-1.0.0-SNAPSHOT.jar), but
> then it proceeds to create a new instance of the connector. Code snippet
> from DelegatingClassLoader.java below. The problem is that the connector
> class relies on other jar files that are in the plugin folder, but not in the
> path. This causes a class not found exception (below).
> I suspect when the plugin connector is actually executed, it is done through
> an isolated context and that context's classpath is set to the plugin folder,
> so i think the connector would actually work. But scanning for the connector
> fails prior to ever getting there.
> I understand that you need a version of the connector and hence why this code
> is running, but i think the new instance creation for version check needs to
> be done in a sandbox to support the classpaths of the plugin connector.
> [2017-10-28 06:04:08,961] INFO Loading plugin from:
> /usr/share/java/kafka-connect-bigquery/kcbq-connector-1.0.0-SNAPSHOT.jar
> (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:176)
> Exception in thread "main" java.lang.NoClassDefFoundError:
> com/google/cloud/bigquery/BigQuery
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:242)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:223)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:190)
> at
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:150)
> at
> org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
> at
> org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:63)
> Caused by: java.lang.ClassNotFoundException:
> com.google.cloud.bigquery.BigQuery
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:62)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 11 more
> -------------------------
> private <T> Collection<PluginDesc<T>> getPluginDesc(
> Reflections reflections,
> Class<T> klass,
> ClassLoader loader
> ) throws InstantiationException, IllegalAccessException {
> Set<Class<? extends T>> plugins = reflections.getSubTypesOf(klass);
> Collection<PluginDesc<T>> result = new ArrayList<>();
> for (Class<? extends T> plugin : plugins) {
> if (PluginUtils.isConcrete(plugin)) {
> // Temporary workaround until all the plugins are versioned.
> if (Connector.class.isAssignableFrom(plugin)) {
> result.add(
> new PluginDesc<>(
> plugin,
> ((Connector)
> plugin.newInstance()).version(),
> loader
> )
> );
> } else {
> result.add(new PluginDesc<>(plugin, "undefined", loader));
> }
> }
> }
> return result;
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
