gharris1727 commented on PR #14064: URL: https://github.com/apache/kafka/pull/14064#issuecomment-1644636341
Here's some sample output from the current implementation: ``` $ ./bin/connect-plugin-path.sh list --plugin-path ~/test/plugin-path/ SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/Users/greg.harris/github/kafka/tools/build/dependant-libs-2.13.11/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/Users/greg.harris/github/kafka/trogdor/build/dependant-libs-2.13.11/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory] /Users/greg.harris/test/plugin-path/a2solutions-oracdc-kafka-1.3.3.1 solutions.a2.cdc.oracle.OraCdcJdbcSinkConnector OraCdcJdbcSink OraCdcJdbcSinkConnector 1.3.3.1 sink true false /Users/greg.harris/test/plugin-path/a2solutions-oracdc-kafka-1.3.3.1 solutions.a2.cdc.oracle.OraCdcLogMinerConnector OraCdcLogMinerConnector OraCdcLogMiner 1.3.3.1 source true false /Users/greg.harris/test/plugin-path/a2solutions-oracdc-kafka-1.3.3.1 solutions.a2.cdc.oracle.OraCdcSourceConnector OraCdcSource OraCdcSourceConnector 1.3.3.1 source true false /Users/greg.harris/test/plugin-path/microsoft-kafka-connect-iothub-0.6 com.microsoft.azure.iot.kafka.connect.sink.IotHubSinkConnector IotHubSink IotHubSinkConnector 0.6 sink true false /Users/greg.harris/test/plugin-path/microsoft-kafka-connect-iothub-0.6 com.microsoft.azure.iot.kafka.connect.IotHubSourceConnector IotHubSourceConnector IotHubSource 0.6 source true false [2023-07-20 14:19:13,855] ERROR Failed to get plugin version for class com.github.jcustenborder.kafka.connect.spooldir.elf.SpoolDirELFSourceConnector (org.apache.kafka.connect.runtime.isolation.PluginScanner) java.lang.NoClassDefFoundError: com/google/common/base/Strings at com.github.jcustenborder.kafka.connect.utils.VersionUtil.version(VersionUtil.java:32) at com.github.jcustenborder.kafka.connect.spooldir.elf.SpoolDirELFSourceConnector.version(SpoolDirELFSourceConnector.java:57) at org.apache.kafka.connect.runtime.isolation.PluginScanner.versionFor(PluginScanner.java:199) at org.apache.kafka.connect.runtime.isolation.ReflectionScanner.versionFor(ReflectionScanner.java:74) at org.apache.kafka.connect.runtime.isolation.ReflectionScanner.getPluginDesc(ReflectionScanner.java:135) at org.apache.kafka.connect.runtime.isolation.ReflectionScanner.scanPlugins(ReflectionScanner.java:88) at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) at org.apache.kafka.tools.ConnectPluginPath.runCommand(ConnectPluginPath.java:213) at org.apache.kafka.tools.ConnectPluginPath.mainNoExit(ConnectPluginPath.java:79) at org.apache.kafka.tools.ConnectPluginPath.main(ConnectPluginPath.java:71) Caused by: java.lang.ClassNotFoundException: com.google.common.base.Strings at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587) at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:136) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ... 11 more ``` The SLF4J lines and the ERROR logs are printed to stderr, so redirecting with `2> /dev/null` is sufficient to clean up the output. Unloadable plugins (due to class structure problems) are printed with error logs. Right now the stdout is formatted like a TSV, but without a header. The KIP only specifies that this output be human-readable, so i'm interested in changing this output slightly before merging. The columns are: * plugin location * full class name * simple name (if unique within the classpath+location), or null * pruned name (if unique within the classpath+location), or null * reported version, or undefined * plugin type (sink/source/converter/etc) * whether the plugin is reflectively loadable * whether the plugin has a manifest file Because the ReflectionScanner is used on individual locations one-at-a-time, the script is slow, but prints results after scanning each location. For the above, the results start to be printed within about 1/4 second, and the whole listing takes 30s for 150 components (400 plugins) downloaded from the internet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
