clintropolis opened a new issue #8591: issue with Hadoop indexing task and jdk9+ URL: https://github.com/apache/incubator-druid/issues/8591 Related to #5589, capturing a Hadoop indexing issue with newer Java versions from [this dev list thread on jdk11 support.](https://lists.apache.org/thread.html/d4a325cb38071f34db413c86150e9f9fc4e7daf91b808d9957dc5785@%3Cdev.druid.apache.org%3E) While most things now appear to be working correctly with recent versions of Java, Hadoop indexing tasks are failing with an error of the form: ``` 2019-08-28T22:14:56,959 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.common.task.HadoopIndexTask - Got invocation target exception in run(), cause: java.lang.NoClassDefFoundError: javax/script/ScriptEngineManager at org.apache.logging.log4j.core.script.ScriptManager.<init>(ScriptManager.java:49) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:179) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:209) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:492) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:562) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:578) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:214) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41) ~[log4j-core-2.5.jar:2.5] at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182) ~[log4j-api-2.5.jar:2.5] at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103) ~[log4j-api-2.5.jar:2.5] at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43) ~[log4j-slf4j-impl-2.5.jar:2.5] at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42) ~[log4j-api-2.5.jar:2.5] at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29) ~[log4j-slf4j-impl-2.5.jar:2.5] at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358) ~[slf4j-api-1.7.25.jar:1.7.25] at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383) ~[slf4j-api-1.7.25.jar:1.7.25] at org.apache.druid.java.util.common.logger.Logger.<init>(Logger.java:38) [druid-core-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.guice.PropertiesModule.<clinit>(PropertiesModule.java:42) ~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.guice.GuiceInjectors.makeDefaultStartupModules(GuiceInjectors.java:39) ~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.guice.GuiceInjectors.makeStartupInjector(GuiceInjectors.java:56) ~[druid-processing-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexer.HadoopDruidIndexerConfig.<clinit>(HadoopDruidIndexerConfig.java:103) ~[druid-indexing-hadoop-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessingRunner.runTask(HadoopIndexTask.java:644) ~[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.apache.druid.indexing.common.task.HadoopIndexTask.runInternal(HadoopIndexTask.java:353) ~[druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexing.common.task.HadoopIndexTask.runTask(HadoopIndexTask.java:287) [druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:137) [druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419) [druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391) [druid-indexing-service-0.16.0-incubating-SNAPSHOT.jar:0.16.0-incubating-SNAPSHOT] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.base/java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.ClassNotFoundException: javax.script.ScriptEngineManager at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[?:?] ... 35 more ``` https://stackoverflow.com/questions/53714010/log4j2-slf4j-and-java-11 suggests adding '--add-modules java.scripting', but it did not appear to help anything when added to java opts of peons. It is not related to the java opts having any sort of failure to accept these module arguments, i was able to successfully pass the option to make jvm gc metrics work, and attempting to run the task with the new 'indexer' service type where the tasks are threads instead of forked processes was not successful with this option set either. I haven't dug in any further yet.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
