sebastienpattyn93 opened a new issue #2913: Got UploaderException when submit topology on kubernetes URL: https://github.com/apache/incubator-heron/issues/2913 I have a Kubernetes cluster with 3 nodes where Heron is deployed. I am able to visit the heron ui and to curl the heron api-server. When I try to submit a topology I keep getting the same error: ``` heron submit kubernetes --verbose ./dist/word-count-topology.pex - Word_Count_Topology [2018-06-07 08:08:26 +0000] [DEBUG]: Input Command Line Args: {'config_property': [], 'dry_run_format': 'colored_table', 'topology-file-name': './dist/word-count-topology.pex', 'cluster/[role]/[env]': 'kubernetes', 'dry_run': False, 'config_path': '/home/ubuntu/.heron/conf', 'subcommand': 'submit', 'deploy_deactivated': False, 'topology_main_jvm_property': [], 'extra_launch_classpath': '', 'service_url': '', 'topology-class-name': '-', 'verbose': 'True'} [2018-06-07 08:08:26 +0000] [DEBUG]: Input Command Line Args: {'config_property': [], 'dry_run_format': 'colored_table', 'topology-file-name': './dist/word-count-topology.pex', 'cluster/[role]/[env]': 'kubernetes', 'dry_run': False, 'config_path': '/home/ubuntu/.heron/conf', 'subcommand': 'submit', 'deploy_deactivated': False, 'topology_main_jvm_property': [], 'extra_launch_classpath': '', 'service_url': '', 'topology-class-name': '-', 'verbose': 'True'} [2018-06-07 08:08:26 +0000] [DEBUG]: Using cluster definition from file /home/ubuntu/.config/heron/kubernetes/cli.yaml [2018-06-07 08:08:26 +0000] [DEBUG]: Processed Command Line Args: {'config_property': [], 'topology-file-name': './dist/word-count-topology.pex', 'verbose': 'True', 'subcommand': 'submit', 'deploy_deactivated': False, 'submit_user': 'ubuntu', 'cluster': 'kubernetes', 'extra_launch_classpath': '', 'deploy_mode': 'server', 'role': 'ubuntu', 'dry_run_format': 'colored_table', 'dry_run': False, 'config_path': '/home/ubuntu/.heron/conf', 'topology_main_jvm_property': [], 'environ': 'default', 'service_url': 'http://10.128.0.20:32674', 'topology-class-name': '-'} [2018-06-07 08:08:26 +0000] [DEBUG]: Submit Args {'config_property': [], 'topology-file-name': './dist/word-count-topology.pex', 'verbose': 'True', 'subcommand': 'submit', 'deploy_deactivated': False, 'submit_user': 'ubuntu', 'cluster': 'kubernetes', 'extra_launch_classpath': '', 'deploy_mode': 'server', 'role': 'ubuntu', 'dry_run_format': 'colored_table', 'dry_run': False, 'config_path': '/home/ubuntu/.heron/conf', 'topology_main_jvm_property': [], 'environ': 'default', 'service_url': 'http://10.128.0.20:32674', 'topology-class-name': '-'} [2018-06-07 08:08:26 +0000] [DEBUG]: Importing - from ./dist/word-count-topology.pex [2018-06-07 08:08:26 +0000] [DEBUG]: Invoking class using command: ``./dist/word-count-topology.pex Word_Count_Topology'' [2018-06-07 08:08:26 +0000] [DEBUG]: Heron options: {cmdline.topologydefn.tmpdirectory=/tmp/tmpW58aFW,cmdline.topology.initial.state=RUNNING} [2018-06-07 08:08:27 +0000] [INFO]: Launching topology: 'Word_Count_Topology' [2018-06-07 08:08:27 +0000] [INFO]: {'config_property': [], 'topology-file-name': './dist/word-count-topology.pex', 'verbose': 'True', 'subcommand': 'submit', 'deploy_deactivated': False, 'submit_user': 'ubuntu', 'cluster': 'kubernetes', 'extra_launch_classpath': '', 'deploy_mode': 'server', 'role': 'ubuntu', 'dry_run_format': 'colored_table', 'dry_run': False, 'config_path': '/home/ubuntu/.heron/conf', 'topology_main_jvm_property': [], 'environ': 'default', 'service_url': 'http://10.128.0.20:32674', 'topology-class-name': '-'} [2018-06-07 08:08:27 +0000] [DEBUG]: Starting new HTTP connection (1): 10.128.0.20 [2018-06-07 08:08:27 +0000] [DEBUG]: http://10.128.0.20:32674 "POST /api/v1/topologies HTTP/1.1" 500 94 [2018-06-07 08:08:27 +0000] [ERROR]: Failed to instantiate uploader class 'org.apache.heron.uploader.dlog.DLUploader' [2018-06-07 08:08:27 +0000] [ERROR]: Failed to launch topology 'Word_Count_Topology' [2018-06-07 08:08:27 +0000] [DEBUG]: Elapsed time: 0.409s. ``` Looking in my Kubernetes POD logs I found the following error: ``` [2018-06-07 08:08:27 +0000] [SEVERE] com.twitter.heron.apiserver.resources.TopologyResource: error submitting topology Word_Count_Topology com.twitter.heron.spi.uploader.UploaderException: Failed to instantiate uploader class 'org.apache.heron.uploader.dlog.DLUploader' at com.twitter.heron.scheduler.SubmitterMain.submitTopology(SubmitterMain.java:411) at com.twitter.heron.apiserver.actions.SubmitTopologyAction.execute(SubmitTopologyAction.java:33) at com.twitter.heron.apiserver.resources.TopologyResource.submit(TopologyResource.java:223) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:564) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128) at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: org.apache.heron.uploader.dlog.DLUploader at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at com.twitter.heron.spi.utils.ReflectionUtils.newInstance(ReflectionUtils.java:34) at com.twitter.heron.spi.utils.ReflectionUtils.newInstance(ReflectionUtils.java:25) at com.twitter.heron.scheduler.SubmitterMain.submitTopology(SubmitterMain.java:408) ... 51 more ``` I tried with other example topologies from heron but got the same error.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
