Working on a Storm project using C# for Bolts and Spouts - as I have a dependency on .Net/Mono for I/O -
We are using Flux for the topologies and FluxShellSpout to invoke a .net 4.5 executable that uses the storm net adapter (https://github.com/ziyunhx/storm-net-adapter) It works when: - We invoke it via mono - We invoke it natively using the full path of the exe It does not work when we invoke it natively with a relative path. We built a .net 4.5 exe using Sotrm.Net.Adapter and binplace it in a resource path: <basedir>/multilang/resources/mono/RdfeExtractSpout.exe The pom includes this resource like this: <resource> <directory>${basedir}/multilang</directory> </resource> We can see that the mono directory is correctly copied into the target\classes folder when we compile the pom. This topology works: spouts: # Spout definitions - id: "availability-csharp-spout" className: "org.apache.storm.flux.wrappers.spouts.FluxShellSpout" constructorArgs: # command line - ["mono", "mono/RdfeExtractSpout.exe"] # output fields - ["Service", "EnvironmentType", "EnvironmentName", "Plane", "TimeEmitted1MinBin", "GeoLocale", "Operation", "MetricLatency"] parallelism: 1 The spout runs under mono correctly on the Windows box. This topology works too: spouts: # Spout definitions - id: "availability-csharp-spout" className: "org.apache.storm.flux.wrappers.spouts.FluxShellSpout" constructorArgs: # command line - ["C:\\OneBranch\\SRE\\DataInsightsHub\\src\\StormDataHub\\Topologies\\Rdfe\\target\\classes\\resources\\mono\\RdfeExtractSpout.exe"] # output fields - ["Service", "EnvironmentType", "EnvironmentName", "Plane", "TimeEmitted1MinBin", "GeoLocale", "Operation", "MetricLatency"] parallelism: 1 The spout runs under .net on the windows box (I know this would never work in production, but I am trying to make a point with the following config). This topology does not work: spouts: # Spout definitions - id: "availability-csharp-spout" className: "org.apache.storm.flux.wrappers.spouts.FluxShellSpout" constructorArgs: # command line - ["mono\\RdfeExtractSpout.exe"] # output fields - ["Service", "EnvironmentType", "EnvironmentName", "Plane", "TimeEmitted1MinBin", "GeoLocale", "Operation", "MetricLatency"] parallelism: 1 I get the following: 16:52:50 [Thread-18-availability-csharp-spout-executor[2 2]] ERROR org.apache.storm.util - Async loop died! java.lang.RuntimeException: Error when launching multilang subprocess at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:94) ~[storm-core-1.1.0.jar:1.1.0] at org.apache.storm.spout.ShellSpout.open(ShellSpout.java:111) ~[storm-core-1.1.0.jar:1.1.0] at org.apache.storm.daemon.executor$fn__4976$fn__4991.invoke(executor.clj:600) ~[storm-core-1.1.0.jar:1.1.0] at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:482) [storm-core-1.1.0.jar:1.1.0] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] Caused by: java.io.IOException: Cannot run program "mono\RdfeExtractSpout.exe" (in directory "C:\Users\maurgi\AppData\Local\Temp\85a06845-1257-433b-a938-8ebe34 e6a43a\supervisor\stormdist\rdfe-sample-topology-1-1491954766\resources"): CreateProcess error=2, The system cannot find the file specified at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[?:1.8.0_121] at org.apache.storm.utils.ShellProcess.launch(ShellProcess.java:87) ~[storm-core-1.1.0.jar:1.1.0] I tried different variants: mono\RdfeExtractSpout.exe , mono/RdfeExtractSpout.exe, I tried to get rid of the mono subfolder and just call RdfeExtractSpout.exe but I get the same error. The file is there in Windows, why can't the FluxShellSpout work with the relative path, but it can work with the absolute path? [cid:[email protected]] Thanks, Mauro Giusti.
