Hey Nikon, I had the same issue days ago, the problem in my case was that I
wasn't declaring the storm-core dependency as provided in the pom file,
therefore the fat jar included the dependency, which is provided by nimbus
(and workers) when deploying in the cluster.
...
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>${storm.version}</version>
<scope>provided</scope>
</dependency>
...
Best,
Jose Luis
On Tue, Feb 17, 2015 at 5:29 PM, Nick R. Katsipoulakis <
[email protected]> wrote:
> Hello,
>
> I am currently trying to automate some experiments on Storm and to do so,
> I have created a Java process which in turn launches three Java
> sub-processes using the ProcessBuilder framework. The three processes are
> mainly executions of jar files (2 of them simple java applications, 1 of
> them Storm topology submitted to a Storm cluster). Therefore, the 2 simple
> java applications are executed as terminal commands:
>
> $>java -jar executable-jar <arg-1> <arg-2> ... <arg-N>
>
> The storm topology is bundled into a jar using Eclipse maven package
> assembly:single goal and the following plugin configuration is added to the
> pom.xml of the project:
>
> <build>
>
> <plugins>
>
> <plugin>
>
> <groupId>org.apache.maven.plugins</groupId>
>
> <artifactId>maven-assembly-plugin</artifactId>
>
> <version>2.5.3</version>
>
> <configuration>
>
> <descriptorRefs>
>
> <descriptorRef>jar-with-dependencies</descriptorRef>
>
> </descriptorRefs>
>
> <archive>
>
> <manifest>
>
> <mainClass>gr.katsip.storm.topology.ExperimentalTopology</mainClass>
>
> </manifest>
>
> </archive>
>
> </configuration>
>
> <executions>
>
> <execution>
>
> <phase>package</phase>
>
> <goals>
>
> <goal>single</goal>
>
> </goals>
>
> </execution>
>
> </executions>
>
> </plugin>
>
> </plugins>
>
> </build>
>
>
> The problem is that when I create the Java sub-process that submits the
> following command:
>
>
> $>storm jar bundled-topology.jar <arg-1> <arg-2> ... <arg-M>
>
>
> at some point I get the following error:
>
>
> Exception in thread "main" java.lang.RuntimeException: Found multiple
> defaults.yaml resources. You're probably bundling the Storm jars with your
> topology jar.
> [jar:file:/Users/nick/Documents/stream-gen/Experiment/synefo-topology.jar!/defaults.yaml,
> jar:file:/Users/nick/apache-storm/lib/storm-core-0.9.2-incubating.jar!/defaults.yaml]
>
> at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:140)
>
> at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:167)
>
> at backtype.storm.utils.Utils.readStormConfig(Utils.java:191)
>
> at backtype.storm.config$read_storm_config.invoke(config.clj:121)
>
> at backtype.storm.command.config_value$_main.invoke(config_value.clj:22)
>
> at clojure.lang.AFn.applyToHelper(AFn.java:161)
>
> at clojure.lang.AFn.applyTo(AFn.java:151)
>
> at backtype.storm.command.config_value.main(Unknown Source)
>
> Therefore, I understand that during packaging, maven includes a
> default.yaml file in my package, that conflicts with the yam file of the
> cluster.
>
> My question is how can I remove the default.yaml from my bundled-topology
> or how can I overcome this issue. The simple solution would be to start my
> topology by itself and then run the other two processes, but I want my
> experiments to be fully automated.
>
> Thank you and I apologize in advance for the long post.
>
> Cheers,
> Nikos
>
> --
> Nikolaos Romanos Katsipoulakis,
> University of Pittsburgh, PhD candidate
>
