Anyone with Gradle experience, kindly help. I've tried for the past 40
minutes with help from posts like this
<http://stackoverflow.com/questions/17421726/making-the-storm-jars-compile-time-only-in-a-gradle-project>,
this
<http://www.michael-noll.com/tutorials/running-multi-node-storm-cluster/#build-a-correct-standalone--fat-jar-file-of-my-storm-code>,
this
<http://kennethjorgensen.com/blog/2014/fat-jars-with-excluded-dependencies-in-gradle>
and this
<https://discuss.gradle.org/t/removing-dependencies-from-a-jar-file-during-jar-task/5521>,
but it isn't working.
This is all my original gradle script contains:
apply plugin: 'java'
apply plugin: 'eclipse'
defaultTasks 'jar'
jar {
from {
(configurations.runtime).collect {
it.isDirectory() ? it : zipTree(it)
}
}
manifest {
attributes 'Main-Class': 'com.slh.myprogram.MyProgram'
}
}
sourceCompatibility = '1.8'
[compileJava, compileTestJava]*.options*.encoding = 'UTF-8'
if (!hasProperty('mainClass')) {
ext.mainClass = 'com.slh.myprogram.MyProgram'
}
repositories {
mavenCentral()
}
dependencies {
compile 'org.apache.storm:storm-core:1.0.1'
testCompile group: 'junit', name: 'junit', version: '4.10'
}
On Mon, May 23, 2016 at 3:18 PM, Matthias J. Sax <[email protected]> wrote:
> You are doing it basically correct. But you use the wrong jar file...
>
> The jar you submit must contain your spouts and bolt classes, but not
> the classes of Storm itself.
>
> I am not too familiar with Gradle, but you need to fix your build script
> to assemble your topology jar file correctly.
>
> It would help, if you post your gradle file here, for other to help you
> out with this.
>
> -Matthias
>
> On 05/23/2016 11:45 AM, Navin Ipe wrote:
> > Bingo. Supervisor.log showed:
> > /Worker Process 17d197c0-80c8-408d-ac70-11970a93d2b9 exited with code: 1
> > Worker Process 204c20f5-a6d2-4739-b010-c90e651cb2cf exited with code: 1
> > 204c20f5-a6d2-4739-b010-c90e651cb2cf still hasn't started
> > /
> > And the worker logs were located deep inside here
> >
> /Users/nav/programming/apache-storm-1.0.1/*logs/workers-artifacts*/MyTopologyName-6-1463986744/6701/worker.log.err
> >
> > *The error:*
> > /Caused by: java.io.IOException: Found multiple defaults.yaml resources.
> > You're probably bundling the Storm jars with your topology jar.
> >
> [jar:file:/Users/nav/programming/apache-storm-1.0.1/storm-local/supervisor/stormdist/MyTopologyName-6-1463986744/stormjar.jar!/defaults.yaml,
> >
> jar:file:/Users/nav/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar!/defaults.yaml]
> > at
> > org.apache.storm.utils.Utils.getConfigFileInputStream(Utils.java:333)
> > at
> > org.apache.storm.utils.Utils.findAndReadConfigFile(Utils.java:287)
> > ... 8 more
> > Exception in thread "main" java.lang.ExceptionInInitializerError/
> >
> > But I don't understand. If I don't include the storm jars in the Gradle
> > dependency, then I won't be able to compile the program itself.
> >
> > *What I'm doing:*
> > *Using this version of Storm via Gradle in Netbeans.*
> > dependencies { compile 'org.apache.storm:storm-core:1.0.1'
> >
> > *and submitting topology like this:*
> > System.setProperty("storm.jar",
> > "/Users/nav/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar");
> >
> > StormSubmitter.submitTopology(c.TOPO_NAME, config,
> > builder.createTopology());
> >
> > Is there some other way to do this?
> >
> >
> >
> > On Mon, May 23, 2016 at 1:36 PM, Matthias J. Sax <[email protected]
> > <mailto:[email protected]>> wrote:
> >
> > How many workers did you configure (ie, storm.yaml parameter
> > "supervisor.slots.ports" -- the number of available slots is also
> shown
> > in the web UI)?
> >
> > If there are free workers available, look into logs/worker*.log -- if
> > worker*.log is not available, no worker was started. You might also
> have
> > a look into logs/supervisor.log and maybe logs/nimbus.log
> >
> >
> > -Matthias
> >
> > On 05/23/2016 08:55 AM, Navin Ipe wrote:
> > > I installed zookeeper on my local system. Started it. Extracted
> Storm
> > > 1.0.1 on my local system and *started it via this script:*
> > > /#!/bin/bash
> > > ./storm nimbus &
> > > ./storm supervisor &
> > > ./storm ui &/
> > >
> > > *storm.yaml has:*
> > > / storm.zookeeper.servers:
> > > - "localhost"
> > > nimbus.seeds: ["localhost"]/
> > >
> > > *and zoo.conf has:*
> > > /tickTime=2000
> > > initLimit=10
> > > syncLimit=5
> > > dataDir=/Users/nav/programming/zookeeper-3.4.8/data
> > > clientPort=2181/
> > > *
> > > *
> > > *Using this version of Storm via Gradle.*
> > > /dependencies { compile 'org.apache.storm:storm-core:1.0.1'
> > >
> > > /
> > > *Firewall on my system
> > > *
> > > Switched off*
> > > *
> > > //
> > >
> > > *and submitting topology like this:*
> > > /System.setProperty("storm.jar",
> > >
> "/Users/nav/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar");
> > >
> > > StormSubmitter.submitTopology(c.TOPO_NAME, config,
> > > builder.createTopology());
> > > /
> > >
> > >
> > > But on running it in my program's build/lib folder with java -jar
> > > myProgram.jar, the topology gets submitted (and I can see the
> submitted
> > > topology on the storm UI via http://localhost:8080),*but after
> > > submitting, no emits happen. The program seems to freeze.*
> > > /BUILD SUCCESSFUL
> > >
> > > Total time: 12.78 secs
> > >
> > > This build could be faster, please consider using the Gradle
> Daemon:
> > > https://docs.gradle.org/2.10/userguide/gradle_daemon.html
> > > 569 [main] INFO c.s.c.s.Queries_ReaderWriter - Waiting few
> seconds
> > > until table is dropped...
> > > 589 [main] INFO c.s.c.s.Queries_ReaderWriter - New table created.
> > > 1007 [main] INFO o.a.s.StormSubmitter - Generated ZooKeeper secret
> > > payload for MD5-digest: -5943870921993217077:-9035830007806332208
> > > 1074 [main] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
> > > 1124 [main] INFO o.a.s.StormSubmitter - Uploading topology jar
> > > /Users/nav/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar
> to
> > > assigned location:
> > >
> /Users/nav/programming/apache-storm-1.0.1/storm-local/nimbus/inbox/stormjar-036472f9-15ce-4f09-8663-b1500942e696.jar
> > > 1271 [main] INFO o.a.s.StormSubmitter - Successfully uploaded
> topology
> > > jar to assigned location:
> > >
> /Users/nav/programming/apache-storm-1.0.1/storm-local/nimbus/inbox/stormjar-036472f9-15ce-4f09-8663-b1500942e696.jar
> > > 1271 [main] INFO o.a.s.StormSubmitter - Submitting topology
> MyTopology
> > > in distributed mode with conf
> > >
> {"storm.zookeeper.topology.auth.scheme":"digest","topology.workers":2,"topology.debug":false,"topology.max.spout.pending":10,"storm.zookeeper.topology.auth.payload":"-5943870921993217077:-9035830007806332208","topology.transfer.buffer.size":32,"topology.executor.send.buffer.size":16384,"topology.executor.receive.buffer.size":16384}
> > > 1427 [main] INFO o.a.s.StormSubmitter - Finished submitting
> topology:
> > > MyTopology
> > > 1427 [main] INFO c.s.c.CrystalStorm - Topology submitted/
> > >
> > > *Two questions:*
> > > 1. What usually causes this kind of problem? I've seen the answers
> on
> > > StackOverflow, but they didn't help.
> > > 2. Which log files would provide the most useful info in such
> > situations?
> > >
> > >
> > > --
> > > Regards,
> > > Navin
> >
> >
> >
> >
> > --
> > Regards,
> > Navin
>
>
--
Regards,
Navin
