Storm : 0.10.1
Streamparse : 2.1.4
I was running storm nimbus in the same IP
(PYTHON) $ sudo netstat -lpn | grep :6627
tcp 0 0 :::6627 :::*
LISTEN 29199/java
(PYTHON) ]$
Now, when I ran topology in different IP, it works fine
{{{
$ sparse submit
Cleaning from prior builds...
Creating topology Uber-JAR...
Uber-JAR created:
/home/ram/PYTHON/test/_build/Data-0.0.1-SNAPSHOT-standalone.jar
Deploying "customer" topology...
[email protected]'s password:
ssh tunnel to Nimbus 10.218.173.100:6627 established.
Routing Python logging to /home/ram/log/storm/streamparse.
Running lein command to submit topology to nimbus:
lein run -m streamparse.commands.submit_topology/-main
topologies/customer.clj --option 'topology.workers=2' --option
'topology.acker.executors=2' --option '
*streamparse.log.path="/home/ram/log/storm/streamparse"'* --option
'streamparse.log.max_bytes=1000000' --option
'streamparse.log.backup_count=10' --option 'streamparse.log.level="info"'
2490 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
{:option {streamparse.log.level info, streamparse.log.backup_count 10,
streamparse.log.max_bytes 1000000, streamparse.log.path
/home/ram/log/storm/streamparse, topology.acker.executors 2,
topology.workers 2}, :debug false, :port 6627, :host localhost, :help false}
3494 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
3526 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload
for MD5-digest: -7825715999852442250:-4871326514649406713
3528 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds []
3540 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
3546 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
3568 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
3610 [main] INFO b.s.StormSubmitter - Uploading topology jar
/home/ram/PYTHON/customer/_build/Data-0.0.1-SNAPSHOT-standalone.jar to
assigned location:
/home/ram/log/nimbus/inbox/stormjar-f35e27cf-460a-4146-ba4b-348ed7c3ffc6.jar
5927 [main] INFO b.s.StormSubmitter - Successfully uploaded topology jar
to assigned location:
/home/ram/log/nimbus/inbox/stormjar-f35e27cf-460a-4146-ba4b-348ed7c3ffc6.jar
5927 [main] INFO b.s.StormSubmitter - Submitting topology data in
distributed mode with conf
{"storm.zookeeper.topology.auth.scheme":"digest","streamparse.log.backup_count":10,"storm.zookeeper.topology.auth.payload":"-7825715999852442250:-4871326514649406713","streamparse.log.path":"\/home\/ram\/storm\/streamparse","topology.debug":false,"nimbus.thrift.port":6627,"topology.max.spout.pending":5000,"nimbus.host":"localhost","topology.acker.executors":2,"topology.workers":2,"streamparse.log.max_bytes":1000000,"streamparse.log.level":"info","topology.message.timeout.secs":60}
6033 [main] INFO b.s.StormSubmitter - *Finished submitting topology: data*
$
}}}
Then I produced message to Kafka broker
But, I can't see the logs for the topology "data"
in /home/ram/log/storm/streamparse
How will I check the submitted topology's logs
Thanks
On Thu, May 19, 2016 at 12:29 AM, cogumelosmaravilha <
[email protected]> wrote:
> Can you tell us your Apache Storm version, and Streamparse version.
> And the size of the jar file created using streamparse?
> How do you check the port lsof, netstat?
>
> Thanks
>
>
>
> On 18-05-2016 15:06, ram kumar wrote:
>
> Hi,
>
> I tried to submit a storm topology in production mode (streamparse)
>
>
>
> $ *sparse submit*
> Cleaning from prior builds...
> Creating topology Uber-JAR...
> Uber-JAR created:
> /home/ram/PYTHON/test/_build/Data-0.0.1-SNAPSHOT-standalone.jar
> Deploying "customer-metadata" topology...
> Traceback (most recent call last):
> File "/home/ram/PYTHON2.7-ENV/bin/sparse", line 11, in <module>
> sys.exit(main())
> File
> "/home/ram/PYTHON2.7-ENV/lib/python2.7/site-packages/streamparse/cli/sparse.py",
> line 57, in main
> args.func(args)
> File
> "/home/ram/PYTHON2.7-ENV/lib/python2.7/site-packages/streamparse/cli/submit.py",
> line 247, in main
> wait=args.wait, simple_jar=args.simple_jar)
> File
> "/home/ram/PYTHON2.7-ENV/lib/python2.7/site-packages/streamparse/cli/submit.py",
> line 197, in submit_topology
> with ssh_tunnel(env_config.get("user"), host, 6627, port):
> File "/usr/local/lib/python2.7/contextlib.py", line 17, in __enter__
> return self.gen.next()
> File
> "/home/ram/PYTHON/lib/python2.7/site-packages/streamparse/contextmanagers.py",
> line 32, in ssh_tunnel
> "tunnel to {}:{}.".format(local_port, host, remote_port))
> *IOError: Local port: 6627 already in use, unable to open ssh tunnel to
> 10.154.9.166:6627 <http://10.154.9.166:6627>*
>
>
> I check if the port 6627 is busy. But its not.
>
> Can anyone help me with this,
>
> Thanks
>
>
>
>
