Erik Weathers created STORM-2191:
------------------------------------
Summary: shorten classpaths in worker and LogWriter commands
Key: STORM-2191
URL: https://issues.apache.org/jira/browse/STORM-2191
Project: Apache Storm
Issue Type: Task
Components: storm-core
Affects Versions: 1.0.2
Reporter: Erik Weathers
Priority: Minor
When launching the worker daemon and its wrapping LogWriter daemon, the
commands can become so long that they eclipse the default Linux linux of 4096
bytes. That results in commands that are cut off in {{ps}} output, and prevents
easily debugging.
The specific scenario in which this problem can be easily triggered: *running
Storm on Mesos*.
h5. Details on why it happens:
# using the default Mesos containerizer instead of Docker containers, which
causes the storm-mesos package to be unpacked into the Mesos executor sandbox.
# The ["expand all jars on
classpath"|https://github.com/apache/storm/blob/master/bin/storm.py#L114-L140]
functionality in the {{bin/storm.py}} script causes every one of the jars that
storm bundles into its lib directory to be explicitly listed in the command.
#* e.g., say the mesos work dir is {{/var/run/mesos/work_dir/}}
#* and say that the original classpath argument in the supervisor cmd includes
the following for the {{lib/}} dir in the binary storm package:
#**
{{/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/*}}
#* That leads to a hugely expanded classpath argument for the LogWriter and
Worker daemons that get launched:
#**
{{/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/asm-4.0.jar:/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/carbonite-1.4.0.jar:...}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
