[
https://issues.apache.org/jira/browse/STORM-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985866#comment-15985866
]
Erik Weathers commented on STORM-2191:
--------------------------------------
[~revans2] sweet. :-) I'm just glad that when Mr. Marz added
{{get_jars_full}}, he didn't put in some sort method. I'm hopeful this just
means it was done for functionality reasons as [I guessed
above|https://issues.apache.org/jira/browse/STORM-2191?focusedCommentId=15981693&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15981693].
I do think we should still address the conflicts we've noted above, and enable
some CI test to bail out if a conflict arises, as that is probably the best way
to avoid such problems arising in the future. That can be separate, I'll file
a ticket for it. I'll send a PR soon with my changes for wildcarding.
> shorten classpaths in worker and LogWriter commands
> ---------------------------------------------------
>
> Key: STORM-2191
> URL: https://issues.apache.org/jira/browse/STORM-2191
> Project: Apache Storm
> Issue Type: Task
> Components: storm-core
> Affects Versions: 1.0.2
> Reporter: Erik Weathers
> Priority: Minor
> Labels: cli, command-line
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When launching the worker daemon and its wrapping LogWriter daemon, the
> commands can become so long that they eclipse the default Linux limit of 4096
> bytes. That results in commands that are cut off in {{ps}} output, and
> prevents easily inspecting the system to see even what processes are running.
> The specific scenario in which this problem can be easily triggered: *running
> Storm on Mesos*.
> h5. Details on why it happens:
> # using the default Mesos containerizer instead of Docker containers, which
> causes the storm-mesos package to be unpacked into the Mesos executor sandbox.
> # The ["expand all jars on
> classpath"|https://github.com/apache/storm/blob/6dc6407a01d032483edebb1c1b4d8b69a304d81c/bin/storm.py#L114-L140]
> functionality in the {{bin/storm.py}} script causes every one of the jars
> that storm bundles into its lib directory to be explicitly listed in the
> command.
> #* e.g., say the mesos work dir is {{/var/run/mesos/work_dir/}}
> #* and say that the original classpath argument in the supervisor cmd
> includes the following for the {{lib/}} dir in the binary storm package:
> #**
> {{/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/*}}
> #* That leads to a hugely expanded classpath argument for the LogWriter and
> Worker daemons that get launched:
> #**
> {{/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/asm-4.0.jar:/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/carbonite-1.4.0.jar:...}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
