[
https://issues.apache.org/jira/browse/STORM-152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rick Kellogg updated STORM-152:
-------------------------------
Component/s: storm-core
> Permissions of resources are not preserved
> ------------------------------------------
>
> Key: STORM-152
> URL: https://issues.apache.org/jira/browse/STORM-152
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/229
> When the resources are copied from the resources directory to something like
> /tmp/09c4e529-7e64-496e-8afa-4990b7aba206/supervisor/stormdist/word-count-1-1337616827/resources
> the permissions of the files are not preserved, but rather all set to 644.
> This causes a problem because then we cannot execute binary files as bolts
> and spouts.
> ----------
> tobigue: I can second this issue. The workaround I ended up with for now is
> creating a shell script like this in the same folder:
> chmod u+x ./relative/path/to/mybinary
> ./relative/path/to/mybinary
> and calling
> super("sh", "myscript.sh")
> in the ShellSpout/ShellBolt instead of calling the binary directly.
> ----------
> analogue: Tobigue,
> I don't see how your workaround actually works because myscript.sh will not
> be executable after after being unzipped and will fail to run.
> My workaround is as follows:
> For a given spout or bolt:
> public static class MySpout extends ShellSpout implements IRichSpout {
> public MySpout() {
> super("bash", "-c", "chmod +x ./pystorm;./pystorm my_spout.py
> arg1 arg2 argN");
> }
> And a shell script multilang/resources/pystorm:
> #!/bin/sh
> PYTHONPATH=`pwd` python $*
> This also handles not being able to set environment variables :-)
> I'm thinking that ShellSpout should have an additional constructor which
> takes a ProcessBuilder to provide maximum flexibility. There are downsides to
> this though. The author of the spout or bolt will then have access to the
> ProcessBuilder and could do something to screw things up like inadvertently
> reading/writing to the InputStream or OutputStream in an effort to debug
> things.
> ----------
> tobigue: Hi!
> I think for me it worked, because I had the permission to execute sh.
> So this works:
> sh myscript.sh
> while this does not
> ./myscript.sh
> as there are no execute permissions on the file itself.
> ----------
> analogue: Ok, I see my problem. bash -c ./myscript.sh fails if the script is
> not executable but bash ./myscript.sh doesn't.
> ----------
> alien2150: I am having the same problem with a C++ programm. The original
> permissions are:
> ls -l target/classes/resources/xy
> -rwxr-xr-x 1 tech tech 1080184 Jul 19 17:26 target/classes/resources/xy
> But later it is (ls -l
> /tmp/448ce6ba-ccd2-4335-a119-903381aee09c/supervisor/stormdist/foo-1374248239/resources):
> -rw-rw-rw- 1 tech tech 1080184 Jul 19 17:37 xy
> Is there any solution for this?
> ----------
> revans2: jar files are really just zip files and do not store any kind of
> permissions. There are some extensions to zip that do store permissions but
> jar does not use those. Because these resources are pushed to the nodes
> through a jar the permissions are being lost. Hadoop gets around this by
> doing the really ugly thing of changing all of the permissions to be user
> executable after unzipping the file. We could try something like this in the
> supervisor when it unzips the files but I would rather provide a different
> way of distributing files beyond a single jar file once the bit-torrent code
> is in place.
> ----------
> patricklucas: We can reproduce this consistently by launching a topology to a
> "fresh" cluster. It appears that if the storm-local directory has to be
> created, unpacked files will not have the right permissions. If it is a
> subsequent run, and the storm-local directory already exists, permissions of
> unpacked files are correct.
> Will this be fixed in 0.9.0?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
