Go ahead and run `env` in your script too, and see if there are any interesting differences when run via Marathon vs. directly. Maybe you're running in a different shell?
On Sun, May 10, 2015 at 2:21 PM, John Omernik <[email protected]> wrote: > I believe the slave IS running as root. FWIW when I ran the script from > above as root, it did work as intended (created the files on the NFS > share). > > On Sun, May 10, 2015 at 9:08 AM, Dick Davies <[email protected]> > wrote: > >> Any idea what user mesos is running as? This could just be a >> filesystem permission >> thing (ISTR last time I used NFS mounts, they had a 'root squash' >> option that prevented >> local root from writing to the NFS mount). >> >> On 9 May 2015 at 22:13, John Omernik <[email protected]> wrote: >> > I am not specifying isolators. The Default? :) Is that a per slave >> setting? >> > >> > On Sat, May 9, 2015 at 3:33 PM, James DeFelice < >> [email protected]> >> > wrote: >> >> >> >> What isolators are you using? >> >> >> >> On Sat, May 9, 2015 at 3:48 PM, John Omernik <[email protected]> wrote: >> >>> >> >>> Marco... great idea... thank you. I just tried it and it worked when >> I >> >>> had a /mnt/permtesting with the same permissions. So it appears >> something >> >>> to do with NFS and Mesos (Remember I tested just NFS that worked >> fine, it's >> >>> the combination that is causing this). >> >>> >> >>> On Sat, May 9, 2015 at 1:09 PM, Marco Massenzio <[email protected]> >> >>> wrote: >> >>>> >> >>>> Out of my own curiousity (sorry, I have no fresh insights into the >> issue >> >>>> here) did you try to run the script and write to a non-NFS mounted >> >>>> directory? (same ownership/permissions) >> >>>> >> >>>> This way we could at least find out whether it's something related to >> >>>> NFS, or a more general permission-related issue. >> >>>> >> >>>> Marco Massenzio >> >>>> Distributed Systems Engineer >> >>>> >> >>>> On Sat, May 9, 2015 at 5:10 AM, John Omernik <[email protected]> >> wrote: >> >>>>> >> >>>>> Here is the testing I am doing. I used a simple script (run.sh) It >> >>>>> writes the user it is running as to stderr (so it's the same log as >> the >> >>>>> errors from file writing) and then tries to make a directory in >> nfs, and >> >>>>> then touch a file in nfs. Note: This script directly run works on >> every >> >>>>> node. You can see the JSON I used in marathon, and in the sandbox >> results, >> >>>>> you can see the user is indeed darkness and the directory cannot be >> created. >> >>>>> However when directly run, it the script, with the same user, >> creates the >> >>>>> directory with no issue. Now, I realize this COULD still be a NFS >> quirk >> >>>>> here, however, this testing points at some restriction in how >> marathon kicks >> >>>>> off the cmd. Any thoughts on where to look would be very helpful! >> >>>>> >> >>>>> John >> >>>>> >> >>>>> >> >>>>> >> >>>>> Script: >> >>>>> >> >>>>> #!/bin/bash >> >>>>> echo "Writing whoami to stderr for one stop logging" 1>&2 >> >>>>> whoami 1>&2 >> >>>>> mkdir /mapr/brewpot/mesos/storm/test/test1 >> >>>>> touch /mapr/brewpot/mesos/storm/test/test1/testing.go >> >>>>> >> >>>>> >> >>>>> >> >>>>> Run Via Marathon >> >>>>> >> >>>>> >> >>>>> { >> >>>>> "cmd": "/mapr/brewpot/mesos/storm/run.sh", >> >>>>> "cpus": 1.0, >> >>>>> "mem": 1024, >> >>>>> "id": "permtest", >> >>>>> "user": "darkness", >> >>>>> "instances": 1 >> >>>>> } >> >>>>> >> >>>>> >> >>>>> I0509 07:02:52.457242 9562 exec.cpp:132] Version: 0.21.0 >> >>>>> I0509 07:02:52.462700 9570 exec.cpp:206] Executor registered on >> slave >> >>>>> 20150505-145508-1644210368-5050-8608-S0 >> >>>>> Writing whoami to stderr for one stop logging >> >>>>> darkness >> >>>>> mkdir: cannot create directory >> `/mapr/brewpot/mesos/storm/test/test1': >> >>>>> Permission denied >> >>>>> touch: cannot touch >> `/mapr/brewpot/mesos/storm/test/test1/testing.go': >> >>>>> No such file or directory >> >>>>> >> >>>>> >> >>>>> Run Via Shell: >> >>>>> >> >>>>> >> >>>>> $ /mapr/brewpot/mesos/storm/run.sh >> >>>>> Writing whoami to stderr for one stop logging >> >>>>> darkness >> >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/ >> >>>>> test1 >> >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/test1/ >> >>>>> testing.go >> >>>>> >> >>>>> >> >>>>> On Sat, May 9, 2015 at 3:14 AM, Adam Bordelon <[email protected]> >> >>>>> wrote: >> >>>>>> >> >>>>>> I don't know of anything inside of Mesos that would prevent you >> from >> >>>>>> writing to NFS. Maybe examine the environment variables set when >> running as >> >>>>>> that user. Or are you running in a Docker container? Those can have >> >>>>>> additional restrictions. >> >>>>>> >> >>>>>> On Fri, May 8, 2015 at 4:44 PM, John Omernik <[email protected]> >> wrote: >> >>>>>>> >> >>>>>>> I am doing something where people may recommend against my course >> of >> >>>>>>> action. However, I am curious if there is "a way" basically I >> have a process >> >>>>>>> being kicked off in marathon that is trying to write to a nfs >> location. The >> >>>>>>> permissions of the user running the task and the nfs location are >> good. So >> >>>>>>> what component of mesos or marathon is keeping me from writing >> here ? ( I >> >>>>>>> am getting permission denied). Is this one of those things that >> is just not >> >>>>>>> allowed, or is there an option to pass to marathon to allow >> this? Thanks ! >> >>>>>>> >> >>>>>>> -- >> >>>>>>> Sent from my iThing >> >>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>> >> >> >> >> >> >> >> >> -- >> >> James DeFelice >> >> 585.241.9488 (voice) >> >> 650.649.6071 (fax) >> > >> > >> > >
