Any idea what user mesos is running as? This could just be a filesystem permission thing (ISTR last time I used NFS mounts, they had a 'root squash' option that prevented local root from writing to the NFS mount).
On 9 May 2015 at 22:13, John Omernik <[email protected]> wrote: > I am not specifying isolators. The Default? :) Is that a per slave setting? > > On Sat, May 9, 2015 at 3:33 PM, James DeFelice <[email protected]> > wrote: >> >> What isolators are you using? >> >> On Sat, May 9, 2015 at 3:48 PM, John Omernik <[email protected]> wrote: >>> >>> Marco... great idea... thank you. I just tried it and it worked when I >>> had a /mnt/permtesting with the same permissions. So it appears something >>> to do with NFS and Mesos (Remember I tested just NFS that worked fine, it's >>> the combination that is causing this). >>> >>> On Sat, May 9, 2015 at 1:09 PM, Marco Massenzio <[email protected]> >>> wrote: >>>> >>>> Out of my own curiousity (sorry, I have no fresh insights into the issue >>>> here) did you try to run the script and write to a non-NFS mounted >>>> directory? (same ownership/permissions) >>>> >>>> This way we could at least find out whether it's something related to >>>> NFS, or a more general permission-related issue. >>>> >>>> Marco Massenzio >>>> Distributed Systems Engineer >>>> >>>> On Sat, May 9, 2015 at 5:10 AM, John Omernik <[email protected]> wrote: >>>>> >>>>> Here is the testing I am doing. I used a simple script (run.sh) It >>>>> writes the user it is running as to stderr (so it's the same log as the >>>>> errors from file writing) and then tries to make a directory in nfs, and >>>>> then touch a file in nfs. Note: This script directly run works on every >>>>> node. You can see the JSON I used in marathon, and in the sandbox >>>>> results, >>>>> you can see the user is indeed darkness and the directory cannot be >>>>> created. >>>>> However when directly run, it the script, with the same user, creates the >>>>> directory with no issue. Now, I realize this COULD still be a NFS quirk >>>>> here, however, this testing points at some restriction in how marathon >>>>> kicks >>>>> off the cmd. Any thoughts on where to look would be very helpful! >>>>> >>>>> John >>>>> >>>>> >>>>> >>>>> Script: >>>>> >>>>> #!/bin/bash >>>>> echo "Writing whoami to stderr for one stop logging" 1>&2 >>>>> whoami 1>&2 >>>>> mkdir /mapr/brewpot/mesos/storm/test/test1 >>>>> touch /mapr/brewpot/mesos/storm/test/test1/testing.go >>>>> >>>>> >>>>> >>>>> Run Via Marathon >>>>> >>>>> >>>>> { >>>>> "cmd": "/mapr/brewpot/mesos/storm/run.sh", >>>>> "cpus": 1.0, >>>>> "mem": 1024, >>>>> "id": "permtest", >>>>> "user": "darkness", >>>>> "instances": 1 >>>>> } >>>>> >>>>> >>>>> I0509 07:02:52.457242 9562 exec.cpp:132] Version: 0.21.0 >>>>> I0509 07:02:52.462700 9570 exec.cpp:206] Executor registered on slave >>>>> 20150505-145508-1644210368-5050-8608-S0 >>>>> Writing whoami to stderr for one stop logging >>>>> darkness >>>>> mkdir: cannot create directory `/mapr/brewpot/mesos/storm/test/test1': >>>>> Permission denied >>>>> touch: cannot touch `/mapr/brewpot/mesos/storm/test/test1/testing.go': >>>>> No such file or directory >>>>> >>>>> >>>>> Run Via Shell: >>>>> >>>>> >>>>> $ /mapr/brewpot/mesos/storm/run.sh >>>>> Writing whoami to stderr for one stop logging >>>>> darkness >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/ >>>>> test1 >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/test1/ >>>>> testing.go >>>>> >>>>> >>>>> On Sat, May 9, 2015 at 3:14 AM, Adam Bordelon <[email protected]> >>>>> wrote: >>>>>> >>>>>> I don't know of anything inside of Mesos that would prevent you from >>>>>> writing to NFS. Maybe examine the environment variables set when running >>>>>> as >>>>>> that user. Or are you running in a Docker container? Those can have >>>>>> additional restrictions. >>>>>> >>>>>> On Fri, May 8, 2015 at 4:44 PM, John Omernik <[email protected]> wrote: >>>>>>> >>>>>>> I am doing something where people may recommend against my course of >>>>>>> action. However, I am curious if there is "a way" basically I have a >>>>>>> process >>>>>>> being kicked off in marathon that is trying to write to a nfs location. >>>>>>> The >>>>>>> permissions of the user running the task and the nfs location are good. >>>>>>> So >>>>>>> what component of mesos or marathon is keeping me from writing here ? >>>>>>> ( I >>>>>>> am getting permission denied). Is this one of those things that is just >>>>>>> not >>>>>>> allowed, or is there an option to pass to marathon to allow this? >>>>>>> Thanks ! >>>>>>> >>>>>>> -- >>>>>>> Sent from my iThing >>>>>> >>>>>> >>>>> >>>> >>> >> >> >> >> -- >> James DeFelice >> 585.241.9488 (voice) >> 650.649.6071 (fax) > >
