https://issues.apache.org/jira/browse/MESOS-1790
On Fri, Sep 12, 2014 at 11:13 PM, John Omernik <j...@omernik.com> wrote: > I would but I am out of pocket for a week, if you put the ticket in, I'll > buy you a drink of your choice at the next Mesos con? ( I don't want this > to get lost as I am traveling) :) > On Sep 12, 2014 9:06 PM, "Vinod Kone" <vinodk...@gmail.com> wrote: > >> Having a "skip chown" option sounds good to me. We'll add the option to >> CommandInfo.URI so that frameworks can override the default if desired. >> Mind filing a ticket? >> >> On Thu, Sep 11, 2014 at 5:00 AM, John Omernik <j...@omernik.com> wrote: >> >>> Vinod - >>> >>> I believe this is EXACTLY the issue. I also understand why in most >>> cases this is ok. If a user is provided, then a fair assumption would be to >>> chown the extracted archive as that user. (Assuming the untar is happening >>> as root in all cases) So that leads to three components we may want to >>> make customizable by the framework: >>> >>> 1. Who untars the archive. Right now, it appears root untars the archive >>> (otherwise, I would imagine that the chown would be unneeded, if the user >>> untared the archive, the user would already have permissions, thus the >>> chown would not be needed). If it is root, perhaps this is "ok" to leave >>> as is? Another option may be to set the untar user separate from the >>> running user, but I am not sure we'd need to if root always untars. >>> >>> 2. Pass a flag from framework that allow a skipping of the chown. For >>> compatibility sakes, the flag would default to "off" so that it wouldn't >>> break existing things, but if the framework wanted, they could tell the >>> slave that the permissions are fine how they are set, and there is no need >>> to chown. I am not sure I understand the architecture of Mesos well enough >>> yet to comment on the best way to do this. Should it be a framework >>> variable? (Frameworks would have to be updated to make use of this) A >>> string in the filename (could this be abused?) Etc. >>> >>> 3. The user that runs the executor. This is already passed, and I am >>> not sure we need to change anything here. As long as A. Root untars the >>> archive, and B. We have the ability to skip the chown, the user stuff >>> should be perfectly ok as is. This way, in my case, root would untar the >>> archive, I could set the skip on chown, and then I'd have the user hadoop >>> run the framework. In this model, the LinuxTaskController should work. >>> >>> Thanks for looking into this, I welcome more thoughts on the subject. >>> >>> John >>> >>> >>> >>> On Wed, Sep 10, 2014 at 4:39 PM, Vinod Kone <vinodk...@gmail.com> wrote: >>> >>>> IanD: Mind helping John out here? >>>> >>>> My hunch here is that this is because the slave does "chown()" after >>>> extracting ( >>>> https://github.com/apache/mesos/blob/master/src/launcher/fetcher.cpp#L258 >>>> )? >>>> >>>> From POSIX standard, it looks like chown() when invoked by root doesn't >>>> clear the setuid bit for ordinary files but clears them for other types >>>> (e.g., binary). >>>> >>>> >>>> http://unix.stackexchange.com/questions/53665/chown-removes-sticky-bit-bug-or-feature >>>> http://pubs.opengroup.org/onlinepubs/009695399/utilities/chown.html >>>> >>>> >>>> On Wed, Sep 10, 2014 at 2:17 PM, John Omernik <j...@omernik.com> wrote: >>>> >>>>> I am wondering about the process of fetching the tgz files and running >>>>> them on slaves. Basically, I am trying to run hadoop-mesos, but still use >>>>> the LinuxTaskController ( >>>>> http://hadoop.apache.org/docs/r1.0.4/cluster_setup.html for details). >>>>> >>>>> When I am using hadoop, I have to swich to the defaultTaskController >>>>> because when Mesos untars the tgz, it loses the setuid bit on the binary. >>>>> I've done a bit of testing around this, and I am unsure why it loses it >>>>> (even if the running process is root) but it does. >>>>> >>>>> Basically, tar by itself works like this: If the user is a super user, >>>>> tar maintain all permissions that are in the tgz. (I've tested this, when >>>>> I >>>>> manually untar with tar zxf myhadoop.tgz it untars properly, including >>>>> permissions and setuid on the Linux Task Controller.) >>>>> >>>>> When I untar as a non-super user, the permissions all get moved to the >>>>> user that untared it, and the setuid bit is lost. It makes sense from a >>>>> security point of view. >>>>> >>>>> So how does this work in mesos and hadoop? >>>>> >>>>> Well, if I run the jobtracker as user hadoop, hadoop is not a super >>>>> user, all the files in the untared hadooop folder are owned by >>>>> hadoop:hadoop, and the setuid bit is lost. >>>>> >>>>> Ok, next test, well, let's run jobtracker as root, and see what >>>>> happens. (remember, when I untared as root, the setuid and all >>>>> permissions >>>>> were preserved). So, when we run JT as root, all the files become >>>>> root:root, and the setuid bit is lost. That's weird? What happened here? >>>>> (This is where I get lost, perhaps the untar/gzipping isn't using the tar >>>>> command thus permissions are not preserved like I would expect) >>>>> >>>>> Either way, when using the LinuxTaskController, tasktrackers WILL NOT >>>>> RUN if the setuid bit is not set. That's a pain, the LinuxTaskController >>>>> is really nice from an impersonation/security setup with hadoop jobs. I >>>>> CAN run my hadoop framework as hadoop:hadoop, but then I am limited in >>>>> how >>>>> things are setup and I get strange permissions issues when trying to run >>>>> certain jobs as other users. >>>>> >>>>> The fix? >>>>> >>>>> I am hoping we can have a discussion around this. As I see it, the >>>>> slaves are running as root, they have the power to run however we need >>>>> them >>>>> to run. Ideally, I'd like to see the untarring happen with the preserve >>>>> permissions bit. I.e. the archives for mesos, at the very least having the >>>>> OPTION to preserve permissions in the tgz. If we could do this, as an >>>>> option somehow, this would be a win. >>>>> >>>>> Also ideally, I don't want to run the framework as root, just untar >>>>> the tgz as root, preserving permissions. There is a difference between >>>>> the >>>>> action of untarring, and the execution of the framework, and the security >>>>> nerd in me would like to ensure while the slave COULD run the framework as >>>>> root, we avoid it if possible. >>>>> >>>>> I am not sure how exactly mesos untars things, nor am I aware how hard >>>>> it would be to do this, but I think from a security perspective, the >>>>> flexibility that untarting/preserving permissions (especially the setuid >>>>> bit) would bring Mesos would warrant the dev time. >>>>> >>>>> Thoughts? >>>>> >>>>> >>>>> >>>> >>> >>
