IanD: Mind helping John out here? My hunch here is that this is because the slave does "chown()" after extracting ( https://github.com/apache/mesos/blob/master/src/launcher/fetcher.cpp#L258)?
>From POSIX standard, it looks like chown() when invoked by root doesn't clear the setuid bit for ordinary files but clears them for other types (e.g., binary). http://unix.stackexchange.com/questions/53665/chown-removes-sticky-bit-bug-or-feature http://pubs.opengroup.org/onlinepubs/009695399/utilities/chown.html On Wed, Sep 10, 2014 at 2:17 PM, John Omernik <[email protected]> wrote: > I am wondering about the process of fetching the tgz files and running > them on slaves. Basically, I am trying to run hadoop-mesos, but still use > the LinuxTaskController ( > http://hadoop.apache.org/docs/r1.0.4/cluster_setup.html for details). > > When I am using hadoop, I have to swich to the defaultTaskController > because when Mesos untars the tgz, it loses the setuid bit on the binary. > I've done a bit of testing around this, and I am unsure why it loses it > (even if the running process is root) but it does. > > Basically, tar by itself works like this: If the user is a super user, tar > maintain all permissions that are in the tgz. (I've tested this, when I > manually untar with tar zxf myhadoop.tgz it untars properly, including > permissions and setuid on the Linux Task Controller.) > > When I untar as a non-super user, the permissions all get moved to the > user that untared it, and the setuid bit is lost. It makes sense from a > security point of view. > > So how does this work in mesos and hadoop? > > Well, if I run the jobtracker as user hadoop, hadoop is not a super user, > all the files in the untared hadooop folder are owned by hadoop:hadoop, and > the setuid bit is lost. > > Ok, next test, well, let's run jobtracker as root, and see what happens. > (remember, when I untared as root, the setuid and all permissions were > preserved). So, when we run JT as root, all the files become root:root, > and the setuid bit is lost. That's weird? What happened here? (This is > where I get lost, perhaps the untar/gzipping isn't using the tar command > thus permissions are not preserved like I would expect) > > Either way, when using the LinuxTaskController, tasktrackers WILL NOT RUN > if the setuid bit is not set. That's a pain, the LinuxTaskController is > really nice from an impersonation/security setup with hadoop jobs. I CAN > run my hadoop framework as hadoop:hadoop, but then I am limited in how > things are setup and I get strange permissions issues when trying to run > certain jobs as other users. > > The fix? > > I am hoping we can have a discussion around this. As I see it, the slaves > are running as root, they have the power to run however we need them to > run. Ideally, I'd like to see the untarring happen with the preserve > permissions bit. I.e. the archives for mesos, at the very least having the > OPTION to preserve permissions in the tgz. If we could do this, as an > option somehow, this would be a win. > > Also ideally, I don't want to run the framework as root, just untar the > tgz as root, preserving permissions. There is a difference between the > action of untarring, and the execution of the framework, and the security > nerd in me would like to ensure while the slave COULD run the framework as > root, we avoid it if possible. > > I am not sure how exactly mesos untars things, nor am I aware how hard it > would be to do this, but I think from a security perspective, the > flexibility that untarting/preserving permissions (especially the setuid > bit) would bring Mesos would warrant the dev time. > > Thoughts? > > >
