Hi, This is the follow-up of the discussion we started yesterday on rootwrap usage at the cross-project meeting.
A bit of history to stage this story first. OpenStack nodes sometimes need to run things with elevated privileges. Nova started out as calling sudo shell commands to execute those, which basically allowed it to run anything as root. Distributions didn't really like that, so packagers started to ship sudoers files that restricted the commands Nova could run with sudo. It created a maintenance nightmare, where the code and the packaging were routinely out of sync. Rootwrap was designed so that we would maintain the allowed commands in the code itself, which facilitated maintenance. It would also allow more complex rules than sudoers allowed, like the ability to only kill a certain executable processes. Rootwrap was then adopted by Cinder and Neutron, and moved to oslo to avoid code duplication. There were still two long-standing problems with it, though. The first one is performance -- each call would spawn a Python interpreter which would then call the system command. This was fine when there were just a few calls here and there, not so much when it's called a hundred times in a row. During the Juno cycle, a daemon mode was added to solve this issue. It is significantly faster than running sudo directly (the often-suggested alternative). Projects still have to start adopting it though. Neutron and Cinder have started work to do that in Kilo. The second problem is the quality of the filter definitions. Rootwrap is a framework to enable isolation. It's only as good as the filters each project defines. Most of them rely on CommandFilters that do not check any argument, instead of using more powerful filters (which are arguably more painful to maintain). Developers routinely add filter definitions that basically remove any isolation that might have been there, like allowing blank dd, tee, chown or chmod. Basically, some nodes run such a variety of commands as root that maintaining proper isolation is a non-trivial amount of work: that's the case for Nova's compute nodes and Cinder's volume nodes. I would argue that for all other types of nodes, rootwrap is still an appropriate way to provide isolation. But for nodes which need to make a very wide variety of root calls, we pretend to isolate, but the filter definitions are so full of holes that we actually don't. What solutions do we have ? (1) we could get our act together and audit and fix those filter definitions. Remove superfluous usage of root rights, make use of advanced filters for where we actually need them. We have been preaching for that at many many design summits. This is a lot of work though... There were such efforts in the past, but they were never completed for some types of nodes. Worse, the bad filter definitions kept coming back, since developers take shortcuts, reviewers may not have sufficient security awareness to detect crappy filter definitions, and I don't think we can design a gate test that would have such awareness. (2) bite the bullet and accept that some types of nodes actually need root rights for so many different things, they should just run as root anyway. I know a few distributions which won't be very pleased by such a prospect, but that would be a more honest approach (rather than claiming we provide efficient isolation when we really don't). An added benefit is that we could replace a number of shell calls by Python code, which would simplify the code and increase performance. (3) intermediary solution where we would run as the nova user but run sudo COMMAND directly (instead of sudo nova-rootwrap CONFIG COMMAND). That would leave it up to distros to choose between a blanket sudoer or maintain their own filtering rules. I think it's a bit hypocritical though (pretend the distros could filter if they wanted it, when we dropped the towel on doing that ourselves). I'm also not convinced it's more secure than solution 2, and it prevents from reducing the number of shell-outs, which I think is a worthy idea. In all cases I would not drop the baby with the bath water, and keep rootwrap for all the cases where root rights are needed on a very specific set of commands (like neutron, or nova's api-metadata). The daemon mode should address the performance issue for the projects making a lot of calls. -- Thierry Carrez (ttx) __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
