Re: [Python-Dev] Python jails
Le 11/06/2011 02:41, R. David Murray a écrit : I haven't read through your post, but if you don't know about it I suspect that you will be interested in the following: http://code.activestate.com/pypm/pysandbox/ I'm pretty sure Victor will be happy to have someone else interested in this topic. Yes, I am happy :-) The project URL is https://github.com/haypo/pysandbox/ Activestate page is wrong: pysanbox does support Python 3 (Python 2.5 - 3.3). pysandbox uses different policy depending on the problem. For example, whitelist for builtins, blacklist for object attributes. pysandbox is based on Tav's ideas. The main idea of pysandbox is to execute untrusted in a new empty namespace, the untrusted namespace. Objects imported into this namespace are imported as proxies to get a read-only view of the Python namespace. Importing modules is protected by a whitelist (modules and symbols names). To protect the namespace, some introspection attributes are hidden like __subclasses__ or __self__. Performances are supposed to be close to a classic Python interpreter (I didn't run a benchmark, I don't really care). An empty namespace is not enough to protect Python: pysandbox denies the execution of arbitrary bytecode, write files, write to stdout/stderr, exit Python, etc. Tav's sandbox is good to deny everything, whereas you can configure pysandbox to enable some features (e.g. exit Python, useful for an interpreter). About restricted mode: you can also configure pysandbox to use it, but the restricted mode is too much restrictive: you cannot open files, whereas pysandbox allows to read files in a whitelist (e.g. useful to display a backtrace). If you would like to implement your own sandbox: great! You should try pysandbox test suite, I'm proud of it :-) I am still not sure that pysandbox approach is the good one: if you find a vulnerability to escape pysandbox jail (see pysandbox Changelog, it would not be the first time), you can do anything. PyPy sandbox and Seccomp nurse (specific to Linux?) use two processes: the Python process cannot do anything, it relies completly in a trusted process which control all operations. I don't understand exactly how it is different: a vulnerability in the trusted process gives also full control, but it's maybe a safer approach. Or the difference is maybe that the implementation is simpler (less code?) and so safer (less code usually means less bugs). Seccomp nurse: http://chdir.org/~nico/seccomp-nurse/ I tested recently AppEngine sandbox (testable online via http://shell.appspot.com/): it is secure *and* powerful, quite all modules are allowed (except not ctypes, as expected). AppEngine is not a Python sandbox: it's a sandbox between the Python process and Linux kernel, so it protects also modules written in C (pysandbox is unable to protect these modules). AppEngine modifies the Python standard library to cooperate with the low-level sandbox, e.g. raise nice error messages with open(filename, w): invalid file mode (instead of an ugly OSError with a cryptic message). Get more information about pysandbox and other sandboxes in pysandbox README file. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python jails
Hello! This is my first posting to the python-dev list, so please
forgive me if I violate any unspoken etiquette here. :)
I was looking at Python 2.x's f_restricted frame flag (or, rather, the
numerous ways around it) and noticed that most (all?)
of the attacks to escape restricted execution involved the attacker
grabbing something he wasn't supposed to have.
IMO, Python's extensive introspection features make that a losing
battle, since it's simply too easy to forget to blacklist
something and the attacker finding it. Not only that, even with a
perfect vacuum-sealed jail, an attacker can still bring down
the interpreter by exhausting memory or consuming excess CPU.
I think I might have a way of securely sealing-in untrusted code. It's a
fairly nascent idea, though, and I haven't worked out
all of the details yet, so I'm posting what I have so far for feedback
and for others to try to poke holes in it.
Absolutely nothing here is final. I'm just framing out what I generally
had in mind. Obviously, it will need to be adjusted to
be consistent with the Python way - my hope is that this can become a
PEP. :)
# It all starts with the introduction of a new type, called a jail.
(I haven't yet worked out whether it should be a builtin type,
... # or a module.) Unjailed code can create jails, which will run the
untrusted code and keep strict limits on it.
...
j = jail()
dir(j)
['__class__', '__delattr__', '__doc__', '__format__',
'__getattribute__', '__hash__',
'__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__',
'__sizeof__', '__str__', '__subclasshook__', 'acquire', 'getcpulimit',
'getcpuusage',
'getmemorylimit', 'getmemoryusage', 'gettimelimit', 'gettimeusage',
'release',
'setcpulimit', 'setmemorylimit', 'settimelimit']
# The jail monitors three things: Memory (in bytes), real time (in
seconds), and CPU time (also in seconds)
... # and it also allows you to impose limits on them. If any limit is
non-zero, code in that jail may not exceed its limit.
... # Exceeding a memory limit will result in a MemoryError. I haven't
decided what CPU/real time limits should raise.
... # The other two calls are acquire and release, which allow you
to seal (any) objects inside the jail, or bust them
# out. Objects inside the jail (i.e. created by code in that jail)
contribute their __sizeof__() to the j.getmemoryusage()
...
def stealPasswd():
... return open('/etc/passwd','r').read()
...
j.acquire(stealPasswd)
j.getmemoryusage() # The stealPasswd function, its code, etc. are
now locked away within the jail.
375
stealPasswd()
Traceback (most recent call last):
File stdin, line 1, in module
JailError: tried to access an object outside of the jail
The object in question is, of course, 'open'. Unlike the f_restricted
model, the jail was freely able to grab
the open() function, but was absolutely unable to touch it: It can't
call it, set/get/delete attributes/items,
or pass it as an argument to any functions. There are three criteria
that determine whether an object can
be accessed:
a. The code accessing the object is not within a jail; or
b. The object belongs to the same jail as the code accessing the object; or
c. The object has an __access__ function, and
theObject.__access__(theJail) returns True.
For the jail to be able to access 'open', it needs to be given access
explicitly. I haven't quite decided
how this should work, but I had in mind the creation of a guard
(essentially a proxy) that allows the jail
to access the object. It belongs to the same jail as the guarded object
(and is therefore impossible to create
within a jail unless the guarded object belongs to the same jail), has a
list of jails (or None for 'any') that the
guard will allow to __access__ it (the guard is immutable, so jails
can't mess with it even though they can
access it), and what the guard will allow though it (read-write,
read-only, call-within-jail, call-outside-jail).
I have a couple remaining issues that I haven't quite sussed out:
* How exactly do guards work? I had in mind a system of proxies (memory
usage is a concern, especially
in memory-limited jails - maybe allow __access__ to return specific
modes of access rather than
all-or-nothing?) that recursively return more guards after
operations. (e.g., if I have a guard allowing
read+call on sys, sys.stdout would return another guard allowing
read+call on sys.stdout, likewise for
sys.stdout.write)
* How are objects sealed in the jail? j.acquire can lead to serious
problems with lots of references
getting recursively sealed in. Maybe disallow sealing in anything
but code objects, or allow explicitly
running code within a jail like j.execute(code, globals(),
locals()), which works fine since any objects
created by jailed code are also jailed.
* How do imports work? Should __import__ be modified so that when a jail
invokes it, the import runs
normally (unjailed), and then returns the module with a special
guard
Re: [Python-Dev] Python jails
On Fri, 10 Jun 2011 18:23:47 -0600, Sam Edwards sam.edwa...@colorado.edu wrote: Hello! This is my first posting to the python-dev list, so please forgive me if I violate any unspoken etiquette here. :) Well, hopefully we won't bite, though of course I can't promise anything for anyone else :) I haven't read through your post, but if you don't know about it I suspect that you will be interested in the following: http://code.activestate.com/pypm/pysandbox/ I'm pretty sure Victor will be happy to have someone else interested in this topic. -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python jails
Hi Sam,
Have you seen this?
http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html
It might relate a similar idea. There were a few iterations of Tav's approach.
--Guido
On Fri, Jun 10, 2011 at 5:23 PM, Sam Edwards sam.edwa...@colorado.edu wrote:
Hello! This is my first posting to the python-dev list, so please
forgive me if I violate any unspoken etiquette here. :)
I was looking at Python 2.x's f_restricted frame flag (or, rather, the
numerous ways around it) and noticed that most (all?)
of the attacks to escape restricted execution involved the attacker
grabbing something he wasn't supposed to have.
IMO, Python's extensive introspection features make that a losing
battle, since it's simply too easy to forget to blacklist
something and the attacker finding it. Not only that, even with a
perfect vacuum-sealed jail, an attacker can still bring down
the interpreter by exhausting memory or consuming excess CPU.
I think I might have a way of securely sealing-in untrusted code. It's a
fairly nascent idea, though, and I haven't worked out
all of the details yet, so I'm posting what I have so far for feedback
and for others to try to poke holes in it.
Absolutely nothing here is final. I'm just framing out what I generally
had in mind. Obviously, it will need to be adjusted to
be consistent with the Python way - my hope is that this can become a
PEP. :)
# It all starts with the introduction of a new type, called a jail.
(I haven't yet worked out whether it should be a builtin type,
... # or a module.) Unjailed code can create jails, which will run the
untrusted code and keep strict limits on it.
...
j = jail()
dir(j)
['__class__', '__delattr__', '__doc__', '__format__',
'__getattribute__', '__hash__',
'__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__',
'__sizeof__', '__str__', '__subclasshook__', 'acquire', 'getcpulimit',
'getcpuusage',
'getmemorylimit', 'getmemoryusage', 'gettimelimit', 'gettimeusage',
'release',
'setcpulimit', 'setmemorylimit', 'settimelimit']
# The jail monitors three things: Memory (in bytes), real time (in
seconds), and CPU time (also in seconds)
... # and it also allows you to impose limits on them. If any limit is
non-zero, code in that jail may not exceed its limit.
... # Exceeding a memory limit will result in a MemoryError. I haven't
decided what CPU/real time limits should raise.
... # The other two calls are acquire and release, which allow you
to seal (any) objects inside the jail, or bust them
# out. Objects inside the jail (i.e. created by code in that jail)
contribute their __sizeof__() to the j.getmemoryusage()
...
def stealPasswd():
... return open('/etc/passwd','r').read()
...
j.acquire(stealPasswd)
j.getmemoryusage() # The stealPasswd function, its code, etc. are
now locked away within the jail.
375
stealPasswd()
Traceback (most recent call last):
File stdin, line 1, in module
JailError: tried to access an object outside of the jail
The object in question is, of course, 'open'. Unlike the f_restricted
model, the jail was freely able to grab
the open() function, but was absolutely unable to touch it: It can't
call it, set/get/delete attributes/items,
or pass it as an argument to any functions. There are three criteria
that determine whether an object can
be accessed:
a. The code accessing the object is not within a jail; or
b. The object belongs to the same jail as the code accessing the object; or
c. The object has an __access__ function, and
theObject.__access__(theJail) returns True.
For the jail to be able to access 'open', it needs to be given access
explicitly. I haven't quite decided
how this should work, but I had in mind the creation of a guard
(essentially a proxy) that allows the jail
to access the object. It belongs to the same jail as the guarded object
(and is therefore impossible to create
within a jail unless the guarded object belongs to the same jail), has a
list of jails (or None for 'any') that the
guard will allow to __access__ it (the guard is immutable, so jails
can't mess with it even though they can
access it), and what the guard will allow though it (read-write,
read-only, call-within-jail, call-outside-jail).
I have a couple remaining issues that I haven't quite sussed out:
* How exactly do guards work? I had in mind a system of proxies (memory
usage is a concern, especially
in memory-limited jails - maybe allow __access__ to return specific
modes of access rather than
all-or-nothing?) that recursively return more guards after
operations. (e.g., if I have a guard allowing
read+call on sys, sys.stdout would return another guard allowing
read+call on sys.stdout, likewise for
sys.stdout.write)
* How are objects sealed in the jail? j.acquire can lead to serious
problems with lots of references
getting recursively sealed in. Maybe disallow sealing in anything
but code
Re: [Python-Dev] Python jails
At 06:23 PM 6/10/2011 -0600, Sam Edwards wrote: I have a couple remaining issues that I haven't quite sussed out: [long list of questions deleted] You might be able to answer some of them by looking at this project: http://pypi.python.org/pypi/RestrictedPython Which implements the necessary ground machinery for doing that sort of thing, in the form of a specialized Python compiler (implemented in Python, for 2.3 through 2.7) that allows you to implement whatever sorts of guards and security policies you want on top of it. Even if it doesn't answer all your questions in and of itself, it may prove a fruitful environment in which you can experiment with various approaches and see which ones you actually like, without first having to write a bunch of code yourself. Discussing an official implementation of this sort of thing as a language feature is probably best left to python-ideas, though, until and unless you actually have a PEP to propose. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python jails
All, Thanks for the quick responses! I've skimmed the pysandbox code yesterday. I think Victor has the right idea with relying on a whitelist, as well as limiting execution time. The fact that untrusted code can still execute memory exhaustion attacks is the only thing that still worries me: It's hard to write a server that will run hundreds of scripts from untrusted users, since one of them can bring down the entire server by writing an infinite loop that allocates tons of objects. Python needs a way to hook the object-allocation process in order to (effectively) limit how much memory untrusted code can consume. Tav's blog post makes some interesting points... The object-capability model definitely has the benefit of efficiency; simply getting the reference to an object means the untrusted code is trusted with full capability to that object (which saves having to query the jail every time the object is touched) - it's just as fast as unrestricted Python, which I like. Perhaps my jails idea should then be refactored into some mechanism for monitoring and limiting memory and CPU usage -- it's the perfect thing to ship as an extension, the only shame is that it requires interpreter support. Anyway, in light of Tav's post which seems to suggest that f_restricted frames are impossible to escape (if used correctly), why was f_restricted removed in Python 3? Is it simply that it's too easy to make a mistake and accidentally give an attacker an unsafe object, or is there some fundamental flaw with it? Could you see something like f_restricted (or f_jail) getting put back in Python 3, if it were a good deal more bulletproof? And, yeah, I've been playing with RestrictedPython. It's pretty good, but it lacks memory- and CPU-limiting, which is my main focus right now. And yes, I should probably have posted this to python-ideas, thanks. :) This is a very long way away from a PEP. PyPy's sandboxing feature is probably closest to what I'd like, but I'm looking for something that can coexist in the same process (since running hundreds of interpreter processes continuously has a lot of system memory overhead, it's better if the many untrusted, but independent, jails could share a single interpreter) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
