Re: [Python-Dev] Pathlib enhancments - method name only
Nick Coghlan wrote: Similar to my proposal for dealing with DirEntry.path being a bytes-like object, I'd like to suggest raising TypeError in __fspath__ if the request is nonsensical for the currently running system - *nix systems can *manipulate* Windows paths (and vice-versa), but actually trying to *use* them with the local filesystem isn't going to work properly, since the syntax and semantics are different. That sounds reasonable, since it would be preferable to fail early if you mistakenly pass a PureWindowsPath to e.g. open(). But there needs to be some way to ask a path object for its native string representation, otherwise there would be no point in using foreign path objects at all. -- Greg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
On 10 April 2016 at 17:12, Greg Ewing wrote:
> Nick Coghlan wrote:
>>
>> Similar to my proposal for dealing with DirEntry.path being a
>> bytes-like object, I'd like to suggest raising TypeError in __fspath__
>> if the request is nonsensical for the currently running system - *nix
>> systems can *manipulate* Windows paths (and vice-versa), but actually
>> trying to *use* them with the local filesystem isn't going to work
>> properly, since the syntax and semantics are different.
>
>
> That sounds reasonable, since it would be preferable to
> fail early if you mistakenly pass a PureWindowsPath to
> e.g. open().
>
> But there needs to be some way to ask a path object for
> its native string representation, otherwise there would
> be no point in using foreign path objects at all.
In addition to the existing "str(pathobj)", a "path" property was
recently added for that purpose:
>>> import pathlib
>>> pathlib.PureWindowsPath(".")
PureWindowsPath('.')
>>> pathlib.PureWindowsPath(".").path
'.'
(The specific property name was chosen to match os.scandir's DirEntry.path)
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
On 10 April 2016 at 08:36, Nick Coghlan wrote:
> In addition to the existing "str(pathobj)", a "path" property was
> recently added for that purpose:
>
>>>> import pathlib
>>>> pathlib.PureWindowsPath(".")
>PureWindowsPath('.')
>>>> pathlib.PureWindowsPath(".").path
>'.'
>
> (The specific property name was chosen to match os.scandir's DirEntry.path)
I believe that under the current proposal, the ".path" property will
be removed again in favour of the new protocol, so the only actual
option would be str(pathobj).
Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pathlib+os/shutil feedback
I talked to my colleague. He didn't remember the concrete use-case, though, he instantly mentioned three possible things (no order of preference): 1) pathlib + mtime 2) os.walk and pathlib 3) creation/removal of paths He wasn't too sure but I checked with the docs and his memories seemed to be correct: - 1) https://docs.python.org/3/library/pathlib.html#pathlib.Path.stat High-level path objects should return high-level [insert type here] objects. Put differently, an API for retrieving time-stats as real date/time objects would be nice. I think that can be expanded to other pathlib methods as well, to make them less "os-wrapper"-like and provide added value. - 2) I remember a discussion on python-ideas about using "glob" or "rglob". However, when searching the docs for "walk" like in "os.walk" or for "iter", I don't find "glob"/"rglob". I can imagine ourselves (pathlib newbies back then), we didn't discover them. It would be great if the docs could be improved like the following: """ Path.rglob(pattern) Walk down a given path; a wrapper for "os.scandir"/"os.listdir". This is like calling glob() with “**” added in front of the given pattern: """ I think it would make "glob" and "rglob" more discoverable to new users. NOTE: """ Using the “**” pattern in large directory trees may consume an inordinate amount of time.""" sounds not really encouraging. This is especially true for "rglob" as it is defined as "like calling glob() with “**”". That leads to wondering whether "rglob" performs slow globbing instead of a "os.scandir"/"os.listdir". https://docs.python.org/3/library/pathlib.html#basic-use even promotes "glob" with "**" in the beginning which seems rather discouraging to use "rglob" as a fast alternative to "os.walk/scandir/listdir". Renaming "rglob"/adding a "scan" method would definitely help here. - 3) Again searching the docs for "create", "delete" (nothing found) and "remove", I found "Path.touch", "Path.rmdir" and "Path.unlink". It would be great if we had an easy way to remove a complete subtree as with "shutil.rmtree". We mostly don't care if a directory is empty. We need the system to be in a state of "this path does not exist anymore". Moreover, touching a file is good enough to "create" it if you don't care about changing its mtime. It you care about its mtime, it's a problem to "touch". -- That's it for our issues with pathlib from the past. Additionally, I got two further observations: A) pathlib tries to mimic/publish some low-level APIs to its users. "unlink" is not something people would expect to use when they want to "delete" or to "remove" a file or a directory. I know where the term stems from but it's the wrong layer of abstraction IMHO. Same for "touch" or "chmod". B) "rename" vs "replace". The difference is not really clear from the docs. You need to read "Path.replace" in order to understand "Path.rename" completely. (one raises an exception, the other don't if I read it correctly). If there's some agreement to change things with respect to those 5 points, I am willing to put some time into it. Best, Sven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pathlib+os/shutil feedback
On 10 April 2016 at 15:07, Sven R. Kunze wrote: > If there's some agreement to change things with respect to those 5 points, I > am willing to put some time into it. In broad terms I agree with these points. Thanks for doing the research. It would certainly be good to try to improve pathlib based on this sort of feedback while it is still provisional. One specific point - you say: """ Path.rglob(pattern) Walk down a given path; a wrapper for "os.scandir"/"os.listdir". """ However, at least in 3.5, Path.rglob does *not* wrap scandir. There's a difference in principle, in that scandir (DirEntry) objects cache stat data, where pathlib does not. Whether that makes using scandir in Path.rglob impossible, I don't know. Ideally I'd like to see pathlib modified to use scandir (because otherwise there will always be people saying "use os.walk rather than scandir, as it's faster) - or if it's not possible to do so because of the difference in principle, then I'd like to see a clear discussion of the issue in the docs, including the recommended approach for people who want scandir performance *without* having to abandon pathlib for lower level functions. Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
On Sun, 10 Apr 2016 18:51:23 +1200, Greg Ewing wrote: > > On 9 April 2016 at 23:02, R. David Murray wrote: > > > >>That is, a 'filename' is the identifier we've assigned to this thing > >>pointed to by an inode in linux, but an os path is a text representation > >>of the path from the root filename to a specified filename. That is, > >>the path *is* the name, so to say "path name" sounds redundant and > >>confusing to me. > > The term "pathname" is what is conventionally used to refer > to a textual string passed to the OS to identify an object > in the file system. > > It's often abbreviated to just "path", but that's ambiguous > for our purposes, because "path" can also refer to one of > our higher-level objects. I find it interesting that in all my years of unix computing I've never run into this (at least so that I became concious of it). I see now that in fact the Posix spec uses 'pathname'. Objection, such as it was, completely withdrawn :) (Nick's point about Path object vs path is also a good one.) --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
On 04/10/2016 12:36 AM, Nick Coghlan wrote:
On 10 April 2016 at 17:12, Greg Ewing wrote:
But there needs to be some way to ask a path object for
its native string representation, otherwise there would
be no point in using foreign path objects at all.
In addition to the existing "str(pathobj)", a "path" property was
recently added for that purpose:
>>> import pathlib
>>> pathlib.PureWindowsPath(".")
PureWindowsPath('.')
>>> pathlib.PureWindowsPath(".").path
'.'
(The specific property name was chosen to match os.scandir's DirEntry.path)
But with the new __fspath__ enhancements wouldn't the .path attribute go
away?
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
> On Apr 10, 2016, at 2:43 AM, Nick Coghlan wrote: > > This does raise a concrete API design question: how should > PurePath.__fspath__ behave when called on a mismatched OS? I think that PurePath.__fspath__ should return a string. There’s no reason why we can’t in my opinion and doing so just limits the usefulness of the method. For instance, it’d prevent it from being possible to serialize a pure windows path and send it over the wire to a process running on a Windows machine, like say if you have a build master running on Linux and a build slave running on Windows. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On 04/09/2016 10:31 PM, Nick Coghlan wrote: On 10 April 2016 at 02:41, Ethan Furman wrote: When somebody hands you bytes rather than text you need to worry about the encoding, and you need to worry about returning bytes rather than text yourself. https://hg.python.org/cpython/rev/e44410e5928e#l4.1 provides an illustration of how fiddly that can get, and that's in the URL context - cross-platform filesystem path handling is worse, since you need to worry about the significant differences between the way Windows and *nix handle binary paths, and you can't use os.sep directly any more (since that's always text). Okay, that makes sense. DirEntry can still get the check, it can just throw TypeError when it represents a binary path (that's one of the advantages of using a method-based protocol - exceptions on method calls are more acceptable than exceptions on property access). I guess I don't see the point of this. Either DirEntry's [1] only get partial support (which is only marginally better than the no support pathlib currently has), or stdlib code will need to catch those errors and then do an isinstance check to see if knows what the type is and how to deal with it [1]. On the other hand, if __fspath__ is allowed to hold bytes then the algorithm gets easier: - get the serialized form - check for bytes or str and act accordingly As a practicality argument that seems a lot easier for everybody. -- ~Ethan~ [1] Being a low-level function I think working with either bytes or str is entirely appropriate for DirEntry. [2] DirEntry? Oh yeah, grab the .path attribute. Something else? Bah, let the exception propogate. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Defining a path protocol
Ethan Furman writes: > It means the stuff in place won't change, but the stuff we're > adding now to integrate with Path will only support str (which is > one reason why os.path isn't going to die). I don't think this is a reason for keeping os.path. (Backward compatibility with existing code is sufficient, of course.) Support of str for all file names is provided by PEP 383. ISTM there's no big loss to using PEP 383's 'surrogateescape' handler to allow un-decode- able filenames in pathlib.Path: they're very rare. AFAIK pathlib doesn't care about surrogates -- after all, they're entirely "consenting adults" stuff. Of course that detracts a bit from the attractiveness of pathlib.Path vs. os.path or bytes methods, but only for a use case most people won't encounter in practice. We continue to support bytes at the os/io/open level for the same reasons you added formatting back to bytes: there are times when it's as least as natural to work with bytes as str (eg, when the path is passed around without manipulation) and more convenient (eg, you don't have to deal with encodings and UnicodeError handling). > After all, the idea is to make these things work with the stdlib, and > the stdlib accepts bytes for path strings. I don't see a problem. In dealing with legacy data (archives that include paths, such as .zips and .isos) we may find un-decode-able paths, or paths that are decode-able but by undetermined encoding, for a while to come (decades). For those, the bytes interfaces are preferable to unlovely expedients like decoding as 'iso8859-1'. But those are specialized use cases. Sane people dealing with current file systems won't need bytes in pathlib, and most "out of bounds" uses for pathlib I can think of in my own experience will be able to use surrogateescape. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sat, Apr 09, 2016 at 02:43:19PM +0200, Victor Stinner wrote: >Please don't loose time trying yet another sandbox inside CPython. It's >just a waste of time. It's broken by design. > >Please read my email about my attempt (pysandbox): >https://lwn.net/Articles/574323/ > >And the LWN article: >https://lwn.net/Articles/574215/ > >There are a lot of safe ways to run CPython inside a sandbox (and not rhe >opposite). > >I started as you, add more and more things to a blacklist, but it doesn't >work. That's the opposite of my approach though - I'm starting small and adding things, not starting with everything and removing stuff. Even if what we end up with is an extremely restricted subset of Python, there are still cases where that could be a useful tool to have. I've read your links above, and indeed everything I can find written by anyone about historical attempts to sandbox Python. I'm aware that others have tried and failed at this in the past, so it's certainly true that there is room for suspicion that this simply cannot be done. However on the other hand, nobody has tried before to do what I am doing (static code analysis), so it's not necessarily a safe assumption that the idea is doomed. For example, as far as I can see, none of the methods used to break out of your pysandbox would work to break out of my experiment. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: > On 9 April 2016 at 22:43, Victor Stinner wrote: > > See pysandbox test suite for a lot of ways to escape a sandbox. CPython has > > a list of know code to crash CPython (I don't recall the dieectory in > > sources), even with the latest version of CPython. > > They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers Thanks. I take your point that sandboxing Python requires CPython to free of code execution bugs. However I will note that none of the crashers in that directory will work inside my experiment (except "infinite_loop_re.py", which isn't a crasher just a long loop). > Even without those considerations though, there are system level > denial of service attacks that untrusted code can perform without even > trying to break out of the sandbox - the most naive is "while 1: > pass", but there are more interesting ones like "from itertools import > count; sum(count())", or even "sum(iter(int, 1))" and "list(iter(int, > 1))". Yes, of course. I have already explicitly noted that infinite loops and memory exhausation are not preventable. > Operating system level security sandboxes still aren't particularly > easy to use correctly, but they're a lot more reliable than language > runtime level sandboxes, can be used to defend against many more > attack vectors, and even offer increased flexibility (e.g. "can write > to these directories, but no others", "can read these files, but no > others", "can contact these IP addresses, but no others"). I don't entirely trust operating system sandboxes either - I generally assume that if someone can execute arbitrary code on my machine, then they can do anything they want to that machine. What I *might* trust, though, would be a "sandbox Python" that is itself running inside an operating system sandbox... ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 506 secrets module
Hi Steven, No probIem with the delay -- it's still before 3.6.0. I do think it's just about a record gap in the PEP review process. :-) I will approve the PEP as soon as you've updated the two function names in the PEP. (If you don't have write access to the peps repo, send the new version to [email protected] -- or send a link to the new draft somewhere online, e.g. github if you're using that. If you do have peps repo write access, just reply here when it's done.) Regarding the alluded vagueness of the PEP on the specs, I think I was mostly about the phrase "At the time of writing, the following functions have been suggested" which doesn't seem to commit very strongly to a specific API. The later phrase "The following pseudo-code can be taken as a possible starting point for the real implementation" doesn't really do much to take away the feeling that the PEP is non-committal on the actual API it proposes. But I don't want to approve the *idea* of a secrets module -- I want to approve a specific API. Maybe you can just change the words a bit to say something like "this PEP proposes the following API; the implementations given here are not final". None of this will prevent adding more functions to secrets.py before 3.6.0 is released (or, of course, in 3.7, 3.8 etc.), but it should send a clear message that we've agreed on these specific names and signatures, and that those are what I'm approving. If we change our minds about the API of the module before releasing 3.6.0, we should treat it as an amendment to the PEP and take it pretty seriously (but it's happened before so it's not impossible). Hopefully this message isn't drowned in the infinity of pathlib and ~bool threads, and we can proceed to add secrets.py to the 3.6 stdlib. You should be proud of that accomplishment! --Guido On Sat, Apr 9, 2016 at 10:08 PM, Steven D'Aprano wrote: > I've just spotted this email from Guido, sorry about the delay in > responding. > > Further comments below. > > > On Thu, Jan 14, 2016 at 10:47:09AM -0800, Guido van Rossum wrote: > >> I think the discussion petered out and nobody asked me to approve it yet >> (or I lost track of it). I'm almost happy to approve it in the current >> state. My only quibble is with some naming -- I'm not sure that a >> super-generic name like 'equal' is better than the original >> ('compare_digest'), > > Changed. > > >> and I would have picked a different name for token_url >> -- probably token_urlsafe. But maybe Steven can convince me that the names >> currently in the PEP are better. > > Changed. > > >> (I also don't like the wishy-washy >> position of the PEP on the actual specs of the proposed functions. But I'm >> fine with the actual implementation shown as the spec.) > > I'm not really sure what you want me to do to improve that. Can you be > more concrete about what you would like the PEP to say? > > > I haven't updated the PEP yet, but the newest version of the secrets > module with the changes requested is here: > > https://bitbucket.org/sdaprano/secrets > > > > -- > Steve > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Apr 10, 2016 11:51 AM, "Jon Ribbens" wrote: > > On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: > > On 9 April 2016 at 22:43, Victor Stinner wrote: > > > See pysandbox test suite for a lot of ways to escape a sandbox. CPython has > > > a list of know code to crash CPython (I don't recall the dieectory in > > > sources), even with the latest version of CPython. > > > > They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers > > Thanks. I take your point that sandboxing Python requires CPython to > free of code execution bugs. However I will note that none of the > crashers in that directory will work inside my experiment (except > "infinite_loop_re.py", which isn't a crasher just a long loop). > > > Even without those considerations though, there are system level > > denial of service attacks that untrusted code can perform without even > > trying to break out of the sandbox - the most naive is "while 1: > > pass", but there are more interesting ones like "from itertools import > > count; sum(count())", or even "sum(iter(int, 1))" and "list(iter(int, > > 1))". > > Yes, of course. I have already explicitly noted that infinite loops > and memory exhausation are not preventable. > > > Operating system level security sandboxes still aren't particularly > > easy to use correctly, but they're a lot more reliable than language > > runtime level sandboxes, can be used to defend against many more > > attack vectors, and even offer increased flexibility (e.g. "can write > > to these directories, but no others", "can read these files, but no > > others", "can contact these IP addresses, but no others"). > > I don't entirely trust operating system sandboxes either - I generally > assume that if someone can execute arbitrary code on my machine, then > they can do anything they want to that machine. > > What I *might* trust, though, would be a "sandbox Python" that is > itself running inside an operating system sandbox... > * https://github.com/jupyter/jupyterhub/wiki/Spawners - Docker LXC Containers - https://github.com/jupyter/jupyterhub/wiki/Authenticators - DOS is still trivial - Segfault is still trivial * http://doc.pypy.org/en/latest/sandbox.html#introduction ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On 10.04.16 19:51, Jon Ribbens wrote: On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: On 9 April 2016 at 22:43, Victor Stinner wrote: See pysandbox test suite for a lot of ways to escape a sandbox. CPython has a list of know code to crash CPython (I don't recall the dieectory in sources), even with the latest version of CPython. They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers Thanks. I take your point that sandboxing Python requires CPython to free of code execution bugs. However I will note that none of the crashers in that directory will work inside my experiment (except "infinite_loop_re.py", which isn't a crasher just a long loop). Try following example: it = iter([1]) for i in range(100): it = filter(None, it) next(it) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Apr 10 2016, Jon Ribbens wrote: > On Sat, Apr 09, 2016 at 02:43:19PM +0200, Victor Stinner wrote: >>Please don't loose time trying yet another sandbox inside CPython. It's >>just a waste of time. It's broken by design. >> >>Please read my email about my attempt (pysandbox): >>https://lwn.net/Articles/574323/ >> >>And the LWN article: >>https://lwn.net/Articles/574215/ >> >>There are a lot of safe ways to run CPython inside a sandbox (and not rhe >>opposite). >> >>I started as you, add more and more things to a blacklist, but it doesn't >>work. > > That's the opposite of my approach though - I'm starting small and > adding things, not starting with everything and removing stuff. That contradicts what you said in another mail: On Apr 08 2016, Jon Ribbens wrote: > Ah, I've not used Python 3.5, and I can't find any documentation on > this cr_frame business, but I've added cr_frame and f_back to the > disallowed attributes list. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Mon, Apr 11, 2016 at 12:07:48AM +0300, Serhiy Storchaka wrote: > On 10.04.16 19:51, Jon Ribbens wrote: > >On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: > >>On 9 April 2016 at 22:43, Victor Stinner wrote: > >>>See pysandbox test suite for a lot of ways to escape a sandbox. CPython has > >>>a list of know code to crash CPython (I don't recall the dieectory in > >>>sources), even with the latest version of CPython. > >> > >>They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers > > > >Thanks. I take your point that sandboxing Python requires CPython to > >free of code execution bugs. However I will note that none of the > >crashers in that directory will work inside my experiment (except > >"infinite_loop_re.py", which isn't a crasher just a long loop). > > Try following example: > > it = iter([1]) > for i in range(100): > it = filter(None, it) > next(it) That does indeed segfault. I guess you should report that as a bug! ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sun, Apr 10, 2016 at 02:08:16PM -0700, Nikolaus Rath wrote:
> On Apr 10 2016, Jon Ribbens wrote:
> > On Sat, Apr 09, 2016 at 02:43:19PM +0200, Victor Stinner wrote:
> > That's the opposite of my approach though - I'm starting small and
> > adding things, not starting with everything and removing stuff.
>
> That contradicts what you said in another mail:
>
> On Apr 08 2016, Jon Ribbens wrote:
> > Ah, I've not used Python 3.5, and I can't find any documentation on
> > this cr_frame business, but I've added cr_frame and f_back to the
> > disallowed attributes list.
No, you've just misunderstood my meaning. Obviously I'm not only
allowing access to whitelisted variable and property names, that
would be ridiculous ("your code may only use variables called
'foo', 'bar' and 'baz'...").
The point is that we can start with, say, only allowing expressions
and not statements, and a __builtins__ that contains literally
nothing. We can even limit ourselves to disallow, say, lambda and
yield and generator expressions if we like. Can this minimal
language be made "safe"? If so, we have already won something - the
ability to use "eval" as a powerful calculator function. Then, can
we allow statements? Can we allow user-defined classes? Can we allow
try/catch? etc.
With regard to names by the way, I suspect that disallowing just
anything starting "_" and the names of the properties of frame
objects would be good enough. Unless someone knows a way to get
to an object's __dict__ or its type without using vars() or type()
or underscore attributes...
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On 10 Apr 2016 22:55, "Jon Ribbens" wrote: > > On Mon, Apr 11, 2016 at 12:07:48AM +0300, Serhiy Storchaka wrote: > > On 10.04.16 19:51, Jon Ribbens wrote: > > >On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: > > >>On 9 April 2016 at 22:43, Victor Stinner wrote: > > >>>See pysandbox test suite for a lot of ways to escape a sandbox. CPython has > > >>>a list of know code to crash CPython (I don't recall the dieectory in > > >>>sources), even with the latest version of CPython. > > >> > > >>They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers > > > > > >Thanks. I take your point that sandboxing Python requires CPython to > > >free of code execution bugs. However I will note that none of the > > >crashers in that directory will work inside my experiment (except > > >"infinite_loop_re.py", which isn't a crasher just a long loop). > > > > Try following example: > > > > it = iter([1]) > > for i in range(100): > > it = filter(None, it) > > next(it) > > That does indeed segfault. I guess you should report that as a bug! There will be always be obscure ways to crash the interpreter. That one can be fixed but if someone really wants to break your sandbox this way then they will be able to. Remember that exploits are often based on bugs and any codebase the size of CPython has bugs. I haven't looked at your sandbox but for a different approach try this one: L = [None] L.extend(iter(L)) On my Linux machine that doesn't just crash Python. -- Oscar ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sun, Apr 10, 2016 at 7:02 PM, Oscar Benjamin wrote: > I haven't looked at your sandbox but for a different approach try this one: > > L = [None] > L.extend(iter(L)) > > On my Linux machine that doesn't just crash Python. For the record: don't try this if you have unsaved files open on your computer, because you will lose them. When I typed these two lines into the Py3.5 interactive prompt, it completely and totally froze Windows to the point that nothing would respond and I had to resort to the old trick of holding the power button down for five seconds to forcibly shut the computer down. Fortunately, I made extra certain everything was fully saved before I opened the Python interpreter, so I'm not TOTALLY dumb. :-P ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/10/2016 06:31 PM, Jon Ribbens wrote: > Unless someone knows a way to get to an object's __dict__ or its type > without using vars() or type() or underscore attributes... Hmm, 'classmethod'-wrapped functions get passed the type. Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJXCwKgAAoJEPKpaDSJE9HYHbAP/ibVrlKBTqkwePFr4n4hfA5Z 6te+FCzYm4RfAiIMq0Mitc9mFzeeAx5J9Z6kxONkbCBoBbhttcngR1uHWHHR/7tk a9OVKCu0fzvQvKM9J1wPWdu6uB50TZ2PmRiZ1nChXG2XKC8F3xnj/JwZod0N+3vK zus1T6/5vB6pm+q/hm9gh1yd9gTRldzoVQ9T2Tp8vo6PiYxe5qBwfhIHKR8xtWVs eUG0OR1w8QzaU97NDTOShotDq9Ekow66zqlhppqUGSmt2nOTDndLekse6q1l/oir nMuPBxgyb/CkQ9+KNXb3UvT5l8MLmCtJaMm/To0n8OUBSXG8sspP0oUSiMLUXc5a F/haZnpD2jLmCFz9ivBxIpFRVkLIajwovzLLItSzePclZHj6TChctSQvGPY0roVD BYVnGa4i7vi46mSzkeWvXKT2XFed2pCklD+FLnS6RnShxaxj1VEct8LVAJHFNAJ4 qg1dyLlTeclWUdoerRdGG2J7oa3Ib04ydh9OxnB1Y5KGa5iDCmfydHw24BU0gzvu DIX8tEpq5XSqzN5QAkIbtIV5nyqFwPj1Jun275ETkESTvI0fdja/8RJvJ5npYZj0 yJ5Gc5iXwQWazF18ALFYdyeV+ZKKv2Q5UiYEOBxG02XYaH8GZypAqMbf5apJKQAj PXHMjfW/YIuASrzcporx =1Wrb -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sun, Apr 10, 2016 at 08:12:30PM -0400, Jonathan Goble wrote: > On Sun, Apr 10, 2016 at 7:02 PM, Oscar Benjamin > wrote: > > I haven't looked at your sandbox but for a different approach try this one: > > > > L = [None] > > L.extend(iter(L)) > > > > On my Linux machine that doesn't just crash Python. > > For the record: don't try this if you have unsaved files open on your > computer, because you will lose them. When I typed these two lines > into the Py3.5 interactive prompt, it completely and totally froze > Windows to the point that nothing would respond and I had to resort to > the old trick of holding the power button down for five seconds to > forcibly shut the computer down. I think this might improve matters: http://bugs.python.org/issue26351 although I must admit I don't understand why the entire OS is effected. -- Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Mon, Apr 11, 2016 at 01:09:19PM +1000, Steven D'Aprano wrote: > On Sun, Apr 10, 2016 at 08:12:30PM -0400, Jonathan Goble wrote: > > On Sun, Apr 10, 2016 at 7:02 PM, Oscar Benjamin > > wrote: > > > I haven't looked at your sandbox but for a different approach try this > > > one: > > > > > > L = [None] > > > L.extend(iter(L)) > > > > > > On my Linux machine that doesn't just crash Python. > > > > For the record: don't try this if you have unsaved files open on your > > computer, because you will lose them. When I typed these two lines > > into the Py3.5 interactive prompt, it completely and totally froze > > Windows to the point that nothing would respond and I had to resort to > > the old trick of holding the power button down for five seconds to > > forcibly shut the computer down. > > > I think this might improve matters: > > http://bugs.python.org/issue26351 > > although I must admit I don't understand why the entire OS is effected. Memory exhaustion? > -- > Steve Oleg. -- Oleg Broytmanhttp://phdru.name/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Sun, Apr 10, 2016 at 10:50 PM, Oleg Broytman wrote: > On Mon, Apr 11, 2016 at 01:09:19PM +1000, Steven D'Aprano < > [email protected]> wrote: > > On Sun, Apr 10, 2016 at 08:12:30PM -0400, Jonathan Goble wrote: > > > On Sun, Apr 10, 2016 at 7:02 PM, Oscar Benjamin > > > wrote: > > > > I haven't looked at your sandbox but for a different approach try > this one: > > > > > > > > L = [None] > > > > L.extend(iter(L)) > > > > > > > > On my Linux machine that doesn't just crash Python. > > > > > > For the record: don't try this if you have unsaved files open on your > > > computer, because you will lose them. When I typed these two lines > > > into the Py3.5 interactive prompt, it completely and totally froze > > > Windows to the point that nothing would respond and I had to resort to > > > the old trick of holding the power button down for five seconds to > > > forcibly shut the computer down. > > > > > > I think this might improve matters: > > > > http://bugs.python.org/issue26351 > > > > although I must admit I don't understand why the entire OS is effected. > >Memory exhaustion? > * https://docs.docker.com/compose/compose-file/#cpu-shares-cpu-quota-cpuset-domainname-hostname-ipc-mac-address-mem-limit-memswap-limit-privileged-read-only-restart-stdin-open-tty-user-working-dir * https://github.com/jupyter/dockerspawner/blob/master/systemuser/Dockerfile > > > -- > > Steve > > Oleg. > -- > Oleg Broytmanhttp://phdru.name/[email protected] >Programmers don't die, they just GOSUB without RETURN. > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
On Mon, Apr 11, 2016 at 12:42:47AM -0500, Wes Turner wrote: > On Sun, Apr 10, 2016 at 10:50 PM, Oleg Broytman wrote: > > > On Mon, Apr 11, 2016 at 01:09:19PM +1000, Steven D'Aprano < > > [email protected]> wrote: > > > On Sun, Apr 10, 2016 at 08:12:30PM -0400, Jonathan Goble wrote: > > > > On Sun, Apr 10, 2016 at 7:02 PM, Oscar Benjamin > > > > wrote: > > > > > I haven't looked at your sandbox but for a different approach try > > this one: > > > > > > > > > > L = [None] > > > > > L.extend(iter(L)) > > > > > > > > > > On my Linux machine that doesn't just crash Python. > > > > > > > > For the record: don't try this if you have unsaved files open on your > > > > computer, because you will lose them. When I typed these two lines > > > > into the Py3.5 interactive prompt, it completely and totally froze > > > > Windows to the point that nothing would respond and I had to resort to > > > > the old trick of holding the power button down for five seconds to > > > > forcibly shut the computer down. > > > > > > > > > I think this might improve matters: > > > > > > http://bugs.python.org/issue26351 > > > > > > although I must admit I don't understand why the entire OS is effected. > > > >Memory exhaustion? > > * > https://docs.docker.com/compose/compose-file/#cpu-shares-cpu-quota-cpuset-domainname-hostname-ipc-mac-address-mem-limit-memswap-limit-privileged-read-only-restart-stdin-open-tty-user-working-dir > > * https://github.com/jupyter/dockerspawner/blob/master/systemuser/Dockerfile I think memory control groups in Linux can be used to limit memory usage. I have mem. c. g. configured and I'll try to find time to experiment with the code above. > > > -- > > > Steve Oleg. -- Oleg Broytmanhttp://phdru.name/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancments - method name only
On 11 April 2016 at 01:50, Donald Stufft wrote: > >> On Apr 10, 2016, at 2:43 AM, Nick Coghlan wrote: >> >> This does raise a concrete API design question: how should >> PurePath.__fspath__ behave when called on a mismatched OS? > > I think that PurePath.__fspath__ should return a string. There’s no > reason why we can’t in my opinion and doing so just limits the usefulness > of the method. For instance, it’d prevent it from being possible to > serialize a pure windows path and send it over the wire to a process running > on a Windows machine, like say if you have a build master running on Linux > and a build slave running on Windows. Yeah, given that you have to go out of your way to create a path object for an alternate platform, this makes sense - the "I know what I'm doing" indicator is calling pathlib.Pure[Windows|Posix]Path instead of ""pathlib.PurePath in the first place, and so __fspath__ can just do its thing as a pure text-based operation, without worrying about the current platform. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On 11 April 2016 at 02:16, Ethan Furman wrote: > On 04/09/2016 10:31 PM, Nick Coghlan wrote: >> >> On 10 April 2016 at 02:41, Ethan Furman wrote: > > >> When somebody hands you bytes rather than text you need to worry about >> the encoding, and you need to worry about returning bytes rather than >> text yourself. https://hg.python.org/cpython/rev/e44410e5928e#l4.1 >> provides an illustration of how fiddly that can get, and that's in the >> URL context - cross-platform filesystem path handling is worse, since >> you need to worry about the significant differences between the way >> Windows and *nix handle binary paths, and you can't use os.sep >> directly any more (since that's always text). > > > Okay, that makes sense. > >> DirEntry can still get the check, it can just throw TypeError when it >> represents a binary path (that's one of the advantages of using a >> method-based protocol - exceptions on method calls are more acceptable >> than exceptions on property access). > > > I guess I don't see the point of this. Either DirEntry's [1] only get > partial support (which is only marginally better than the no support pathlib > currently has), or stdlib code will need to catch those errors and then do > an isinstance check to see if knows what the type is and how to deal with it > [1]. What's wrong with only gaining partial support? Standard library code that doesn't currently support DirEntry at all will gain the ability to support str-based DirEntry objects, while bytes-based DirEntry objects will continue to be a low level object that isn't interoperable with most other APIs (which is fine - anyone writing low level POSIX-specific code can deal with unpacking the values explicitly, it just won't happen implicitly anywhere). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 506 secrets module
> On Apr 10, 2016, at 11:43 AM, Guido van Rossum wrote: > > I will approve the PEP as soon as you've updated the two function > names in the PEP. Congratulations Steven. Raymond ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
