Re: Why exception from os.path.exists()?

2018-06-02 Thread Gregory Ewing
Paul Moore wrote: Windows (the kernel) has the capability to implement fork(), but this isn't exposed via the Win32 API. To implement fork() you need to go to the raw kernel layer. Which is basically what the Windows Linux subsystem (bash on Windows 10) does What people usually mean by "POSIX

Re: Why exception from os.path.exists()?

2018-06-02 Thread Dan Stromberg
On Sat, Jun 2, 2018 at 4:28 AM, Chris Angelico wrote: > On Sat, Jun 2, 2018 at 9:13 PM, Steven D'Aprano > wrote: > > On Sat, 02 Jun 2018 20:58:43 +1000, Chris Angelico wrote: > Can someone confirm whether or not all the listed signals are actually > supported? We know that Ctrl-C maps to the

Re: Why exception from os.path.exists()?

2018-06-02 Thread Tim Chase
On 2018-06-02 00:14, Steven D'Aprano wrote: > Since /wibble doesn't exist, neither does /wibble/a\0b > > > py> os.path.exists("/wibble") > False > py> os.path.exists("/wibble/a\0b") > Traceback (most recent call last): > File "", line 1, in > File

Re: Why exception from os.path.exists()?

2018-06-02 Thread Peter J. Holzer
On 2018-06-02 01:51:07 +, Grant Edwards wrote: > On 2018-06-01, Steven D'Aprano wrote: > > But since "\0" is the correct type (a string), and the fact that it > > happens to be illegal on POSIX is a platform-dependent detail of no more > > importance than the fact that "?" is illegal on

Re: Why exception from os.path.exists()?

2018-06-02 Thread Paul Moore
On 2 June 2018 at 12:28, Chris Angelico wrote: > On Sat, Jun 2, 2018 at 9:13 PM, Steven D'Aprano > wrote: >> On Sat, 02 Jun 2018 20:58:43 +1000, Chris Angelico wrote: >> > Windows isn't POSIX compliant. Technically, Windows is POSIX compliant. You have to turn off a bunch of

Re: Why exception from os.path.exists()?

2018-06-02 Thread Chris Angelico
On Sat, Jun 2, 2018 at 9:13 PM, Steven D'Aprano wrote: > On Sat, 02 Jun 2018 20:58:43 +1000, Chris Angelico wrote: > Windows isn't POSIX compliant. >>> >>> Technically, Windows is POSIX compliant. You have to turn off a bunch >>> of features, turn on another bunch of features, and what you

Re: Why exception from os.path.exists()?

2018-06-02 Thread Steven D'Aprano
On Sat, 02 Jun 2018 20:58:43 +1000, Chris Angelico wrote: >>> Windows isn't POSIX compliant. >> >> Technically, Windows is POSIX compliant. You have to turn off a bunch >> of features, turn on another bunch of features, and what you get is the >> bare minimum POSIX compliance possible, but it's

Re: Why exception from os.path.exists()?

2018-06-02 Thread Steven D'Aprano
On Sat, 02 Jun 2018 01:51:07 +, Grant Edwards wrote: > What about the case where somebody calls > > os.path.exists("/tmp/foo\x00bar") > > If /tmp/foo exists should it return True? That depends on whether /tmp/foo is a directory containing a file \0bar or not. Since that is not a legal

Re: Why exception from os.path.exists()?

2018-06-02 Thread Chris Angelico
On Sat, Jun 2, 2018 at 8:27 PM, Steven D'Aprano wrote: > On Sat, 02 Jun 2018 10:32:55 +1000, Chris Angelico wrote: > >> On Sat, Jun 2, 2018 at 10:14 AM, Steven D'Aprano >> wrote: But assuming you're right, POSIX is still a set of minimum requirements - not maximums, to my

Re: Why exception from os.path.exists()?

2018-06-02 Thread Steven D'Aprano
On Sat, 02 Jun 2018 10:32:55 +1000, Chris Angelico wrote: > On Sat, Jun 2, 2018 at 10:14 AM, Steven D'Aprano > wrote: >>> But >>> assuming you're right, POSIX is still a set of minimum requirements - >>> not maximums, to my knowledge. >> >> It isn't even a set of minimum requirements. "<" is

Re: Why exception from os.path.exists()?

2018-06-01 Thread Serhiy Storchaka
02.06.18 03:05, Chris Angelico пише: The permissions error is reported differently by stat, but then exists() just says "oh that's an OS error so we're going to say it doesn't exist". If you truly want the reliable form of os.path.exists, it would be this: try: os.stat(path) return

Re: Why exception from os.path.exists()?

2018-06-01 Thread Serhiy Storchaka
01.06.18 16:58, Chris Angelico пише: Possibly more confusing, though, is this: os.path.exists(1) True os.path.exists(2) True os.path.exists(3) False I think it's testing that the file descriptors exist, because os.path.exists is defined in terms of os.stat, which can stat a path or an

Re: Why exception from os.path.exists()?

2018-06-01 Thread Grant Edwards
On 2018-06-01, Steven D'Aprano wrote: > On Thu, 31 May 2018 17:43:28 +, Grant Edwards wrote: > >> Except on the platform in quetion filenames _don't_ contain an embedded >> \0. What was passed was _not_ a path/filename. > > "/wibble/rubbish/nobodyexpectsthespanishinquistion" is not a

Re: Why exception from os.path.exists()?

2018-06-01 Thread Grant Edwards
On 2018-06-01, Steven D'Aprano wrote: > But since "\0" is the correct type (a string), and the fact that it > happens to be illegal on POSIX is a platform-dependent detail of no more > importance than the fact that "?" is illegal on Windows, it should be > treated as any other

Re: Why exception from os.path.exists()?

2018-06-01 Thread eryk sun
On Sat, Jun 2, 2018 at 12:14 AM, Steven D'Aprano wrote: > > It isn't even a set of minimum requirements. "<" is legal under POSIX, > but not Windows. "<" (i.e. the DOS_STAR wildcard character) is valid in device and stream names. It's only invalid in filenames, since it's reserved for wildcard

Re: Why exception from os.path.exists()?

2018-06-01 Thread Chris Angelico
On Sat, Jun 2, 2018 at 10:14 AM, Steven D'Aprano wrote: >> But >> assuming you're right, POSIX is still a set of minimum requirements - >> not maximums, to my knowledge. > > It isn't even a set of minimum requirements. "<" is legal under POSIX, > but not Windows. Windows isn't POSIX compliant.

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Sat, 02 Jun 2018 09:56:58 +1000, Chris Angelico wrote: > On Sat, Jun 2, 2018 at 9:37 AM, Steven D'Aprano > wrote: >> On Thu, 31 May 2018 17:43:28 +, Grant Edwards wrote: >> >>> Except on the platform in quetion filenames _don't_ contain an >>> embedded \0. What was passed was _not_ a

Re: Why exception from os.path.exists()?

2018-06-01 Thread Chris Angelico
On Sat, Jun 2, 2018 at 9:54 AM, Steven D'Aprano wrote: > On Sat, 02 Jun 2018 08:50:38 +1000, Chris Angelico wrote: > >> My ideal preference would be for True to mean "we know for certain that >> this exists" and False "we know for certain that this doesn't exist" > > We cannot make that promise,

Re: Why exception from os.path.exists()?

2018-06-01 Thread Chris Angelico
On Sat, Jun 2, 2018 at 9:37 AM, Steven D'Aprano wrote: > On Thu, 31 May 2018 17:43:28 +, Grant Edwards wrote: > >> Except on the platform in quetion filenames _don't_ contain an embedded >> \0. What was passed was _not_ a path/filename. > > "/wibble/rubbish/nobodyexpectsthespanishinquistion"

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Sat, 02 Jun 2018 08:50:38 +1000, Chris Angelico wrote: > My ideal preference would be for True to mean "we know for certain that > this exists" and False "we know for certain that this doesn't exist" We cannot make that promise, because we might not have permission to view the file. Since we

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Fri, 01 Jun 2018 11:58:42 -0400, Richard Damon wrote: > I would say that one way to look at it is that os.path.exists > fundamentally (at the OS level) expects a parameter of the 'type' of > either a nul terminated string or a file descriptor (aka fixed width > integer). One issue we have is

Re: Why exception from os.path.exists()?

2018-06-01 Thread eryk sun
On Fri, Jun 1, 2018 at 3:58 PM, Richard Damon wrote: > > The fundamental question is about case 2. Should os.path.exist, having > been give a value of the right 'Python Type' but not matching the type > of the operating system parameter identify this as an error (as it > currently does), or

Re: Why exception from os.path.exists()?

2018-06-01 Thread Gregory Ewing
Grant Edwards wrote: Python allows floating point numbers, so it is possible to express this question in python: os.path.exists(3.14159). Is the fact that the underlying OS/filesystem can't identify files via a floating point number relevent? Should it return False or raise ValueError? I

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Thu, 31 May 2018 17:43:28 +, Grant Edwards wrote: > Except on the platform in quetion filenames _don't_ contain an embedded > \0. What was passed was _not_ a path/filename. "/wibble/rubbish/nobodyexpectsthespanishinquistion" is not a pathname on my system either, and os.path.exists()

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Fri, 01 Jun 2018 14:38:56 +, Grant Edwards wrote: > On 2018-06-01, Paul Moore wrote: > >> Python allows strings with embedded \0 characters, so it's possible to >> express that question in Python - os.path.exists('a\0b'). What can be >> expressed in terms of the low-level (C-based)

Re: Why exception from os.path.exists()?

2018-06-01 Thread Gregory Ewing
Marko Rauhamaa wrote: Interestingly, you get a False even for existing files if you don't have permissions to access the file. Obviously in that case, instead of True or False it should return FileNotFound. :-) https://thedailywtf.com/articles/What_Is_Truth_0x3f_ -- Greg --

Re: Why exception from os.path.exists()?

2018-06-01 Thread Chris Angelico
On Sat, Jun 2, 2018 at 8:10 AM, Steven D'Aprano wrote: > But it *isn't* the wrong type. It is the same type: > > > py> type("abc") is type("a\0c") > True > > > So TypeError is out, since the type is right, only the value is wrong. I agree with you so far. This parallels the fact that

Re: Why exception from os.path.exists()?

2018-06-01 Thread Steven D'Aprano
On Fri, 01 Jun 2018 09:41:04 -0400, Richard Damon wrote: > I think this is a key point. os.path.exists needs to pass a null > terminated string to the system to ask it about the file. That's an implementation detail utterly irrelevant to Python programmers. If the OS expects a pointer to a

Re: Why exception from os.path.exists()?

2018-06-01 Thread Richard Damon
On 6/1/18 9:58 AM, Chris Angelico wrote: > On Fri, Jun 1, 2018 at 11:41 PM, Richard Damon > wrote: >> The confusion is that in python, a string with an embedded null is >> something pretty much like a string without an embedded null, so the >> programmer might not think of it as being the wrong

Re: Why exception from os.path.exists()?

2018-06-01 Thread Paul Moore
On 1 June 2018 at 15:38, Grant Edwards wrote: > On 2018-06-01, Paul Moore wrote: > >> Python allows strings with embedded \0 characters, so it's possible >> to express that question in Python - os.path.exists('a\0b'). What >> can be expressed in terms of the low-level (C-based) operating >>

Re: Why exception from os.path.exists()?

2018-06-01 Thread Grant Edwards
On 2018-06-01, Paul Moore wrote: > Python allows strings with embedded \0 characters, so it's possible > to express that question in Python - os.path.exists('a\0b'). What > can be expressed in terms of the low-level (C-based) operating > system API shouldn't be relevant. Python allows floating

Re: Why exception from os.path.exists()?

2018-06-01 Thread Marko Rauhamaa
Chris Angelico : > Possibly more confusing, though, is this: > os.path.exists(1) > True os.path.exists(2) > True os.path.exists(3) > False > > I think it's testing that the file descriptors exist, because > os.path.exists is defined in terms of os.stat, which can stat a path > or

Re: Why exception from os.path.exists()?

2018-06-01 Thread Chris Angelico
On Fri, Jun 1, 2018 at 11:41 PM, Richard Damon wrote: > The confusion is that in python, a string with an embedded null is > something pretty much like a string without an embedded null, so the > programmer might not think of it as being the wrong type. Thus we have > several options. > > 1) we

Re: Why exception from os.path.exists()?

2018-06-01 Thread Richard Damon
On 5/31/18 1:43 PM, Grant Edwards wrote: > On 2018-05-31, Paul Moore wrote: >> On 31 May 2018 at 15:01, Chris Angelico wrote: >> >> Honestly, I think the OP's point is correct. os.path.exists should >> simply return False if the filename has an embedded \0 - at least on >> Unix. > Except on the

Re: Why exception from os.path.exists()?

2018-06-01 Thread Marko Rauhamaa
Paul Moore : > On 1 June 2018 at 13:15, Barry Scott wrote: >> Once you know that all of the string you provided is given to the >> operating system it can then do whatever checks it sees fit to and >> return a suitable result. > > As the programmer, I don't care. The Python interpreter should

Re: Why exception from os.path.exists()?

2018-06-01 Thread Paul Moore
On 1 June 2018 at 13:15, Barry Scott wrote: > I think the reason for the \0 check is that if the string is passed to the > operating system with the \0 you can get surprising results. > > If \0 was not checked for you would be able to get True from: > > os.file.exists('/home\0ignore me')

Re: Why exception from os.path.exists()?

2018-06-01 Thread Barry Scott
On Thursday, 31 May 2018 14:03:01 BST Marko Rauhamaa wrote: > Chris Angelico : > > On Thu, May 31, 2018 at 10:03 PM, Marko Rauhamaa wrote: > >> This surprising exception can even be a security issue: > >>>>> os.path.exists("\0") > >> > >>Traceback (most recent call last): > >>

Re: Why exception from os.path.exists()?

2018-05-31 Thread Grant Edwards
On 2018-05-31, Paul Moore wrote: > On 31 May 2018 at 15:01, Chris Angelico wrote: >> Can someone on Windows see if there are other path names that raise >> ValueError there? Windows has a whole lot more invalid characters, and >> invalid names as well. > > On Windows: > os.path.exists('\0')

Re: Why exception from os.path.exists()?

2018-05-31 Thread Chris Angelico
On Fri, Jun 1, 2018 at 12:51 AM, MRAB wrote: > On 2018-05-31 14:38, Marko Rauhamaa wrote: >> >> Chris Angelico : >>> >>> Do you have an actual use-case where it is correct for an invalid path >>> to be treated as not existing? >> >> >> Note that os.path.exists() returns False for other types of

Re: Why exception from os.path.exists()?

2018-05-31 Thread Marko Rauhamaa
Terry Reedy : > On 5/31/2018 8:03 AM, Marko Rauhamaa wrote: >> Is the behavior a bug? Shouldn't it be: >> >> >>> os.path.exists("\0") >> False > > Please open an issue on the tracker if there is not one for this > already. issue 33721 created Marko --

Re: Why exception from os.path.exists()?

2018-05-31 Thread Terry Reedy
On 5/31/2018 8:03 AM, Marko Rauhamaa wrote: This surprising exception can even be a security issue: >>> os.path.exists("\0") Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python3.6/genericpath.py", line 19, in exists os.stat(path)

Re: Why exception from os.path.exists()?

2018-05-31 Thread Paul Moore
On 31 May 2018 at 16:11, Steven D'Aprano wrote: > On Thu, 31 May 2018 22:46:35 +1000, Chris Angelico wrote: > [...] >>> Most other analogous reasons *don't* generate an exception, nor is that >>> possibility mentioned in the specification: >>> >>>

Re: Why exception from os.path.exists()?

2018-05-31 Thread Steven D'Aprano
On Thu, 31 May 2018 22:46:35 +1000, Chris Angelico wrote: [...] >> Most other analogous reasons *don't* generate an exception, nor is that >> possibility mentioned in the specification: >> >>https://docs.python.org/3/library/os.path.html?#os.path.exists >> >> Is the behavior a bug? Shouldn't

Re: Why exception from os.path.exists()?

2018-05-31 Thread Paul Moore
On 31 May 2018 at 15:01, Chris Angelico wrote: > Can someone on Windows see if there are other path names that raise > ValueError there? Windows has a whole lot more invalid characters, and > invalid names as well. On Windows: >>> os.path.exists('\0') ValueError: stat: embedded null character

Re: Why exception from os.path.exists()?

2018-05-31 Thread MRAB
On 2018-05-31 14:38, Marko Rauhamaa wrote: Chris Angelico : Do you have an actual use-case where it is correct for an invalid path to be treated as not existing? Note that os.path.exists() returns False for other types of errors including: * File might exist but you have no access rights

Re: Why exception from os.path.exists()?

2018-05-31 Thread Gregory Ewing
Chris Angelico wrote: A Unix path name cannot contain a null byte, so what you have is a fundamentally invalid name. ValueError is perfectly acceptable. It would also make sense for it could simply return False, since a file with such a name can't exist. This is analogous to the way

Re: Why exception from os.path.exists()?

2018-05-31 Thread Chris Angelico
On Thu, May 31, 2018 at 11:38 PM, Marko Rauhamaa wrote: > Chris Angelico : >> Do you have an actual use-case where it is correct for an invalid path >> to be treated as not existing? > > Note that os.path.exists() returns False for other types of errors > including: > > * File might exist but

Re: Why exception from os.path.exists()?

2018-05-31 Thread Marko Rauhamaa
Chris Angelico : > Do you have an actual use-case where it is correct for an invalid path > to be treated as not existing? Note that os.path.exists() returns False for other types of errors including: * File might exist but you have no access rights * The pathname is too long for the file

Re: Why exception from os.path.exists()?

2018-05-31 Thread Chris Angelico
On Thu, May 31, 2018 at 11:03 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Thu, May 31, 2018 at 10:03 PM, Marko Rauhamaa wrote: >>> >>> This surprising exception can even be a security issue: >>> >>>>>> os.path.exists("\0") >>>Traceback (most recent call last): >>> File "",

Re: Why exception from os.path.exists()?

2018-05-31 Thread Marko Rauhamaa
Chris Angelico : > On Thu, May 31, 2018 at 10:03 PM, Marko Rauhamaa wrote: >> >> This surprising exception can even be a security issue: >> >>>>> os.path.exists("\0") >>Traceback (most recent call last): >> File "", line 1, in >> File "/usr/lib64/python3.6/genericpath.py",

Re: Why exception from os.path.exists()?

2018-05-31 Thread Chris Angelico
On Thu, May 31, 2018 at 10:03 PM, Marko Rauhamaa wrote: > > This surprising exception can even be a security issue: > >>>> os.path.exists("\0") >Traceback (most recent call last): > File "", line 1, in > File "/usr/lib64/python3.6/genericpath.py", line 19, in exists >

Why exception from os.path.exists()?

2018-05-31 Thread Marko Rauhamaa
This surprising exception can even be a security issue: >>> os.path.exists("\0") Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python3.6/genericpath.py", line 19, in exists os.stat(path) ValueError: embedded null byte Most other analogous

<    1   2