Re: [Python-Dev] file system path protocol PEP

Greg Ewing writes: > Paul Moore wrote: > > On 13 May 2016 at 17:57, Ethan Furman wrote: > > > >>1) What is a wallet garden? > > > > I assumed he meant "walled garden" > > Works either way -- you'd want a wall around your wallet > garden to stop people stealing your

Re: [Python-Dev] file system path protocol PEP

On Fri, May 13, 2016 at 7:43 PM, Chris Angelico wrote: [...] > "Check" accepts subclasses; "CheckExact" doesn't (it's like "type(x) > is str"). The question is, which one SHOULD be being done? Indeed it should do "Check", so that path libraries that do inherit from str will

Re: [Python-Dev] file system path protocol PEP

On 14 May 2016 at 19:36, Sven R. Kunze wrote: > Fine example. Thinking naively, I would say, when somebody made an effort to > write __fspath__, it should be respected. Maybe, that's just me. Over time, we've generally moved towards treating subclasses as if they were the base

Re: [Python-Dev] file system path protocol PEP

On 13.05.2016 18:43, Chris Angelico wrote: https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_Check https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_CheckExact Thanks for pointing me at this. I searched via github and found usages only:

Re: [Python-Dev] file system path protocol PEP

Chris Angelico writes: > AFAICT, the compatibility layer would simply decode the bytes using > surrogateescape handling, which should round-trip anything. By design. See PEP 383. Or rather, the OP should; he has not done his homework and is confused by his own FUD. This whole subthread is

Re: [Python-Dev] file system path protocol PEP

Paul Moore wrote: On 13 May 2016 at 17:57, Ethan Furman wrote: 1) What is a wallet garden? I assumed he meant "walled garden" Works either way -- you'd want a wall around your wallet garden to stop people stealing your wallets. -- Greg

Re: [Python-Dev] file system path protocol PEP

On 13 May 2016 at 17:57, Ethan Furman wrote: > 1) What is a wallet garden? I assumed he meant "walled garden" (for people who don't know the phrase, it refers to an area or capability that's only made accessible to "favoured" clients - think of something like Apple's App

Re: [Python-Dev] file system path protocol PEP

On Fri, May 13, 2016 at 7:06 PM, Sven R. Kunze wrote: > On 13.05.2016 11:48, Koos Zevenhoven wrote: >> >> This issue is coupled with the future optimization questions. >> > > AFAIC coupling API design to optimization is called premature optimization. > I suppose so, but

Re: [Python-Dev] file system path protocol PEP

On 05/13/2016 09:06 AM, Sven R. Kunze wrote: On 13.05.2016 11:48, Koos Zevenhoven wrote: >> Sven R Kunze had previously queried: However, the proposed semantics will change if the checks are swapped. So, my actual question is: Is that an intended API inconsistency or a known bug supposed to

Re: [Python-Dev] file system path protocol PEP

On 13 May 2016 at 17:06, Sven R. Kunze wrote: > As far as I remember, one goal of this proposal was to avoid wallet gardens. > During the lengthy discussion on python-ideas people brought up that some > third-party libs indeed subclass from str. They are currently locked out.

Re: [Python-Dev] file system path protocol PEP

On Sat, May 14, 2016 at 2:33 AM, Sven R. Kunze wrote: > On 13.05.2016 17:29, Brett Cannon wrote: >> >> Purposeful change. It was what I had in my head after I posted my "groups" >> breakdown email and what Guido suggested as well independently. This helps >> alleviate any perf

Re: [Python-Dev] file system path protocol PEP

On 13.05.2016 17:29, Brett Cannon wrote: Purposeful change. It was what I had in my head after I posted my "groups" breakdown email and what Guido suggested as well independently. This helps alleviate any perf worries as type checks in C are pointer checks that are cheap to make compared to

Re: [Python-Dev] file system path protocol PEP

On 13.05.2016 11:48, Koos Zevenhoven wrote: This issue is coupled with the future optimization questions. AFAIC coupling API design to optimization is called premature optimization. However, the proposed semantics will change if the checks are swapped. So, my actual question is: Is that an

Re: [Python-Dev] file system path protocol PEP

On Fri, 13 May 2016 at 02:25 Sven R. Kunze wrote: > On 13.05.2016 10:36, Koos Zevenhoven wrote: > > This has just been discussed very recently in this thread (and earlier > > too). > > Could you point me to that? It seems I missed that part. I only found > posts related to

Re: [Python-Dev] file system path protocol PEP

> On 13 May 2016, at 08:07, Ethan Furman wrote: > > On 05/12/2016 01:59 PM, Sjoerd Job Postmus wrote: >>> On 12 May 2016, at 21:30, Ethan Furman wrote: >>> >>> If you need bytes support for your paths, there's at least one [1] that has >>> that

Re: [Python-Dev] file system path protocol PEP

On Fri, May 13, 2016 at 8:19 PM, Steven D'Aprano wrote: > I feel this is Not Our Problem. Surely the stdlib cannot be held > responsible for all the poor decisions made by third-party libraries? If > a library hard-codes "\\" as their directory separator, because everyone >

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 07:22:25PM +0200, Sjoerd Job Postmus wrote: > The whole point of adding `os.fspath` is to make it easier to use Path > objects. This is in an effort to gain greater adoption of pathlib in > libraries. Now, this is an excellent idea. > > However, if it were to reject

Re: [Python-Dev] file system path protocol PEP

On Fri, May 13, 2016 at 12:24 PM, Sven R. Kunze wrote: > On 13.05.2016 10:36, Koos Zevenhoven wrote: >> >> This has just been discussed very recently in this thread (and earlier >> too). > > > Could you point me to that? It seems I missed that part. I only found posts > related

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 05:42 PM, Ethan Furman wrote: And even given all that, for smoother interoperability with the rest of the stdlib, or at least the os.* portion, those functions would still need to be upgraded to check for .path on the incoming arguments -- at which point we may as well make a

Re: [Python-Dev] file system path protocol PEP

On 13.05.2016 10:36, Koos Zevenhoven wrote: This has just been discussed very recently in this thread (and earlier too). Could you point me to that? It seems I missed that part. I only found posts related to performance degradation. However, the proposed semantics will change if the checks

Re: [Python-Dev] file system path protocol PEP

This has just been discussed very recently in this thread (and earlier too). It may make sense, but it's not among our current worries. Besides, we already added the new fspath semantics to the PEP. While I hope Brett is asleep in his time zone, I'm guessing he will agree (just saying this

Re: [Python-Dev] file system path protocol PEP

On 12.05.2016 18:24, Guido van Rossum wrote: def fspath(p: Union[str, bytes, PathLike]) -> Union[str, bytes]: if isinstance(p, (str, bytes)): return p try: return p.__fspath__ except AttributeError: raise TypeError(...) @Brett Would you think it makes sense

Re: [Python-Dev] file system path protocol PEP

On 12.05.2016 19:27, Ethan Furman wrote: Maybe, but a bad idea for two reasons: 1) Reducing a str to the exact same str is silly; and, more importantly Finding something silly is no technical argument. Best, Sven ___ Python-Dev mailing list

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 01:59 PM, Sjoerd Job Postmus wrote: On 12 May 2016, at 21:30, Ethan Furman wrote: If you need bytes support for your paths, there's at least one [1] that has that support. So if I would need bytes support, I should submit a pull request to > library> which

Re: [Python-Dev] file system path protocol PEP

On Thu, 12 May 2016 at 14:04 Sjoerd Job Postmus wrote: > > > > On 12 May 2016, at 21:30, Ethan Furman wrote: > > > > If you need bytes support for your paths, there's at least one [1] that > has that support. > > So if I would need bytes support, I

Re: [Python-Dev] file system path protocol PEP

> On 12 May 2016, at 21:30, Ethan Furman wrote: > > If you need bytes support for your paths, there's at least one [1] that has > that support. So if I would need bytes support, I should submit a pull request to which replaces usage of the stdlib pathlib with another

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 10:22 AM, Sjoerd Job Postmus wrote: However, if it were to reject bytes, that would mean that when libraries start to use pathlib, it would suddenly become harder for people that actually need bytes-support to use pathlib. pathlib is not about bytes support. While bytes are

Re: [Python-Dev] file system path protocol PEP

There's no need for typing.PathLike. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] file system path protocol PEP

On Thu, 12 May 2016 at 12:03 Guido van Rossum wrote: > On Thu, May 12, 2016 at 11:51 AM, Brett Cannon wrote: > >> >> Anyway, with your strong preference of how to tweak os.fspath() what >> specifically would you like to see discussed at this point? >> > >

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 11:51 AM, Brett Cannon wrote: > > Anyway, with your strong preference of how to tweak os.fspath() what > specifically would you like to see discussed at this point? > Preferably nothing. :-) There's been too much discussion already. > Assuming the

Re: [Python-Dev] file system path protocol PEP

On Thu, 12 May 2016 at 11:36 Guido van Rossum wrote: > On Thu, May 12, 2016 at 10:18 AM, Brett Cannon wrote: > >> On Thu, 12 May 2016 at 09:25 Guido van Rossum wrote: >> > def fspath(p: Union[str, bytes, PathLike]) -> Union[str, bytes]: >>>

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 10:18 AM, Brett Cannon wrote: > > On Thu, 12 May 2016 at 09:25 Guido van Rossum wrote: > >> def fspath(p: Union[str, bytes, PathLike]) -> Union[str, bytes]: >> if isinstance(p, (str, bytes)): >> return p >> try: >>

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 8:22 PM, Sjoerd Job Postmus wrote: > I would like to make just 1 comment regarding the question of accepting > (or not) bytes as output of `os.fspath`. > > The whole point of adding `os.fspath` is to make it easier to use Path > objects. This is in

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 7:24 PM, Guido van Rossum wrote: > I am glad this is finally happening. There's quite a bit of noise in the > thread which I have to ignore. The two issues that I want to respond to are > speed and whether os.fspath() can return bytes. > > - Speed: We

Re: [Python-Dev] file system path protocol PEP

I would like to make just 1 comment regarding the question of accepting (or not) bytes as output of `os.fspath`. The whole point of adding `os.fspath` is to make it easier to use Path objects. This is in an effort to gain greater adoption of pathlib in libraries. Now, this is an excellent idea.

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 10:21 AM, Sven R. Kunze wrote: On 12.05.2016 18:56, Ethan Furman wrote: On 05/12/2016 09:26 AM, Sven R. Kunze wrote: str and bytes will receive the __fspath__ attribute when this PEP is accepted? No, they won't. The __fspath__ protocol will reduce the rich path object down

Re: [Python-Dev] file system path protocol PEP

On 12.05.2016 18:56, Ethan Furman wrote: On 05/12/2016 09:26 AM, Sven R. Kunze wrote: str and bytes will receive the __fspath__ attribute when this PEP is accepted? No, they won't. The __fspath__ protocol will reduce the rich path object down to a str/bytes object. Would this make the

Re: [Python-Dev] file system path protocol PEP

On Thu, 12 May 2016 at 09:25 Guido van Rossum wrote: > I am glad this is finally happening. There's quite a bit of noise in the > thread which I have to ignore. > Don't worry, I'm not ignoring it on your behalf. :) > The two issues that I want to respond to are speed and

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 09:26 AM, Sven R. Kunze wrote: On 12.05.2016 17:42, Ethan Furman wrote: On 05/12/2016 01:31 AM, Sven R. Kunze wrote: I think the "Rationale" section ignores the fact the Path also supports the .path attribute now. Which indeed defines a common interface between path objects.

Re: [Python-Dev] file system path protocol PEP

On 12.05.2016 17:42, Ethan Furman wrote: On 05/12/2016 01:31 AM, Sven R. Kunze wrote: I think the "Rationale" section ignores the fact the Path also supports the .path attribute now. Which indeed defines a common interface between path objects. The version of Python that has Path.path has

Re: [Python-Dev] file system path protocol PEP

I am glad this is finally happening. There's quite a bit of noise in the thread which I have to ignore. The two issues that I want to respond to are speed and whether os.fspath() can return bytes. - Speed: We should trust our ability to optimize the implementations where necessary. First the API

Re: [Python-Dev] file system path protocol PEP

On 11.05.2016 23:57, Brett Cannon wrote: On Wed, 11 May 2016 at 14:29 Nikolaus Rath > wrote: On May 11 2016, Brett Cannon > wrote: > This PEP proposes a protocol for classes which represent a

Re: [Python-Dev] file system path protocol PEP

On 05/12/2016 01:31 AM, Sven R. Kunze wrote: I think the "Rationale" section ignores the fact the Path also supports the .path attribute now. Which indeed defines a common interface between path objects. The version of Python that has Path.path has not been released yet. And even so, .path

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 4:20 PM, Nick Coghlan wrote: > > It's not unusual for me to encounter "POSIX oughtta be enough for > anyone" folks that are not yet entirely convinced that > bytes-are-not-text, so I'm actually in favour of making the default > Python-level API str-only

Re: [Python-Dev] file system path protocol PEP

On 12 May 2016 at 22:40, Koos Zevenhoven wrote: > On Thu, May 12, 2016 at 3:04 PM, Nick Coghlan wrote: >> >> I'd still like to see this exposed to Python code as os._raw_fspath() >> (with the leading underscore just meaning "this probably isn't the API >>

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 3:04 PM, Nick Coghlan wrote: > > I'd still like to see this exposed to Python code as os._raw_fspath() > (with the leading underscore just meaning "this probably isn't the API > you want" rather than indicating a private or unstable API), and then >

Re: [Python-Dev] file system path protocol PEP

On 12 May 2016 at 02:43, Brett Cannon wrote: > **deep, calming breath** > > Here is the PEP for __fspath__(). The draft lives at > https://github.com/brettcannon/path-pep so feel free to send me PRs for > spelling mistakes, grammatical errors, etc. Thanks for putting this

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 11:31 AM, Sven R. Kunze wrote: > On 11.05.2016 18:43, Brett Cannon wrote: >> >> Rationale >> = >> >> Historically in Python, file system paths have been represented as >> strings or bytes. This choice of representation has stemmed from C's >> own

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 11:14 AM, Serhiy Storchaka wrote: > > This is cheap in C, but os.path functions are implemented in Python. They > have to make at least one function call (os.fspath(), hasattr() or > isinstance()), not counting a bytecode for retrieving arguments,

Re: [Python-Dev] file system path protocol PEP

Thanks Brett for your hard work. My comments below: On 11.05.2016 18:43, Brett Cannon wrote: Rationale = Historically in Python, file system paths have been represented as strings or bytes. This choice of representation has stemmed from C's own decision to represent file system paths

Re: [Python-Dev] file system path protocol PEP

On 12.05.16 10:54, Ethan Furman wrote: Currently, any of these functions that already take a string have to do a couple pointer comparisons to make sure they have a string; any of these functions that take both a string and a bytes have to do a couple pointer comparisons to make sure they have a

Re: [Python-Dev] file system path protocol PEP

On 12.05.2016 00:13, Brett Cannon wrote: I see this whole discussion breaking down into a few groups which changes what gets done upfront and what might be done farther down the line: 1. Maximum acceptance: do whatever we can to make all representation of paths just work, which means

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 09:55 PM, Serhiy Storchaka wrote: [...] But for example os.walk() was significantly boosted with using os.scandir(), it would be sad to make it slower again. scandir's speed improvement is due to not not throwing away data the OS was already giving us. os.path is used in

Re: [Python-Dev] file system path protocol PEP

On 12.05.16 01:13, Brett Cannon wrote: On Wed, 11 May 2016 at 13:45 Serhiy Storchaka > wrote: On 11.05.16 19:43, Brett Cannon wrote: > os.path > ''' > > The various path-manipulation functions of ``os.path``

Re: [Python-Dev] file system path protocol PEP

On 11.05.16 23:51, Ethan Furman wrote: On 05/11/2016 01:44 PM, Serhiy Storchaka wrote: I afraid that this will hit a performance. Some os.path functions are used in tight loops, they are hard optimized, and adding support of path protocol can have visible negative effect. Do you have an

Re: [Python-Dev] file system path protocol PEP

On Wed, 11 May 2016 at 15:13 Brett Cannon wrote: > On Wed, 11 May 2016 at 13:45 Serhiy Storchaka wrote: > >> On 11.05.16 19:43, Brett Cannon wrote: >> > os.path >> > ''' >> > >> > The various path-manipulation functions of ``os.path`` [#os-path]_ >> >

Re: [Python-Dev] file system path protocol PEP

On Wed, 11 May 2016 at 16:08 Koos Zevenhoven wrote: > On Thu, May 12, 2016 at 1:13 AM, Brett Cannon wrote: > > > > > > On Wed, 11 May 2016 at 13:45 Serhiy Storchaka > wrote: > >> > >> On 11.05.16 19:43, Brett Cannon wrote: > >> >

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 05:13 PM, Arthur Darcet wrote: os.path.getmtime could be used in a tight loop, to sync directories with a lot of files for instance. % python3 -m timeit -s "import os.path; p = 'out'" "hasattr(p, '__fspath__'), os.path.getmtime(p)" 10 loops, best of 3: 2.67 usec per loop %

Re: [Python-Dev] file system path protocol PEP

On 11 May 2016 at 22:51, Ethan Furman wrote: > On 05/11/2016 01:44 PM, Serhiy Storchaka wrote: > > os.path >>> ''' >>> >>> The various path-manipulation functions of ``os.path`` [#os-path]_ >>> will be updated to accept path objects. For polymorphic functions that >>>

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 2:53 AM, Koos Zevenhoven wrote: > On Thu, May 12, 2016 at 2:49 AM, Koos Zevenhoven wrote: >> On Thu, May 12, 2016 at 2:05 AM, Ethan Furman wrote: >>> On 05/11/2016 03:13 PM, Brett Cannon wrote: >>> If [...] I

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 2:49 AM, Koos Zevenhoven wrote: > On Thu, May 12, 2016 at 2:05 AM, Ethan Furman wrote: >> On 05/11/2016 03:13 PM, Brett Cannon wrote: >> >>> If [...] I would drop os.path changes and make os.fspath() do what >> >>> Ethan and Koos

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 2:05 AM, Ethan Furman wrote: > On 05/11/2016 03:13 PM, Brett Cannon wrote: > >> If [...] I would drop os.path changes and make os.fspath() do what > >> Ethan and Koos have suggested and simply pass through without checks >> >> whatever path.__fspath__()

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 1:13 AM, Brett Cannon wrote: > > > On Wed, 11 May 2016 at 13:45 Serhiy Storchaka wrote: >> >> On 11.05.16 19:43, Brett Cannon wrote: >> > os.path >> > ''' >> > >> > The various path-manipulation functions of ``os.path``

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 03:13 PM, Brett Cannon wrote: If [...] I would drop os.path changes and make os.fspath() do what > Ethan and Koos have suggested and simply pass through without checks whatever path.__fspath__() returned if the argument wasn't str or bytes. Not to derail the conversation too

Re: [Python-Dev] file system path protocol PEP

On Wed, 11 May 2016 at 13:45 Serhiy Storchaka wrote: > On 11.05.16 19:43, Brett Cannon wrote: > > os.path > > ''' > > > > The various path-manipulation functions of ``os.path`` [#os-path]_ > > will be updated to accept path objects. For polymorphic functions that > >

Re: [Python-Dev] file system path protocol PEP

On Wed, 11 May 2016 at 14:29 Nikolaus Rath wrote: > On May 11 2016, Brett Cannon wrote: > > This PEP proposes a protocol for classes which represent a file system > > path to be able to provide a ``str`` or ``bytes`` representation. > [...] > > As I said

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 02:28 PM, Nikolaus Rath wrote: On May 11 2016, Brett Cannon wrote: This PEP proposes a protocol for classes which represent a file system path to be able to provide a ``str`` or ``bytes`` representation. [...] As I said before, to me this seems like a lot of effort for a very

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 12:28 AM, Nikolaus Rath wrote: > On May 11 2016, Brett Cannon wrote: >> This PEP proposes a protocol for classes which represent a file system >> path to be able to provide a ``str`` or ``bytes`` representation. > [...] > > As I said

Re: [Python-Dev] file system path protocol PEP

On May 11 2016, Brett Cannon wrote: > This PEP proposes a protocol for classes which represent a file system > path to be able to provide a ``str`` or ``bytes`` representation. [...] As I said before, to me this seems like a lot of effort for a very specific use-case. So let me

Re: [Python-Dev] file system path protocol PEP

On Thu, May 12, 2016 at 12:15 AM, Ethan Furman wrote: > On 05/11/2016 01:51 PM, Ethan Furman wrote: >> >> On 05/11/2016 01:44 PM, Serhiy Storchaka wrote: > > os.path ''' The various path-manipulation functions of ``os.path`` [#os-path]_ will be

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 01:51 PM, Ethan Furman wrote: On 05/11/2016 01:44 PM, Serhiy Storchaka wrote: os.path ''' The various path-manipulation functions of ``os.path`` [#os-path]_ will be updated to accept path objects. For polymorphic functions that accept both bytes and strings, they will be

Re: [Python-Dev] file system path protocol PEP

On Wed, May 11, 2016 at 11:04 PM, Brett Cannon wrote: > A quick comment about sending me fixes. While I do appreciate them, sending > them as a pull request is much easier for me as (a) I don't have to hunt the > changes down in the text, and (b) you will see the fixes others

Re: [Python-Dev] file system path protocol PEP

On 05/11/2016 01:44 PM, Serhiy Storchaka wrote: os.path ''' The various path-manipulation functions of ``os.path`` [#os-path]_ will be updated to accept path objects. For polymorphic functions that accept both bytes and strings, they will be updated to simply use code very much similar to

Re: [Python-Dev] file system path protocol PEP

On 11.05.16 19:43, Brett Cannon wrote: os.path ''' The various path-manipulation functions of ``os.path`` [#os-path]_ will be updated to accept path objects. For polymorphic functions that accept both bytes and strings, they will be updated to simply use code very much similar to

Re: [Python-Dev] file system path protocol PEP

A quick comment about sending me fixes. While I do appreciate them, sending them as a pull request is much easier for me as (a) I don't have to hunt the changes down in the text, and (b) you will see the fixes others have done already to the PEP and I then don't have to figure out what changes

Re: [Python-Dev] file system path protocol PEP

**another deep, calming breath** On Wed, May 11, 2016 at 7:43 PM, Brett Cannon wrote: > Open Issues > === > > Should os.fspath() return bytes? > > In most cases, it of course should not. The section (or the title) do not represent my

[Python-Dev] file system path protocol PEP

**deep, calming breath** Here is the PEP for __fspath__(). The draft lives at https://github.com/brettcannon/path-pep so feel free to send me PRs for spelling mistakes, grammatical errors, etc. - PEP: NNN Title: Adding a file system path protocol Version: $Revision$ Last-Modified: $Date$