Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
To keep this a manageable length, I've trimmed vigourously. Apologies in advance if I've been too enthusiastic with the trimming :-) On Sat, Mar 24, 2018 at 05:09:54AM +1100, Chris Angelico wrote: > No, I haven't yet. Sounds like a new section is needed. Thing is, > there's a HUGE family of C-like and C-inspired languages that allow > assignment expressions, and for the rest, I don't have any personal > experience. So I need input from people: what languages do you know of > that have small-scope name bindings like this? I don't know if this counts as "like this", but Lua has a do...end block that introduces a new scope. Something like this: x = 1 do: x = 2 print(x) # prints 2 end print(x) # prints 1 I think that's a neat concept, but I'm struggling to think what I would use it for. [...] > > result = (func(x), func(x)+1, func(x)*2) > > True, but outside of comprehensions, the most obvious response is > "just add another assignment statement". You can't do that in a list > comp (or equivalently in a genexp or dict comp). Yes you can: your PEP gives equivalents that work fine for list comps, starting with factorising the duplicate code out into a helper function, to using a separate loop to get assignment: [(spam, spam+1) for x in values for spam in (func(x),)] [(spam, spam+1) for spam in (func(x) for x in values)] They are the equivalent to "just add another assignment statement" for comprehensions. I acknowledge that comprehensions are the motivating example here, but I don't think they're the only justification for the concept. Strictly speaking, there's never a time that we cannot use a new assignment statement. But sometimes it is annoying or inconvenient. Consider a contrived example: TABLE = [ alpha, beta, gamma, delta, ... func(omega) + func(omega)**2 + func(omega)**3, ] Yes, I can pull out the duplication: temp = function(omega) TABLE = [ alpha, beta, gamma, delta, ... temp + temp**2 + temp**3, ] but that puts the definition of temp quite distant from its use. So this is arguably nicer: TABLE = [ alpha, beta, gamma, delta, ... (func(omega) as temp) + temp**2 + temp**3, ] > >> Just as function-local names shadow global names for the scope of the > >> function, statement-local names shadow other names for that statement. > >> (They can technically also shadow each other, though actually doing this > >> should not be encouraged.) > > > > That seems weird. > > Which part? That they shadow, or that they can shadow each other? Shadowing themselves. I'm still not convinced these should just shadow local variables. Of course locals will shadow nonlocals, which shadow globals, which shadow builtins. I'm just not sure that we gain much (enough?) to justify adding a new scope between what we already have: proposed statement-local local nonlocal class (only during class statement) global builtins I think that needs justification by more than just "it makes the implementation easier". > Shadowing is the same as nested functions (including comprehensions, > since they're implemented with functions); and if SLNBs are *not* to > shadow each other, the only way is to straight-up disallow it. Or they can just rebind to the same (statement-)local. E.g.: while ("spam" as x): assert x == "spam" while ("eggs" as x): assert x == "eggs" break assert x == "eggs" > > Why can they not be used in closures? I expect that's going to cause a > > lot of frustration. > > Conceptually, the variable stops existing at the end of that > statement. It makes for some oddities, but fewer oddities than every > other variant that I toyed with. For example, does this create one > single temporary or many different temporaries? > > def f(): > x = "outer" > funcs = {} > for i in range(10): > if (g(i) as x) > 0: > def closure(): > return x > funcs[x] = closure I think the rule should be either: - statement-locals actually *are* locals and so behave like locals; - statement-locals introduce a new scope, but still behave like locals with respect to closures. No need to introduce two separate modes of behaviour. (Or if there is such a need, then the PEP should explain it.) > > I think there's going to be a lot of confusion about which uses of "as" > > bind to a new local and which don't. > > That's the exact point of "statement-local" though. I don't think so. As I say: > > I think this proposal is conflating two unrelated concepts: > > > > - introducing new variables in order to meet DRY requirements; > > > > - introducing a new scope. If you're going to champion *both* concepts, then you need to justify them both in the PEP, not just assume its obvious why we want
Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
FWIW, I thought another way which provides cache object library, it seems to just work in some cases. But it doesn't create statement local scope and might be difficult to read because looks ordinary expression doing magic. Chris, would you append the library to alternative proposal section? class Cache: """ Return a object which stores a value called with a keyword and immediately return it, then get the value using the attribute. """ def __call__(self, **kwargs): (k, v), = kwargs.items() # require one keyword setattr(self, k, v) return v c = Cache() print(c(spam="ham")) # => ham print(c.spam) # => ham L = [c(y=f(x)) + g(c.y) for x in range(5)] num = int(c.s) if c(s=eggs()).isdigit() else 0 if c(match=search(string)) is not None: print(c.match.groups()) while c(command=input()) != "quit": print("command is", c.command) -- Masayuki 2018-03-02 20:43 GMT+09:00 Chris Angelico : > After dozens of posts and a wide variety of useful opinions and > concerns being raised, here is the newest version of PEP 572 for your > debating pleasure. > > Formatted version: > > https://www.python.org/dev/peps/pep-0572/ > > There are now several more examples, greater clarity in edge cases, > and improved wording of the actual proposal and specifications. Also, > the reference implementation has been significantly enhanced, for > those who wish to try this themselves. > > ChrisA > > PEP: 572 > Title: Syntax for Statement-Local Name Bindings > Author: Chris Angelico > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 28-Feb-2018 > Python-Version: 3.8 > Post-History: 28-Feb-2018, 02-Mar-2018 > > > Abstract > > > Programming is all about reusing code rather than duplicating it. When > an expression needs to be used twice in quick succession but never again, > it is convenient to assign it to a temporary name with small scope. > By permitting name bindings to exist within a single statement only, we > make this both convenient and safe against name collisions. > > > Rationale > = > > When a subexpression is used multiple times in a list comprehension, there > are currently several ways to spell this, none of which is universally > accepted as ideal. A statement-local name allows any subexpression to be > temporarily captured and then used multiple times. > > Additionally, this syntax can in places be used to remove the need to > write an > infinite loop with a ``break`` in it. Capturing part of a ``while`` loop's > condition can improve the clarity of the loop header while still making the > actual value available within the loop body. > > > Syntax and semantics > > > In any context where arbitrary Python expressions can be used, a named > expression can appear. This must be parenthesized for clarity, and is of > the form ``(expr as NAME)`` where ``expr`` is any valid Python expression, > and ``NAME`` is a simple name. > > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that NAME is bound to that > value in all retrievals for the remainder of the current statement. > > Just as function-local names shadow global names for the scope of the > function, statement-local names shadow other names for that statement. > They can also shadow each other, though actually doing this should be > strongly discouraged in style guides. > > Assignment to statement-local names is ONLY through this syntax. Regular > assignment to the same name will remove the statement-local name and > affect the name in the surrounding scope (function, class, or module). > > Statement-local names never appear in locals() or globals(), and cannot be > closed over by nested functions. > > > Execution order and its consequences > > > Since the statement-local name binding lasts from its point of execution > to the end of the current statement, this can potentially cause confusion > when the actual order of execution does not match the programmer's > expectations. Some examples:: > > # A simple statement ends at the newline or semicolon. > a = (1 as y) > print(y) # NameError > > # The assignment ignores the SLNB - this adds one to 'a' > a = (a + 1 as a) > > # Compound statements usually enclose everything... > if (re.match(...) as m): > print(m.groups(0)) > print(m) # NameError > > # ... except when function bodies are involved... > if (input("> ") as cmd): > def run_cmd(): > print("Running command", cmd) # NameError > > # ... but function *headers* are executed immediately > if (input("> ") as cmd): > def run_cmd(cmd=cmd): # Capture the value in the default arg > print("Running command", cmd) # Works > > Some of these examples should be considered *bad code* and rejected by code > review and/or linters; they are not, however, illegal. > > > Exa
Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
On Sat, Mar 24, 2018 at 2:09 PM, Ethan Furman wrote: > On 03/23/2018 07:38 PM, Chris Angelico wrote: >> >> On Sat, Mar 24, 2018 at 3:58 AM, Guido van Rossum wrote: >>> >>> On Fri, Mar 23, 2018 at 9:34 AM, Christoph Groth wrote: > > But wouldn't it be a good alternative to PEP 572 to *add* an assignment operator that is an expression to Python, and is distinct from "=", for example ":="? > >>> >>> >>> I also think it's fair to at least reconsider adding inline assignment, >>> with >>> the "traditional" semantics (possibly with mandatory parentheses). This >>> would be easier to learn and understand for people who are familiar with >>> it >>> from other languages (C++, Java, JavaScript). >> >> >> Thank you; both of these have now been incorporated into the document. > > > I'm certainly hoping PEP 572 is rejected so we can have a follow-up PEP that > only deals with the assignment-as-expression portion. > > No offense intended, Chris! :) In fact, maybe you could write that one > too, and then have an accepted PEP to your name? ;) > Okay, maybe I *do* need to split them. There are other uses for statement-local names, so they're both potentially viable. Would that be the best way forward? If I do, sooner would be better than later - I'd want to grab PEP 573 while it's still available. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
On 03/23/2018 07:38 PM, Chris Angelico wrote: On Sat, Mar 24, 2018 at 3:58 AM, Guido van Rossum wrote: On Fri, Mar 23, 2018 at 9:34 AM, Christoph Groth wrote: But wouldn't it be a good alternative to PEP 572 to *add* an assignment operator that is an expression to Python, and is distinct from "=", for example ":="? >> I also think it's fair to at least reconsider adding inline assignment, with the "traditional" semantics (possibly with mandatory parentheses). This would be easier to learn and understand for people who are familiar with it from other languages (C++, Java, JavaScript). Thank you; both of these have now been incorporated into the document. I'm certainly hoping PEP 572 is rejected so we can have a follow-up PEP that only deals with the assignment-as-expression portion. No offense intended, Chris! :) In fact, maybe you could write that one too, and then have an accepted PEP to your name? ;) -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
On Sat, Mar 24, 2018 at 3:58 AM, Guido van Rossum wrote: > I also think it's fair to at least reconsider adding inline assignment, with > the "traditional" semantics (possibly with mandatory parentheses). This > would be easier to learn and understand for people who are familiar with it > from other languages (C++, Java, JavaScript). > > On Fri, Mar 23, 2018 at 9:34 AM, Christoph Groth > wrote: >> But wouldn't it be a good alternative to PEP 572 to *add* an assignment >> operator that is an expression to Python, and is distinct from "=", for >> example ":="? Thank you; both of these have now been incorporated into the document. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On Fri, Mar 23, 2018 at 3:38 PM, Wes Turner wrote: > Here's a comparison table of os, os.path, shutil, pathlib, and path.py. > Darn, that's a big list -- exactly what we want to avoid :-( Though there are bunch of string methods in there that we can dump right off the bat. > ... trio wraps pathlib methods with async; which also might as well be done in pathlib? hmm --that's s touch one -- a full set of async file operations would be great. But where to put them and most of what we are talking about doesn't need asnyc, does it? deleting/renaming a single, file, etc. So my thought is that an asnyc lib should mirror / overwrite this file-operations-api. Which makes it a problem for the async lib(s) later on down the road. BTW -- this is a nice argument for making this API in the first place -- then we have a single API to make an async version of! - Does it make sense to copy the docstrings at import time every time? copy from where? is this for the async version? or are you thinking that the new methods will simply be wrappers for the olds ones -- in which case, I think no -- this should be a new API with its own docs, whether or not a particular call is any different that the "old" one. The big challenge is to identity what is "basic" or "normal" ratehr than advanced. here's a quick take on trimming down that whole table -- jsut my quick HO: attr table == == == === == === === attr os os.path shutil pathlib path.py == == === == === === `absolute`_ X `abspath`_X X `access`_ X X `atime`_ X `basename`_ X X `cd`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copytree`_ X X `curdir`_ X X `dirname`_X X `drive`_ XX `exists`_ X XX `expanduser`_ X XX `expandvars`_ X X `ext`_X `fnmatch`_X X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_X X `glob`_ XX `home`_ X `is_absolute`_ X `is_dir`_X `is_symlink`_X `isabs`_ X X `isdir`_ X X `isfile`_ X X `islink`_ X X `iterdir`_ X `lchmod`_X `link`_X X `listdir`_ X X `lstat`_ X XX `mkdir`_ X XX `makedirs`_X X `match`_ X `merge_tree`_ X `move`_ X X `mtime`_ X `normcase`_ X X `normpath`_ X X `open`_X XX `owner`_ XX `parent`_XX `parents`_ X `relative_to`_ X `relpath`_X X `relpathto`_ X `remove`_ X X `removedirs`_ X X `rename`_ X XX `replace`_ X XX `resolve`_ X `rglob`_ X `root`_ X `samefile`_ X XX `size`_ X `suffix`_X `suffixes`_ X `symlink`_ X X `symlink_to`_X `touch`_ XX `utime`_ X X `walk`_X X `with_name`_ X `with_suffix`_ XX `write_bytes`_ XX `write_text`_XX These are about permissions -- challenging how to do that in a X-platform way... And it would be good to group this sort of th
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On Fri, Mar 23, 2018 at 1:47 PM, Jason Maldonis > > I can find the time I'll write the PEP (although I'd rather just > contribute to it). Figuring out what to include and how to change the > methods' APIs seems like one of the easier PEPs to write in terms of > required knowledge (I don't know much about python's C code), so hopefully > I would have a chance at doing an adequate job. If you think not, please > tell me and I won't bother :) > Most of PEP writing (at least for a PEP like this) is capturing the debate and conclusions and capturing and moderating the discussion to some extent -- technical knowledge is the least of it. I expect you'll get plenty of help on the technical issues if there are any (I'm say if, 'cause at this point it doesn't look like anyone is proposing any *new* functionality, it's really just moving and renaming things. I suggest by starting with the a PEP outpine from a previous PEP, getting it into a a gitHub repo (it can start in your own account somewhere, then go through this thread to start sorting out the issues. I say gitHub repo 'cause then it's easy to publish and collect comments and improvements in a central place - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On 23 March 2018 at 22:22, Mike Miller wrote: > > On 2018-03-23 13:47, Jason Maldonis wrote: >> >> I’ve found it odd that there doesn’t even seem to be acknowledgment >> among >> longtime python users that the current hodgepodge is pretty >> dysfunctional >> for new users. > > > I'll acknowledge it too, just that after X years it becomes second nature as > you know. It's certainly easy to get so used to the current state of affairs that you forget that it's not "obvious" to a newcomer. I've been teaching a colleague Python and it's certainly something he would have problems with. > Tinkered around at lunchtime with a module to bring them under one roof. > Named it "f2" because "fs" was taken on PyPI and thought the name should be > very short since it could potentially be used in every script, like os and > sys are. > > https://github.com/mixmastamyk/f2 > > Just poking around these questions came up about what to include: > > - Include file descriptors apis? > - chroot? > - l- functions to not follow links? > - Unix dev files? obscure > - pathconf? > - supports_* api? > - Only scandirs or walk too? > > - Should unpack os.path functions into the root? Handy but many of > them. > - path and Path in same module could confuse. > > - Unpack file-related shutil functions? Or leave as submodule? > > > Perhaps this is enough to generate some comments. Yeah, this is the real issue. It's not that the general principle "have a single place where all filesystem stuff is found" is a particularly bad one, it's more that it's genuinely hard to know what the right answer is here. I'll add some other questions: - Should the module only contain cross-platform functions, or are platform-specific ones OK? - How about operations that appear on multiple levels (Steve Dower mentioned copy "that works like drag and drop", there's also about delete that handles recycle bins vs lower-level delete)? and the big one: - How will we handle backward compatibility? This is why a generic proposal like the one that started this thread is pretty useless in practice - it prompts debate, but doesn't really answer any of the important questions. Discussing functions on a case by case basis *will*, but it'll take a long time, and may miss the "big picture". A 3rd party module, like the f2 module you have created, is another approach - try it out and refine it over time, then when the various questions have been resolved, propose it for merging into the stdlib. There may be other ways to make progress, too. So I'm also OK with the general idea, I just don't see the value in simply rehashing all the old questions - we need something actionable if this is to go anywhere (and the current proposal simply isn't). Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
Here's a comparison table of os, os.path, shutil, pathlib, and path.py. The full version is at https://github.com/westurner/pyfilemods (README.rst) and at https://westurner.github.io/pyfilemods. I ran a few set intersections and went ahead and wrote a report to compare function/method signatures and sources. (cc'ed ironically, for the last time, I promise, from the other fork of this thread: https://mail.python.org/pipermail/python-ideas/2018-March/049375.html ) ... trio wraps pathlib methods with async; which also might as well be done in pathlib? - Does it make sense to copy the docstrings at import time every time? attr table == == == === == === === attr os os.path shutil pathlib path.py == == === == === === `__div__`_X `__rdiv__`_ X `absolute`_ X `abspath`_X X `access`_ X X `altsep`_ X X `anchor`_X `as_posix`_ X `as_uri`_X `atime`_ X `basename`_ X X `bytes`_ X `capitalize`_ X `casefold`_ X `cd`_ X `center`_ X `chdir`_ X X `chmod`_ X XX `chown`_ X X X `chroot`_ X X `chunks`_ X `commonpath`_ X `commonprefix`_ X `copy`_ X X `copy2`_ X X `copyfile`_ X X `copymode`_ X X `copystat`_ X X `copytree`_ X X `count`_ X `ctime`_ X `curdir`_ X X `cwd`_ X `defpath`_ X X `devnull`_ X X `dirname`_X X `dirs`_ X `drive`_ XX `encode`_ X `endswith`_ X `exists`_ X XX `expand`_ X `expandtabs`_ X `expanduser`_ X XX `expandvars`_ X X `ext`_X `extsep`_ X X `files`_ X `find`_ X `fnmatch`_X X `format`_ X `format_map`_ X `get_owner`_ X `getatime`_ X X `getctime`_ X X `getcwd`_ X X `getmtime`_ X X `getsize`_X X `glob`_ XX `group`_ X `home`_ X `in_place`_ X `index`_ X `is_absolute`_ X `is_block_device`_ X `is_char_device`_X `is_dir`_X `is_fifo`_ X `is_file`_ X `is_reserved`_ X `is_socket`_ X `is_symlink`_X `isabs`_ X X `isalnum`_X `isalpha`_X `isdecimal`_ X `isdigit`_X `isdir`_ X X `isfile`_ X X `isidentifier`_ X `islink`_ X X `islower`_X `ismount`_X X `isnumeric`_ X `isprintable`_X `isspace`_X `istitle`_X `isupper`_X `iterdir`_ X `join`_ X X `joinpath`_ XX `lchmod`_X `lexists`_X `lines`_ X `link`_X
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On 2018-03-23 13:47, Jason Maldonis wrote: I’ve found it odd that there doesn’t even seem to be acknowledgment among longtime python users that the current hodgepodge is pretty dysfunctional for new users. I'll acknowledge it too, just that after X years it becomes second nature as you know. Tinkered around at lunchtime with a module to bring them under one roof. Named it "f2" because "fs" was taken on PyPI and thought the name should be very short since it could potentially be used in every script, like os and sys are. https://github.com/mixmastamyk/f2 Just poking around these questions came up about what to include: - Include file descriptors apis? - chroot? - l- functions to not follow links? - Unix dev files? obscure - pathconf? - supports_* api? - Only scandirs or walk too? - Should unpack os.path functions into the root? Handy but many of them. - path and Path in same module could confuse. - Unpack file-related shutil functions? Or leave as submodule? Perhaps this is enough to generate some comments. -Mike ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
> > I’ve found it odd that there doesn’t even seem to be acknowledgment among > longtime python users that the current hodgepodge is pretty dysfunctional > for new users. > I find this odd too. There have been a few comments along the lines of this being a problem for newbies or for *some* people (not sure who *some* is referring to), but I think that having these functions in different modules is weird for everyone. It just so happens that if you use file-operations a lot that you've gotten used to the system and know where to look, but it's still weird to have to use 2-3 different modules. Additionally, if you know where to look then it isn't really a hassle to import another module, whereas it is a hassle if you are less familiar with the functionality (regardless of whether you are a newbie or not). > Heck, I’ve been using Python for almost 20 years and I still have go look > up whether it’s “delete” or “remove” or “unlink”. And there is so much in > os that ipython or IDE completion doesn’t help much. > Me too, although I've only been using python for ~ 10 years. > Of course, this will require a substantial amount of work to work out the > details in a PEP. I don’t have the time to do that, but if the OP or > someone else does, I’ll help. > I'd be happy to help as well, and if I can find the time I'll write the PEP (although I'd rather just contribute to it). Figuring out what to include and how to change the methods' APIs seems like one of the easier PEPs to write in terms of required knowledge (I don't know much about python's C code), so hopefully I would have a chance at doing an adequate job. If you think not, please tell me and I won't bother :) - Jason ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
On Mar 23, 2018, at 7:26 AM, Steve Dower wrote: I had a colleague complaining to me the other day about having to search multiple packages for the right function to move a file (implying: with the same semantics as drag-drop). Thanks Steve — I know there isn’t any kind of consensus about exactly what should be part of a “most file operations” package/class, but I’ve found it odd that there doesn’t even seem to be acknowledgment among longtime python users that the current hodgepodge is pretty dysfunctional for new users. One thing that’s been hammered home for me teaching newbies is that people get very far with computing these days without ever having touched a command line — certainly not a *nix one. Even the concept of a working directory is foreign to many. So telling them that they can create and manipulate paths in nifty ways with the Path object, but then telling them to go to the os module to do simple things like rename or delete a file, and then looking at the docs for the os module, which starts with a bunch of references to other modules you need for not quite as simple things (shutil) or more complex things (tempfile). Then you look through the module and see a LOT of low level cryptic and semi-platform dependent functions — not newbie friendly. Heck, I’ve been using Python for almost 20 years and I still have go look up whether it’s “delete” or “remove” or “unlink”. And there is so much in os that ipython or IDE completion doesn’t help much. So as not to only write a critical rant — here is a proposal of sorts: The python standard library should have one-stop shopping for the basic file system manipulations. “Basic” could be guided by “what users typically do with a GUI file manager” This could be yet another module, but given that navigating the file system ( I.e. path manipulation) is one of the things people typically do with a file manager, it makes sense to add it all to pathlib, maybe even all to the Path object. ( though maybe more advanced classes/functions could be added to pathlib as way to put it all in one place, while not expanding the Path namespace too much) Of course, this will require a substantial amount of work to work out the details in a PEP. I don’t have the time to do that, but if the OP or someone else does, I’ll help. -CHB If there isn’t a pathtools library on PyPI yet, this would certainly be valuable for newer developers. My view on Path is to either have everything on it or nothing on it (without removing what’s already there, of course), and since everything is so popular we should at least put everything in the one place. Top-posted from my Windows phone *From: *Mike Miller *Sent: *Monday, March 19, 2018 10:51 *To: *python-ideas@python.org *Subject: *Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2 On 2018-03-18 10:55, Paul Moore wrote: >> Should Path() have methods to access all file operations? > > No, (Counterexample, having a Path operation to set Windows ACLs for a path). Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
On Sat, Mar 24, 2018 at 2:00 AM, Steven D'Aprano wrote: > On Fri, Mar 23, 2018 at 09:01:01PM +1100, Chris Angelico wrote: > >> PEP: 572 >> Title: Syntax for Statement-Local Name Bindings > [...] >> Abstract >> >> >> Programming is all about reusing code rather than duplicating it. > > I don't think that editorial comment belongs here, or at least, it is > way too strong. I'm pretty sure that programming is not ALL about reusing > code, > and code duplication is not always wrong. > > Rather, we can say that *often* we want to avoid code duplication, and > this proposal is way way to do so. And this should go into the > Rationale, not the Abstract. The abstract should describe what this > proposal *does*, not why, for example: > > This is a proposal for permitting temporary name bindings > which are limited to a single statement. > > What the proposal *is* goes in the Abstract; reasons *why* we want it > go in the Rationale. Thanks. I've never really been happy with my "Abstract" / "Rationale" split, as they're two sections both designed to give that initial 'sell', and I'm clearly not good at writing the distinction :) Unless you object, I'm just going to steal your Abstract wholesale. Seems like some good words there. > I see you haven't mentioned anything about Nick Coglan's (long ago) > concept of a "where" block. If memory serves, it would be something > like: > > value = x**2 + 2*x where: > x = some expression > > These are not necessarily competing, but they are relevant. Definitely relevant, thanks. This is exactly what I'm looking for - related proposals that got lost in the lengthy threads on the subject. I'll mention it as another proposal, but if anyone has an actual post for me to reference, that would be appreciated (just to make sure I'm correctly representing it). > Nor have you done a review of any other languages, to see what similar > features they already offer. Not even the C's form of "assignment as an > expression" -- you should refer to that, and explain why this would not > similarly be a bug magnet. No, I haven't yet. Sounds like a new section is needed. Thing is, there's a HUGE family of C-like and C-inspired languages that allow assignment expressions, and for the rest, I don't have any personal experience. So I need input from people: what languages do you know of that have small-scope name bindings like this? >> Rationale >> = >> >> When a subexpression is used multiple times in a list comprehension, > > I think that list comps are merely a single concrete example of a more > general concept that we sometimes want or need to apply the DRY > principle to a single expression. > > This is (usually) a violation of DRY whether it is inside or outside of > a list comp: > > result = (func(x), func(x)+1, func(x)*2) True, but outside of comprehensions, the most obvious response is "just add another assignment statement". You can't do that in a list comp (or equivalently in a genexp or dict comp). Syntactically you're right that they're just one example of a general concept; but they're one of the original motivating reasons. I've tweaked the rationale wording some; the idea is now "here's a general idea" followed by two paragraphs of specific use-cases (comprehensions and loops). Let me know if that works better. >> Syntax and semantics >> >> >> In any context where arbitrary Python expressions can be used, a **named >> expression** can appear. This must be parenthesized for clarity, and is of >> the form ``(expr as NAME)`` where ``expr`` is any valid Python expression, >> and ``NAME`` is a simple name. >> >> The value of such a named expression is the same as the incorporated >> expression, with the additional side-effect that NAME is bound to that >> value for the remainder of the current statement. > > > Examples should go with the description. Such as: > > x = None if (spam().ham as eggs) is None else eggs Not sure what you gain out of that :) Maybe a different first expression would help. > y = ((spam() as eggs), (eggs.method() as cheese), cheese[eggs]) Sure. I may need to get some simpler examples to kick things off though. >> Just as function-local names shadow global names for the scope of the >> function, statement-local names shadow other names for that statement. >> (They can technically also shadow each other, though actually doing this >> should not be encouraged.) > > That seems weird. Which part? That they shadow, or that they can shadow each other? Shadowing is the same as nested functions (including comprehensions, since they're implemented with functions); and if SLNBs are *not* to shadow each other, the only way is to straight-up disallow it. For the moment, I'm not forbidding it, as there's no particular advantage to popping a SyntaxError. >> Assignment to statement-local names is ONLY through this syntax. Regular >> assignment to the same name will remove the statement-local name a
Re: [Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
I also think it's fair to at least reconsider adding inline assignment, with the "traditional" semantics (possibly with mandatory parentheses). This would be easier to learn and understand for people who are familiar with it from other languages (C++, Java, JavaScript). On Fri, Mar 23, 2018 at 9:34 AM, Christoph Groth wrote: > Disclaimer: I skimmed/searched through the PEP 572 threads (or should I > say "literature"?) and did not find a discussion of the following point. > If it has been discussed already, I'd be glad to be pointed to it. > > I am aware that Python, in contrast to C-like languages, has chosen not > to treat assignments (=) as an expression with a value. The motivation > for this is to avoid bugs due to confusion with the equality operator > (==). > > But wouldn't it be a good alternative to PEP 572 to *add* an assignment > operator that is an expression to Python, and is distinct from "=", for > example ":="? Then, instead of PEP 572's: > > stuff = [[(f(x) as y), x/y] for x in range(5)] > > one could write > > stuff = [[(y := f(x)), x/y] for x in range(5)] > > In difference to PEP 572, the variable y would not be statement-local, > which IMHO would be a welcome simplification. (PEP 572 introduces a > third class of variables to Python.) > > Overall, it seems to me that introducing a new operator ":=" would serve > the same purpose as PEP 572, but in a simpler and arguably cleaner way, > while eliminating the risk of confusion with "==". The operator "=" > would be left around, but could be deprecated in Python 5 and removed in > Python 6. It would certainly suit a language that is widely used in > education to sharpen the distinction between assignment and equality > operators. > > Cheers, > Christoph > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] PEP 572 version 2: Statement-Local Name Bindings
Disclaimer: I skimmed/searched through the PEP 572 threads (or should I say "literature"?) and did not find a discussion of the following point. If it has been discussed already, I'd be glad to be pointed to it. I am aware that Python, in contrast to C-like languages, has chosen not to treat assignments (=) as an expression with a value. The motivation for this is to avoid bugs due to confusion with the equality operator (==). But wouldn't it be a good alternative to PEP 572 to *add* an assignment operator that is an expression to Python, and is distinct from "=", for example ":="? Then, instead of PEP 572's: stuff = [[(f(x) as y), x/y] for x in range(5)] one could write stuff = [[(y := f(x)), x/y] for x in range(5)] In difference to PEP 572, the variable y would not be statement-local, which IMHO would be a welcome simplification. (PEP 572 introduces a third class of variables to Python.) Overall, it seems to me that introducing a new operator ":=" would serve the same purpose as PEP 572, but in a simpler and arguably cleaner way, while eliminating the risk of confusion with "==". The operator "=" would be left around, but could be deprecated in Python 5 and removed in Python 6. It would certainly suit a language that is widely used in education to sharpen the distinction between assignment and equality operators. Cheers, Christoph ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
On Fri, Mar 23, 2018 at 09:01:01PM +1100, Chris Angelico wrote: > PEP: 572 > Title: Syntax for Statement-Local Name Bindings [...] > Abstract > > > Programming is all about reusing code rather than duplicating it. I don't think that editorial comment belongs here, or at least, it is way too strong. I'm pretty sure that programming is not ALL about reusing code, and code duplication is not always wrong. Rather, we can say that *often* we want to avoid code duplication, and this proposal is way way to do so. And this should go into the Rationale, not the Abstract. The abstract should describe what this proposal *does*, not why, for example: This is a proposal for permitting temporary name bindings which are limited to a single statement. What the proposal *is* goes in the Abstract; reasons *why* we want it go in the Rationale. I see you haven't mentioned anything about Nick Coglan's (long ago) concept of a "where" block. If memory serves, it would be something like: value = x**2 + 2*x where: x = some expression These are not necessarily competing, but they are relevant. Nor have you done a review of any other languages, to see what similar features they already offer. Not even the C's form of "assignment as an expression" -- you should refer to that, and explain why this would not similarly be a bug magnet. > Rationale > = > > When a subexpression is used multiple times in a list comprehension, I think that list comps are merely a single concrete example of a more general concept that we sometimes want or need to apply the DRY principle to a single expression. This is (usually) a violation of DRY whether it is inside or outside of a list comp: result = (func(x), func(x)+1, func(x)*2) > Syntax and semantics > > > In any context where arbitrary Python expressions can be used, a **named > expression** can appear. This must be parenthesized for clarity, and is of > the form ``(expr as NAME)`` where ``expr`` is any valid Python expression, > and ``NAME`` is a simple name. > > The value of such a named expression is the same as the incorporated > expression, with the additional side-effect that NAME is bound to that > value for the remainder of the current statement. Examples should go with the description. Such as: x = None if (spam().ham as eggs) is None else eggs y = ((spam() as eggs), (eggs.method() as cheese), cheese[eggs]) > Just as function-local names shadow global names for the scope of the > function, statement-local names shadow other names for that statement. > (They can technically also shadow each other, though actually doing this > should not be encouraged.) That seems weird. > Assignment to statement-local names is ONLY through this syntax. Regular > assignment to the same name will remove the statement-local name and > affect the name in the surrounding scope (function, class, or module). That seems unnecessary. Since the scope only applies to a single statement, not a block, there can be no other assignment to that name. Correction: I see further in that this isn't the case. But that's deeply confusing, to have the same name refer to two (or more!) scopes in the same block. I think that's going to lead to some really confusing scoping problems. > Statement-local names never appear in locals() or globals(), and cannot be > closed over by nested functions. Why can they not be used in closures? I expect that's going to cause a lot of frustration. > Execution order and its consequences > > > Since the statement-local name binding lasts from its point of execution > to the end of the current statement, this can potentially cause confusion > when the actual order of execution does not match the programmer's > expectations. Some examples:: > > # A simple statement ends at the newline or semicolon. > a = (1 as y) > print(y) # NameError That error surprises me. Every other use of "as" binds to the current local namespace. (Or global, if you use the global declaration first.) I think there's going to be a lot of confusion about which uses of "as" bind to a new local and which don't. I think this proposal is conflating two unrelated concepts: - introducing new variables in order to meet DRY requirements; - introducing a new scope. Why can't we do the first without the second? a = (1 as y) print(y) # prints 1, as other uses of "as" would do That would avoid the unnecessary (IMO) restriction that these variables cannot be used in closures. > # The assignment ignores the SLNB - this adds one to 'a' > a = (a + 1 as a) "SLNB"? Undefined acronym. What is it? I presume it has something to do with the single-statement variable. I know it would be legal, but why would you write something like that? Surely your examples must at least have a pretence of being useful (even if the examples are only toy exam
Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2
I had a colleague complaining to me the other day about having to search multiple packages for the right function to move a file (implying: with the same semantics as drag-drop). If there isn’t a pathtools library on PyPI yet, this would certainly be valuable for newer developers. My view on Path is to either have everything on it or nothing on it (without removing what’s already there, of course), and since everything is so popular we should at least put everything in the one place. Top-posted from my Windows phone From: Mike Miller Sent: Monday, March 19, 2018 10:51 To: python-ideas@python.org Subject: Re: [Python-ideas] New PEP proposal -- Pathlib Module ShouldContain All File Operations -- version 2 On 2018-03-18 10:55, Paul Moore wrote: >> Should Path() have methods to access all file operations? > > No, (Counterexample, having a Path operation to set Windows ACLs for a path). Agreed, not a big fan of everything filesystem-related in pathlib, simply because it doesn't read well. Having them scattered isn't a great experience either. Perhaps it would be better to have a filesystem package instead, maybe named "fs" that included all this stuff in one easy to use location. File stuff from os, path stuff from os.path, pathlib, utils like stat, and shutil etc? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
On 23/03/18 10:01, Chris Angelico wrote: Apologies for letting this languish; life has an annoying habit of getting in the way now and then. Feedback from the previous rounds has been incorporated. From here, the most important concern and question is: Is there any other syntax or related proposal that ought to be mentioned here? If this proposal is rejected, it should be rejected with a full set of alternatives. Thank you very much, Chris. I think you've won me over on most points, though I'm not sure whether I'm overall +0 or -0 on the whole PEP :-) -- Rhodri James *-* Kynesim Ltd ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
On Fri, Mar 23, 2018 at 9:38 PM, Paul Moore wrote: > On 23 March 2018 at 10:01, Chris Angelico wrote: >> # ... except when function bodies are involved... >> if (input("> ") as cmd): >> def run_cmd(): >> print("Running command", cmd) # NameError >> >> # ... but function *headers* are executed immediately >> if (input("> ") as cmd): >> def run_cmd(cmd=cmd): # Capture the value in the default arg >> print("Running command", cmd) # Works > > What about > > cmd = "Something else" > if (input("> ") as cmd): > def run_cmd(): > print("Running command", cmd) # Closes over the "outer" > cmd, not the statement-local one? > > Did I get that right? I don't really like it if so (I think it's > confusing) but I guess I could live with "well, don't do that then" as > an answer. And I don't have a better interpretation. Yes, that would be it. And I agree: Don't do that. It's the same sort of confusion you'd get here: def f(): spam = 1 class C: spam = 2 def g(x=spam): print(spam) # prints 1 print(x) # prints 2 C.g() A class creates a scope that function bodies inside it don't close over, but their headers are still executed in that scope. So default argument values "see" those inner variables, but the body of the function doesn't. It's the same with SLNBs. > I'm still not convinced I like the proposal, but it's a lot cleaner > than previous versions, so thanks for that. Far fewer places where I > said "hmm, I don't understand the implications". Cool, thanks. That's the idea here. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
On 23 March 2018 at 10:01, Chris Angelico wrote: > # ... except when function bodies are involved... > if (input("> ") as cmd): > def run_cmd(): > print("Running command", cmd) # NameError > > # ... but function *headers* are executed immediately > if (input("> ") as cmd): > def run_cmd(cmd=cmd): # Capture the value in the default arg > print("Running command", cmd) # Works What about cmd = "Something else" if (input("> ") as cmd): def run_cmd(): print("Running command", cmd) # Closes over the "outer" cmd, not the statement-local one? Did I get that right? I don't really like it if so (I think it's confusing) but I guess I could live with "well, don't do that then" as an answer. And I don't have a better interpretation. I'm still not convinced I like the proposal, but it's a lot cleaner than previous versions, so thanks for that. Far fewer places where I said "hmm, I don't understand the implications". Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP proposal: unifying function/method classes
On Fri, 23 Mar 2018 07:25:33 +0100 Jeroen Demeyer wrote: > On 2018-03-23 00:36, Antoine Pitrou wrote: > > It does make sense, since the proposal sounds ambitious (and perhaps > > impossible without breaking compatibility). > > Well, *some* breakage of backwards compatibility will be unavoidable. > > > My plan (just a plan for now!) is to preserve backwards compatibility in > the following ways: > > * Existing Python attributes of functions/methods should continue to > exist and behave the same > > * The inspect module should give the same results as now (by changing > the implementation of some of the functions in inspect to match the new > classes) > > * Everything from the documented Python/C API. > > > This means that I might break compatibility in the following ways: > > * Changing the classes of functions/methods (this is the whole point of > this PEP). So anything involving isinstance() checks might break. > > * The undocumented parts of the Python/C API, in particular the C structure. One breaking change would be to add __get__ to C functions. This means e.g. the following: class MyClass: my_open = open would make my_open a MyClass method, therefore you would need to spell it: class MyClass: my_open = staticmethod(open) ... if you wanted MyClass().my_open('some file') to continue to work. Of course that might be considered a minor annoyance. Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] PEP 572: Statement-Local Name Bindings, take three!
Apologies for letting this languish; life has an annoying habit of getting in the way now and then. Feedback from the previous rounds has been incorporated. From here, the most important concern and question is: Is there any other syntax or related proposal that ought to be mentioned here? If this proposal is rejected, it should be rejected with a full set of alternatives. Text of PEP is below; formatted version will be live shortly (if it isn't already) at: https://www.python.org/dev/peps/pep-0572/ ChrisA PEP: 572 Title: Syntax for Statement-Local Name Bindings Author: Chris Angelico Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018 Abstract Programming is all about reusing code rather than duplicating it. When an expression needs to be used twice in quick succession but never again, it is convenient to assign it to a temporary name with small scope. By permitting name bindings to exist within a single statement only, we make this both convenient and safe against name collisions. Rationale = When a subexpression is used multiple times in a list comprehension, there are currently several ways to spell this, none of which is universally accepted as ideal. A statement-local name allows any subexpression to be temporarily captured and then used multiple times. Additionally, this syntax can in places be used to remove the need to write an infinite loop with a ``break`` in it. Capturing part of a ``while`` loop's condition can improve the clarity of the loop header while still making the actual value available within the loop body. Syntax and semantics In any context where arbitrary Python expressions can be used, a **named expression** can appear. This must be parenthesized for clarity, and is of the form ``(expr as NAME)`` where ``expr`` is any valid Python expression, and ``NAME`` is a simple name. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that NAME is bound to that value for the remainder of the current statement. Just as function-local names shadow global names for the scope of the function, statement-local names shadow other names for that statement. (They can technically also shadow each other, though actually doing this should not be encouraged.) Assignment to statement-local names is ONLY through this syntax. Regular assignment to the same name will remove the statement-local name and affect the name in the surrounding scope (function, class, or module). Statement-local names never appear in locals() or globals(), and cannot be closed over by nested functions. Execution order and its consequences Since the statement-local name binding lasts from its point of execution to the end of the current statement, this can potentially cause confusion when the actual order of execution does not match the programmer's expectations. Some examples:: # A simple statement ends at the newline or semicolon. a = (1 as y) print(y) # NameError # The assignment ignores the SLNB - this adds one to 'a' a = (a + 1 as a) # Compound statements usually enclose everything... if (re.match(...) as m): print(m.groups(0)) print(m) # NameError # ... except when function bodies are involved... if (input("> ") as cmd): def run_cmd(): print("Running command", cmd) # NameError # ... but function *headers* are executed immediately if (input("> ") as cmd): def run_cmd(cmd=cmd): # Capture the value in the default arg print("Running command", cmd) # Works Function bodies, in this respect, behave the same way they do in class scope; assigned names are not closed over by method definitions. Defining a function inside a loop already has potentially-confusing consequences, and SLNBs do not materially worsen the existing situation. Differences from regular assignment statements -- Using ``(EXPR as NAME)`` is similar to ``NAME = EXPR``, but has a number of important distinctions. * Assignment is a statement; an SLNB is an expression whose value is the same as the object bound to the new name. * SLNBs disappear at the end of their enclosing statement, at which point the name again refers to whatever it previously would have. SLNBs can thus shadow other names without conflict (although deliberately doing so will often be a sign of bad code). * SLNBs cannot be closed over by nested functions, and are completely ignored for this purpose. * SLNBs do not appear in ``locals()`` or ``globals()``. * An SLNB cannot be the target of any form of assignment, including augmented. Attempting to do so will remove the SLNB and assign to the fully-scoped name. In many respects, an SLNB is akin to a local variable in an imaginary nested