Re: [Python-ideas] PEP 572: about the operator precedence of :=

[Tim Peters]

[Guido]
> (I vaguely recall this has been brought up before, but I'm too lazy to find
> the subtread. So it goes.)
>
> PEP 572 currently seems to specify that when used in expressions, the
> precedence of `:=` is lower (i.e. it binds more tightly)

Umm ... that's the opposite of what the Reference Manual says "lower":means:

"""
6.16. Operator precedence

The following table summarizes the operator precedence in Python, from
lowest precedence (least binding) to highest precedence (most
binding).
"""


> than all operators except for the comma.

Which gets more potentially confusing become the comma isn't listed at
all as "an operator".   Instead the meaning of commas for building
tuples is captured by the higher-level `expression-list` grammar
production:

expression_list ::=  expression ( "," expression )* [","]

So _every_ mere operator "binds more tightly" (if you view it in those
terms) than a comma in an expression list.

What was mostly discussed before - after giving up on fully generally
"assignment expressions" - was whether ":=" should get a precedence
between comparison and bitwise OR operators.  So that, e.g., parens
wouldn't be needed in

if x := match() is not None:

to get the intended

if (x := match()) is not None:

But that never gained much traction, and it was dropped quickly.  It
was left with a (possibly unwritten!) consensus that ":=" should bind
very weakly as an operator.  Of the binary infix operators, the most
weakly binding (the weakest of which now is Boolean OR).


> I derive this from the single example
> `stuff = [[y := f(x), x/y] for x in range(5)]`.`

As above, the part you're looking at there falls under the
expression_list part of the grammar, and no binary operator could
group as

y OP (f(x), x/y)

instead.  All binary operators group as

(y OP f(x)), (x/y)


> From this it would follow that `f(a := 1, a)`

And now you're throwing in the entirely different meaning of commas in
argument lists ;-)  But same thing:  no operator can cross comma
boundaries in argument lists, because the grammar  of argument lists
doesn't allow for that either.  Even; e.g.,

f(lambda x: x, 2)

groups as

f((lambda x: x), 2)


> is equivalent to `a = 1; f(1, 1)`,

Yup.

> and also that `(a := 1, a)` is equivalent to `a = 1; (1, 1)`. (Although
> M.A.L. objected to this.)

That's just another instance of expression_list.

Nothing so far has surprised me, because I have been viewing ":=" as
an operator for some time now.


> But what should `a := 1, 1` at the top level (as a statement) do? On the one
> hand, analogy with the above suggest that it is equivalent to
>`a = 1; (1, 1)`.

If it's an operator, there's really no choice about that.


> But on the other hand, it would be really strange if the following two
> lines had different meanings:
>
> a = 1, 1   # a = (1, 1)
> a := 1, 1  # a = 1; (1, 1)

Agreed.


> I now think that the best way out is to rule `:=` in the top level
> expression of an expression statement completely (it would still be okay
> inside parentheses, where it would bind tighter than comma).

Probably prudent, and no objections here.


> An alternative would be to make `:=` bind less tight than comma (like `=`)
> everywhere, so that `a := 1, 1` indeed meant the same as `a = 1, 1`. But
> that feels very wrong at least for the case `f(a := 1, 1)` -- I believe Tim
> already mentioned that we've been conditioned by keyword arguments to parse
> this as `f((a := 1), 1)`. (I could add to this that we have done various
> things to generator expression syntax to avoid ever having to deal with
> ambiguities like `a, a+1 for a in range(10)` or `a for a in x, y`.)

Offhand, since comma is not formally "an operator" now, I expect it
would require major surgery to the grammar to have

a := 1, 1

group as

a := (1, 1)

in any context.  At least if ";=" is treated as "just another
operator" and doesn't grow its own unique-to-it pile of grammar rules.


> Another alternative would be to always require parentheses around `:=`, so
> that we would have to write `f((a := 1), 1)`. That's unambiguous, but
> otherwise just gets on the nerves.

I hoped that rigidly calling these "binding expressions" would break
the connection in peoples' minds that these somehow "should behave"
like assignment statements, but that seems futile.  There's really
nothing surprising here _if_ it's viewed as just another operator.
Nobody would be tempted to, e.g., add parens to f(a + 1, 1), or if `+`
were any other binary operator either.  So, over time, it would be
just as annoying need to type them for the `:=` operator.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: about the operator precedence of :=

[Guido van Rossum]

On Wed, May 9, 2018 at 8:42 PM, Chris Angelico  wrote:

> On Thu, May 10, 2018 at 1:33 PM, Guido van Rossum 
> wrote:
> > (I vaguely recall this has been brought up before, but I'm too lazy to
> find
> > the subtread. So it goes.)
> >
> > PEP 572 currently seems to specify that when used in expressions, the
> > precedence of `:=` is lower (i.e. it binds more tightly) than all
> operators
> > except for the comma. I derive this from the single example `stuff = [[y
> :=
> > f(x), x/y] for x in range(5)]`.
> >
> > From this it would follow that `f(a := 1, a)` is equivalent to `a = 1;
> f(1,
> > 1)`, and also that `(a := 1, a)` is equivalent to `a = 1; (1, 1)`.
> (Although
> > M.A.L. objected to this.)
> >
> > But what should `a := 1, 1` at the top level (as a statement) do? On the
> one
> > hand, analogy with the above suggest that it is equivalent to `a = 1; (1,
> > 1)`. But on the other hand, it would be really strange if the following
> two
> > lines had different meanings:
> >
> > a = 1, 1   # a = (1, 1)
> > a := 1, 1  # a = 1; (1, 1)
> >
> > I now think that the best way out is to rule `:=` in the top level
> > expression of an expression statement completely (it would still be okay
> > inside parentheses, where it would bind tighter than comma).
>
> I would have := bind more tightly than the comma. Consider:
>
> a = 1, x := 2, 3
>
> IMO the only sane interpretation is "x = 2; a = 1, 2, 3". Effectively,
> the := operator does not like to play with commas; we've already ruled
> out "a, b := range(2)" as a means of unpacking, so it makes more sense
> to have that simply mean "b = range(2); a, b".
>

Oh, I hadn't even though of combining the two in one statement. That
example is truly horrible (on first skim I didn't even notice it used two
different assignment operators!) and strengthens my confidence that we
should just disallow an un-parenthesized `:=` operator at the top level,
where now the top level includes the RHS of a classic assignment.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: about the operator precedence of :=

[Chris Angelico]

On Thu, May 10, 2018 at 1:33 PM, Guido van Rossum  wrote:
> (I vaguely recall this has been brought up before, but I'm too lazy to find
> the subtread. So it goes.)
>
> PEP 572 currently seems to specify that when used in expressions, the
> precedence of `:=` is lower (i.e. it binds more tightly) than all operators
> except for the comma. I derive this from the single example `stuff = [[y :=
> f(x), x/y] for x in range(5)]`.
>
> From this it would follow that `f(a := 1, a)` is equivalent to `a = 1; f(1,
> 1)`, and also that `(a := 1, a)` is equivalent to `a = 1; (1, 1)`. (Although
> M.A.L. objected to this.)
>
> But what should `a := 1, 1` at the top level (as a statement) do? On the one
> hand, analogy with the above suggest that it is equivalent to `a = 1; (1,
> 1)`. But on the other hand, it would be really strange if the following two
> lines had different meanings:
>
> a = 1, 1   # a = (1, 1)
> a := 1, 1  # a = 1; (1, 1)
>
> I now think that the best way out is to rule `:=` in the top level
> expression of an expression statement completely (it would still be okay
> inside parentheses, where it would bind tighter than comma).

I would have := bind more tightly than the comma. Consider:

a = 1, x := 2, 3

IMO the only sane interpretation is "x = 2; a = 1, 2, 3". Effectively,
the := operator does not like to play with commas; we've already ruled
out "a, b := range(2)" as a means of unpacking, so it makes more sense
to have that simply mean "b = range(2); a, b".

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Raymond Hettinger]

On May 9, 2018, at 7:39 AM, Facundo Batista  wrote:
> 
> This way, I could do:
> 
 authors = ["John", "Mary", "Estela"]
 "Authors: {:, j}".format(authors)
> 'Authors: John, Mary, Estela'
> 
...
> 
> What do you think?

That is an inspired idea.  I like it :-)


Raymond
 

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP 572: about the operator precedence of :=

[Guido van Rossum]

(I vaguely recall this has been brought up before, but I'm too lazy to find
the subtread. So it goes.)

PEP 572 currently seems to specify that when used in expressions, the
precedence of `:=` is lower (i.e. it binds more tightly) than all operators
except for the comma. I derive this from the single example `stuff = [[y :=
f(x), x/y] for x in range(5)]`.

>From this it would follow that `f(a := 1, a)` is equivalent to `a = 1; f(1,
1)`, and also that `(a := 1, a)` is equivalent to `a = 1; (1, 1)`.
(Although M.A.L. objected to this.)

But what should `a := 1, 1` at the top level (as a statement) do? On the
one hand, analogy with the above suggest that it is equivalent to `a = 1;
(1, 1)`. But on the other hand, it would be really strange if the following
two lines had different meanings:

a = 1, 1   # a = (1, 1)
a := 1, 1  # a = 1; (1, 1)

I now think that the best way out is to rule `:=` in the top level
expression of an expression statement completely (it would still be okay
inside parentheses, where it would bind tighter than comma).

An alternative would be to make `:=` bind less tight than comma (like `=`)
everywhere, so that `a := 1, 1` indeed meant the same as `a = 1, 1`. But
that feels very wrong at least for the case `f(a := 1, 1)` -- I believe Tim
already mentioned that we've been conditioned by keyword arguments to parse
this as `f((a := 1), 1)`. (I could add to this that we have done various
things to generator expression syntax to avoid ever having to deal with
ambiguities like `a, a+1 for a in range(10)` or `a for a in x, y`.)

Another alternative would be to always require parentheses around `:=`, so
that we would have to write `f((a := 1), 1)`. That's unambiguous, but
otherwise just gets on the nerves.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Terry Reedy]

On 5/9/2018 9:02 PM, Steven D'Aprano wrote:

On Thu, May 10, 2018 at 10:50:13AM +1000, Chris Angelico wrote:

On Thu, May 10, 2018 at 10:45 AM, Steven D'Aprano  wrote:

I too spent some time scratching my head trying to guess what you expect
this "j" format code to do, including the fundamental question "why j
and not some other letter?".


Interestingly, that question didn't faze me in the slightest. But
maybe my instant and automatic "oh you mean join" coloured my
expectations elsewhere, and possibly not correctly.


Oh, I didn't think of that! I was thinking i, j, k as integers, and
thinking "i is used, and j is the next letter".


I took me several minutes to *guess* that j abbreviated join.  It would 
have been better if it were given in the proposal.


--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Ken Kundert]

Facundo,
I think this is the beginning of a great feature. And it fills a hole in 
the 
current string formatting. Specifically, we can carefully control the 
formatting 
of the base data types, but not collections.

I would like to see you flesh out the idea. In particular, I'd like to see you 
address cases where:
1. The underlying members in the collection are not strings. Besides the basic 
   types such as numbers, it would also be nice to be able to apply formats 
   recursively so that one can construct a string using the attributes of 
   members that are objects or items or other collections.
2. The ability to handle collections other than simple lists or iterables, such 
   as dictionaries.

-Ken


On Wed, May 09, 2018 at 09:39:08AM -0300, Facundo Batista wrote:
> This way, I could do:
> 
> >>> authors = ["John", "Mary", "Estela"]
> >>> "Authors: {:, j}".format(authors)
> 'Authors: John, Mary, Estela'
> 
> In this case the join can be made in the format yes, but this proposal
> would be very useful when the info to format comes inside a structure
> together with other stuff, like...
> 
> >>> info = {
> ...   'title': "A book",
> ...   'price': Decimal("2.34"),
> ...   'authors: ["John", "Mary", "Estela"],
> ... }
> ...
> >>> print("{title!r} (${price}) by {authors:, j}".format(**info))
> "A book" ($2.34) by John, Mary, Estela
> 
> What do you think?
> 
> -- .Facundo
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Steven D'Aprano]

On Thu, May 10, 2018 at 10:50:13AM +1000, Chris Angelico wrote:
> On Thu, May 10, 2018 at 10:45 AM, Steven D'Aprano  wrote:
> > I too spent some time scratching my head trying to guess what you expect
> > this "j" format code to do, including the fundamental question "why j
> > and not some other letter?".
> 
> Interestingly, that question didn't faze me in the slightest. But
> maybe my instant and automatic "oh you mean join" coloured my
> expectations elsewhere, and possibly not correctly.

Oh, I didn't think of that! I was thinking i, j, k as integers, and 
thinking "i is used, and j is the next letter".


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Chris Angelico]

On Thu, May 10, 2018 at 10:45 AM, Steven D'Aprano  wrote:
> I too spent some time scratching my head trying to guess what you expect
> this "j" format code to do, including the fundamental question "why j
> and not some other letter?".

Interestingly, that question didn't faze me in the slightest. But
maybe my instant and automatic "oh you mean join" coloured my
expectations elsewhere, and possibly not correctly.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Steven D'Aprano]

On Wed, May 09, 2018 at 04:56:38PM -0300, Facundo Batista wrote:
> 2018-05-09 13:48 GMT-03:00 Rhodri James :
> 
> > -1 until you give me an actual spec rather than a curious example.
> >
> > Sorry if that sounds a bit rude, but I spend most of my time trying to find
> 
> Be sorry, it was rude.

I do not agree that Rhodri's comments were rude. They were frank and 
honest constructive criticism of the presentation of your idea.

I have no patience with hair-trigger accusations of "rudeness" as a 
debating tactic to curtail criticism, and your response to Rhodri's mild 
and fair words looks exactly like that. Whether you intended it that way 
or not, that is what it looks like to me.

I too spent some time scratching my head trying to guess what you expect 
this "j" format code to do, including the fundamental question "why j 
and not some other letter?". Even after I worked out some of it, I was 
still left with questions. So I think that Rhodri's criticism is 
completely valid: your post was vague, with a lack of detail and not 
even an attempt to provide a partial specification.

That doesn't have to be a formal specification, but it should be at 
least an attempt to say "this is what the format code means", rather 
than expect people to guess from a single example.

(By the way, if you answered my earlier questions, I haven't seen the 
answer.)

Dealing with under-specified, vague instructions, and the invariable 
failure to guess correctly what was intended, is frustrating, annoying 
and painful. We should not be surprised that vague proposals can trigger 
an angry response, but Rhodri's post showed great restraint.

It isn't just the waste of time and effort when when we guess wrong, it 
is also the implied disrespect: "I am too important to bother explaining 
myself to minions like you. Work out what I mean."

If an idea is important enough to post here, we should be willing to 
spend some time presenting the idea in sufficient detail so that people 
don't have to guess what we mean. As the Zen says, "Explicit is better 
than implicit".

No, that doesn't mean that we have to write a PEP before posting. There 
is a middle-ground between giving a single example and expecting people 
to extrapolate from it, and writing a full PEP.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Facundo Batista]

2018-05-09 13:48 GMT-03:00 Rhodri James :

> -1 until you give me an actual spec rather than a curious example.
>
> Sorry if that sounds a bit rude, but I spend most of my time trying to find

Be sorry, it was rude.

This list is for throwing ideas and see if they gain momentum... if
yes, it will be time for a PEP.

I know how to do a PEP, believe me.

But if we'll be asking for a Spec for every idea we think may be
valuable, we'll stall.

Regards,

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Twitter: @facundobatista
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Rhodri James]

On 09/05/18 13:39, Facundo Batista wrote:

This way, I could do:


authors = ["John", "Mary", "Estela"]
"Authors: {:, j}".format(authors)

'Authors: John, Mary, Estela'

In this case the join can be made in the format yes, but this proposal
would be very useful when the info to format comes inside a structure
together with other stuff, like...


info = {

...   'title': "A book",
...   'price': Decimal("2.34"),
...   'authors: ["John", "Mary", "Estela"],
... }
...

print("{title!r} (${price}) by {authors:, j}".format(**info))

"A book" ($2.34) by John, Mary, Estela

What do you think?


-1 until you give me an actual spec rather than a curious example.

Sorry if that sounds a bit rude, but I spend most of my time trying to 
find polite ways to inform people that I'm perfectly capable of 
implementing the fluffy collection of vague aspirations and inconsistent 
terminology they have given me, but the results won't be what they 
expect and probably won't be what they want either.  And that's just 
talking to my boss :-)  If you want something specific you'll have to 
specify it, otherwise you'll get something else.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A comprehension scope issue in PEP 572

[Tim Peters]

...

[Guido]
>> We should probably define what happens when you write [p := p for p in
>> range(10)]. I propose that this overwrites the loop control variable rather
>> than creating a second p in the containing scope -- either way it's probably
>> a typo anyway.

[Jacco van Dorp ]
> My naive assumption would be both.

Since this is all about scope, while I'm not 100% sure of what Guido
meant, I assumed he was saying "p can only have one scope in the
synthetic function:  local or non-local, not both, and local is what I
propose".  For example, let's flesh out his example a bit more:

p = 42
[p := p for p in range(10) if p == 3]
print(p) # 42?  3?  9?

If `p` is local to the listcomp, it must print 42.  If `p` is
not-local, it must print 9.  If it's some weird mixture of both, 3
makes most sense (the only time `p := p` is executed is when the `for`
target `p` is 3).

 > If it's just the insertion of a nonlocal statement like Tim suggested,

Then all occurrences of `p` in the listcomp are not-local, and the
example above prints 9..

> wouldn't the comprehension blow up to:
>
> def implicitfunc()
>   nonlocal p
>   templist = []
>   for p in range(10):
> p = p
> templist.append(p)
>   return templist
>
> ?

Yes.


> If it were [q := p for p in range(10)], it would be:
>
> def implicitfunc()
>   nonlocal q
>   templist = []
>   for p in range(10):
> q = p
> templist.append(q)
>   return templist

There's no question about that one, because `q` isn't _also_ used as a
`for` target.  There are two "rules" here:

1. A name appearing as a `for` target is local.  That's already the case.

2. All other names (including a name appearing as a binding-expression
target) are not local.

Clearer?  If a name appears as both, which rule applies?  "Both" is
likely the worst possible answer, since it's incoherent ;-)  If a name
appears as both a `for` target and as a binding-expression target,
that particular way of phrasing "the rules" suggests #1 (it's local,
period) is the more natural choice.  And, whether Guido consciously
knows it or not, that's why he suggested it ;-)


> Why would it need to be treated differently ?

Because it's incoherent.  It's impossible to make the example above
print 3 _merely_ by fiddling the scope of `p`.  Under the covers, two
distinct variables would need to be created, both of which are named
`p` as far as the user can see.  For my extension of Guido's example:

def implicitfunc()
nonlocal p
templist = []
for hidden_loop_p in range(10):
if hidden_loop_p == 3:
p = hidden_loop_p
templist.append(hidden_loop_p)
return templist


[Tim]
>> A compile-time error would be fine by me too.  Creating two meanings
>> for `p` is nuts - pick one in case of conflict.  I suggested before
>> that the first person with a real use case for this silliness should
>> get the meaning their use case needs, but nobody bit, so "it's local
>> then" is fine.

> x = x is legal. Why wouldn't p := p be ?

It's easy to make it "legal":  just say `p is local, period` or `p is
not local, period`.  The former will confuse people who think "but
names appearing as binding-expression targets are not local", and the
latter will confuse people who think "but names appearing as `for`
targets are local".

Why bother?  In the absence of an actual use case (still conspicuous
by absence), I'd be happiest refusing to compile such pathological
code.  Else `p is local, period` is the best pointless choice ;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Nathaniel Smith]

There are hidden directories, and then there are hidden directories :-). It
makes sense to me to add an option to the stdlib functions to skip
directories (and files) that the system considers hidden, so I guess that
means dotfiles on Unix and files with the hidden attribute on Windows. But
if you want "smart" matching that has special knowledge of CVS directories
and so forth, then that seems like something that would fit better as a
library on PyPI.

The rust "ignore" crate has a pretty good set of semantics, for reference.
It's not trivial, but it sure is handy :-):

https://docs.rs/ignore/0.4.2/ignore/struct.WalkBuilder.html

-n

On Tue, May 8, 2018, 00:43 Steve Barnes  wrote:

> In a lot of uses of os.walk it is desirable to skip version control
> directories, (which are usually hidden directories), to the point that
> almost all of the examples given look like:
>
> import os
> for root, dirs, files in os.walk(some_dir):
>  if 'CVS' in dirs:
>  dirs.remove('CVS')  # or .svn or .hg etc.
>  # do something...
>
> But of course there are many version control systems to the point that
> much of my personal code looks like, (note that I have to use a
> multitude of version control systems due to project requirements):
>
>
> import os
> vcs_dirs = ['.hg', '.svn', 'CSV', '.git', '.bz']  # Version control
> directory names I know
>
>
> for root, dirs, files in os.walk(some_dir):
>  for dirname in vcs_dirs:
>  dirs.remove(dirname)
>
> I am sure that I am missing many other version control systems but the
> one thing that all of the ones that I am familiar with default to
> creating their files in hidden directories. I know that the above
> sometimes hits problems on Windows if someone manually created a
> directory and you end up with abortions such as Csv\ or .SVN 
>
> Since it could be argued that hidden directories are possibly more
> common than simlinks, (especially in the Windows world of course), and
> that hidden directories have normally been hidden by someone for a
> reason it seems to make sense to me to normally ignore them in directory
> traversal.
>
> Obviously there are also occasions when it makes sense to include VCS,
> or other hidden, directories files, (e.g. "Where did all of my disk
> space go?" or "delete recursively"), so I would like to suggest
> including in the os.walk family of functions an additional parameter to
> control skipping all hidden directories - either positively or negatively.
>
> Names that spring to mind include:
>   * nohidden
>   * nohidden_dirs
>   * hidden
>   * hidden_dirs
>
> This change could be made with no impact on current behaviour by
> defaulting to hidden=True (or nohidden=False) which would just about
> ensure that no existing code is broken or quite a few bugs in existing
> code could be quietly fixed, (and some new ones introduced), by
> defaulting to this behaviour.
>
> Since the implementation of os.walk has changed to use os.scandir which
> exposes the returned file statuses in the os.DirEntry.stat() the
> overhead should be minimal.
>
> An alternative would be to add another new function, say os.vwalk(), to
> only walk visible entries.
>
> Note that a decision would have to be made on whether to include such
> filtering when topdown is False, personally I am tempted to include the
> filtering so as to maintain consistency but ignoring the filter when
> topdown is False, (or if topdown is False and the hidden behaviour is
> unspecified), might make sense if the skipping of hidden directories
> becomes the new default (then recursively removing files & directories
> would still include processing hidden items by default).
>
> If this receives a positive response I would be happy to undertake the
> effort involved in producing a PR.
> --
> Steve (Gadget) Barnes
> Any opinions in this message are my personal opinions and do not reflect
> those of my employer.
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Paul Moore]

On 9 May 2018 at 15:29, Eric V. Smith  wrote:
> On 5/9/18 10:01 AM, Paul Moore wrote:
>>
>> On 9 May 2018 at 14:49, Eric V. Smith  wrote:
>>>
>>> I would object to changing the format machinery. Any format spec should
>>> be
>>> interpreted by some object's __format__ method.
>>
>>
>> Agreed. In theory this is a nice idea, but the way formatting is
>> implemented (and the fact that join is a method on strings taking an
>> arbitrary iterable as an argument) means that it's a bad fit for the
>> format mini-language.
>>
>> I don't think the improved convenience is sufficient to warrant the
>> change that would be required in practice. (But if someone found a way
>> to make it work *without* changes to the underlying format machinery,
>> that would be a different matter...)
>
>
> Well, since you asked, let's combine this with dataclasses, because we can!
>
> from dataclasses import dataclass
> from typing import List
>
> @dataclass
> class Join:
> o: List[str]  # or similar
> def __format__(self, spec):
> return spec.join(self.o)
>
> l = ['a', 'b']
>
> print('{:, }'.format(Join(l)))
> print(f'{Join(l):-}')
>
> Gives:
> a, b
> a-b

:-)

Sorry, I should have been clearer. Given that

print('{}'.format(', '.join(l)))

isn't that difficult, at least for me the attraction of the proposal
was the possibility of not having to wrap the argument (whether in a
function call, or in a custom class). But the

print(f'{Join(l):-}')

approach is certainly cleaner looking than print(f'{", ".join(l)}').

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Eric V. Smith]

On 5/9/18 10:01 AM, Paul Moore wrote:

On 9 May 2018 at 14:49, Eric V. Smith  wrote:

I would object to changing the format machinery. Any format spec should be
interpreted by some object's __format__ method.


Agreed. In theory this is a nice idea, but the way formatting is
implemented (and the fact that join is a method on strings taking an
arbitrary iterable as an argument) means that it's a bad fit for the
format mini-language.

I don't think the improved convenience is sufficient to warrant the
change that would be required in practice. (But if someone found a way
to make it work *without* changes to the underlying format machinery,
that would be a different matter...)


Well, since you asked, let's combine this with dataclasses, because we can!

from dataclasses import dataclass
from typing import List

@dataclass
class Join:
o: List[str]  # or similar
def __format__(self, spec):
return spec.join(self.o)

l = ['a', 'b']

print('{:, }'.format(Join(l)))
print(f'{Join(l):-}')

Gives:
a, b
a-b

Eric
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Paul Moore]

On 9 May 2018 at 14:49, Eric V. Smith  wrote:
> On 5/9/18 9:28 AM, Chris Angelico wrote:
>> On Wed, May 9, 2018 at 11:06 PM, Steven D'Aprano 
>> wrote:
>>
>>> - do you truly mean lists *only*, or is any iterable acceptible?
>>
>> With the letter being "j" and the semantics being lifted from
>> str.join(), I would guess the latter.
>
> Since '{:spec}'.format(obj) basically becomes obj.__format__('spec'), this
> would have to be implemented on a concrete type (in the above example,
> list).

[...]

> I would object to changing the format machinery. Any format spec should be
> interpreted by some object's __format__ method.

Agreed. In theory this is a nice idea, but the way formatting is
implemented (and the fact that join is a method on strings taking an
arbitrary iterable as an argument) means that it's a bad fit for the
format mini-language.

I don't think the improved convenience is sufficient to warrant the
change that would be required in practice. (But if someone found a way
to make it work *without* changes to the underlying format machinery,
that would be a different matter...)

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Eric V. Smith]

On 5/9/18 9:28 AM, Chris Angelico wrote:

On Wed, May 9, 2018 at 11:06 PM, Steven D'Aprano  wrote:

On Wed, May 09, 2018 at 09:39:08AM -0300, Facundo Batista wrote:

This way, I could do:


authors = ["John", "Mary", "Estela"]
"Authors: {:, j}".format(authors)

'Authors: John, Mary, Estela'




Looks interesting, but I think we need to know the semantics in more
detail. For example:

- if the items of the list aren't already strings, how are they
  converted?


I'd expect that they'd be converted using format(), which by default
would just call str(). How you'd go about specifying a format string,
though, I'm not sure.


- do you truly mean lists *only*, or is any iterable acceptible?


With the letter being "j" and the semantics being lifted from
str.join(), I would guess the latter.


Since '{:spec}'.format(obj) basically becomes obj.__format__('spec'), 
this would have to be implemented on a concrete type (in the above 
example, list).



From the sound of it, this would be a change made to format(), or
rather the underlying C level function, PyObject_Format(). If done
there, it would also automatically apply to f-strings and anything
else that calls format(). Perhaps the right way is not a colon marker,
but an enhancement to the ! notation? We currently have !s and !r to
do str() and repr(), and this could be !j followed by a join string.
Combining this with a colon would allow the individual elements to be
formatted with the given string, and then joined. For instance:


I would object to changing the format machinery. Any format spec should 
be interpreted by some object's __format__ method.


Eric


x = [1,2,3]
msg = '#{x:3d!j, }#'.format(x=x)
# or equivalently
msg = f'#{x:3d!j, }#'
assert msg == '#  1,   2,   3#'

+0.5 on this. I don't currently yearn for it, but I'd probably use it
if it were available.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Eric V. Smith]

On 5/9/18 8:39 AM, Facundo Batista wrote:

This way, I could do:


authors = ["John", "Mary", "Estela"]
"Authors: {:, j}".format(authors)

'Authors: John, Mary, Estela'

In this case the join can be made in the format yes, but this proposal
would be very useful when the info to format comes inside a structure
together with other stuff, like...


info = {

...   'title': "A book",
...   'price': Decimal("2.34"),
...   'authors: ["John", "Mary", "Estela"],
... }
...

print("{title!r} (${price}) by {authors:, j}".format(**info))

"A book" ($2.34) by John, Mary, Estela

What do you think?


Among other things, I think you should use .format_map(info) instead of 
.format(**info)!


Eric

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Chris Angelico]

On Wed, May 9, 2018 at 11:06 PM, Steven D'Aprano  wrote:
> On Wed, May 09, 2018 at 09:39:08AM -0300, Facundo Batista wrote:
>> This way, I could do:
>>
>> >>> authors = ["John", "Mary", "Estela"]
>> >>> "Authors: {:, j}".format(authors)
>> 'Authors: John, Mary, Estela'
>
>
>
> Looks interesting, but I think we need to know the semantics in more
> detail. For example:
>
> - if the items of the list aren't already strings, how are they
>   converted?

I'd expect that they'd be converted using format(), which by default
would just call str(). How you'd go about specifying a format string,
though, I'm not sure.

> - do you truly mean lists *only*, or is any iterable acceptible?

With the letter being "j" and the semantics being lifted from
str.join(), I would guess the latter.


>From the sound of it, this would be a change made to format(), or
rather the underlying C level function, PyObject_Format(). If done
there, it would also automatically apply to f-strings and anything
else that calls format(). Perhaps the right way is not a colon marker,
but an enhancement to the ! notation? We currently have !s and !r to
do str() and repr(), and this could be !j followed by a join string.
Combining this with a colon would allow the individual elements to be
formatted with the given string, and then joined. For instance:

x = [1,2,3]
msg = '#{x:3d!j, }#'.format(x=x)
# or equivalently
msg = f'#{x:3d!j, }#'
assert msg == '#  1,   2,   3#'

+0.5 on this. I don't currently yearn for it, but I'd probably use it
if it were available.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Wolfgang Maier]

On 05/09/2018 02:39 PM, Facundo Batista wrote:

This way, I could do:


authors = ["John", "Mary", "Estela"]
"Authors: {:, j}".format(authors)

'Authors: John, Mary, Estela'

In this case the join can be made in the format yes, but this proposal
would be very useful when the info to format comes inside a structure
together with other stuff, like...


info = {

...   'title': "A book",
...   'price': Decimal("2.34"),
...   'authors: ["John", "Mary", "Estela"],
... }
...

print("{title!r} (${price}) by {authors:, j}".format(**info))

"A book" ($2.34) by John, Mary, Estela

What do you think?



For reference (first message of a rather long previous thread): 
https://mail.python.org/pipermail/python-ideas/2015-September/035787.html


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Steven D'Aprano]

On Wed, May 09, 2018 at 09:39:08AM -0300, Facundo Batista wrote:
> This way, I could do:
> 
> >>> authors = ["John", "Mary", "Estela"]
> >>> "Authors: {:, j}".format(authors)
> 'Authors: John, Mary, Estela'



Looks interesting, but I think we need to know the semantics in more 
detail. For example:

- if the items of the list aren't already strings, how are they
  converted?

- do you truly mean lists *only*, or is any iterable acceptible?

Here's a tiny proof-of-concept for the feature:

import string
class Template(string.Formatter):
def format_field(self, value, spec):
if spec.endswith('j'):
value = ', '.join(map(str, value))
spec = spec[:-1] + 's'
return super(Template, self).format_field(value, spec)

Template().format('{:j} => {:d}', ['alpha', 'beta', 42, 'delta'], 99)

# returns 'alpha, beta, 42, delta => 99'



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Have a "j" format option for lists

[Joao S. O. Bueno]

On 9 May 2018 at 09:39, Facundo Batista  wrote:
> This way, I could do:
>
 authors = ["John", "Mary", "Estela"]
 "Authors: {:, j}".format(authors)
> 'Authors: John, Mary, Estela'

+1.
I bit concerned about the relevant whitespace in there,
but just a little bit - it is already inside a string-literal anyway.


>
> In this case the join can be made in the format yes, but this proposal
> would be very useful when the info to format comes inside a structure
> together with other stuff, like...
>
 info = {
> ...   'title': "A book",
> ...   'price': Decimal("2.34"),
> ...   'authors: ["John", "Mary", "Estela"],
> ... }
> ...
 print("{title!r} (${price}) by {authors:, j}".format(**info))
> "A book" ($2.34) by John, Mary, Estela
>
> What do you think?
>
> --
> .Facundo
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Have a "j" format option for lists

[Facundo Batista]

This way, I could do:

>>> authors = ["John", "Mary", "Estela"]
>>> "Authors: {:, j}".format(authors)
'Authors: John, Mary, Estela'

In this case the join can be made in the format yes, but this proposal
would be very useful when the info to format comes inside a structure
together with other stuff, like...

>>> info = {
...   'title': "A book",
...   'price': Decimal("2.34"),
...   'authors: ["John", "Mary", "Estela"],
... }
...
>>> print("{title!r} (${price}) by {authors:, j}".format(**info))
"A book" ($2.34) by John, Mary, Estela

What do you think?

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Twitter: @facundobatista
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Wes Turner]

fnmatch.filter does Unix filename pattern matching.
https://docs.python.org/3/library/fnmatch.html#fnmatch.filter

grin and grind are like grep and find with options to filter hidden files
and VCS directories by default.
https://pypi.org/project/grin/

There's an example of using the Python API here:
https://github.com/rkern/grin/blob/master/examples/grinpython.py

- grin.get_regex(args)
- grin.get_filenames(args)

https://github.com/rkern/grin/blob/master/grin.py

On Wednesday, May 9, 2018, Steve Barnes  wrote:

>
>
> On 08/05/2018 15:53, Giampaolo Rodola' wrote:
> >
> >
> > On Tue, May 8, 2018 at 2:00 PM, David Mertz  > > wrote:
> >
> > I like the idea. I think an argument to os.walk() is the simplest
> > option for most users. But per some comments, "hidden" is actually
> > more subtle than the filesystem bit sometimes. I.e. dot-files, ~
> > suffix, maybe .bak, etc.
> >
> > I'd suggest meeting the ideas slightly and making the new argument
> > 'filter' or 'skip' that takes a callable. Default to None, but
> > provide an os.is_hidden that users don't need to figure out how to
> > implement. E.g.
> >
> > os.walk(PATH, skip=os.is_hidden)
> >
> > os.walk(PATH, skip=lambda entry: entry.name.endswith(('~', '.bak',
> > '.tmp')))
> >
> >
> > I think this would be a good addition because it gives direct access to
> > the underlying os.scandir() objects which are currently inaccessible and
> > discarded (if os.walk() were to be written today it'd probably yield
> > (root, os.DirEntry) instead of (root, dirs, files)). As such one can
> > implement advanced filtering logic without having to call os.stat() for
> > each path string yielded by os.walk() (faster).
> >
> > IMO the callback should accept a (root, os.DirEntry) pair though,
> > because the "root" path can also be part of the filtering logic.
> >
> > --
> >
> > Giampaolo - http://grodola.blogspot.com
> >
> >
>
> I like the idea of extending the original idea to a filtered walk
> possibly with some predefined filters.
>
> As there does not seem to be a lot of strong opposition so far to the
> basic idea, (other than a some "why bother it is too easy to do
> yourself"), it seems like there is a choice now is between:
>
>a) raising an enhancement request on the tracker (I am not sure if
> this is major enough to require a PEP) or
>b) setting up a new library on PyPi and putting it out there to see
> if it sinks or swims.
>
> What is the general feeling between the two options?
>
> --
> Steve (Gadget) Barnes
> Any opinions in this message are my personal opinions and do not reflect
> those of my employer.
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A comprehension scope issue in PEP 572

[Jacco van Dorp]

My apologies for something unclear in my previous mail - the second
block I quoted (the one without a name) originated from Guido, not
from Tim.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Steve Barnes]

On 08/05/2018 15:53, Giampaolo Rodola' wrote:
> 
> 
> On Tue, May 8, 2018 at 2:00 PM, David Mertz  > wrote:
> 
> I like the idea. I think an argument to os.walk() is the simplest
> option for most users. But per some comments, "hidden" is actually
> more subtle than the filesystem bit sometimes. I.e. dot-files, ~
> suffix, maybe .bak, etc.
> 
> I'd suggest meeting the ideas slightly and making the new argument
> 'filter' or 'skip' that takes a callable. Default to None, but
> provide an os.is_hidden that users don't need to figure out how to
> implement. E.g.
> 
> os.walk(PATH, skip=os.is_hidden)
> 
> os.walk(PATH, skip=lambda entry: entry.name.endswith(('~', '.bak',
> '.tmp')))
> 
> 
> I think this would be a good addition because it gives direct access to 
> the underlying os.scandir() objects which are currently inaccessible and 
> discarded (if os.walk() were to be written today it'd probably yield 
> (root, os.DirEntry) instead of (root, dirs, files)). As such one can 
> implement advanced filtering logic without having to call os.stat() for 
> each path string yielded by os.walk() (faster).
> 
> IMO the callback should accept a (root, os.DirEntry) pair though, 
> because the "root" path can also be part of the filtering logic.
> 
> -- 
> 
> Giampaolo - http://grodola.blogspot.com
> 
> 

I like the idea of extending the original idea to a filtered walk 
possibly with some predefined filters.

As there does not seem to be a lot of strong opposition so far to the 
basic idea, (other than a some "why bother it is too easy to do 
yourself"), it seems like there is a choice now is between:

   a) raising an enhancement request on the tracker (I am not sure if 
this is major enough to require a PEP) or
   b) setting up a new library on PyPi and putting it out there to see 
if it sinks or swims.

What is the general feeling between the two options?

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A comprehension scope issue in PEP 572

[Jacco van Dorp]

> [Tim]
> {About binding the for loop variable}

Yeah, that binding is the one I attempted to refer to. So I did
understand you after all.

> We should probably define what happens when you write [p := p for p in
> range(10)]. I propose that this overwrites the loop control variable rather
> than creating a second p in the containing scope -- either way it's probably
> a typo anyway.

My naive assumption would be both. If it's just the insertion of a
nonlocal statement like Tim suggested, wouldn't the comprehension blow
up to:

def implicitfunc()
  nonlocal p
  templist = []
  for p in range(10):
p = p
templist.append(p)
  return templist

?

If it were [q := p for p in range(10)], it would be:

def implicitfunc()
  nonlocal q
  templist = []
  for p in range(10):
q = p
templist.append(q)
  return templist

Why would it need to be treated differently ?
(type checkers probably should, though.)

> [Tim]
> A compile-time error would be fine by me too.  Creating two meanings
> for `p` is nuts - pick one in case of conflict.  I suggested before
> that the first person with a real use case for this silliness should
> get the meaning their use case needs, but nobody bit, so "it's local
> then" is fine.

x = x is legal. Why wouldn't p := p be ?

> [Juancarlo]
> Maybe my distrust is just don't like the new syntax, or that I'am biased 
> towards using "as".

I share your bias, but not your distrust. (go as !)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/