[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[David Mertz]

On Sat, Jul 27, 2019 at 9:28 PM Steven D'Aprano  wrote:

> If you're going to add support for a placeholder, there is no need to
> create a new kind of partial function?
>

Exactly! This is just an enhanced functools.partial.  An enhancement that
has already been done in several 3rd party libraries, in fact.  I like the
ellipsis, which most 3rd parties use, but it could be configurable what the
sentinel is.  Maybe a default brand new `functools.SKIP` object, but with
the docs explaining the way to use the ellipsis for those 99% of users for
whom that wouldn't break anything.

Someone posted an example implementation that used a configurable
placeholder like this.


> > Sure we could also use a `lambda` instead
> I believe that partial is faster than lambda.
>

I don't really care about faster.  I think the "partial with placeholder"
is simply nicer to read for many cases.  Sure this is a toy, but:

>>> lastname = 'Mertz'
>>> greet_david = partial(print, ..., 'David', lastname)  # In Python 3.9
>>> greet_david('Verwelkoming')
Verwelkoming David Mertz

vs.

>>> greet_david = lambda greeting: print(greeting, "David", lastname)
>>> greet_david('Cześć')
Cześć David Mertz

OK, that's a bad example because we are using it for the side effect too,
but I think it illustrates how the more flexible partial() would look
better than lambda.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XG27FN6BSHNJVTS5YRLEZBRICRZLGT43/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Andrew Barnert via Python-ideas]

On Jul 27, 2019, at 12:47, Anders Hovmöller  wrote:
> 
> 
>> On 27 Jul 2019, at 21:10, Andrew Barnert  wrote:
>> 
>> Many of the compelling examples for PEP 570 are good examples here. The 
>> following are all impossible intentionally, and for different reasons:
>> 
>>   skipper = partial(range, step=n)
>>   powbase = partial(pow, mod=base)
>>   clscheck = partial(isinstance, class_or_tuple=cls)
> 
> Those seem like great cases for changing the standard library to use normal 
> parameters! range and pow are both nicer with keyword arguments imo.

Look at the signatures for range and isinstance: range takes an optional 
argument on the left (or, if you prefer, it’s an overload of two different 
signatures), which can’t even be expressed in Python syntax; isinstance takes a 
class or a tuple of classes in the same parameter, so there’s no good name for 
it. There are a few other functions just like range, and a bunch like 
isinstance, and some other classes of functions with different kinds of weird 
signatures (although pow isn’t one of them).

You could argue that these are just bad designs, but better designs weren’t 
possible when they were added to Python. For example, anything designed to take 
1 or many classes since 2.2 would just use *classes instead of a single 
argument, but isinstance-style functions predate 2.2, so there was no *args. 
Changing those existing functions would break a lot of existing code. And if 
that wasn’t acceptable for 2.2 or even 3.0, it’s probably never going to be 
acceptable.

>> Also, even for cases like the OP’s where there’s no semantics reason the 
>> argument couldn’t have a keyword, there may still be other reasons. When 
>> argclinic was added in PEP 436, it was specifically argued that it shouldn’t 
>> be used as an opportunity to phase out positional-only params, in part 
>> because for many functions the performance cost of keyword processing 
>> outweighs the uncommon potential use of the keywords; IIRC, the OP’s 
>> specific method was even one of the core examples. And, as PEP 570 points 
>> out, METH_FASTCALL makes that potentially true for pure Python functions as 
>> well, to the extent that some other stdlib functions might actually want to 
>> lose their keyword args.
> 
> Well that sounds pretty terrible to me. I’ve tried to write calls with 
> keywords of many many functions in the standard library because it just isn’t 
> really readable with positional :(

I think many of the functions in the stdlib do get it wrong, for legacy reasons 
(nobody thought about the tradeoff consciously until the argclinic discussion, 
and also it was a pain to add keywords in C before argclinic—and many of the 
functions you use most often date back even farther, to before keywords 
existing at all), but there is an actual tradeoff, so it shouldn’t be 100% of 
functions take keywords. The two PEPs both make the case for that.

And nobody’s going to sweep through the stdlib evaluating every function. If 
you’ve got a group of functions that particularly annoys you, and can make the 
case that the arg parsing performance isn’t important for that function and the 
keywords are useful, you can file it on b.p.o. (ideally with a patch) and I’m 
sure someone will take a look.

Meanwhile, there are going to be functions that take positional arguments, 
whether for semantic reasons, for performance reasons, or for pure legacy 
issues. As long as any of those cases exist, the OP’s proposal makes sense, at 
least to me.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VGA5FYQ4AIMINQ27Y5QUTYWXRFFKMXDW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Steven D'Aprano]

On Sat, Jul 27, 2019 at 12:47:39PM -, Dominik Vilsmeier wrote:

> I propose adding a function to `functools` that works with 
> placeholders and thus offers even greater flexibility. The Ellipsis 
> literal `...` seems a intuitive choice for that task. When eventually 
> calling such a "partial placeholder" object, it would fill in 
> placeholders from the left and add remaining `args` to the right. In 
> terms of implementation this can be realized as a subclass of 
> `partial` itself.

If you're going to add support for a placeholder, there is no need to 
create a new kind of partial function?

Support for a placeholder is completely backwards compatible, if you use 
a brand new special sentinel that doesn't yet exist.

# May want to give it a nicer repr, but this is the 
# minimal version that will work.
SKIP = object()

There's only one public function, ``partial``, regardless of whether the 
caller uses the sentinel or not:


def function(a, b, c, d): pass

# Existing use is unchanged:
from functools import partial
partial(function, 1, 2)  # Fill in a, b.


# New functionality is triggered by the use of the sentinel:
from functools import partial, SKIP
partial(function, SKIP, 1, SKIP, 2)  # Fill in b, d.


If you don't like the use of a named sentinel, we could still use 
Ellipsis, but that will break anyone's partial functions that happen to 
already use Ellipsis as a value. That will probably require a 
warning in 3.9 and the new functionality only gets added in 3.10.


> Sure we could also use a `lambda` instead

I believe that partial is faster than lambda.


[steve@ando cpython]$ ./python -m timeit \
-s "from functools import partial" \
-s "f = partial(lambda x,y: x+y, 1)" \
"f(100)"
10 loops, best of 5: 1.98 usec per loop


[steve@ando cpython]$ ./python -m timeit \
-s "g = lambda x,y: x+y" \
-s "f = lambda a: g(1, a)" \
"f(100)"
10 loops, best of 5: 2.44 usec per loop


To avoid the name lookup of "g", I tried this, but it was even slower:

[steve@ando cpython]$ ./python -m timeit \
-s "f = lambda a: (lambda x,y: x+y)(1, a)" \
"f(100)"
10 loops, best of 5: 3.4 usec per loop


(Not surprising, as we've changed a name lookup into creating a new 
function object.)

Obviously this example is simple enough that we don't need partial at 
all, but its just an illustration demonstrating that partial seems to 
have lower overhead than normal function objects.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3IEMJSDGPACW2OBPP4OPUCAM7Q77EE7N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Guido van Rossum]

Before we go too far down this route, let's consider whether these can't be
solved with lambda rather than introducing new special cases to partial().

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5Z7QB7RAWODXPNPF774YL4BNXOQRJU5G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[MRAB]

On 2019-07-27 20:10, Andrew Barnert via Python-ideas wrote:

On Jul 27, 2019, at 08:04, Anders Hovmöller  wrote:



On 27 Jul 2019, at 14:47, Dominik Vilsmeier  wrote:

currently, regarding positional arguments, `partial` gives us the option to 
partialize functions from the left. There's been some interest about 
partializing functions from the right instead (e.g. [SO post, 9k views, 39 
upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. the 
various `str` methods.


Do you have other examples? That (and most likely similar) examples are just that the standard library contains methods and functions that could be fixed to accept keyword arguments. This would be less confusing and more coherent. 


Many of the compelling examples for PEP 570 are good examples here. The 
following are all impossible intentionally, and for different reasons:

 skipper = partial(range, step=n)
 powbase = partial(pow, mod=base)
 clscheck = partial(isinstance, class_or_tuple=cls)

Plus, there’s also one very general example: anything that semantically has to 
take *args can’t be changed to use keywords:

 partial(format, 2=x)
 partial(map, 1=x, 2=y)
 partial(executor.submit, 1=arg)

And similarly for itertools.product, min/max, and everything that acts as a 
proxy like submit.


[snip]

I was thinking that 'partial' is like a function definition, so why not 
have a variation on the function definition:


def powbase(x, y) is pow(x, y, base)
def clscheck(obj) is isinstance(obj, cls)

It would capture the current reference of any name that's not passed via 
the parameter list, e.g. base and cls in the above examples.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PIVGTG455RCAKLKSZFY5PMZK7SQJZIMW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Anders Hovmöller]

> On 27 Jul 2019, at 21:10, Andrew Barnert  wrote:
> 
> On Jul 27, 2019, at 08:04, Anders Hovmöller  wrote:
>> 
>>> On 27 Jul 2019, at 14:47, Dominik Vilsmeier  
>>> wrote:
>>> 
>>> currently, regarding positional arguments, `partial` gives us the option to 
>>> partialize functions from the left. There's been some interest about 
>>> partializing functions from the right instead (e.g. [SO post, 9k views, 39 
>>> upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. 
>>> the various `str` methods.
>> 
>> Do you have other examples? That (and most likely similar) examples are just 
>> that the standard library contains methods and functions that could be fixed 
>> to accept keyword arguments. This would be less confusing and more coherent. 
> 
> Many of the compelling examples for PEP 570 are good examples here. The 
> following are all impossible intentionally, and for different reasons:
> 
>skipper = partial(range, step=n)
>powbase = partial(pow, mod=base)
>clscheck = partial(isinstance, class_or_tuple=cls)

Those seem like great cases for changing the standard library to use normal 
parameters! range and pow are both nicer with keyword arguments imo.

> Plus, there’s also one very general example: anything that semantically has 
> to take *args can’t be changed to use keywords:
> 
>partial(format, 2=x)
>partial(map, 1=x, 2=y)
>partial(executor.submit, 1=arg)

Ok now THAT is a compelling argument! 

> And similarly for itertools.product, min/max, and everything that acts as a 
> proxy like submit.
> 
> Also, even for cases like the OP’s where there’s no semantics reason the 
> argument couldn’t have a keyword, there may still be other reasons. When 
> argclinic was added in PEP 436, it was specifically argued that it shouldn’t 
> be used as an opportunity to phase out positional-only params, in part 
> because for many functions the performance cost of keyword processing 
> outweighs the uncommon potential use of the keywords; IIRC, the OP’s specific 
> method was even one of the core examples. And, as PEP 570 points out, 
> METH_FASTCALL makes that potentially true for pure Python functions as well, 
> to the extent that some other stdlib functions might actually want to lose 
> their keyword args.

Well that sounds pretty terrible to me. I’ve tried to write calls with keywords 
of many many functions in the standard library because it just isn’t really 
readable with positional :(

/ Anders
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/24W72RAQFPWSL7U7GMNMH6JLI3GTVYFK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Andrew Barnert via Python-ideas]

On Jul 27, 2019, at 08:04, Anders Hovmöller  wrote:
> 
>> On 27 Jul 2019, at 14:47, Dominik Vilsmeier  wrote:
>> 
>> currently, regarding positional arguments, `partial` gives us the option to 
>> partialize functions from the left. There's been some interest about 
>> partializing functions from the right instead (e.g. [SO post, 9k views, 39 
>> upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. 
>> the various `str` methods.
> 
> Do you have other examples? That (and most likely similar) examples are just 
> that the standard library contains methods and functions that could be fixed 
> to accept keyword arguments. This would be less confusing and more coherent. 

Many of the compelling examples for PEP 570 are good examples here. The 
following are all impossible intentionally, and for different reasons:

skipper = partial(range, step=n)
powbase = partial(pow, mod=base)
clscheck = partial(isinstance, class_or_tuple=cls)

Plus, there’s also one very general example: anything that semantically has to 
take *args can’t be changed to use keywords:

partial(format, 2=x)
partial(map, 1=x, 2=y)
partial(executor.submit, 1=arg)

And similarly for itertools.product, min/max, and everything that acts as a 
proxy like submit.

Also, even for cases like the OP’s where there’s no semantics reason the 
argument couldn’t have a keyword, there may still be other reasons. When 
argclinic was added in PEP 436, it was specifically argued that it shouldn’t be 
used as an opportunity to phase out positional-only params, in part because for 
many functions the performance cost of keyword processing outweighs the 
uncommon potential use of the keywords; IIRC, the OP’s specific method was even 
one of the core examples. And, as PEP 570 points out, METH_FASTCALL makes that 
potentially true for pure Python functions as well, to the extent that some 
other stdlib functions might actually want to lose their keyword args.

All that being said, this proposal seems like something that could easily be 
put on PyPI to see how much uptake it gets, instead of put immediately into the 
stdlib. I think people will find it useful. But maybe, say, funcy/toolz/etc. 
will borrow the idea and it’ll turn out that almost everyone who wants it 
already wants one of those libs anyway. Or maybe the bikeshedding potential 
will be higher than expected, and a variety of different modules that all 
handle the placeholders differently will compete, and it’ll turn out that 
everyone loves some different design that allows both single-arg and multi-arg 
placeholders 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AAA32XZ3O7HNESRDQ437OS35MPAFUPTT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Cartesian Product on `__mul__`

[Ethan Furman]

On 07/27/2019 03:17 AM, Batuhan Taskaya wrote:


Its more clear and understandable than itertools.product for people from outside of 
a programming background. The simplicity is useful when we are working with 
multiple expressions at sametime. (cset(a & b) * cset(a | b))


So build that functionality into cset.  I believe that's what NumPy does.

--
~Ethan~
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GUXBAEWSHE75E7SP2PBRCVNNI7H5WQVF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: for ... except, with ... except

[Ethan Furman]

On 07/27/2019 01:19 AM, Serhiy Storchaka wrote:

26.07.19 23:57, Guido van Rossum пише:



However, I worry that when people see this syntax, they will think that the 
except clause is for handling exceptions in the loop body. (That's certainly 
what I assumed when I read just your subject line. :-)


And it's what I thought when I saw the psuedo-code.


I think people can make a mistake only when they see it for the first time. 
After you learn what this syntax means, you will not make this mistake the 
second time. The same applies to any other syntactic construction.


Sorry, but that mistake will be made many, many times by the same people.  It's taken me 
several years to sort out `for...else`, and even now I have to think "it's a search 
loop, adapt your algorithm".

Another classic example is allowing single-"=" assignment in tests:

  if some_var = 7:
  ...

Sure, folks /know/ what it means, but it's a common bug because it doesn't read as "if 
some_var is assigned 7" but as "if some_var is equal to 7".

--
~Ethan~
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WGIGILBBHKAQ4CJHMBLRBYPNNOM3ZERT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a "partial with placeholders" function to the functools module

[Anders Hovmöller]

> On 27 Jul 2019, at 14:47, Dominik Vilsmeier  wrote:
> 
> Hello,
> 
> currently, regarding positional arguments, `partial` gives us the option to 
> partialize functions from the left. There's been some interest about 
> partializing functions from the right instead (e.g. [SO post, 9k views, 39 
> upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. the 
> various `str` methods.

Do you have other examples? That (and most likely similar) examples are just 
that the standard library contains methods and functions that could be fixed to 
accept keyword arguments. This would be less confusing and more coherent. 

/ Anders 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZRAGJJF7TKRYNLHXBL7ZW6MZZTKFA4YN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add a "partial with placeholders" function to the functools module

[Dominik Vilsmeier]

Hello,

currently, regarding positional arguments, `partial` gives us the option to 
partialize functions from the left. There's been some interest about 
partializing functions from the right instead (e.g. [SO post, 9k views, 39 
upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. the 
various `str` methods.

I propose adding a function to `functools` that works with placeholders and 
thus offers even greater flexibility. The Ellipsis literal `...` seems a 
intuitive choice for that task. When eventually calling such a "partial 
placeholder" object, it would fill in placeholders from the left and add 
remaining `args` to the right. In terms of implementation this can be realized 
as a subclass of `partial` itself.

## Implementation

from functools import partial
from reprlib import recursive_repr

class partial_placehold(partial):
placeholder = Ellipsis

def __call__(self, /, *args, **keywords):
args = iter(args)
try:
old_args = [x if x is not self.placeholder else next(args) for 
x in self.args]
except StopIteration:
raise TypeError('too few arguments were supplied') from None
keywords = {**self.keywords, **keywords}
return self.func(*old_args, *args, **keywords)

@recursive_repr()
def __repr__(self):
qualname = type(self).__qualname__
args = [repr(self.func)]
args.extend(repr(x) if x is not self.placeholder else '...' for x 
in self.args)  # Only this line deviates from `partial.__repr__`; could also 
factor that out into a separate method.
args.extend(f"{k}={v!r}" for (k, v) in self.keywords.items())
if type(self).__module__ == "functools":
return f"functools.{qualname}({', '.join(args)})"
return f"{qualname}({', '.join(args)})"

# Would need to add something for compatibility with `partial`, i.e. for 
partializing a placeholder function.

## Example

This allows for example the following usage:

replace_dots_with_underscore = partial_placehold(str.replace, ..., '.', '_')
replace_dots_with_underscore('foo.bar.baz')

## Relevance

Sure we could also use a `lambda` instead ([as discussed 
here](https://mail.python.org/archives/list/python-ideas@python.org/message/YD5OQEPXRL6LIK3DRVIZR6IIMHATCMVC/))
 but there was a reason `partial` was introduced and I think the same arguments 
apply here too. Though most functions allow partializing via keyword arguments 
and this is undoubtedly a cleaner way, some might not and for example 
built-ins' methods won't allow it. Especially Python 3.8's introduction of 
positional-only parameters (PEP 570) might give rise to cases where `partial` 
is not sufficient.
In case inspection is desired a `lambda` does not provide much information 
(sure you could always dig deeper with `inspect` for example but that's not the 
point). Consider the following example of a pre-defined sequence of default 
postprocessing steps and the user might add their own or remove existing ones, 
as appropriate:

postprocessing_steps = [
lambda s: s.replace('foo', 'bar'),
]
print(postprocessing_steps[0])  #  at 0x7f94a850dd30>

This doesn't give a lot of information about what the lambda actually does (and 
thus whether the user should remove it or not). Using the `partial_placehold` 
instead, it's clear what is happening:

postprocessing_steps = [
partial_placehold(str.replace, ..., 'foo', 'bar'),
]
print(postprocessing_steps[0])  # partial_placehold(, ..., 'foo', 'bar')

## Compatibility

The proposed solution works with the current syntax and the usage of Ellipsis 
as a placeholder object is likely not to collide with actually used values (in 
any case the user might still reassign the `.placeholder` attribute).
Because the direction of partializing is unchanged (still left to right) this 
doesn't introduce ambiguities which might come with a "right partial" function. 
Creating a placeholder function from a `partial` object is possible without any 
changes, the opposite way requires an additional check to result in a 
placeholder object again.

## Possible confusion

Regarding the usage of Ellipsis right now, in `numpy` or `typing` for example, 
it always represents a placeholder for multiple "things", not a single one:

array[..., None]  # All the dimensions of `array` plus a new one.
typing.Tuple[str, ...]  # Any number of str objects.

So the expectations might be biased in that sense. For example:

def foo(a, b, c, d):
pass

p_foo = partial_placehold(foo, ..., 1, 2)
p_foo(3, 4)

Someone else reviewing the code might now assume that the `...` means to act as 
a placeholder for all arguments except the last two (and hence `p_foo(3, 4)` 
would be equivalent to `foo(3, 4, 1, 2)` while it actually is equivalent to 
`foo(3, 1, 2, 4)`). But this would be again some kind of 

[Python-ideas] Re: for ... except, with ... except

[Anders Hovmöller]

> On 27 Jul 2019, at 10:19, Serhiy Storchaka  wrote:
> 
> 26.07.19 23:57, Guido van Rossum пише:
>> These are interesting ideas. It looks like you intend the except clause of 
>> the for loop to *only* cover the iter() and next() calls that are implicit 
>> in the for loop. You're right that it's awkward to catch exceptions there.
> 
> Right. If you want to catch an exception in the loop body you have to wrap 
> the loop body with "try ... except". If you want to catch an exception in the 
> loop body *and* the 'for' clause you have to wrap the whole "for" statement 
> with "try ... except". No new syntax is needed for this. But if you want to 
> catch an exception *only* in the 'for' clause you can't (without getting rid 
> of the "for" statement at all).
> 
>> However, I worry that when people see this syntax, they will think that the 
>> except clause is for handling exceptions in the loop body. (That's certainly 
>> what I assumed when I read just your subject line. :-)
> 
> I think people can make a mistake only when they see it for the first time. 
> After you learn what this syntax means, you will not make this mistake the 
> second time. The same applies to any other syntactic construction.


I don't think that's correct. I know many people who have learned what for-else 
does and promptly forgot it because it doesn't make sense to them. I'm fairly 
certain I know what it does after having this discussion many many times but 
I'm not 100% sure. 

If something reads one way and is in reality something else it's super hard to 
not read it as the plain meaning.  In the for-else case it could have been 
"for.. on not break" (and "on empty", "on break", and such) and it would be 
very nice. 

I'm supportive of the proposal but not the syntax. 

/ Anders 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D464RQJMR5RK74JVE7DFHHU6INOEB6RR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: for ... except, with ... except

[Serhiy Storchaka]

26.07.19 21:52, Bruce Leban пише:


On Fri, Jul 26, 2019 at 11:27 AM Serhiy Storchaka > wrote:



So you will be able to add errors handling like in:

     with connect() as stream:
         for data in stream:
             try:
                 write(data)
             except OSError:
                 handle_write_error()
         except OSError:
             handle_read_error()
     except OSError:
         handle_connection_error()


To put this in a simpler way: the proposal is to add an except clause 
that applies ONLY to the direct operation of the with or for statement 
and not to the block. That's an interesting idea.


The one thing I find confusing about your proposal is that the 
proposed syntax does not imply the behavior. In a try statement, the 
except appears at the end and after all possible statements that it 
could cover. The proposal mimics that syntax but with different 
semantics. Something like this would be much more clear what is going on:


    for VARIABLE in EXPRESSION:
        except EXCEPTION:
            BLOCK
        BLOCK

    with EXPRESSION as VARIABLE:
        except EXCEPTION:
            BLOCK
        BLOCK

    while EXPRESSION:
        except EXCEPTION:
            BLOCK
        BLOCK



Besides an unusual for Python layout (a clause has different indentation 
than the initial clause of the statement to which it belongs) there is 
other problem. The exception block is not the part of the "for" or 
"with" block. After handling an exception in the "for" clause you do not 
continue to execute the "for" block, but leave the loop. After handling 
an exception in the "with" clause you do not continue to execute the 
"with" block and do not call `__exit__` when leave it. To me, this 
syntax is much more confusing than my initial proposition.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IVNQ73IUIH2TKUJMZTXVHX4A75RCTB37/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Skip modules by default in star-import

[Serhiy Storchaka]

26.07.19 23:53, Guido van Rossum пише:
Serhiy proposed a relatively minor change to the behavior of `import *` 
in the absence of __all__. This sounds like an idea we could try though 
we should have a look at the implications for various well-known 
packages. It is also a relatively minor problem that he's trying to 
solve, so it's not worth breaking lots of stuff over. (We can just add 
an __all__ to Tkinter that contains the desired list.)


I have tested this change on the stdlib, and it had very small effect. 
It broke `test_pkg` [1] which tested star-import of package and the 
`xml.parsers.expat` module [2] which is a thin wrapper around the 
`pyexpat` module, including its submodules `model` and `errors`.


It is not easy to determine the effect on third-party code, but seems 
that well-known packages has defined `__all__` in they main modules. It 
may affect small projects which do not care much about linters and 
`__all__`. But taking to account that the effect on the stdlib (where 
many modules do not have `__all__`) is small, there is a hope that it is 
small for other projects too.


It may be worth to mention that help() does not show imported module. It 
shows only submodules.


`__all__` has been added to Tkinter recently. [3]  Excluding imported 
modules was not the only reason, there was other problem which can be 
solved only by `__all__`.


[1] https://github.com/python/cpython/blob/master/Lib/test/test_pkg.py
[2] 
https://github.com/python/cpython/blob/master/Lib/xml/parsers/expat.py[3] 
https://bugs.python.org/issue29446

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/555D4MJ7G664LW6CGGKOUDJYZJMPDMFT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: for ... except, with ... except

[Serhiy Storchaka]

26.07.19 23:57, Guido van Rossum пише:
These are interesting ideas. It looks like you intend the except clause 
of the for loop to *only* cover the iter() and next() calls that are 
implicit in the for loop. You're right that it's awkward to catch 
exceptions there.


Right. If you want to catch an exception in the loop body you have to 
wrap the loop body with "try ... except". If you want to catch an 
exception in the loop body *and* the 'for' clause you have to wrap the 
whole "for" statement with "try ... except". No new syntax is needed for 
this. But if you want to catch an exception *only* in the 'for' clause 
you can't (without getting rid of the "for" statement at all).


However, I worry that when people see this syntax, 
they will think that the except clause is for handling exceptions in the 
loop body. (That's certainly what I assumed when I read just your 
subject line. :-)


I think people can make a mistake only when they see it for the first 
time. After you learn what this syntax means, you will not make this 
mistake the second time. The same applies to any other syntactic 
construction.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MPMI4WIQREY4D647XWBD6PTFAHQKTEJM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: for ... except, with ... except

[Serhiy Storchaka]

26.07.19 23:46, MRAB пише:

On 2019-07-26 19:26, Serhiy Storchaka wrote:
[snip]

I propose to add "except" clause to "for" and "with" statement to catch
exceptions in the code that can't be wrapped with "try ... except".

  for VAR in EXPR:
  BLOCK
  except EXC:
  HANDLER

should be equivalent to

  try:
  _it = iter(EXPR)
  except EXC:
  HANDLER
  else:
  while True:
  try:
  VAR = next(_it)
  except StopIteration:
  break
  except EXC:
  HANDLER
  break
  BLOCK


[snip]
1. The 'for' loop can have an 'else' clause, and so can the 'try' 
statement. Is there any ambiguity over its meaning?


An 'else' clause is already have different meanings in different 
statements: 'if', 'try' and 'while'/'for'. This proposition does not add 
anything new to this.


If an exception is not raised in 'for' or 'with' then its body is 
executed. So no need to add an 'else' clause in the meaning of 'try'. 
There will be no conflict between two 'else'.


2. In your example you have it catching the exception if EXPR or 
iter(EXPR) raises. Is that a good idea?


Definitely we should catch the exception if EXPR is raised in the 'with' 
statement, because different content managers can acquire resources 
either in `__enter__` or in constructor (like open()). There is less 
confidence about 'for', but I think that it is better to catch it, 
because iter() is called implicitly and there is no way to catch an 
exception in it. iter() can raise an exception:


>>> f = open('python')
>>> f.close()
>>> iter(f)
Traceback (most recent call last):
File "", line 1, in 
ValueError: I/O operation on closed file.

But all this is discussable.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NCBMXN6N7HDDXGPTJ2O374BEJ7JQ7DR2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Skip modules by default in star-import

[Anders Hovmöller]

> On 26 Jul 2019, at 22:53, Guido van Rossum  wrote:
> 
> Serhiy proposed a relatively minor change to the behavior of `import *` in 
> the absence of __all__. This sounds like an idea we could try though we 
> should have a look at the implications for various well-known packages. It is 
> also a relatively minor problem that he's trying to solve, so it's not worth 
> breaking lots of stuff over. (We can just add an __all__ to Tkinter that 
> contains the desired list.)
> 
> Anders then stole Serhiy's thread to advocate fora much more radical idea. 
> That's not good netiquette. That idea is going to break a lot more code than 
> Serhiy's. But my key message here is to Anders: stay on topic or start a new 
> thread. You're welcome to discuss your idea in a separate thread. But don't 
> steal existing threads.

I respectfully disagree. Sure I advocated for a more radical idea, but one that 
solves the original problem and also other problems. That doesn't seem like 
stealing a thread to me but offering another perspective on the problem at 
hand. Taking a step back and getting a bigger picture seems like a good thing. 
We're just throwing ideas around after all. 

/ Anders 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ICKVRBAMWUYXFI77JH66J7KX5J5ZVTBL/
Code of Conduct: http://python.org/psf/codeofconduct/