Re: [Python-ideas] Add "default" kwarg to list.pop()

[Steven D'Aprano]

On Wed, Oct 31, 2018 at 02:25:25AM +0200, Serhiy Storchaka wrote:
> 31.10.18 01:44, Giampaolo Rodola' пише:
> >Sorry in advance if this has been proposed in the past but I couldn't 
> >find anything on python-ideas:
> >
> > >>> l = []
> > >>> l.pop(default=1)
> >1
[...]

> It is just
> 
> l.pop() if l else default

It might *do* the same thing, but it doesn't communicate the 
programmer's intention as well.

{}.pop('key', default) could be written using LBYL too, but the 
intention is much clearer given an explicit default argument.

The only advantage of the "if l" version is that if the default is 
expensive to calculate, we can short-circuit it.


> or
> 
> (l or [default]).pop()

That's clever, but it is also wasteful, building a single-item list only 
to immediately pop the item out of it and throw the list away.

[steve@ando ~]$ python3.5 -m timeit -s "l = []" "l.pop() if l else None"
1000 loops, best of 3: 0.0739 usec per loop

[steve@ando ~]$ python3.5 -m timeit -s "l = []" "(l or [None]).pop()"
100 loops, best of 3: 0.421 usec per loop



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add "default" kwarg to list.pop()

[Serhiy Storchaka]

31.10.18 01:44, Giampaolo Rodola' пише:
Sorry in advance if this has been proposed in the past but I couldn't 
find anything on python-ideas:


 >>> l = []
 >>> l.pop(default=1)
1

FWIW my use case consists in reading entries from /proc/diskstats where 
lines can have a variable number of fields depending on the kernel version:

https://github.com/giampaolo/psutil/issues/1354#issuecomment-434495870
https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cacb2127/psutil/_pslinux.py#L1068
As such it would be convenient to set missing fields to 0 as "reads = 
fields.pop(default=0)" instead of catching IndexError every time. Extra: 
for consistency set.pop() should probably have the same.


Thoughts?


It is just

l.pop() if l else default

or

(l or [default]).pop()

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[julien tayon]

On Wed, 31 Oct 2018 at 00:20, David Mertz  wrote:

> Counter doesn't QUITE do the same thing as this `mdict`.  But it's pretty
> close.
>
> I think if .__add__() became a synonym for .update() that wouldn't break
> anything that currently works.  But I'm probably wrong, and missing a case
> in my quick thought:
>
> My quick thoughts too is that it achieve coincidently Counter features as
a subset of its features. I never noticed it, and both approaches seem
consistent in their results (pfiou, close one since I did not thought of
checking it)

And  if you add the trait to Counter  you have the following
results :
>>> from collections import Counter
>
> >>> from archery.quiver import LinearAlgebrae
>>> class ACounter(LinearAlgebrae, Counter): pass

> >>> c = ACounter(a=[2], b='a')
> >>> c.update(c)
> >>> c
>

 ACounter({'a': [2, 2], 'b': 'aa'})
 (same)

>>> c2 = ACounter(a=1, b=2)
> >>> c2 + c2
>
ACounter({'b': 4, 'a': 2})

> >>> c2.update(c2)
> >>> c2
>
 ACounter({'b': 4, 'a': 2})

> >>> c2 + c2
>
ACounter({'a': 4, 'b': 8})
>>> c2 + .5 * c2
 ACounter({'a': 1.5, 'b': 3.0})


On Tue, Oct 30, 2018 at 6:54 PM Alexander Belopolsky <
> alexander.belopol...@gmail.com> wrote:
>
>> > In [12]: a= mdict(a=[2], b='a')
>> > In [13]: a+a
>>
>> Aren't you reinventing the Counter type?
>>
>> nop. It is an unintended subset of the possibilities.
I do have though
>>> c2 / 2
Out[17]: ACounter({'a': 0.5, 'b': 1.0})
>>> c / 2
TypeError: can't multiply sequence by non-int of type 'float'

And talking about Counter, by inheriting from the mixins of Vector (dot,
abs, cos) we give it out of the box the cosine simlarities.

Which given its wide use in textual indexation is pretty reassuring. It
would also enable to normalize Counter (with value that supports truediv)
easily by writing

>>> class VCounter(LinearAlgebrae,Vector, Counter): pass
>>> c2 = VCounter(a=1, b=2)
>>> c2/abs(c2)
Out[20]: VCounter({'a': 0.4472135954999579, 'b': 0.894427190159})
And since it is mixins it touches nothing of the MutableMapping class it
relies on. It just gives behaviours associated with operators.
(ofc c2.cos(c) willl normally raise a TypeError since it would have no
sense)

It really is a proof of concept of adding linear/vectorial algebrae to ANY
kind of mutable mapping be it : dict, Counter, OrderedDict, defaultDict ...
It only relies on what mutableMapping (from abc) offers and does its life
with it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add "default" kwarg to list.pop()

[Giampaolo Rodola']

Sorry in advance if this has been proposed in the past but I couldn't find
anything on python-ideas:

>>> l = []
>>> l.pop(default=1)
1

FWIW my use case consists in reading entries from /proc/diskstats where
lines can have a variable number of fields depending on the kernel version:
https://github.com/giampaolo/psutil/issues/1354#issuecomment-434495870
https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cacb2127/psutil/_pslinux.py#L1068
As such it would be convenient to set missing fields to 0 as "reads =
fields.pop(default=0)" instead of catching IndexError every time. Extra:
for consistency set.pop() should probably have the same.

Thoughts?

-- 
Giampaolo - http://grodola.blogspot.com
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[David Mertz]

Actually, they are definitely different as in-place mutation versus
returning a new Counter. But in some arithmetic way they look mostly the
same.

On Tue, Oct 30, 2018, 7:19 PM David Mertz  Counter doesn't QUITE do the same thing as this `mdict`.  But it's pretty
> close.
>
> I think if .__add__() became a synonym for .update() that wouldn't break
> anything that currently works.  But I'm probably wrong, and missing a case
> in my quick thought:
>
> >>> from collections import Counter
> >>> c = Counter(a=[2], b='a')
> >>> c.update(c)
> >>> c
> Counter({'a': [2, 2], 'b': 'aa'})
> >>> c2 = Counter(a=1, b=2)
> >>> c2 + c2
> Counter({'b': 4, 'a': 2})
> >>> c2.update(c2)
> >>> c2
> Counter({'b': 4, 'a': 2})
> >>> c + c
> Traceback (most recent call last):
>   File "", line 1, in 
> c + c
>   File "/anaconda3/lib/python3.6/collections/__init__.py", line 705, in
> __add__
> if newcount > 0:
> TypeError: '>' not supported between instances of 'list' and 'int'
>
> On Tue, Oct 30, 2018 at 6:54 PM Alexander Belopolsky <
> alexander.belopol...@gmail.com> wrote:
>
>> > In [12]: a= mdict(a=[2], b='a')
>> > In [13]: a+a
>>
>> Aren't you reinventing the Counter type?
>>
>> >>> from collections import Counter
>> >>> c = Counter(a=1,b=2)
>> >>> c + c
>> Counter({'b': 4, 'a': 2})
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[David Mertz]

Counter doesn't QUITE do the same thing as this `mdict`.  But it's pretty
close.

I think if .__add__() became a synonym for .update() that wouldn't break
anything that currently works.  But I'm probably wrong, and missing a case
in my quick thought:

>>> from collections import Counter
>>> c = Counter(a=[2], b='a')
>>> c.update(c)
>>> c
Counter({'a': [2, 2], 'b': 'aa'})
>>> c2 = Counter(a=1, b=2)
>>> c2 + c2
Counter({'b': 4, 'a': 2})
>>> c2.update(c2)
>>> c2
Counter({'b': 4, 'a': 2})
>>> c + c
Traceback (most recent call last):
  File "", line 1, in 
c + c
  File "/anaconda3/lib/python3.6/collections/__init__.py", line 705, in
__add__
if newcount > 0:
TypeError: '>' not supported between instances of 'list' and 'int'

On Tue, Oct 30, 2018 at 6:54 PM Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

> > In [12]: a= mdict(a=[2], b='a')
> > In [13]: a+a
>
> Aren't you reinventing the Counter type?
>
> >>> from collections import Counter
> >>> c = Counter(a=1,b=2)
> >>> c + c
> Counter({'b': 4, 'a': 2})
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[Alexander Belopolsky]

> In [12]: a= mdict(a=[2], b='a')
> In [13]: a+a

Aren't you reinventing the Counter type?

>>> from collections import Counter
>>> c = Counter(a=1,b=2)
>>> c + c
Counter({'b': 4, 'a': 2})
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[julien tayon]

On Tue, 30 Oct 2018 at 22:33, Greg Ewing 
wrote:

> julien tayon wrote:
> > like the + of [] could be the + of "RecordAlgebrae"
>
> If you're proposing to change the behaviour of '+' on the
> built-in list type, that's not going to happen.
>
> I dont suggest to change something that already exists and works (I am
pretty conservative too, and expect stuff to not be broken by any changes)
And all behaviours can coexists quite peacefully.

"RecordAlgebra"
In [5]: [2] + [2]
Out[5]: [2, 2]
In [6]: [2] * 2
Out[6]: [2, 2]
In [7]: "a" + "a"
Out[7]: 'aa'
In [8]: "a" * 2
Out[8]: 'aa'
(adding n times the same value is equal to multiplying by n // that is
totally consistent to me)

Mixed scenario :
In [12]: a= mdict(a=[2], b='a')
In [13]: a+a
Out[14]: {'a': [2, 2], b='aa'}
In [17]: a * 4
Out[17]: {'a': [2, 2, 2, 2], b=''}
I propose the operators to be propagated, and any value to still follow its
logic.
LibearAlgebraic MutableMapping would be as algebraic as their values. No
more.


-- 
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[Greg Ewing]

julien tayon wrote:

like the + of [] could be the + of "RecordAlgebrae"


If you're proposing to change the behaviour of '+' on the
built-in list type, that's not going to happen.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[julien tayon]

Thanks robert for the praise. It feels nice.

I may be bold, but I really hate to come empty handed to a discussion. So
this lib is nothing more than doing my homework when I don't have a PhD.

Actually, science (in my opinion) is about measuring. What I propose is
nothing more than (if you add Vector traits) giving native metrics to
objects (having coded in Perl for too long I still see objects as a
hierarchy of blessed MutableMappings, I am sorry). And I think that
measurements are a corner stone of science, thus of data science. (my
opinion you may not share).

Thus it could be kind of extending some of the concepts of datasets :
https://www.python.org/dev/peps/pep-0557/ to additionnal default behaviour
(that could be subscribed optionnally).

As an everyday coder, this behaviour does solve problems I can illustrate
with code (like aggregating data, or measuring if I might have doubon in a
set of dataset, transforming objects into objects).

I do not want to force feed the community with my "brilliant" ideas, I much
more would like to plead my case on how adopting "consistent geometric
behaviours" at the language level would ease our lives as coders, if this
is not inappropriate.

Please don't look at the lib. Look at the idea of making operators behave
in a consistent way that gives the property of well known mathematic
constructions to the core of the language.

It also enables parallelisation without side effects (aka the map reduce of
the poors), which are a first order consequence of the linear algebrae.

I may not be gifted with writing long dissertations, however, I have a
pragmatic mind. So I don't mind being challenged a tad, as long as we talk
about stuffs like : how does it profit python coders to be standard, can
you show me real life example ?

However, if a "no (answer)" is a "no", I do understand. I like python the
way it is, and I don't want to introduce friction in the process of
improving python by being off topic.

Thus if no one is interested, I still have a last word : keep up the good
work! And thank you all for what you bring us.


Cheers


On Tue, 30 Oct 2018 at 19:11, Robert Vanden Eynde 
wrote:

> Julien, your article is very pleasant to read (and funny) but as other say
> the mailing list is not there to share some articles, but for proposition
> to the standard python library,
>
> do our own lib on github and pypi first if you want to Share some code to
> the world !
>
> And if project becomes super useful to everyone one day, it may come one
> day to the standard library so that everybody will have it.
>
> Cheers,
>
> Robert
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[Robert Vanden Eynde]

Julien, your article is very pleasant to read (and funny) but as other say
the mailing list is not there to share some articles, but for proposition
to the standard python library,

do our own lib on github and pypi first if you want to Share some code to
the world !

And if project becomes super useful to everyone one day, it may come one
day to the standard library so that everybody will have it.

Cheers,

Robert
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[Anders Hovmöller]

> What are your opinions ?
> I don't actually see a lot of use case except it was funny to build. But
> maybe it can be of use.

This list is for suggesting additions and changes to python. Broad usefulness 
is a prerequisite. So please build your lib but this seems off topic on this 
list. 

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[julien tayon]

Hello :)

the idea is described here:http://jul.github.io/cv/pres.html#printable

Summary of the idea :

Take a linear algebrae book, and implements all the rules as a
TDD.https://github.com/jul/archery/blob/master/consistent_algebrae.py

make it works based on abstract base class and sets of
Mixins.https://archery.readthedocs.io/en/latest/

And see if we can make cos/__abs__/dot and if it gives naively the intended
results ? (spoiler: yes)

Making it work with dict, and "other" dictionary like counter by using
ineritancehttps://archery.readthedocs.io/en/latest/#advanced-usage

My idea is : wouldn't it be nice if we introduced geometries as sets of
mixins for objects ?
(Hilbertian algebrae could be nice too, and we could make MutableMapping
behave like bra/kets).

So I was proposing a soft discussion on : could we agree that it would be
nice to consider operation overloading as a whole set of behaviours that
could profit from being consistent in a categorized way ? (like the + of []
could be the + of "RecordAlgebrae")
Meaning we could define sets of "expected behaviour consistent interaction
between operators" as we defined the abc and call them algebrae?

I offer the LinearAlgebrae Mixins as a POC, and was thinking of creating a
unittest to qualify if an object is following the rules of linear algebrae.

What are your opinions ?
I don't actually see a lot of use case except it was funny to build. But
maybe it can be of use.


Cordialement

-- 
Julien
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] gevent-like Coroutines in Python

[Mark E. Haase]

Python's coroutines are designed to make suspension points visible, which
enhances "local reasoning" about code. This concept has been written up
very well over here: https://glyph.twistedmatrix.com/2014/02/unyielding.html

On Tue, Oct 30, 2018 at 8:37 AM Ron Reiter  wrote:

> You are right that they are different, I was actually assuming that
> developers by default don't try to parallelize and would rather go ahead
> and write code to yield one function at a time, which is fine. The need to
> separate "await" from the invocation is something which is rarely used. Not
> sure what you mean about "threading" as it is still more efficient and
> lightweight to parallelize workers on an event loop rather than using
> blocking threads.
>
> As I said - I don't think we should downgrade Python's current ability to
> do so, my suggestion is to create something like the "codef" proposal,
> which will also await on every function invocation - for readability.
>
> - Ron
>
>
> [image: Facebook]  [image: Twitter]
>  [image: LinkedIn]
> 
>
>
> On Tue, Oct 30, 2018 at 12:41 PM Chris Angelico  wrote:
>
>> On Tue, Oct 30, 2018 at 6:01 PM Ron Reiter  wrote:
>> >
>> > ... most developers would always mean they prefer to do this:
>> >
>> > result = [await fun(x) for fun in funcs]
>> >
>> > versus:
>> >
>> > result = [fun(x) for fun in funcs]
>> > await asyncio.gather(*result)
>> >
>> > Moreso, having it become the default makes statements like this:
>> >
>> > result = [await fun(x) for fun in funcs if await smth]
>> >
>> > Look like this:
>> >
>> > result = [fun(x) for fun in funcs if smth]
>> >
>> > Therefore, my suggestion is to create a new "async" definition which
>> basically turns every function invocation into an "await" if a generator is
>> returned.
>> >
>>
>> I'm not sure what you're driving at here. From your first example, I
>> gather that (pun intended) you're expecting the 'result' list to
>> contain all the results from the different function calls, running
>> them all in parallel; but your second example and described suggestion
>> seem to imply that the waitings would continue to be sequential.
>>
>> Unless you're asking for straight-up magic ("do everything in parallel
>> unless they need to be serialized"), there still needs to be a clear
>> way to differentiate between "wait for this right now and give me a
>> result before this function continues" and "gather all these jobs
>> together, get me the results, and then move on once you have them
>> all". It might perhaps be nice to have an easier/more obvious syntax
>> for gather(), but it definitely needs to have some form of spelling.
>>
>> If you're not asking for them to be run in parallel, you're asking for
>> an implicit way for a function call to block its caller, and for the
>> calling function to act sequentially. Python already has that - it's
>> called threading :)
>>
>> ChrisA
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] gevent-like Coroutines in Python

[Chris Angelico]

On Tue, Oct 30, 2018 at 11:36 PM Ron Reiter  wrote:
>
> You are right that they are different, I was actually assuming that 
> developers by default don't try to parallelize and would rather go ahead and 
> write code to yield one function at a time, which is fine. The need to 
> separate "await" from the invocation is something which is rarely used. Not 
> sure what you mean about "threading" as it is still more efficient and 
> lightweight to parallelize workers on an event loop rather than using 
> blocking threads.
>

Okay, so it's actually nothing to do with asyncio.gather(). Sure.

So what you're looking for is JUST the removal of the "await"
keywords. As Greg already said, Guido considers the explicit await
markers as a feature, not a bug; these are the exact points where an
intrathread context switch can occur.

As to the efficiency of parallelizing on an event loop rather than
using threads, that's a tradeoff; threads aren't going anywhere just
because asyncio is here. When you want the extreme simplicity of "just
do this stuff, okay?", the easiest way to get it is to just use
threads, and pay a bit of overhead. You'll often find that the
overhead isn't actually all that significant until you get to extremes
of throughput - most Python apps are not trying to run tens of
thousands of concurrent TCP sockets, for instance.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] gevent-like Coroutines in Python

[Ron Reiter]

You are right that they are different, I was actually assuming that
developers by default don't try to parallelize and would rather go ahead
and write code to yield one function at a time, which is fine. The need to
separate "await" from the invocation is something which is rarely used. Not
sure what you mean about "threading" as it is still more efficient and
lightweight to parallelize workers on an event loop rather than using
blocking threads.

As I said - I don't think we should downgrade Python's current ability to
do so, my suggestion is to create something like the "codef" proposal,
which will also await on every function invocation - for readability.

- Ron


[image: Facebook]  [image: Twitter]
 [image: LinkedIn]



On Tue, Oct 30, 2018 at 12:41 PM Chris Angelico  wrote:

> On Tue, Oct 30, 2018 at 6:01 PM Ron Reiter  wrote:
> >
> > ... most developers would always mean they prefer to do this:
> >
> > result = [await fun(x) for fun in funcs]
> >
> > versus:
> >
> > result = [fun(x) for fun in funcs]
> > await asyncio.gather(*result)
> >
> > Moreso, having it become the default makes statements like this:
> >
> > result = [await fun(x) for fun in funcs if await smth]
> >
> > Look like this:
> >
> > result = [fun(x) for fun in funcs if smth]
> >
> > Therefore, my suggestion is to create a new "async" definition which
> basically turns every function invocation into an "await" if a generator is
> returned.
> >
>
> I'm not sure what you're driving at here. From your first example, I
> gather that (pun intended) you're expecting the 'result' list to
> contain all the results from the different function calls, running
> them all in parallel; but your second example and described suggestion
> seem to imply that the waitings would continue to be sequential.
>
> Unless you're asking for straight-up magic ("do everything in parallel
> unless they need to be serialized"), there still needs to be a clear
> way to differentiate between "wait for this right now and give me a
> result before this function continues" and "gather all these jobs
> together, get me the results, and then move on once you have them
> all". It might perhaps be nice to have an easier/more obvious syntax
> for gather(), but it definitely needs to have some form of spelling.
>
> If you're not asking for them to be run in parallel, you're asking for
> an implicit way for a function call to block its caller, and for the
> calling function to act sequentially. Python already has that - it's
> called threading :)
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a way to test for a descriptor in Python Code

[Joy Diamond]

Clarifications:

1.  I miswrote part of my first post where I wrote "I want to test if
symbol found in __dict__ is an attribute or not in python code".

I meant to write "is a DESCRIPTOR" or not.

2.  The example in https://docs.python.org/3/howto/descriptor.html for
reproducing `type.__getattribute__` has a second bug, in that it does not
look at the class inheritance properly.  (See fixed example below named
`type_getattro` and based on the same function in the C code).

In the example below if you call ` __getattribute__(Child, 'x')` it will
incorrectly fail with "Catch `AttributeError: 'type' object has no
attribute 'x'`"

Responding to Steve:

On Tue, Oct 30, 2018 at 6:31 AM Steven D'Aprano  wrote:

> Actually, on further investigation, I think it ought to be:
>
> if inspect.hasattr_static(type(v), '__get__')
>
> except that there is no hasattr_static, there's only a getattr_static.
> So perhaps there ought to be a hasattr_static as well.
>
>
`inspect.hasattr_static` gets me half way there (but it still looks in two
chains of inheritance, where I only want to look in one).

In particular if the metaclass of `type(v)` has a `.__get__` method it will
incorrectly find that.

So it will still misidentify if an instance is a descriptor or not.


> > The question is *NOT* whether 'v' has an attribute '__get__'; *BUT*
> whether
> > `v` has a symbol `__get__` in any of the classes in it's method
> resolution
> > order.
>
> What's the difference as you see it?
>

I want to be able to look in the method resolution order (one inheritance
chain).

`getattr` and `inspect.getattr_static` both look in two inheritance chains
(the instance & it's type; or the case of a class, the class and it's
metaclass).

I need to look in only one chain (and disable descriptors like
`inspect.getattr_static` does).

To put it succinctly: I am trying to reproduce the behavior of
`_PyType_Lookup` from "Objects/typeobject.c" (see example below).

Below is a full reproduction of `object.__getattribute__` and
`type.__getattribute__` based on reading the Python source code.

Note this reproduction of `type.__getattribute__` is much more accurate
than what is at:

https://docs.python.org/2/howto/descriptor.html
https://docs.python.org/3/howto/descriptor.html

Both of which need to be updated.

(This is not yet filed as a bug report; as first I am requesting a call to
something like `_PyType_Lookup` that is efficent; and once we agree on
that, we can created an updated reproduction of `type.__getattribute__`).

Thanks,

Joy Diamond.

#
#  The following reproduces (and tests) `object.__getattribute__` and
`type.__getattribute__` based on reading the C source code.
#
absent = object()

def _PyType_Lookup(model, name):
'''Based on `_PyType_Lookup` in "Objects/typeobject.c"'''

mro = model.__mro__

if mro is None:
return absent

for m in mro:
symbol_table = m.__dict__

if name in symbol_table:
return symbol_table[name]

return absent


def lookup__tp_descr_get(model):
tp_descr_get = _PyType_Lookup(model, '__get__')

return tp_descr_get


def has__tp_descr_set(model):
tp_descr_set = _PyType_Lookup(model, '__set__')

return tp_descr_set is not absent

#
#   Reproduction of `object.__getattribute__`
#
def PyObject_GenericGetAttr(instance, name):
'''Based on `PyObject_GenericGetAttr` in "Objects/object.c"'''

instance_type = type(instance)

descriptor = _PyType_Lookup(instance_type, name)

if descriptor is absent:
get = absent
else:
descriptor_type = type(descriptor)

get = lookup__tp_descr_get(descriptor_type)

if (get is not absent) and (has__tp_descr_set(descriptor_type)):
#
#   "Data Descriptor" (a `__set__` method exists) has
precedence.
#
return get(descriptor, instance, instance_type)

if instance_type.__dictoffset__:
instance__mapping = instance.__dict__

if name in instance__mapping:
return instance__mapping[name]

if get is not absent:
return get(descriptor, instance, instance_type)

raise AttributeError("cannot find attribute `{}` in instance of
`{}`".format(name, instance_type.__name__))

#
#   Reproduction of `type.__getattribute__`
#
def type_getattro(model, name):
'''Based on `type_getattro` in "Objects/type_object.c"'''

metatype = type(model)

descriptor = _PyType_Lookup(metatype, name)

if descriptor is absent:
get = absent
else:
descriptor_type = type(descriptor)

get = lookup__tp_descr_get(descriptor_type)

if (get is not absent) and (has__tp_descr_set(descriptor_type)):
#
#   "Data Descriptor" (a `__set__` method exists) has
precedence.
#
return get(descriptor, instance, instance_type)

symbol = _PyType_Lookup(model, name)

if symbol is not absent:
#
#   Implement descriptor functionality, 

[Python-ideas] Make fnmatch.filter accept a tuple of patterns

[Andre Delfino]

Frequently, while globbing, one needs to work with multiple extensions. I’d
like to propose for fnmatch.filter to handle a tuple of patterns (while
preserving the single str argument functionality, alas str.endswith), as a
first step for glob.i?glob to accept multiple patterns as well.

Here is the implementation I came up with:

https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-multiple-patterns?expand=1

If this is deemed reasonable, I’ll write tests and documentation updates.

Any opinion?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] gevent-like Coroutines in Python

[Chris Angelico]

On Tue, Oct 30, 2018 at 6:01 PM Ron Reiter  wrote:
>
> ... most developers would always mean they prefer to do this:
>
> result = [await fun(x) for fun in funcs]
>
> versus:
>
> result = [fun(x) for fun in funcs]
> await asyncio.gather(*result)
>
> Moreso, having it become the default makes statements like this:
>
> result = [await fun(x) for fun in funcs if await smth]
>
> Look like this:
>
> result = [fun(x) for fun in funcs if smth]
>
> Therefore, my suggestion is to create a new "async" definition which 
> basically turns every function invocation into an "await" if a generator is 
> returned.
>

I'm not sure what you're driving at here. From your first example, I
gather that (pun intended) you're expecting the 'result' list to
contain all the results from the different function calls, running
them all in parallel; but your second example and described suggestion
seem to imply that the waitings would continue to be sequential.

Unless you're asking for straight-up magic ("do everything in parallel
unless they need to be serialized"), there still needs to be a clear
way to differentiate between "wait for this right now and give me a
result before this function continues" and "gather all these jobs
together, get me the results, and then move on once you have them
all". It might perhaps be nice to have an easier/more obvious syntax
for gather(), but it definitely needs to have some form of spelling.

If you're not asking for them to be run in parallel, you're asking for
an implicit way for a function call to block its caller, and for the
calling function to act sequentially. Python already has that - it's
called threading :)

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a way to test for a descriptor in Python Code

[Steven D'Aprano]

On Tue, Oct 30, 2018 at 04:40:40AM -0400, Joy Diamond wrote:

> """
> For classes, the machinery is in type.__getattribute__() which transforms
> B.x into B.__dict__['x'].__get__(None, B). In pure Python, it looks like:
> 
> def __getattribute__(self, key):
> "Emulate type_getattro() in Objects/typeobject.c"
> v = object.__getattribute__(self, key)
> if hasattr(v, '__get__'):
> return v.__get__(None, self)
> return v
> 
> """
[...]
> However, the call to `hasattr(v, '__get__')` appears to me to be incorrect.

I agree, but only because it fails to take into account that dunder 
methods like __get__ are only looked up on the class, not the instance. 
I believe a more accurate eumulation would be:

if hasattr(type(v), '__get__'):
return type(v).__get__(None, self)

Actually, on further investigation, I think it ought to be:

if inspect.hasattr_static(type(v), '__get__')

except that there is no hasattr_static, there's only a getattr_static. 
So perhaps there ought to be a hasattr_static as well.


> The question is *NOT* whether 'v' has an attribute '__get__'; *BUT* whether
> `v` has a symbol `__get__` in any of the classes in it's method resolution
> order.

What's the difference as you see it?




-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] gevent-like Coroutines in Python

[Greg Ewing]

Ron Reiter wrote:
I feel like having the await syntax trigger by default on any awaitable 
invocation in a coroutine context makes much more sense.


Guido seems to regard the requirement to use 'await' as a
feature, not a bug. He says he likes to be able to see where
all the potential suspension points are.

Therefore, my suggestion is to create a new "async" definition which 
basically turns every function invocation into an "await" if a generator 
is returned. Instead of "async def" I propose the alternative "coroutine 
def" syntax. However, a better solution may be to imply the word "async" 
in every function definition given some sort of trigger (however I 
assume that this won't be the preferable approach as it is not something 
that can be implemented at the parsing level).


I don't think it's feasible to automatically infer both 'async'
*and* 'await', even at run time. An async function needs to be
treated differently from the moment it starts running, so there
must be something that statically identifies it as such.

Anyway, if you want to pursue these ideas further, you should
take a look at PEP 3152, which was my attempt at a nicer syntax
for generator-based coroutines, before async/await came along.
I think it would have been better in some ways, but it was
rejected.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add a way to test for a descriptor in Python Code

[Joy Diamond]

Greetings,

I am trying to emulate attribute lookup, and want to test if symbol found
in __dict__ is an attribute or not in python code (i.e.: does its have a
`tp_descr_get` slot?)

Reading the documentation carefully, I am supposed to test if the attribute
has a `.__get__` method; however, I see no way to test for this properly
(easily).

Currently the only way I know to test for this is (See code at end of this
message):

any('__get__' in  m.__dict__   for m in type(v).__mro__)

Which seems terribly inefficient.

The documentation at:

https://docs.python.org/2/howto/descriptor.html
https://docs.python.org/3/howto/descriptor.html

Both says:

"""
For classes, the machinery is in type.__getattribute__() which transforms
B.x into B.__dict__['x'].__get__(None, B). In pure Python, it looks like:

def __getattribute__(self, key):
"Emulate type_getattro() in Objects/typeobject.c"
v = object.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v

"""

However, the call to `hasattr(v, '__get__')` appears to me to be incorrect.

The question is *NOT* whether 'v' has an attribute '__get__'; *BUT* whether
`v` has a symbol `__get__` in any of the classes in it's method resolution
order.

Looking at `type_getattro` in "Objects/typeobject.c" here:

https://github.com/python/cpython/blob/master/Objects/typeobject.c#L3177

Reads:

meta_get = Py_TYPE(meta_attribute)->tp_descr_get

So I really want to know if the `tp_descr_get` slot is set or not.

(Which is a different question than whether `v` has a `__get__` attribute).

The code below shows that:

1.  The valid value of `Point.y` is 
2.  The valid value is returned by `Point.y`, `type.__getattribute__(Point,
y)`, and `fixed__Type__getattribute`
3.  The invalid value of `Point.y` is `2` as returned by the [emulated]
`__getattribute__` documented
https://docs.python.org/3/howto/descriptor.html

So I am requesting:

1.  An efficient python way to test for `tp_descr_get` slot (and other
descriptor) slots.
2.  Fix the documentation referenced above.

Thanks,

Joy Diamond.

NOTE #1:  This email describes the very subtle difference between whether
an instance has a `__get__` symbol or not, which I believe is *NOT* the
same question as whether `hasattr(instance, "__get__")` returns true or
not.  The first question is does it have the symbol `__get_` [Which python
put in the `tp_descr_slot`] while `hasattr` answers the question does it
have the `__get__` attribute.

NOTE #2: Also using `hasattr(type(v), "__get__")` would not answer the
question I want, because then it might find a `__get__` method in the
meta-class of `type(v)` which again would return an incorrect answer.

Example program that shows that using `hasattr(v, '__get__')` is not a
valid way to test if something is a descriptor (works in python 2, python
3, and pypy):

def get_1(self, a, b):
return 1

def get_2(a, b):
return 2

class Descriptor(object):
__get__ = get_1

class Not_A_Descriptor(object):
def __init__(self):
self.__get__ = get_2

def __repr__(self):
return ''

class Point(object):
__slots__ = (())

x = Descriptor()
y = Not_A_Descriptor()


#
#   Copied from https://docs.python.org/3/howto/descriptor.html
#
def __getattribute__(self, key):
"Emulate type_getattro() in Objects/typeobject.c"
v = object.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v


#
#   My fixed version
#
def fixed__Type__getattribute(self, key):
"FIXED: Emulate type_getattro() in Objects/typeobject.c"
v = object.__getattribute__(self, key)

if any('__get__' in  m.__dict__   for m in type(v).__mro__):
return v.__get__(None, self)

return v

print('Point.x: %s' % Point.x)
print('Point.y: %s' % Point.y)

print("type.__getattribute__(Point, 'x'): %s" %
type.__getattribute__(Point, 'x'))
print("type.__getattribute__(Point, 'y'): %s" %
type.__getattribute__(Point, 'y'))

print("__getattribute__(Point, 'x'): %s" % __getattribute__(Point, 'x'))
print("__getattribute__(Point, 'y'): %s  ***WRONG***" %
__getattribute__(Point, 'y'))

print("fixed__Type__getattribute(Point, 'x'): %s" %
fixed__Type__getattribute(Point, 'x'))
print("fixed__Type__getattribute(Point, 'y'): %s  ***CORRECT***" %
fixed__Type__getattribute(Point, 'y'))
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] gevent-like Coroutines in Python

[Ron Reiter]

One of Golang's advantages is that goroutines act like gevent's coroutines
instead of relying on an async/await syntax. In my opinion, it makes Golang
code much more readable.

I feel like having the await syntax trigger by default on any awaitable
invocation in a coroutine context makes much more sense. I consider
async/await syntax to be too complicated for the average developer since it
opens up too many abilities whereas most developers would always mean they
prefer to do this:

result = [await fun(x) for fun in funcs]

versus:

result = [fun(x) for fun in funcs]
await asyncio.gather(*result)

Moreso, having it become the default makes statements like this:

result = [await fun(x) for fun in funcs if await smth]

Look like this:

result = [fun(x) for fun in funcs if smth]

Therefore, my suggestion is to create a new "async" definition which
basically turns every function invocation into an "await" if a generator is
returned. Instead of "async def" I propose the alternative "coroutine def"
syntax. However, a better solution may be to imply the word "async" in
every function definition given some sort of trigger (however I assume that
this won't be the preferable approach as it is not something that can be
implemented at the parsing level).

For me, I believe that Python should aspire to be both concise and
straightforward, which means no boilerplate code just because it's more
"correct" but rather assume there's some logical default behavior going on
in the back - and to me this logical behavior is that every invocation can
trigger an await and go back to the main loop. Our assumption that a
function invocation has to tie back to instant execution unless an "await"
statement has been placed should be challenged in favor of readability,
conciseness, and to always aim to appeal to novice developers (which I
believe is the reason Python is on such a rise these days).

I do have to admit I have not thought through what is the best syntax or
the implications of this because I would like to see what is the general
opinion on this idea first.

- Ron
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/