[Python-ideas] wrapping up 4 years of open source python > 12 pep ideas

[Sylvain MARIE via Python-ideas]

Dear python enthusiasts,

--(you can skip this part if you're in a hurry)--
4 years ago I was discovering this great language. Already an experienced 
software developer and data scientist at that time, accustomed with both 
low-level (C++), Object-Oriented (Java), Query (T-SQL), and Script (MATLAB) 
languages, I still have to admit that I was blown away by the capabilities and 
expressiveness of python ! 

Our team leveraged python during this period to develop our second cloud-based 
Analytics as a Service platform. As I found out, even if the quantity of 
available open source python libraries was already huge, many elements were 
missing in order to cover all of our needs in an elegant, "pythonic" way. So I 
first developed some common libraries for internal use, and eventually made 
some of them open source to make them more mature and to get community 
feedback. At first it was difficult to create elegant libs, but after a dozen I 
think that I began to "get it right".

Now that we are reaching the 4th year anniversary (and I am about to take some 
rest for a second paternity leave), I thought that it would be a great time to 
do a summary of the main issues that I came across with python, and share this 
with you. Most of these issues were so blocking/frustrating that I had to 
create a library to fix them, when I could. Still, I believe that some deserve 
a better fix, directly in the core python stdlib or, for one or two of them 
maybe, in the language itself. 
---

The result is a list of 12 PEP topics/ideas that can be found and discussed 
here: https://github.com/smarie/python-pep-ideas/issues . For reference, the 
list of open source libraries that were at the origin of the discovery or the 
fix of these issues is here: 
https://github.com/smarie/ALL_OF_THE_ABOVE#overview . I tried to use git labels 
to relate ideas with their related lib.

I hope that this will serve the python community and maybe make python even 
greater in the future!
Kind regards - stay safe

--
Sylvain
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T3DSXIKWX64HFMEVAYVBQ7NKOPYWW2QF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Getting the version number for a package or module, reliably

[Sylvain MARIE via Python-ideas]

Thanks for the feedback Xavier

Indeed, this “last resort” strategy only works if your project source is a git 
repository. I rely on the excellent setuptools_scm to handle that special case. 
By the way if someone knows an equivalent for other popular scm systems, please 
let me know (or better suggest it here: 
https://github.com/smarie/python-getversion/issues )

I forgot to add something in my previous email: the original reason for 
developing this library was to provide a version-aware persistency layer based 
on json. Basically when you deserialize an object, your code would receive the 
associated version, which allows for legacy-aware deserialization (typically 
storing a machine learning model for 6 months and needing to upgrade your code 
while still being able to deserialize it).

I guess that this use case could also apply to pickle – from what I remember, 
pickle does not like it too much when the object to deserialize does not 
correspond to the same versions of the classes used.

Best

Sylvain

De : Xavier Combelle 
Envoyé : samedi 6 juillet 2019 11:34
À : Sylvain MARIE 
Objet : Re: [Python-ideas] Getting the version number for a package or module, 
reliably


[External email: Use caution with links and attachments]




I'm certainly wrong, but version of a development version of a typical library 
is probably not reliable, as typically the version number is bumped when a new 
version is shipped so the code can be very different of the version given.

Le sam. 6 juil. 2019 11:13, Sylvain MARIE via Python-ideas 
mailto:python-ideas@python.org>> a écrit :
Dear python enthusiasts,

For some industrial project a few years ago we needed a reliable way to get the 
version number for a package or module.
I found out that there was actually no method working in all edge cases, 
including:


  *   Built-in modules
  *   Unzipped wheels and eggs added to the python patch
  *   Non-installed project under development with version control information 
available
  *   Packages both installed and added to the python path (typically a 
developer working on a new version)

So I created one, and finally found the time to publish it.
No rocket science here, but you may find this new package useful: 
https://smarie.github.io/python-getversion/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-getversion%2F=02%7C01%7Csylvain.marie%40se.com%7C20dc338bee554edb628108d701f519e8%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C1%7C636980024523465904=tw6DivTCfYf5xFmEOUCkVjSdAK9ZB%2BVjjpfHwij%2Ba8s%3D=0>

It works with any imported module for now, including submodules.
Along with the version you get details about why a given version number is 
returned (is it because of the __version__ attribute that was found, or because 
of the Version metadata in the installed package, etc.)
Also, if one edge case is missing, it is fairly easy to add it.

If I missed something in the stdlib (I acme across the importlib.metadata 
initiative but as of now it does not seem to cover all of the above cases), 
please let me know so that I can cite it in the documentation and even redirect 
to it if it happens to already cover all the cases.

Happy summer to all !
--
Sylvain
___
Python-ideas mailing list -- 
python-ideas@python.org<mailto:python-ideas@python.org>
To unsubscribe send an email to 
python-ideas-le...@python.org<mailto:python-ideas-le...@python.org>
https://mail.python.org/mailman3/lists/python-ideas.python.org/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman3%2Flists%2Fpython-ideas.python.org%2F=02%7C01%7Csylvain.marie%40se.com%7C20dc338bee554edb628108d701f519e8%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C1%7C636980024523465904=Qwl176QO8d1S8%2FumncRb7YA9Nvo0G5eASUvYyYxvZhs%3D=0>
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OEXCPM45R2YBE5GBVRXK54CZGZFSBNDI/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Farchives%2Flist%2Fpython-ideas%40python.org%2Fmessage%2FOEXCPM45R2YBE5GBVRXK54CZGZFSBNDI%2F=02%7C01%7Csylvain.marie%40se.com%7C20dc338bee554edb628108d701f519e8%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C1%7C636980024523475900=YyOpqRKVEIlHkxU73p%2B6eTRm0mOW%2Fssll%2B963z8GowU%3D=0>
Code of Conduct: 
http://python.org/psf/codeofconduct/<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F=02%7C01%7Csylvain.marie%40se.com%7C20dc338bee554edb628108d701f519e8%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C1%7C636980024523475900=FAeFYR6fjm%2FWGWnowP7boQC7kSgjXJVqRGoaEhdYk40%3D=0>

__
This email has been scanned by the Symantec Email Security.cloud service.
__
__

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

All of your answers are true,

> (Chris)
> "my_decorator(x=foo)" is not going to look like "@my_decorator \n def foo".

That’s one of the many ways that `decopatch` uses to perform the 
disambiguation, indeed. But that’s already the user helping the lib, not the 
other way round

> (Christopher)
> @something applied on def fun is exactly the same as something(fun)

True. However applying decorator manually is already for advanced users, so 
users of a decopatche-d decorator will not mind calling something()(fun). In 
fact it is consistent with when they use it with arguments : 
something(arg)(fun).

Another criterion is : how easy would it be to implement an 
inspect.is_decorator_call(frame) method returning True if and only if frame is 
the @ statement ? If that’s fairly easy, well, I’m pretty sure that this is 
good stuff. From a naïve user, not accustomed with the long history of this 
language, is very strange that an operator such as @ (the decorator one, not 
the other one) is completely not detectable by code, while there are so many 
hooks available in python for all other operators (+, -, etc.).

Eventually that’s obviously your call, I’m just there to give feedback from 
what I see of the python libs development community.

--
Sylvain

De : Python-ideas  De la 
part de Christopher Barker
Envoyé : mercredi 20 mars 2019 06:50
À : Greg Ewing 
Cc : python-ideas 
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators


[External email: Use caution with links and attachments]




Also:

@something
def fun():
...

Is exactly the same as:

def fun()
...

fun = something(fun)

So you can’t make a distinction based whether a given usage  is as a decoration.

-CHB

On Tue, Mar 19, 2019 at 12:26 PM Greg Ewing 
mailto:greg.ew...@canterbury.ac.nz>> wrote:
Sylvain MARIE via Python-ideas wrote:
> `my_decorator(foo)` when foo is a callable will always look like
> `@my_decorator` applied to function foo, because that's how the language is
> designed.

I don't think it's worth doing anything to change that. Everywhere
else in the language, there's a very clear distinction between
'foo' and 'foo()', and you confuse them at your peril. I don't see
why decorators should be any different.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org<mailto:Python-ideas@python.org>
https://mail.python.org/mailman/listinfo/python-ideas<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas=02%7C01%7Csylvain.marie%40se.com%7C2e9308c9904c40f3702408d6acf7fc3a%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636886578423739625=%2BSHOm4PYwt6p%2F6Of69z0o19sH%2FimE4YsUFFtaEMlYCc%3D=0>
Code of Conduct: 
http://python.org/psf/codeofconduct/<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F=02%7C01%7Csylvain.marie%40se.com%7C2e9308c9904c40f3702408d6acf7fc3a%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636886578423739625=uO%2Bq8AahHYhCVObE%2Fq3Tn%2BODC0vsdjWfX2niuuNgemE%3D=0>
--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

Stephen

> If the answer is "maybe", IMO PyPI is the right solution for distribution.

Very wise words, I understand this point.
However as of today it is *not* possible to write such a library in a complete 
way, without an additional tool from the language itself. Trust me, I tried 
very hard :). Indeed `my_decorator(foo)` when foo is a callable will always 
look like `@my_decorator` applied to function foo, because that's how the 
language is designed. So there is always one remaining ambiguous case to cover, 
and I have to rely on some ugly tricks such as asking users for a custom 
disambiguator.

So if the decision is to let community-provided libraries like `decopatch` 
solve this problem, would you please consider providing us with the missing 
information? For example a new method in the `inspect` package could be 
`inspect.is_decorator_call(frame)`, that would return True if and only if the 
given frame is a decorator usage call as in `@my_decorator`. That piece would 
be enough - I would gladly take care of the rest in `decopatch`.

Thanks for the feedback !

Sylvain

-Message d'origine-
De : Stephen J. Turnbull  
Envoyé : vendredi 15 mars 2019 04:56
À : Sylvain MARIE 
Cc : David Mertz ; python-ideas 
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

[External email: Use caution with links and attachments]

________



Sylvain MARIE via Python-ideas writes:

 > I totally understand your point of view. However on the other hand,  > many 
 > very popular open source projects out there have the opposite  > point of 
 > view and provide decorators that can seamlessly be used  > with and without 
 > arguments (pytest, attrs, click, etc.). So after a  > while users get used 
 > to this behavior and expect it from all  > libraries. Making it easy to 
 > implement is therefore something quite  > important for developers not to 
 > spend time on this useless  > “feature”.

That doesn't follow.  You can also take it that "educating users to know the 
difference between a decorator and a decorator factory is therefore something 
quite important for developers not to spend time on this useless 'feature'."

I'm not a fan of either position.  I don't see why developers of libraries who 
want to provide this to their users shouldn't have "an easy way to do it", but 
I also don't see a good reason to encourage syntactic ambiguity by providing it 
in the standard library.  I think this is a feature that belongs in the area of 
"you *could* do it, but
*should* you?"  If the answer is "maybe", IMO PyPI is the right solution for 
distribution.

Steve

--
Associate Professor  Division of Policy and Planning Science
https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fturnbull.sk.tsukuba.ac.jp%2Fdata=02%7C01%7Csylvain.marie%40se.com%7C1c5b73f96ee240ef288608d6a8fa2030%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636882189569411152sdata=EcCuG%2Bad0oJoT1Fupd7086wkxHUPapCEKN2x3zDvCw0%3Dreserved=0
 Faculty of Systems and Information
Email: turnb...@sk.tsukuba.ac.jp   University of Tsukuba
Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

Thanks David

Sorry for not getting back to you earlier, I made a bunch of releases of 
makefun in the meantime. In particular I fixed a few bugs and added an 
equivalent of  `functools.partial` .

> One of the nice things in wrapt is that Dumpleton lets you use the same 
> decorator for functions, regular methods, static methods, and class methods.  
> Does yours handle that sort of "polymorphism"?

It should, but I will check it thoroughly – I’ll let you know

> I don't think I will want the specific with-or-without parens feature, since 
> it feels too implicit.  Typing `@deco_factory()` really isn't too much work 
> for me to use the two characters extra.

I totally understand your point of view. However on the other hand, many very 
popular open source projects out there have the opposite point of view and 
provide decorators that can seamlessly be used with and without arguments 
(pytest, attrs, click, etc.). So after a while users get used to this behavior 
and expect it from all libraries. Making it easy to implement is therefore 
something quite important for developers not to spend time on this useless 
“feature”.

Kind regards

Sylvain


De : David Mertz 
Envoyé : mardi 12 mars 2019 19:15
À : Sylvain MARIE 
Cc : Steven D'Aprano ; python-ideas 

Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators


[External email: Use caution with links and attachments]




One of the nice things in wrapt is that Dumpleton lets you use the same 
decorator for functions, regular methods, static methods, and class methods.  
Does yours handle that sort of "polymorphism"?

FWIW, thanks for the cool work with your libraries!

I don't think I will want the specific with-or-without parens feature, since it 
feels too implicit.  Typing `@deco_factory()` really isn't too much work for me 
to use the two characters extra.  But given that I feel the option is an 
antipattern, I don't want to add core language features to make the pattern 
easier.  Both you and Graham Dumpleton have found workarounds to get that 
behavior when it is wanted, but I don't want it to be "too easy."

FWIW... I think I'd be tempted to use a metaclass approach so that both the 
class and instance are callable.  The class would be called with a single 
function argument (i.e. a decorator), but if called with any other signature it 
would manufacture a callable instance that was parameterized by the 
initialization arguments (i.e. a decorator factory).  Actually, I haven't 
looked at your actual code, maybe that's what you do.

Best, David...

On Tue, Mar 12, 2019 at 12:44 PM Sylvain MARIE 
mailto:sylvain.ma...@se.com>> wrote:
Thanks David,

> I did write the book _Functional Programming in Python_, so I'm not entirely 
> unfamiliar with function wrappers.

Nice ! I did not realize ; good job here, congrats! ;)

--
I carefully read the documentation you pointed at, 
https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwrapt.readthedocs.io%2Fen%2Flatest%2Fdecorators.html%23decorators-with-optional-arguments=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229744945=hpuzWdGlzmfJypjI7tIMubrUYPaLN8MoS9BBlMIHGAY%3D=0>
This is the example shown by the author:

def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2):
if wrapped is None:
return functools.partial(with_optional_arguments,
myarg1=myarg1, myarg2=myarg2)

@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)

return wrapper(wrapped)

As you can see:

  *   the developer has to explicitly handle the no-parenthesis case (the first 
two lines of code).
  *   And in the next lines of the doc you see his recommendations “For this to 
be used in this way, it is a requirement that the decorator arguments be 
supplied as keyword arguments. If using Python 3, the requirement to use 
keyword only arguments can again be enforced using the keyword only argument 
syntax.”
  *   Finally, but this is just a comment: this is not “flat” mode but nested 
mode (the code returns a decorator that returns a function wrapper)

So if I’m not misleading, the problem is not really solved. Or at least, not 
the way I would like the problem to be solved : it is solved here (a) only if 
the developer takes extra care and (b) reduces the way the decorator can be 
used (no positional args). This is precisely because I was frustrated by all 
these limitations that depend on the desired signature that I wrote decopatch. 
As a developer I do not want to care about which trick to use in which 
situation (mandatory args, optional args, var-positional args..). My decorators 
may change signature during the development cycle, and if

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

Thanks David,

> I did write the book _Functional Programming in Python_, so I'm not entirely 
> unfamiliar with function wrappers.

Nice ! I did not realize ; good job here, congrats! ;)

--
I carefully read the documentation you pointed at, 
https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments
This is the example shown by the author:

def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2):
if wrapped is None:
return functools.partial(with_optional_arguments,
myarg1=myarg1, myarg2=myarg2)

@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)

return wrapper(wrapped)

As you can see:

  *   the developer has to explicitly handle the no-parenthesis case (the first 
two lines of code).
  *   And in the next lines of the doc you see his recommendations “For this to 
be used in this way, it is a requirement that the decorator arguments be 
supplied as keyword arguments. If using Python 3, the requirement to use 
keyword only arguments can again be enforced using the keyword only argument 
syntax.”
  *   Finally, but this is just a comment: this is not “flat” mode but nested 
mode (the code returns a decorator that returns a function wrapper)

So if I’m not misleading, the problem is not really solved. Or at least, not 
the way I would like the problem to be solved : it is solved here (a) only if 
the developer takes extra care and (b) reduces the way the decorator can be 
used (no positional args). This is precisely because I was frustrated by all 
these limitations that depend on the desired signature that I wrote decopatch. 
As a developer I do not want to care about which trick to use in which 
situation (mandatory args, optional args, var-positional args..). My decorators 
may change signature during the development cycle, and if I frequently had to 
change trick during development as I changed the signature - that is a bit 
tiring.

--
Concerning creation of signature-preserving wrappers: @wrapt.decorator is not 
signature preserving, I just checked it. You can check it with the following 
experiment:


def dummy(wrapped):
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
print("wrapper called")
return wrapped(*args, **kwargs)
return wrapper(wrapped)


@dummy
def function(a, b):
pass

If you call


function(1)

you will see that “wrapper called” is displayed before the TypeError is raised…

The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, 
is the source of inspiration for makefun. You can see `makefun` as a 
generalization of the core of `decorator`.

--
> I'm not sure I ever want them (decopatch and makefun) independently in 
> practice

I totally understand.
But some projects actually need makefun and not decopatch because their need is 
different: they just want to create a function dynamically. This is low-level 
tooling, really.
So at least now there is a clear separation of concerns (and dedicated issues 
management/roadmap, which is also quite convenient. Not to mention readability 
!).
To cover your concern: decopatch depends on makefun, so both come at the same 
time when you install decopatch, and decopatch by default relies on makefun 
when you use it in “double-flat” mode to create wrappers as explained here 
https://smarie.github.io/python-decopatch/#even-simpler
--

Thanks again for this discussion! It is challenging but it is necessary, to 
make sure I did not answer a non-existent need ;)
Kind regards

--
Sylvain

De : David Mertz 
Envoyé : mardi 12 mars 2019 15:30
À : Sylvain MARIE 
Cc : Steven D'Aprano ; python-ideas 

Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators


[External email: Use caution with links and attachments]




The documentation for wrapt mentions:

Decorators With Optional Arguments

Although opinion can be mixed about whether the pattern is a good one, if the 
decorator arguments all have default values, it is also possible to implement 
decorators which have optional arguments.
As Graham hints in his docs, I think repurposing decorator factories as 
decorators is an antipattern. Explicit is better than implicit.

While I *do* understands that what decotools and makefun do are technically 
independent, I'm not sure I ever want them independently in practice. I did 
write the book _Functional Programming in Python_, so I'm not entirely 
unfamiliar with function wrappers.
On Tue, Mar 12, 2019, 10:18 AM David Mertz 
mailto:me...@gnosis.cx>> wrote:
The wrapt module I linked to (not funtools.wraps) provides all the capabilities 
you mention since 2013. It allows mixed use of decorators as decorator 
factories. It has a flat style.

There are some minor API difference between your libraries and wrapt, but the 
concept is very similar. Since yours is something new, I imagine you 

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

David
I realize that you were pointing at the 'wrapt'  library not the 
functools.wrapt library.
>From what I have tried with that library, it does not solve the issue solved 
>with decopatch.

Sylvain 

-Message d'origine-
De : Sylvain MARIE 
Envoyé : mardi 12 mars 2019 14:53
À : Steven D'Aprano ; python-ideas@python.org; David Mertz 

Objet : RE: [Python-ideas] Problems (and solutions?) in writing decorators

David, Steven,

Thanks for your interest !

As you probably know, decorators and function wrappers are *completely 
different concepts*. A decorator can directly return the decorated function (or 
class), it does not have to return a wrapper. Even more, it can entirely 
replace the decorated item with something else (not even a function or class!). 
Try it: it is possible to write a decorator to replace a function with an 
integer, even though it is probably not quite useful :)

`decopatch` helps you write decorators, whatever they are. It "just" solves the 
annoying issue of having to handle the no-parenthesis and with-parenthesis 
calls. In addition as a 'goodie', it proposes two development styles: *nested* 
(you have to return a function) and *flat* (you directly write what will happen 
when the decorator is applied to something).
--
Now about creating signature-preserving function wrappers (in a decorator, or 
outside a decorator - again, that's not related). That use case is supposed to 
be covered by functools.wrapt. Unfortunately as explained here 
https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697
 this is not the case because with functools.wrapt:
 - the wrapper code will execute even when the provided arguments are invalid.
 - the wrapper code cannot easily access an argument using its name, from the 
received *args, **kwargs. Indeed one would have to handle all cases 
(positional, keyword, default) and therefore to use something like 
Signature.bind().

For this reason I proposed a replacement in `makefun`: 
https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
--
Now bridging the gap. Of course a very interesting use cases for decorators is 
to create decorators that create a signature-preserving wrapper. It is possible 
to combine decopatch and makefun for this: 
https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
Decopatch even proposes a "double-flat" development style where you directly 
write the wrapper body, as explained in the doc.

Did I answer your questions ?
Thanks again for the quick feedback !
Best,

Sylvain 

-Message d'origine-
De : Python-ideas  De la 
part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : 
python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in 
writing decorators

[External email: Use caution with links and attachments]





On Tue, Mar 12, 2019 at 09:36:41AM +, Sylvain MARIE via Python-ideas wrote:

> I therefore proposed
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
> rie.github.io%2Fpython-makefun%2Fdata=02%7C01%7Csylvain.marie%40s
> e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
> 68fef%7C0%7C0%7C636879872385158085sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
> Fekk35bnYGvmEFJyCXaLDyLm9I%3Dreserved=0 . In particular it 
> provides an equivalent of `@functools.wraps` that is truly 
> signature-preserving

Tell us more about that please. I'm very interested in getting decorators 
preserve the original signature.


--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideasdata=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3Dreserved=0
Code of Conduct: 
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2Fdata=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3Dreserved=0

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

David, Steven,

Thanks for your interest !

As you probably know, decorators and function wrappers are *completely 
different concepts*. A decorator can directly return the decorated function (or 
class), it does not have to return a wrapper. Even more, it can entirely 
replace the decorated item with something else (not even a function or class!). 
Try it: it is possible to write a decorator to replace a function with an 
integer, even though it is probably not quite useful :)

`decopatch` helps you write decorators, whatever they are. It "just" solves the 
annoying issue of having to handle the no-parenthesis and with-parenthesis 
calls. In addition as a 'goodie', it proposes two development styles: *nested* 
(you have to return a function) and *flat* (you directly write what will happen 
when the decorator is applied to something).
--
Now about creating signature-preserving function wrappers (in a decorator, or 
outside a decorator - again, that's not related). That use case is supposed to 
be covered by functools.wrapt. Unfortunately as explained here 
https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697
 this is not the case because with functools.wrapt:
 - the wrapper code will execute even when the provided arguments are invalid.
 - the wrapper code cannot easily access an argument using its name, from the 
received *args, **kwargs. Indeed one would have to handle all cases 
(positional, keyword, default) and therefore to use something like 
Signature.bind().

For this reason I proposed a replacement in `makefun`: 
https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers 
--
Now bridging the gap. Of course a very interesting use cases for decorators is 
to create decorators that create a signature-preserving wrapper. It is possible 
to combine decopatch and makefun for this: 
https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
Decopatch even proposes a "double-flat" development style where you directly 
write the wrapper body, as explained in the doc.

Did I answer your questions ?
Thanks again for the quick feedback !
Best,

Sylvain 

-Message d'origine-
De : Python-ideas  De la 
part de Steven D'Aprano
Envoyé : mardi 12 mars 2019 12:30
À : python-ideas@python.org
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

[External email: Use caution with links and attachments]





On Tue, Mar 12, 2019 at 09:36:41AM +, Sylvain MARIE via Python-ideas wrote:

> I therefore proposed
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
> rie.github.io%2Fpython-makefun%2Fdata=02%7C01%7Csylvain.marie%40s
> e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
> 68fef%7C0%7C0%7C636879872385158085sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
> Fekk35bnYGvmEFJyCXaLDyLm9I%3Dreserved=0 . In particular it 
> provides an equivalent of `@functools.wraps` that is truly 
> signature-preserving

Tell us more about that please. I'm very interested in getting decorators 
preserve the original signature.


--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideasdata=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3Dreserved=0
Code of Conduct: 
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2Fdata=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3Dreserved=0

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

Dear python enthusiasts,

Writing python decorators is indeed quite a tideous process, in particular when 
you wish to add arguments, and in particular in two cases : all optional 
arguments, and one mandatory argument. Indeed in these two cases there is a 
need to disambiguate between no-parenthesis and with-parenthesis usage.

After having struggled with this pattern for two years in various open source 
and industrial projects, I ended up writing a library to hopefully solve this 
once and for all: https://smarie.github.io/python-decopatch/ . It is 
extensively tested (203 tests) against many combinations of signature/calls.
I would gladly appreciate any feedback !

Please note that there is a "PEP proposal draft" in the project page because I 
belive that the best a library can do will always be a poor workaround, where 
the interpreter or stdlib could really fix it properly. 
Sorry for not providing too much implementation details in that page, my 
knowledge of the python interpreter is unfortunately quite limited.

--

Finally there is an additional topic around decorators : people tend to believe 
that decorators and function wrappers are the same, which is absolutely not the 
case. I used the famous `decorator` lib in many projects but I was not 
satisfied because it was solving both issues at the same time, maintaining the 
confusion. I therefore proposed https://smarie.github.io/python-makefun/ . In 
particular it provides an equivalent of `@functools.wraps` that is truly 
signature-preserving (based on the same recipe than `decorator`).
Once again, any feedback would be gladly appreciated ! 

Kind regards

Sylvain

-Message d'origine-
De : Python-ideas  De la 
part de Greg Ewing
Envoyé : vendredi 26 octobre 2018 00:04
À : python-ideas 
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

[External email: Use caution with links and attachments]





Jonathan Fine wrote:
> I also find writing decorators a bit
> hard. It seems to be something I have to learn anew each time I do it.
> Particularly for the pattern
>
> @deco(arg1, arg2) def fn(arg3, arg4):
 > # function body
>
> Perhaps doing something with partial might help here. Anyone here 
> interested in exploring this?
>

I can't think of a way that partial would help. But would you find it easier if 
you could do something like this?

 class deco(Decorator):

 def __init__(self, arg1, arg2):
 self.arg1 = arg1
 self.arg2 = arg2

 def invoke(self, func, arg3, arg4):
 # function body

Implementation:

 class Decorator:

 def __call__(self, func):
 self._wrapped_function = func
 return self._wrapper

 def _wrapper(self, *args, **kwds):
 return self.invoke(self._wrapped_function, *args, **kwds)

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideasdata=02%7C01%7Csylvain.marie%40se.com%7C295cecd58fc3461f42c108d63ac5e03e%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C63676101605889sdata=Z8OET1CZZWnmN5czi0rZ1X57%2FDd4a9IDbbSujsDNWzk%3Dreserved=0
Code of Conduct: 
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2Fdata=02%7C01%7Csylvain.marie%40se.com%7C295cecd58fc3461f42c108d63ac5e03e%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C63676101615894sdata=u0hvM5cR%2BR1Vni%2BV48WoNF%2FpriCaOG5%2BFAXayaTGsYY%3Dreserved=0

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

https://bugs.python.org/issue32886 created.
Don't hesitate to correct if anything is wrong in the text or associated tags

Sylvain 

-Message d'origine-
De : Stephen J. Turnbull [mailto:turnbull.stephen...@u.tsukuba.ac.jp] 
Envoyé : mardi 20 février 2018 06:53
À : gu...@python.org
Cc : Sylvain MARIE <sylvain.ma...@schneider-electric.com>; Python-Ideas 
<python-ideas@python.org>
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module

Guido van Rossum writes:

 > Hm, perhaps Integral is an adjective, just like Boolean?

I would guess so.  This is the same idiom we use when we call [1, 2, 3] a 
"truth-y".


__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

> A thought just occurred to me. Maybe we should just add a Boolean class to 
> numbers?

That would be great indeed

> It's a subclass of Integral, presumably. And normally only builtins.bool is 
> registered with it. But np.bool can be added at the same point you register 
> the other np integral types.

I would rather suggest to keep that Boolean ABC class independent of Integral 
(see proposal in first post) to let it remain 'pure', i.e. represent logical 
booleans only. However nothing prevents us to register python bool as a virtual 
subclass of *both* Integral and Boolean - while np.bool would be registered as 
a virtual subclass of Boolean only. This would reflect quite well the reality - 
the fact that python bool is both a Boolean and an Integer, while numpy bool is 
only a Boolean.

By the way, is there a reason for the name "Integral" (algebraic theory) 
instead of "Integer" (computer science) ? Would it be practically feasible to 
add "Integer" as an alias to "Integral" in the numbers package ? 

Sylvain 

-Message d'origine-
De : Chris Barker - NOAA Federal [mailto:chris.bar...@noaa.gov] 
Envoyé : vendredi 16 février 2018 22:42
À : gu...@python.org
Cc : Sylvain MARIE <sylvain.ma...@schneider-electric.com>; Python-Ideas 
<python-ideas@python.org>
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module

Sent from my iPhone
> A thought just occurred to me. Maybe we should just add a Boolean class to 
> numbers?

This makes lots of sense to me.

Bool is a subclass of int — might as well embrace that fact.

-CHB

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

I see :)

This does not seem to happen with PyCharm IDE + Anaconda distribution. Is 
PyCharm relying on MyPy under the hood ?
I actually have no knowledge at all about MyPy and how it relates to PyCharm 
static code analysis warnings. I’m pretty sure though that the runtime checkers 
(enforce, pytypes) are not dependent on MyPy.

Sylvain

De : gvanros...@gmail.com [mailto:gvanros...@gmail.com] De la part de Guido van 
Rossum
Envoyé : mercredi 14 février 2018 19:47
À : Sylvain MARIE <sylvain.ma...@schneider-electric.com>
Cc : python-ideas <python-ideas@python.org>
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module

I am mystified how you can be using the numbers package with mypy. Example:

import numbers
def f(a: numbers.Integral, b: numbers.Integral) -> numbers.Integral:
  return a + b
f(12, 12)
This gives an two errors on the last line when checked by mypy:

_.py:10: error: Argument 1 to "f" has incompatible type "int"; expected 
"Integral"
_.py:10: error: Argument 2 to "f" has incompatible type "int"; expected 
"Integral"

On Tue, Feb 13, 2018 at 1:21 AM, Sylvain MARIE 
<sylvain.ma...@schneider-electric.com<mailto:sylvain.ma...@schneider-electric.com>>
 wrote:
The main use case I had in mind was PEP484-based type hinting/checking actually:

def my_function(foo: Boolean):
pass

explicitly states that my_function accepts any Boolean value, whether it is a 
python bool or a np.bool that would come from a numpy array or pandas dataframe.
Note that type hinting is also the use case for which I make extensive use of 
the types from the ‘numbers’ package, for the same reasons.

Sylvain

De : Python-ideas 
[mailto:python-ideas-bounces+sylvain.marie<mailto:python-ideas-bounces%2Bsylvain.marie>=schneider-electric@python.org<mailto:schneider-electric@python.org>]
 De la part de David Mertz
Envoyé : mardi 13 février 2018 07:08
À : Nick Coghlan <ncogh...@gmail.com<mailto:ncogh...@gmail.com>>
Cc : python-ideas <python-ideas@python.org<mailto:python-ideas@python.org>>
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module

I'm not sure I'm convinced by Sylvain that Boolean needs to be an ABC in the 
standard library; Guido expresses skepticism.  Of course it is possible to 
define it in some other library that actually needs to use `isinstance(x, 
Boolean)` as Sylvain demonstraits in his post.  I'm not sure I'm unconvinced 
either, I can see a certain value to saying a given value is "fully 
round-trippable to bool" (as is np.bool_).

But just for anyone who doesn't know NumPy, here's a quick illustration of what 
I alluded to:

In [1]: import numpy as np
In [2]: arr = np.array([7,8,12,33])
In [3]: ndx1 = np.array([0,1,1,0], dtype=int)
In [4]: ndx2 = np.array([0,1,1,0], dtype=bool)
In [5]: arr[ndx1]
Out[5]: array([7, 8, 8, 7])
In [6]: arr[ndx2]
Out[6]: array([ 8, 12])

ndx1 and ndx2 are both nice things (and are both often programmatically 
constructed by operations in NumPy).  But indexing using ndx1 gives us an array 
of the things in the listed positions in arr.  In this case, we happen to 
choose two each of the things an index 0 and index 1 in the result.

Indexing by ndx2 gives us a filter of only those positions in arr corresponding 
to 'True's.  These are both nice things to be able to do, but if NumPy's True 
was a special kind of 1, it wouldn't work out unambiguously.  However, recent 
versions of NumPy have gotten a bit smarter about recognizing the special type 
of Python bools, so it's less of a trap than it used to be.  Still, contrast 
these (using actual Python lists for the indexes:

In [10]: arr[[False, True, True, False]]
Out[10]: array([ 8, 12])
In [11]: arr[[False, True, 1, 0]]
Out[11]: array([7, 8, 8, 7])



On Mon, Feb 12, 2018 at 7:50 PM, Nick Coghlan 
<ncogh...@gmail.com<mailto:ncogh...@gmail.com>> wrote:
On 13 February 2018 at 02:14, David Mertz 
<me...@gnosis.cx<mailto:me...@gnosis.cx>> wrote:
> NumPy np.bool_ is specifically not a subclass of any np.int_.  If it we're,
> there would be an ambiguity between indexing with a Boolean array and an
> array of ints. Both are meaningful, but they mean different things (mask vs
> collection of indices).
>
> Do we have other examples a Python ABC that exists to accommodate something
> outside the standard library or builtins? Even if not, NumPy is special...
> the actual syntax for '@' exists primarily for that library!

collections.abc.Sequence and collections.abc.Mapping come to mind -
the standard library doesn't tend to distinguish between different
kinds of subscriptable objects, but it's a distinction some third
party libraries and tools want to be able to make reliably.

The other comparison that comes to mind would be the distinction
between "__int__" ("ca

Re: [Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

The main use case I had in mind was PEP484-based type hinting/checking actually:

def my_function(foo: Boolean):
pass

explicitly states that my_function accepts any Boolean value, whether it is a 
python bool or a np.bool that would come from a numpy array or pandas dataframe.
Note that type hinting is also the use case for which I make extensive use of 
the types from the ‘numbers’ package, for the same reasons.

Sylvain

De : Python-ideas 
[mailto:python-ideas-bounces+sylvain.marie=schneider-electric@python.org] 
De la part de David Mertz
Envoyé : mardi 13 février 2018 07:08
À : Nick Coghlan 
Cc : python-ideas 
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module

I'm not sure I'm convinced by Sylvain that Boolean needs to be an ABC in the 
standard library; Guido expresses skepticism.  Of course it is possible to 
define it in some other library that actually needs to use `isinstance(x, 
Boolean)` as Sylvain demonstraits in his post.  I'm not sure I'm unconvinced 
either, I can see a certain value to saying a given value is "fully 
round-trippable to bool" (as is np.bool_).

But just for anyone who doesn't know NumPy, here's a quick illustration of what 
I alluded to:

In [1]: import numpy as np
In [2]: arr = np.array([7,8,12,33])
In [3]: ndx1 = np.array([0,1,1,0], dtype=int)
In [4]: ndx2 = np.array([0,1,1,0], dtype=bool)
In [5]: arr[ndx1]
Out[5]: array([7, 8, 8, 7])
In [6]: arr[ndx2]
Out[6]: array([ 8, 12])

ndx1 and ndx2 are both nice things (and are both often programmatically 
constructed by operations in NumPy).  But indexing using ndx1 gives us an array 
of the things in the listed positions in arr.  In this case, we happen to 
choose two each of the things an index 0 and index 1 in the result.

Indexing by ndx2 gives us a filter of only those positions in arr corresponding 
to 'True's.  These are both nice things to be able to do, but if NumPy's True 
was a special kind of 1, it wouldn't work out unambiguously.  However, recent 
versions of NumPy have gotten a bit smarter about recognizing the special type 
of Python bools, so it's less of a trap than it used to be.  Still, contrast 
these (using actual Python lists for the indexes:

In [10]: arr[[False, True, True, False]]
Out[10]: array([ 8, 12])
In [11]: arr[[False, True, 1, 0]]
Out[11]: array([7, 8, 8, 7])



On Mon, Feb 12, 2018 at 7:50 PM, Nick Coghlan 
> wrote:
On 13 February 2018 at 02:14, David Mertz 
> wrote:
> NumPy np.bool_ is specifically not a subclass of any np.int_.  If it we're,
> there would be an ambiguity between indexing with a Boolean array and an
> array of ints. Both are meaningful, but they mean different things (mask vs
> collection of indices).
>
> Do we have other examples a Python ABC that exists to accommodate something
> outside the standard library or builtins? Even if not, NumPy is special...
> the actual syntax for '@' exists primarily for that library!

collections.abc.Sequence and collections.abc.Mapping come to mind -
the standard library doesn't tend to distinguish between different
kinds of subscriptable objects, but it's a distinction some third
party libraries and tools want to be able to make reliably.

The other comparison that comes to mind would be the distinction
between "__int__" ("can be coerced to an integer, but may lose
information in the process") and "__index__" ("can be losslessly
converted to and from a builtin integer").

Right now, we only define boolean coercion via "__bool__" - there's no
mechanism to say "this *is* a boolean value that can be losslessly
converted to and from the builtin boolean constants". That isn't a
distinction the standard library makes, but it sounds like it's one
that NumPy cares about (and NumPy was also the main driver for
introducing __index__).

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   
Brisbane, Australia



--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

My point is just that today, I use the ‘numbers’ package classes (Integral, 
Real, …) for PEP484 type-hinting, and I find it quite useful in term of input 
type validation (in combination with PEP484-compliant type checkers, whether 
static or dynamic). Adding a Boolean ABC with a similar behavior would 
certainly add consistency to that ‘numbers’ package – only for users who 
already find it useful, of course.

Note that my use case is not about converting an object to a Boolean, I’m just 
speaking about type validation of a ‘true’ boolean object, for example to be 
received as a function argument for a flag option. This is for example for 
users who want to define strongly-typed APIs for interaction with the ‘outside 
world’, and keep using duck-typing for internals.

Sylvain

De : Python-ideas 
[mailto:python-ideas-bounces+sylvain.marie=schneider-electric@python.org] 
De la part de Chris Barker
Envoyé : mardi 13 février 2018 21:12
À : David Mertz 
Cc : python-ideas 
Objet : Re: [Python-ideas] Boolean ABC similar to what's provided in the 
'numbers' module



On Mon, Feb 12, 2018 at 10:07 PM, David Mertz 
> wrote:
I'm not sure I'm convinced by Sylvain that Boolean needs to be an ABC in the 
standard library; Guido expresses skepticism.  Of course it is possible to 
define it in some other library that actually needs to use `isinstance(x, 
Boolean)` as Sylvain demonstraits in his post.  I'm not sure I'm unconvinced 
either, I can see a certain value to saying a given value is "fully 
round-trippable to bool" (as is np.bool_).

But is an ABC the way to do it? Personally, I'm skeptical that ABCs are a 
solution to, well, anything (as apposed to duck typing and EAFTP). Take Nick's 
example:

"""
The other comparison that comes to mind would be the distinction
between "__int__" ("can be coerced to an integer, but may lose
information in the process") and "__index__" ("can be losslessly
converted to and from a builtin integer").
"""

I suppose we could have had an Index ABC -- but that seems painful to me.

so maybe we could use a __true_bool__ special method?

(and an operator.true_bool() function ???)

(this all makes me wish that python bools were more pure -- but way to late for 
that!)

I guess it comes down to whether you want to:

 - Ask the question: "is this object a boolean?"

or

 - Make this object a boolean

__index__ (and operator.index())  is essentially the later -- you want to make 
an index out of whatever object you have, if you can do so.

-CHB



--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Boolean ABC similar to what's provided in the 'numbers' module

[Sylvain MARIE]

The numbers module provides very useful ABC for the 'numeric tower', able to 
abstract away the differences between python primitives and for example numpy 
primitives.
I could not find any equivalent for Booleans.
However numpy defines np.bool too, so being able to have an abstract Boolean 
class for both python bool and numpy bool would be great.

Here is a version that I included in valid8 in the meantime

---
class Boolean(metaclass=ABCMeta):
"""
An abstract base class for booleans, similar to what is available in numbers
see https://docs.python.org/3.5/library/numbers.html
"""
__slots__ = ()

@abstractmethod
def __bool__(self):
"""Return a builtin bool instance. Called for bool(self)."""

@abstractmethod
def __and__(self, other):
"""self & other"""

@abstractmethod
def __rand__(self, other):
"""other & self"""

@abstractmethod
def __xor__(self, other):
"""self ^ other"""

@abstractmethod
def __rxor__(self, other):
"""other ^ self"""

@abstractmethod
def __or__(self, other):
"""self | other"""

@abstractmethod
def __ror__(self, other):
"""other | self"""

@abstractmethod
def __invert__(self):
"""~self"""


# register bool and numpy bool_ as virtual subclasses
# so that issubclass(bool, Boolean) = issubclass(np.bool_, Boolean) = True
Boolean.register(bool)

try:
import numpy as np
Boolean.register(np.bool_)
except ImportError:
# silently escape
pass

---

If that topic was already discussed and settled in the past, please ignore this 
thread - apologies for not being able to find it.
Best regards

Sylvain

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dataclasses, keyword args, and inheritance

[Sylvain MARIE]

George
For what’s worth if it can help for your need – I know that this is not part of 
the dataclasses PEP –  inheritance now works with @autoclass 
(https://smarie.github.io/python-autoclass/ ), as demonstrated below.
Note that unfortunately for mutable default field values you still have to 
perform self-assignment. But that’s the same problem that you have with 
‘standard’ python.

# with an unmutable default value
from autoclass import autoclass


@autoclass
class Foo:
def __init__(self, some_default: str = 'unmutable_default'):
pass


@autoclass
class Bar(Foo):
def __init__(self, other_field: int, some_default: str = 
'unmutable_default'):
super(Bar, self).__init__(some_default=some_default)


a = Bar(2)
assert a.other_field == 2
assert a.some_default == 'unmutable_default'


# with a mutable default value

@autoclass
class Foo:
def __init__(self, some_default: str = None):
# you have to assign, you can not delegate to @autoclass for this :(
self.some_default = some_default or ['mutable_default']


@autoclass
class Bar(Foo):
def __init__(self, other_field: int, some_default: str = None):
super(Bar, self).__init__(some_default=some_default)


a = Bar(2)
assert a.other_field == 2
assert a.some_default == ['mutable_default']

By the way is there any plan or idea to allow users to provide “default values 
factories (generators)” directly in the function signature, when their default 
value is mutable ? That could be a great improvement of the python language. I 
could not find anything googling around… let me know if I should create a 
dedicated thread to discuss this.
Kind regards

Sylvain

De : Python-ideas 
[mailto:python-ideas-bounces+sylvain.marie=schneider-electric@python.org] 
De la part de Guido van Rossum
Envoyé : lundi 29 janvier 2018 20:44
À : George Leslie-Waksman 
Cc : Eric V. Smith ; python-ideas 
Objet : Re: [Python-ideas] Dataclasses, keyword args, and inheritance

That's fair. Let me then qualify my statement with "in the initial release". 
The initial release has enough functionality to deal with without considering 
your rather esoteric use case. (And I consider it esoteric because attrs has 
apparently never seen the need to solve it either.) We can reconsider for 3.8.

On Mon, Jan 29, 2018 at 11:38 AM, George Leslie-Waksman 
> wrote:
Given I started this thread from a perspective of this is a feature that I 
would like because I need it, it feels a little dismissive to take attrs not 
having the feature to mean "there's no reason to try to implement this."

On Mon, Jan 29, 2018 at 11:05 AM Guido van Rossum 
> wrote:
I think that settles it -- there's no reason to try to implement this.

On Mon, Jan 29, 2018 at 10:51 AM, George Leslie-Waksman 
> wrote:
attrs' seems to also not allow mandatory attributes to follow optional one:

In [14]: @attr.s
...: class Baz:
...: a = attr.ib(default=attr.Factory(list))
...: b = attr.ib()
...:
---
ValueErrorTraceback (most recent call last)
 in ()
> 1 @attr.s
  2 class Baz:
  3 a = attr.ib(default=attr.Factory(list))
  4 b = attr.ib()
  5

/Users/waksman/.pyenv/versions/3.6.1/envs/temp/lib/python3.6/site-packages/attr/_make.py
 in attrs(maybe_cls, these, repr_ns, repr, cmp, hash, init, slots, frozen, str, 
auto_attribs)
700 return wrap
701 else:
--> 702 return wrap(maybe_cls)
703
704

/Users/waksman/.pyenv/versions/3.6.1/envs/temp/lib/python3.6/site-packages/attr/_make.py
 in wrap(cls)
669 raise TypeError("attrs only works with new-style classes.")
670
--> 671 builder = _ClassBuilder(cls, these, slots, frozen, auto_attribs)
672
673 if repr is True:

/Users/waksman/.pyenv/versions/3.6.1/envs/temp/lib/python3.6/site-packages/attr/_make.py
 in __init__(self, cls, these, slots, frozen, auto_attribs)
369
370 def __init__(self, cls, these, slots, frozen, auto_attribs):
--> 371 attrs, super_attrs = _transform_attrs(cls, these, auto_attribs)
372
373 self._cls = cls

/Users/waksman/.pyenv/versions/3.6.1/envs/temp/lib/python3.6/site-packages/attr/_make.py
 in _transform_attrs(cls, these, auto_attribs)
335 "No mandatory attributes allowed after an attribute 
with a "
336 "default value or factory.  Attribute in question: 
{a!r}"
--> 337 .format(a=a)
338 )
339 elif had_default is False and \

ValueError: No mandatory attributes allowed after an attribute with a default 
value or factory.  Attribute in question: Attribute(name='b', default=NOTHING, 

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

[Sylvain MARIE]

> I haven't yet seen any justification for syntax here. The nearest I've seen 
> is that this "ensure" action is more like:
>
> try:
> cond = x >= 0
> except BaseException:
> raise AssertionError("x must be positive")
> else:
> if not cond:
> raise AssertionError("x must be positive")
>
> Which, IMO, is a bad idea, and I'm not sure anyone was actually advocating it 
> anyway.
> 
> ChrisA

Indeed, I was the one advocating for it :) 

Based on all the feedback I received from this discussion, I realized that my 
implementation was completely flawed by the fact that I had done the class and 
functions decorators first, and wanted to apply the same pattern to the inline 
validator, resulting in this assert_valid with overkill delayed evaluation. 
Resulting in me saying that the only way out would be a new python language 
element.

I tried my best to update valid8 and reached a new stable point with version 
3.0.0, providing 2 main utilities for inline validation:
 - the simple but not so powerful `quick_valid` function 
 - the more verbose (2 lines) but much more generic `wrap_valid` context 
manager (that's the best I can do today !)

The more capable but delayed-evaluation based `assert_valid` is not recommended 
anymore, or just a tool to replicate what is done in the function and class 
validation decorators. Like the decorators, it adds the ability to blend two 
styles of base functions (boolean testers and failure raisers) with boolean 
operators seamlessly. But the complexity is not worth it for inline validation 
(it seems to be worth it for decorators).

See https://smarie.github.io/python-valid8 for the new updated documentation. I 
also updated the problem description page at 
https://smarie.github.io/python-valid8/why_validation/ so as to keep a 
reference of the problem description and "wishlist" (whether it is implemented 
by this library or by new language elements in the future). Do not hesitate to 
contribute or send me your edits (off-list).

I would love to get feedback from anyone concerning this library, whether you 
consider it's useless or "interesting but...". We should probably take this 
offline though, so as not to pollute the initial thread.

Thanks again, a great weekend to all (end of the day here in france ;) )
Kind regards

Sylvain 

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Repurpose `assert' into a general-purpose check

[Sylvain MARIE]

(trying with direct reply this time)

> Why do you do this? What's the requirement for delaying evaluation of the 
> condition?

Thanks for challenging my poorly chosen examples :)

The primary requirement is about *catching* unwanted/uncontrolled/heterogenous 
exceptions happening in the underlying functions that are combined together to 
provide the validation means, so as to provide a uniform/consistent outcome 
however diverse the underlying functions are (they can return booleans or raise 
exceptions, or both).

In your proposal, if 'is_foo_compliant' raises an exception, it will not be 
caught by 'assert_valid', therefore the ValidationError will not be raised. So 
this is not what I want as an application developer.

--
Sylvain

-Message d'origine-
De : Paul Moore [mailto:p.f.mo...@gmail.com] 
Envoyé : mardi 16 janvier 2018 18:01
À : Sylvain MARIE <sylvain.ma...@schneider-electric.com>
Cc : Python-Ideas <python-ideas@python.org>
Objet : Re: [Python-ideas] Repurpose `assert' into a general-purpose check

I fixed the reply-to this time, looks like you're still getting messed up by 
Google Groups.

On 16 January 2018 at 16:25, smarie
<sylvain.ma...@schneider-electric.com> wrote:
> Let's consider this example where users want to define on-the-fly one 
> of the validation functions, and combine it with another with a 'or':
>
> assert_valid('surface', surf, or_(lambda x: (x >= 0) & (x < 
> 1), is_foo_compliant), help_msg="surface should be 0=<x<1 or 
> foo compliant")
>
> How ugly for something so simple ! I tried to make it slightly more 
> compact by developping a mini lambda syntax but it obviously makes it slower.

Why do you do this? What's the requirement for delaying evaluation of the 
condition? A validate statement in Python wouldn't be any better able to do 
that, so it'd be just as ugly with a statement. There's no reason I can see why 
I'd ever need delayed evaluation, so what's wrong with just

assert_valid(0 <= surf < 1 and is_foo_compliant(surf), 
help_msg="surface should be 0=<x<1 or foo compliant")



> There are three reasons why having a 'validate' statement would 
> improve
> this:
>
>  * no more parenthesis: more elegant and readable
>  * inline use of python (1): no more use of lambda or mini_lambda, no 
> performance overhead
>  * inline use of python (2): composition would not require custom 
> function composition operators such as 'or_' (above) or mini-lambda 
> composition anymore, it could be built-in in any language element used 
> after 
>
> resulting in
>
> validate (surf >= 0) & (surf < 1) or 
> is_foo_compliant(surf), "surface should be 0=<x<1 or foo compliant"

So how will that work any differently than the function version I gave above? 
In terms of the language syntax, it would just be

validate EXPR, EXPR

and the first EXPR will be evaluated as a boolean, and if it's false an 
exception will be raised with the second expression as the message.
There's no delayed evaluation involved, so a function would work exactly the 
same.

>
> (I removed the variable name alias 'surface' since I don't know if it 
> should remain or not)
>
> Elegant, isn't it ?

No more so than my function version, but yes far more so than yours...

Paul

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/