Re: [Python-Dev] Finding Guido's replacement
On Sun, Jul 22, 2018 at 11:18 PM Chris Angelico wrote: > > * Lately, all Guido's actions have been to benefit his employer, not > the Common Pythonista. We have proof of this from reliable reporting > sources such as Twitter and social media. > This accusation is ridiculous and not appreciated. Type hinting is one of the most praised Python features in pretty much any big company, where managing millions of lines of Python is challenging. Next time, I also suggest you to cite and name your 'reliable reporting sources' otherwise this is just bs. Yury -- Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
On Mon, 23 Jul 2018 11:47:25 +0300 Yury Selivanov wrote: > On Sun, Jul 22, 2018 at 11:18 PM Chris Angelico wrote: > > > > > * Lately, all Guido's actions have been to benefit his employer, not > > the Common Pythonista. We have proof of this from reliable reporting > > sources such as Twitter and social media. > > > > This accusation is ridiculous and not appreciated. Type hinting is one of > the most praised Python features in pretty much any big company, where > managing millions of lines of Python is challenging. Next time, I also > suggest you to cite and name your 'reliable reporting sources' otherwise > this is just bs. I suspect Chris A. was merely joking, though I'm not sure what the joke ultimately is supposed to be about. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
On Mon, Jul 23, 2018 at 12:03 PM Antoine Pitrou wrote: > > I suspect Chris A. was merely joking, though I'm not sure what the joke > ultimately is supposed to be about. > Ah, right, I stopped reading his email after the quoted line. Well executed. Yury > > -- Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
Hi! On Mon, Jul 23, 2018 at 11:47:25AM +0300, Yury Selivanov wrote: > On Sun, Jul 22, 2018 at 11:18 PM Chris Angelico wrote: > > > > > * Lately, all Guido's actions have been to benefit his employer, not > > the Common Pythonista. We have proof of this from reliable reporting > > sources such as Twitter and social media. > > This accusation is ridiculous and not appreciated. Yury, the entire message from Chris was a joke. Of bad taste, in my opinion. And I also didn't get initially that it was a joke, it took me a few minutes to understand. > Yury > -- > Yury Oleg. -- Oleg Broytmanhttps://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
I did exactly the same benchmark again with Python 3.7 and the results are similar. I'm copying and editing the original post for completeness: I finally managed to get some real-life benchmarks for why we need a faster C calling protocol (see PEPs 576, 579, 580). I focused on the Cython compilation of SageMath. By default, a function in Cython is an instance of builtin_function_or_method (analogously, method_descriptor for a method), which has special optimizations in the CPython interpreter. But the option "binding=True" changes those to a custom class which is NOT optimized. I ran the full SageMath testsuite several times on Python 2.7 without and with binding=True to find out any significant differences. I then checked if those differences could be reproduced on Python 3.7 (SageMath has not been fully ported to Python 3 yet). The most dramatic difference is multiplication for generic matrices. More precisely, with the following command: python3 -m timeit -s "from sage.all import MatrixSpace, GF; M = MatrixSpace(GF(9), 200).random_element()" "M * M" With binding=False, I got 1 loop, best of 5: 1.19 sec per loop With binding=True, I got 1 loop, best of 5: 1.83 sec per loop This is a big regression which should be gone completely with PEP 580. I used Python 3.7, SageMath 8.3.rc1 (plus a few patches to make it work with binding=True and with Python 3.7) and Cython 0.28.4. I hope that this finally shows that the problems mentioned in PEP 579 are real. Jeroen. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On Mon, 23 Jul 2018 11:51:41 +0200 Jeroen Demeyer wrote: > I did exactly the same benchmark again with Python 3.7 and the results > are similar. I'm copying and editing the original post for completeness: > > [...] > > I hope that this finally shows that the problems mentioned in PEP 579 > are real. Just for the record, personally I have no doubt that the problems are real. IMHO the main point of discussion should be judging the solution you are proposing (and thank you, really, for being persistent, since this isn't an easy discussion). Best regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
On Mon, Jul 23, 2018 at 7:07 PM, Oleg Broytman wrote: > Hi! > > On Mon, Jul 23, 2018 at 11:47:25AM +0300, Yury Selivanov > wrote: >> On Sun, Jul 22, 2018 at 11:18 PM Chris Angelico wrote: >> >> > >> > * Lately, all Guido's actions have been to benefit his employer, not >> > the Common Pythonista. We have proof of this from reliable reporting >> > sources such as Twitter and social media. >> >> This accusation is ridiculous and not appreciated. > >Yury, the entire message from Chris was a joke. Of bad taste, in my > opinion. And I also didn't get initially that it was a joke, it took me > a few minutes to understand. If the reference to PEP 401 didn't tip you off and you don't recognize the references to famous Battle Royale games, then perhaps the fact that I called social media "reliable" might have been a good hint that I wasn't being entirely serious? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
On Mon, Jul 23, 2018 at 08:37:05PM +1000, Chris Angelico wrote: > On Mon, Jul 23, 2018 at 7:07 PM, Oleg Broytman wrote: > > I also didn't get initially that it was a joke, it took me > > a few minutes to understand. > > If the reference to PEP 401 didn't tip you off and you don't recognize > the references to famous Battle Royale games, then perhaps the fact > that I called social media "reliable" might have been a good hint that > I wasn't being entirely serious? I was sure you've been serious until I got to the last paragraph. > ChrisA Oleg. -- Oleg Broytmanhttps://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
Please, take this thread off-list. Nobody is interested in you explaining a joke. On Mon, 23 Jul 2018 20:37:05 +1000 Chris Angelico wrote: > On Mon, Jul 23, 2018 at 7:07 PM, Oleg Broytman wrote: > > Hi! > > > > On Mon, Jul 23, 2018 at 11:47:25AM +0300, Yury Selivanov > > wrote: > >> On Sun, Jul 22, 2018 at 11:18 PM Chris Angelico wrote: > >> > >> > > >> > * Lately, all Guido's actions have been to benefit his employer, not > >> > the Common Pythonista. We have proof of this from reliable reporting > >> > sources such as Twitter and social media. > >> > >> This accusation is ridiculous and not appreciated. > > > >Yury, the entire message from Chris was a joke. Of bad taste, in my > > opinion. And I also didn't get initially that it was a joke, it took me > > a few minutes to understand. > > If the reference to PEP 401 didn't tip you off and you don't recognize > the references to famous Battle Royale games, then perhaps the fact > that I called social media "reliable" might have been a good hint that > I wasn't being entirely serious? > > ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On 2018-07-23 01:54, Ivan Pozdeev via Python-Dev wrote: All the material to discuss that we have in this thread is a single test result that's impossible to reproduce and impossible to run in Py3. I just posted that it can be reproduced on Python 3.7: https://mail.python.org/pipermail/python-dev/2018-July/154740.html I admit that it's not entirely trivial to do that. The Python 3 port of SageMath is still work in progress and the Python 3.7 port even more so. So it requires a few patches. If somebody wants to reproduce those results right now, I could give more details. But really, I would recommend to wait a month or so and then hopefully those patches will be merged. It's however impossible to say from this how frequent these scenarios are in practice And how would you suggest that we measure that? All benchmarks are artificial in some way: for every benchmark, one can find reasons why it's not relevant. and how consistent the improvement is among them. I only posted the most serious regression. As another data point, the total time to run the full SageMath testsuite increased by about 1.8% when compiling the Cython code with binding=True. So one could assume that there is an average improvement of 1.8% with a much larger improvement in a few specific cases. Likewise, it's impossible to say anything about the complexity the changes will reduce/introduce without a proof-of-concept implementation. Why do you think that there is no implementation? As mentioned in PEP 580, there is an implementation at https://github.com/jdemeyer/cpython/tree/pep580 Jeroen. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On Mon, Jul 23, 2018 at 7:20 PM Antoine Pitrou wrote: > > Just for the record, personally I have no doubt that the problems are > real. IMHO the main point of discussion should be judging the solution > you are proposing (and thank you, really, for being persistent, since > this isn't an easy discussion). > Me too. My interest is what is the best benefit / complexity ratio. -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On 2018-07-23 00:28, Guido van Rossum wrote:
So does your implementation of the PEP result in a net increase or
decrease of the total lines of code?
12 files changed, 918 insertions(+), 704 deletions(-)
That's a net increase, so there is no obvious win here. Still, I have
various excuses for this increased number of lines of code:
1. I'm adding more comments and lines containing only characters from
the set " {};". This accounts for about 60% in the increase in number of
lines of code. Clearly, this shouldn't count against me.
2. I still need to support some old ways of doing things. For backwards
compatibility, functions and methods are still defined using a
PyMethodDef. So I need to convert this old structure to the new
protocol. I also keep support for some functions that my PEP makes
obsolete, such as PyCFunction_Call(). All this requires code, but that
code is simple.
3. In a few cases, specialized code is replaced by general code. For
example, code that needs the __name__ of a function changes from
accessing a known field in a C structure (func->m_ml->ml_name) to an
actual Python attribute lookup. And Python code like
def __qualname__(func):
name = func.__name__
try:
parent_qualname = func.__parent__.__qualname__
except AttributeError:
return name
return str(parent_qualname) + "." + name
is conceptually pretty simple, but it becomes 37 lines of C code in my
implementation.
4. The PEP does actually add a completely new feature: the flag
CCALL_FUNCARG. That is something that I could easily remove for now, but
PEP 580 would be a lot less useful without it. So it's something that I
would certainly want to add in a later PEP anyway. It's also required
for PEP 573.
Jeroen.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On Sun, Jul 22, 2018 at 7:42 PM Jeroen Demeyer wrote: > > On 2018-07-22 08:27, INADA Naoki wrote: > > It's interesting... But I failed to build sage. > > What went wrong? > I can't install Sage into my virtual environment, so I can't run > python -m timeit -s "from sage.all import MatrixSpace, GF; M = > MatrixSpace(GF(9), 200).random_element()" "M * M" But I can run it finally, with `sage -python` command. And I profiled what is the bottleneck of this oneliner. https://gist.github.com/methane/91533f68e88f89b7ec7f71855c069792 Cython does optimizations like CPython does internally. When calling function with one arg, and if the function is PyCFunction, Cython call underlaying C function directly. cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); result = cfunc(self, arg); This is why such regression happened on even Python 2.7. And I think I can write small benchmark emulate it for now. -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Benchmarks why we need PEP 576/579/580
On 2018-07-23 12:13, Antoine Pitrou wrote: IMHO the main point of discussion should be judging the solution you are proposing Yes please. I would very much welcome a discussion about the actual content of the PEP instead of meta-discussions like how complicated it is. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [Windows] how to prevent the wrong version of zlib1.dll to be used by lib-dynload modules
Hello list, I encountered a problem with the Windows packaging of gPodder[1] using msys2: basic libraries (zlib, openssl) depended upon by python platform-specific modules are loaded preferably : 1. from lib-dynload (where they are not) 2. from the Windows directory (can be any version) 3. from the binary directory, next to gpodder.exe (where they are) So an old zlib1.dll installed by another application in c:\Windows was loaded, incompatible with libpng and gPodder couldn't start. I don't know what's the best approach to solve it: - copy those libraries to lib\pythonxx\lib-dynload (works) - preload them in my main script before they are loaded by the module (works) - patch something in python (dynload_win.c ?) to search first in the executable directory (not tried) Please can you provide me with insight on this? Details in the issue: [1] Thanks, [1] https://github.com/gpodder/gpodder/issues/478 pgppBbXVsZUpz.pgp Description: Signature digitale OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Windows] how to prevent the wrong version of zlib1.dll to be used by lib-dynload modules
In general, if the dependent DLL is in the same directory as the module loading it (the .pyd or .exe), then it should be loaded first. If it's alongside the .exe, it should be loaded before any of the other search paths. If it's being loaded directly from Python, your best option is to resolve the full path before trying to load it (via ctypes or whatever). I have argued in the past about trying to hack the importer in order to "fix" this (since any fix is highly likely to break currently working install layouts), and this sounds like it is best solved by putting dependencies in an expected location. However, I don't know how much impact the msys2 aspect of this has. You mention "lib\pythonxx\lib-dynload", which is not present in a normal Python install on Windows. So possibly you're already on a non-standard build, which means my advice is pretty useless to you. There are also significant parts of both zlib and openssl available in a normal Python install on Windows, so perhaps you don't need to include alternate copies of those with your package? Are there particular features or APIs missing that you need? Cheers, Steve On 23Jul2018 1531, Eric Le Lay wrote: Hello list, I encountered a problem with the Windows packaging of gPodder[1] using msys2: basic libraries (zlib, openssl) depended upon by python platform-specific modules are loaded preferably : 1. from lib-dynload (where they are not) 2. from the Windows directory (can be any version) 3. from the binary directory, next to gpodder.exe (where they are) So an old zlib1.dll installed by another application in c:\Windows was loaded, incompatible with libpng and gPodder couldn't start. I don't know what's the best approach to solve it: - copy those libraries to lib\pythonxx\lib-dynload (works) - preload them in my main script before they are loaded by the module (works) - patch something in python (dynload_win.c ?) to search first in the executable directory (not tried) Please can you provide me with insight on this? Details in the issue: [1] Thanks, [1] https://github.com/gpodder/gpodder/issues/478 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Windows] how to prevent the wrong version of zlib1.dll to be used by lib-dynload modules
On Mon, Jul 23, 2018, 08:22 Eric Le Lay wrote: > Hello list, > > I encountered a problem with the Windows packaging of gPodder[1] > using msys2: > > basic libraries (zlib, openssl) depended upon by python > platform-specific modules are loaded preferably : > 1. from lib-dynload (where they are not) > 2. from the Windows directory (can be any version) > 3. from the binary directory, next to gpodder.exe (where they are) > > So an old zlib1.dll installed by another application in c:\Windows was > loaded, incompatible with libpng and gPodder couldn't start. > > I don't know what's the best approach to solve it: > - copy those libraries to lib\pythonxx\lib-dynload (works) > - preload them in my main script before they are loaded by the module >(works) > - patch something in python (dynload_win.c ?) to search first in the >executable directory (not tried) > The way we avoid these kinds of issues on Linux is to rename included libraries to unique names (e.g. zlib1-aef3742bc3e.dll), and patch the extension modules to look for the new names. There's a script (auditwheel) that does the heavy lifting. On MacOS there's a similar script people use, called "delocate". Unfortunately no one has written a script like this for Windows yet. I think it'd be neat if eventually we could consolidate all this into auditwheel, but there aren't that many people interested in working on it. If you wanted to help, the hardest part is already done: https://github.com/njsmith/machomachomangler/blob/master/README.rst#pe-features Alternatively if you just want a hack that works quick, I'd do the preload thing. In general you need to do this anyway, if you don't have all your extension modules in the same directory. (Unfortunately Windows doesn't have the equivalent of RPATH.) -n > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding Guido's replacement
> fun fact: weirdly enough after BDFL1 took a vac (for life?), google made > it's appearance on the mailing list > As the Googler who appeared on the mailing list, I can say this was just a coincidence. I was a bit nervous no one would respond though, given Guido's vacation :) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
