Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Nathaniel Smith 
On Mon, Jul 9, 2018 at 10:20 AM, Steve Dower  wrote:
> On 09Jul2018 0922, Antoine Pitrou wrote:
>>
>> On Mon, 9 Jul 2018 09:01:00 -0700
>> Steve Dower  wrote:
>>>
>>> I've thought a bit about making a single installer that can offer the
>>> option of 32-bit/64-bit at install time, but I don't actually think it's
>>> that big a problem to deserve that much effort as a solution.
>>>
>>> Perhaps we should add non-button text below the button saying "Get the
>>> 64-bit version"?
>>
>>
>> Or perhaps the 32-bit installer could detect a 64-bit system and add
>> an info box at the beginning?
>
>
> That's not a bad idea. Needs a bpo issue, but shouldn't be too hard to do.
> And it should be safe to backport for 3.7.1, as there's no real behaviour
> change.

And if the 64-bit installer could detect 32-bit systems and explain to
users that they got the wrong version and direct them to the correct
one, that would help anyone who does get confused by the 64-bit
installer becoming more prominent.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Encoding variable-length integers/counts in pickle

2018-07-10 Thread Andrew McLean 
Google Protocol Buffers use something similar, which they call "
Base
128 Varints"

https://developers.google.com/protocol-buffers/docs/encoding#varints

I prefer the way this handles negative numbers.

- Andrew

On 10 July 2018 at 02:53, MRAB  wrote:

> In the regex module I use an encoding scheme when pickling pattern objects
> which is based on the way MIDI encodes variable-length integers, and I
> think it might have a use in a pickle protocol.
>
> In the basic format, an integer is split up into 7-bit chunks, each chunk
> put into a byte, and the most-significant bit of the byte used to signal
> whether the value continues into the following byte.
>
> And integer must be encoded into the minimum number of bytes, so an
> encoded sequence of bytes would never start with 0x80.
>
> MIDI deviates from the basic idea in order to reduce the number of bytes,
> so as sequence of bytes in MIDI _can_ start with x080; this is fine for
> MIDI because it doesn't need to represent negative integers.
>
> The format I'm suggesting uses an initial 0x80 as a way of letting it
> encode negative integers.
>
> Here are a couple of Python functions that summarise the encoding and
> decoding (minus obvious optimisations for simplicity):
>
>
> def encode_varint(value: int) -> List[int]:
> negative = value < 0
> encoded = []
>
> if negative:
> final = -1
> else:
> final = 0
>
> while True:
> encoded.append(0x80 | (value & 0x7F))
> value >>= 7
>
> if value == final:
> break
>
> if negative:
> encoded.append(0x80)
>
> encoded.reverse()
>
> encoded[-1] &= 0x7F
>
> return encoded
>
>
> def decode_varint(encoded: Iterable[int]) -> int:
> byte = next(encoded)
>
> if byte == 0x80:
> value = -1
> byte = next(encoded)
> else:
> value = 0
>
> value = (value << 7) | (byte & 0x7F)
>
> while (byte & 0x80) != 0:
> byte = next(encoded)
> value = (value << 7) | (byte & 0x7F)
>
> return value
>
>
> The advantage of encoding integers in this way is that there's no limit to
> their size, so there's no need to add a new protocol to support larger
> counts.
>
> They can also make pickles smaller.
>
> Example:
>
> # Pickle (None, )
> 0: \x80 PROTO  4
> 2: \x95 FRAME  4
>11: NNONE
>12: \x85 TUPLE1
>13: \x94 MEMOIZE(as 0)
>14: .STOP
>
> Here, FRAME takes an argument of 8 bytes. If you replaced FRAME with a
> version that accepted a variable-length count, you could reduce that
> argument to 1 byte.
>
> You also wouldn't need to have different fixed-length versions of an
> opcode, e.g. BINUNICODE and SHORT_BINUNICODE.
>
> Whether you do anything with this is entirely up to the core devs, I just
> thought someone might find it useful.
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/lists%
> 40andros.org.uk
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Encoding variable-length integers/counts in pickle

2018-07-10 Thread Antoine Pitrou 
On Tue, 10 Jul 2018 02:53:47 +0100
MRAB  wrote:
> In the regex module I use an encoding scheme when pickling pattern 
> objects which is based on the way MIDI encodes variable-length integers, 
> and I think it might have a use in a pickle protocol.
> 
> In the basic format, an integer is split up into 7-bit chunks, each 
> chunk put into a byte, and the most-significant bit of the byte used to 
> signal whether the value continues into the following byte.
> 
> And integer must be encoded into the minimum number of bytes, so an 
> encoded sequence of bytes would never start with 0x80.

The problem with variable-length encoding is that you need more
individual reads to fetch a datum.  The whole point of pickle framing is
to replace many small reads with a few large reads, and variable-length
encoding is adversial to that goal.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Jeroen Demeyer 

OK, I tried with --duplicate 200 and you can see the results at
https://gist.github.com/jdemeyer/f0d63be8f30dc34cc989cd11d43df248

In short, the timings with and without PEP 580 are roughly the same 
(which is to be expected). Interestingly, a small but significant 
improvement can be seen when calling *unbound* methods.


The real improvement comes from supporting a new calling protocol: 
formerly custom classes could only implement tp_call, but now they can 
use FASTCALL just like built-in functions/methods. For this, there is an 
improvement of roughly a factor 1.2 for calls without arguments, 1.6 for 
calls with positional arguments and 2.8 for calls with keywords.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread INADA Naoki 
On Tue, Jul 10, 2018 at 8:55 PM Jeroen Demeyer  wrote:
>
> OK, I tried with --duplicate 200 and you can see the results at
> https://gist.github.com/jdemeyer/f0d63be8f30dc34cc989cd11d43df248
>
> In short, the timings with and without PEP 580 are roughly the same
> (which is to be expected). Interestingly, a small but significant
> improvement can be seen when calling *unbound* methods.
>
> The real improvement comes from supporting a new calling protocol:
> formerly custom classes could only implement tp_call, but now they can
> use FASTCALL just like built-in functions/methods. For this, there is an
> improvement of roughly a factor 1.2 for calls without arguments, 1.6 for
> calls with positional arguments and 2.8 for calls with keywords.

We know it when we introduced FASTCALL.

What I want know is "how often" tp_call in custom type is called
in real application.  Does it boost real application performance
significantly?  5%? 10%?

If it's not significant enough, I want to wait make FASTCALL public until
more evolutionary optimization happened.  There are some remaining
possible optimizations.

For example, let's assume cfunction like this:

static PyObject*
myfunc_impl(PyObject *self, Py_ssize_t i)
{
...
}

static PyObject*
myfunc(PyObject *self, PyObject *arg)
{
Py_ssize_t i;
if (!PyArg_Parse(arg, "n;myfunc", &i)) {
return NULL;
}
return myfunc_impl(self, i);
}

Then, the function is called from another C extension like this:

PyObject_CallFunction(func, "n", 42);

Currently, we create temporary long object for passing argument.
If there is protocol for exposeing format used by PyArg_Parse*, we can
bypass temporal Python object and call myfunc_impl directly.

I think optimization like this idea can boost application performance
using Cython heavily.
But in Python and stdlib, there are no enough "call C function from C function"
scenarios, compared with Cython based applications.  We really
need help from Cython world for this area.

Regards,
-- 
INADA Naoki  
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Antoine Pitrou 
On Tue, 10 Jul 2018 21:59:28 +0900
INADA Naoki  wrote:
> 
> Then, the function is called from another C extension like this:
> 
> PyObject_CallFunction(func, "n", 42);
> 
> Currently, we create temporary long object for passing argument.
> If there is protocol for exposeing format used by PyArg_Parse*, we can
> bypass temporal Python object and call myfunc_impl directly.

This is another can of worms to open.  If you're worried about the
added complexity of PEP 580, what you're proposing is one or two orders
of magnitude more complicated.

> I think optimization like this idea can boost application performance
> using Cython heavily.

You can already define the C signature of a function in Cython and
intra-Cython calls will benefit from it where possible.  Cooperation
from core Python is not necessary for that.

The main point of PEP 580 is to make Cython functions faster when
called from pure Python, not from other Cython functions.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Jeroen Demeyer 

On 2018-07-10 14:59, INADA Naoki wrote:

Currently, we create temporary long object for passing argument.
If there is protocol for exposeing format used by PyArg_Parse*, we can
bypass temporal Python object and call myfunc_impl directly.


Indeed, I already mentioned this at
https://www.python.org/dev/peps/pep-0579/#allowing-native-c-arguments

The way I see it, these kind of improvements should be done on top of 
PEP 580. Maybe I didn't emphasize this enough, but one of the goals of 
PEP 580 is to create an *extensible* protocol where such features can 
easily be added later in a compatible way.



Jeroen.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions -- intention to accept, near-final draft

2018-07-10 Thread Nick Coghlan 
On 10 July 2018 at 11:00, Guido van Rossum  wrote:
> A lot has been said about PEP 572. I am planning to accept it soon,
> hopefully within a week. I realize I should have posted the draft from May
> 22 (when Tim and I were added as authors and it was significantly updated --
> see https://github.com/python/peps/pull/654). For this I apologize. Since
> then we've been diligently updating various details, tightening the spec
> without really changing the intended design.

Thanks for the updates, especially the new Appendix B that walks
through the comprehension scoping cases with worked examples.

I've submitted a couple of minor PRs, but this version of the PEP
looks solid to me, and makes the case for change well (I'm still only
a +0 personally, but I can also see myself some day saying "Yeah, that
turned out to be worth it in the long run")

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Victor Stinner 
2018-07-09 18:01 GMT+02:00 Steve Dower :
> The difficulty is that they *definitely* can use the 32-bit version, and
> those few who are on older machines or older installs of Windows may not
> understand why the link we provide didn't work for them.

Let's say that only 10% of users still use 32-bit version. If they
download a default 64-bit binary, I'm quite sure that running the
binary will emit an error no? Such users should be used to such error,
and be able to get the 64-bit version, no?

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Victor Stinner 
2018-07-10 14:59 GMT+02:00 INADA Naoki :
> PyObject_CallFunction(func, "n", 42);
>
> Currently, we create temporary long object for passing argument.
> If there is protocol for exposeing format used by PyArg_Parse*, we can
> bypass temporal Python object and call myfunc_impl directly.

I'm not sure that it's worth it. It seems complex to implement.

I proposed something simpler, but nobody tried to implement it.
Instead of calling the long and complex PyArg_Parse...() functions,
why not generating C code to parse arguments instead? The idea looks
like "inlining" PyArg_Parse...() in its caller, but technically it
means that Argument Clinic generates C code to parse arguments.

PyArg_Parse...() is cheap and has been optimized, but on very fast
functions (less than 100 ns), it might be significant. Well, to be
sure, someone should run a benchmark :-)

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Victor Stinner 
About your benchmark results:

"FASTCALL unbound method(obj, 1, two=2): Mean +- std dev: 42.6 ns +- 29.6 ns"

That's a very big standard deviation :-( Are you using CPU pinning and
other technics explaining in my doc?
http://perf.readthedocs.io/en/latest/run_benchmark.html#how-to-get-reproductible-benchmark-results

Victor

2018-07-10 13:53 GMT+02:00 Jeroen Demeyer :
> OK, I tried with --duplicate 200 and you can see the results at
> https://gist.github.com/jdemeyer/f0d63be8f30dc34cc989cd11d43df248
>
> In short, the timings with and without PEP 580 are roughly the same (which
> is to be expected). Interestingly, a small but significant improvement can
> be seen when calling *unbound* methods.
>
> The real improvement comes from supporting a new calling protocol: formerly
> custom classes could only implement tp_call, but now they can use FASTCALL
> just like built-in functions/methods. For this, there is an improvement of
> roughly a factor 1.2 for calls without arguments, 1.6 for calls with
> positional arguments and 2.8 for calls with keywords.
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for function calls (PEP 576/579/580)

2018-07-10 Thread Victor Stinner 
The pyperformance benchmark suite had micro benchmarks on function
calls, but I removed them because they were sending the wrong signal.
A function call by itself doesn't matter to compare two versions of
CPython, or CPython to PyPy. It's also very hard to measure the cost
of a function call when you are using a JIT compiler which is able to
inline the code into the caller... So I removed all these stupid
"micro benchmarks" to a dedicated Git repository:
https://github.com/vstinner/pymicrobench

Sometimes, I add new micro benchmarks when I work on one specific
micro optimization.

But more generally, I suggest you to not run micro benchmarks and
avoid micro optimizations :-)

Victor

2018-07-10 0:20 GMT+02:00 Jeroen Demeyer :
> Here is an initial version of a micro-benchmark for C function calling:
>
> https://github.com/jdemeyer/callbench
>
> I don't have results yet, since I'm struggling to find the right options to
> "perf timeit" to get a stable result. If somebody knows how to do this, help
> is welcome.
>
>
> Jeroen.
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-10 Thread Victor Stinner 
2018-07-07 0:26 GMT+02:00 Victor Stinner :
> I designed FASTCALL with the help of Serhiy for keywords. I prepared a long
> email reply, but I found an opportunity for optimisation on **kwargs and I
> need time to see how to optimize it.

I just created: "Python function call optimization: avoid temporary
tuple to pass **kwargs"
https://bugs.python.org/issue34090

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-10 Thread Victor Stinner 
2018-07-07 10:55 GMT+02:00 Serhiy Storchaka :
> There is my idea. Split every of keyword argument parsing functions on two
> parts. The first part linearize keyword arguments, it converts positional
> and keyword arguments (in whatever form they were presented) into a linear
> array of PyObject* (with NULLs for not specified optional arguments). The
> second part is common and similar to _PyArg_ParseStack(), but supports
> NULLs. It converts an array of PyObject* to a sequence of C values. I tried
> to implement this idea, is is not simple, and results were mixed, but I
> don't loss a hope.
>
> And here we return to METH_FASTCALL|METH_KEYWORDS. The first part of
> handling arguments can be made outside of the C function, by the calling
> API. Then the signature of the C function can be simpler, the same as for
> METH_FASTCALL. But we need to expose the list of keyword parameter names as
> an attribute of CFunction.
>
> I don't know whether this ides is vital or dead, but I' going to try it. And
> implementing it will change the METH_FASTCALL|METH_KEYWORDS calling
> convention a lot.

I don't understand why changing the implementation of PyArg_Parse*()
would require to require the FASTCALL calling convention?

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread INADA Naoki 
On Wed, Jul 11, 2018 at 7:47 AM Victor Stinner  wrote:
>
> 2018-07-10 14:59 GMT+02:00 INADA Naoki :
> > PyObject_CallFunction(func, "n", 42);
> >
> > Currently, we create temporary long object for passing argument.
> > If there is protocol for exposeing format used by PyArg_Parse*, we can
> > bypass temporal Python object and call myfunc_impl directly.
>
> I'm not sure that it's worth it. It seems complex to implement.
>

Both of my idea and PEP 580 are complicated.

For Python stdlibs, I expect no significant benefit.
We already bypass Python function calling by typecheck + concrete
function call idiom.
But for Cython users, especially people using Cython on Jupyter, I expect
there are many extension-to-extension calls.

Both of this idea and PEP 580 is complicated.  And we don't have
realistic example to demonstrate real world benefit of them.
Without application benchmark, I think both of idea and PEP 580
shouldn't be happened.
That's why I requested application benchmark again and again.

PEP 576 seems minimalistic, straightforward way to allow FASTCALL
for Cython and other 3rd party libraries.
But if we accept PEP 576, it makes harder to allow more optimization
in the future.

I expect best balance is between PEP 576 and 580.  Maybe, adding
new slot as struct pointer with some flags, but don't add per-instance data.
But I'm not sure because I'm not data scientist.
I don't know what's the typical usage and where is main bottleneck of
their application.

Jeroen seems want we to discuss on PEP 576 and 580.
So I explained to him why we need example application first.


> I proposed something simpler, but nobody tried to implement it.
> Instead of calling the long and complex PyArg_Parse...() functions,
> why not generating C code to parse arguments instead? The idea looks
> like "inlining" PyArg_Parse...() in its caller, but technically it
> means that Argument Clinic generates C code to parse arguments.
>

I believe Cython did it already.

But I have worrying about it.  If we do it for all function, it makes Python
binary fatter, consume more CPU cache.  Once CPU cache is start
stashing, application performance got slower quickly.
And benchmarking CPU cache efficiency is very difficult.  Current
Python benchmark is too small.  We benchmarked HTTP server,
SQLAlchemy, JSON, template engine individually.  But real application
do all of them in loop.   And many processes share L3 cache.
Even L1 cache is shared by several processes by HyperThreading
and context switch.

> PyArg_Parse...() is cheap and has been optimized, but on very fast
> functions (less than 100 ns), it might be significant. Well, to be
> sure, someone should run a benchmark :-)
>
> Victor

-- 
INADA Naoki  
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread INADA Naoki 
On Tue, Jul 10, 2018 at 10:20 PM Antoine Pitrou  wrote:
>
> On Tue, 10 Jul 2018 21:59:28 +0900
> INADA Naoki  wrote:
> >
> > Then, the function is called from another C extension like this:
> >
> > PyObject_CallFunction(func, "n", 42);
> >
> > Currently, we create temporary long object for passing argument.
> > If there is protocol for exposeing format used by PyArg_Parse*, we can
> > bypass temporal Python object and call myfunc_impl directly.
>
> This is another can of worms to open.  If you're worried about the
> added complexity of PEP 580, what you're proposing is one or two orders
> of magnitude more complicated.

This is just an example of possible optimization, to explain why I want
example application first.
I know Cython is important for data scientists.  But I don't know how
it used typically.

If my idea has 50% gain and current PEP 580 has only 5% gain,
why we should accept PEP 580?
But no one know real gain, because there are no realistic application
which bottleneck is calling overhead.

Without example application, I can't consider PEP 580 seriously.
Microbenchemarks doesn't attract me.
And PEP 576 seems much simpler and straightforward way to expose
FASTCALL.


>
> > I think optimization like this idea can boost application performance
> > using Cython heavily.
>
> You can already define the C signature of a function in Cython and
> intra-Cython calls will benefit from it where possible.  Cooperation
> from core Python is not necessary for that.

Why not allow it for extensions written in C, without Cython?

>
> The main point of PEP 580 is to make Cython functions faster when
> called from pure Python, not from other Cython functions.

Really?  If so, I prefer PEP 576 to PEP 580.  PEP 580 is too complicated.

Anyway, I want more example applications which uses Cython heavily
for further discussion.

Regards,
-- 
INADA Naoki  
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Ivan Pozdeev via Python-Dev 

On 11.07.2018 1:41, Victor Stinner wrote:

2018-07-09 18:01 GMT+02:00 Steve Dower :

The difficulty is that they *definitely* can use the 32-bit version, and
those few who are on older machines or older installs of Windows may not
understand why the link we provide didn't work for them.

Let's say that only 10% of users still use 32-bit version. If they
download a default 64-bit binary, I'm quite sure that running the
binary will emit an error no? Such users should be used to such error,
and be able to get the 64-bit version, no?


Attached the image of what happens. The message is:

"One or more issues caused the setup to fail. Please fix the issues and 
the retry setup. For more information see the log file .


0x80070661 - This installation package is not supported by this 
processor type. Contact your product vendor."


Pretty descriptive in my book.


Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Stefan Behnel 
INADA Naoki schrieb am 11.07.2018 um 02:12:
> On Tue, Jul 10, 2018 at 10:20 PM Antoine Pitrou wrote:
>> On Tue, 10 Jul 2018 21:59:28 +0900
>> INADA Naoki wrote:
>>>
>>> Then, the function is called from another C extension like this:
>>>
>>> PyObject_CallFunction(func, "n", 42);
>>>
>>> Currently, we create temporary long object for passing argument.
>>> If there is protocol for exposeing format used by PyArg_Parse*, we can
>>> bypass temporal Python object and call myfunc_impl directly.

Note that this is not fast at all. It actually has to parse the format
description at runtime. For really fast calls, this should be avoided, and
it can be avoided by using a str object for the signature description and
interning it. That relies on signature normalisation, which requires a
proper formal specification of C/C++ signature strings, which ... is pretty
much the can of worms that Antoine mentioned.


>> This is another can of worms to open.  If you're worried about the
>> added complexity of PEP 580, what you're proposing is one or two orders
>> of magnitude more complicated.
> 
> This is just an example of possible optimization, to explain why I want
> example application first.
> I know Cython is important for data scientists.  But I don't know how
> it used typically.
> 
> If my idea has 50% gain and current PEP 580 has only 5% gain,
> why we should accept PEP 580?

Because PEP 580 is also meant as a preparation for a fast C-to-C call
interface in Python.

Unpacking C callables is quite an involved protocol, and we have been
pushing the idea around and away in the Cython project for some seven
years. It's about time to consider it more seriously now, and there are
plans to implement it on top of PEP 580 (which is also mentioned in the PEP).


> And PEP 576 seems much simpler and straightforward way to expose
> FASTCALL.

Except that it does not get us one step forward on the path towards what
you proposed. So, why would *you* prefer it over PEP 580?


>>> I think optimization like this idea can boost application performance
>>> using Cython heavily.
>>
>> You can already define the C signature of a function in Cython and
>> intra-Cython calls will benefit from it where possible.  Cooperation
>> from core Python is not necessary for that.
> 
> Why not allow it for extensions written in C, without Cython?

It should be. They just need a simpler protocol, which is PEP 580.

Stefan

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Steven D'Aprano 
On Wed, Jul 11, 2018 at 05:14:34AM +0300, Ivan Pozdeev via Python-Dev wrote:
> On 11.07.2018 1:41, Victor Stinner wrote:
> >2018-07-09 18:01 GMT+02:00 Steve Dower :
> >>The difficulty is that they *definitely* can use the 32-bit version, and
> >>those few who are on older machines or older installs of Windows may not
> >>understand why the link we provide didn't work for them.

I think Steve's comment is right on the money.

Although professional programmers should be a bit more technically 
competent than the average user, many are just hobbiest programmers, or 
school kids who are just as clueless as the average user since they 
*are* average users.


> >Let's say that only 10% of users still use 32-bit version. If they
> >download a default 64-bit binary, I'm quite sure that running the
> >binary will emit an error no? Such users should be used to such error,
> >and be able to get the 64-bit version, no?

That's a lot of assumptions there. Here are a few things which might 
break those assumptions:

- inexperienced users who haven't installed much software;

- or who don't know that 32- versus 64-bit is a thing;

- conservative users who don't install much software aside
  from using their vendor's official packages;

- users who install from app stores which automatically detect
  the right version of the installer so they don't need to think
  about it (how do app stores handle this issue?);

- or those who expect the default installer to work by default, so long 
  as they pick the right OS (Windows or Mac).

I don't remember what CPU my PC has, and when I can't be bothered to 
look it up, I always go for the default install option expecting that it 
ought to work regardless of whether I have a 32- or 64-bit OS. So far 
that's a strategy that has never done me wrong :-)


> Attached the image of what happens. The message is:
> 
> "One or more issues caused the setup to fail. Please fix the issues and 
> the retry setup. For more information see the log file .
> 
> 0x80070661 - This installation package is not supported by this 
> processor type. Contact your product vendor."
> 
> Pretty descriptive in my book.

Are you being sarcastic?

I would expect that "this processor type" refers to incompatible chip 
sets like ARM versus Intel, not the 32- versus 64-bitness of the 
operating system.

And I certainly wouldn't associate the problem:

"I downloaded and ran the wrong installer"

with the appropriate solution:

"I need to hunt for a 32-bit installer, rather than using
the default"

given that error message.



-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Stefan Behnel 
INADA Naoki schrieb am 11.07.2018 um 02:01:
> On Wed, Jul 11, 2018 at 7:47 AM Victor Stinner wrote:
>> I proposed something simpler, but nobody tried to implement it.
>> Instead of calling the long and complex PyArg_Parse...() functions,
>> why not generating C code to parse arguments instead? The idea looks
>> like "inlining" PyArg_Parse...() in its caller, but technically it
>> means that Argument Clinic generates C code to parse arguments.
> 
> But I have worrying about it.  If we do it for all function, it makes Python
> binary fatter, consume more CPU cache.  Once CPU cache is start
> stashing, application performance got slower quickly.

Now, I'd like to see benchmark numbers for that before I believe it. Macro
benchmarks, not micro benchmarks! *wink*

Stefan

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Stephen J. Turnbull 
Ivan Pozdeev via Python-Dev writes:

 > "One or more issues caused the setup to fail. Please fix the issues and 
 > the retry setup. For more information see the log file .
 > 
 > 0x80070661 - This installation package is not supported by this 
 > processor type. Contact your product vendor."
 > 
 > Pretty descriptive in my book.

Experience shows that's definitely not descriptive enough for my
university's students (and starting from AY 2021 we're moving to
Python 3 as the university-wide programming course language, yay!)
They have no idea that "processor type" means "word size", or what
alternative package to look for.  Sometimes they take the "contact
vendor" wording to mean "package is broken".  I don't think the
Japanese or Chinese students will have 32-bit machines (haven't seen
one among my advisees since March 2016), but we do get some students
from less wealthy countries who may be using older machines.

So I think it would be really nice if the installer detects the
wordsize mismatch, and issues an explicit message like

This package is intended for a 64-it machine, but yours is a 32-bit
machine.

Please download and install the package specifically for 32-bit
machines instead.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Glenn Linderman 

On 7/10/2018 11:14 PM, Stephen J. Turnbull wrote:

Ivan Pozdeev via Python-Dev writes:

  > "One or more issues caused the setup to fail. Please fix the issues and
  > the retry setup. For more information see the log file .
  >
  > 0x80070661 - This installation package is not supported by this
  > processor type. Contact your product vendor."
  >
  > Pretty descriptive in my book.

Experience shows that's definitely not descriptive enough for my
university's students (and starting from AY 2021 we're moving to
Python 3 as the university-wide programming course language, yay!)
They have no idea that "processor type" means "word size", or what
alternative package to look for.  Sometimes they take the "contact
vendor" wording to mean "package is broken".  I don't think the
Japanese or Chinese students will have 32-bit machines (haven't seen
one among my advisees since March 2016), but we do get some students
from less wealthy countries who may be using older machines.

So I think it would be really nice if the installer detects the
wordsize mismatch, and issues an explicit message like

 This package is intended for a 64-it machine, but yours is a 32-bit
 machine.

 Please download and install the package specifically for 32-bit
 machines instead.
Which would be far, far better, regardless of which bitness(es) of 
installer is(are) displayed (prominently) on the web site.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com