Re: [Python-Dev] MAKE_FUNCTION simplification

[Ethan Furman]

On 04/14/2016 10:02 AM, Random832 wrote:


"between versions" is ambiguous. It could mean that there's no guarantee
that there will be no changes from one version to the next, or it could
mean, even more strongly, that there's no guarantee that there will be
no changes in a maintenance release (which are, after all, released
*between* minor releases)


I don't see us making a breaking change in a maintenance release except 
to fix something that was already broken.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Random832]

On Thu, Apr 14, 2016, at 12:56, Terry Reedy wrote:
> https://docs.python.org/3/library/dis.html#module-dis
> CPython implementation detail: Bytecode is an implementation detail of 
> the CPython interpreter. No guarantees are made that bytecode will not 
> be added, removed, or changed between versions of Python.
> 
> Version = minor release, as opposed to maintenance release.

"between versions" is ambiguous. It could mean that there's no guarantee
that there will be no changes from one version to the next, or it could
mean, even more strongly, that there's no guarantee that there will be
no changes in a maintenance release (which are, after all, released
*between* minor releases)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Terry Reedy]

On 4/14/2016 12:03 PM, Nikita Nemkin wrote:


I think that Python should make bytecode explicitly unstable and subject
to change with any major release.


https://docs.python.org/3/library/dis.html#module-dis
CPython implementation detail: Bytecode is an implementation detail of 
the CPython interpreter. No guarantees are made that bytecode will not 
be added, removed, or changed between versions of Python.


Version = minor release, as opposed to maintenance release.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Brett Cannon]

On Thu, 14 Apr 2016 at 09:16 Nikita Nemkin  wrote:

> On Thu, Apr 14, 2016 at 8:32 PM, Victor Stinner
>  wrote:
> >
> > Would you like to work on a patch to implement that change?
>
> I'll work on a patch. Should I post it to bugs.python.org?
>

Yep.


>
> > Since Python 3.6 may get a new bytecode format  (wordcode, see the
> > other thread on this mlailing list), I think that it's ok to change
> > MAKE_FUNCTION in the same release.
>
> Wordcode looks like pure win from (projected) 25% bytecode size
> reduction alone.
>

CPU performance is more the worry here (which looks mostly unaffected,
maybe even faster), but reduced .pyc files is a nice perk. :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Nikita Nemkin]

On Thu, Apr 14, 2016 at 8:27 PM, Guido van Rossum  wrote:
> Great analysis! What might stand in the way of adoption is concern for
> bytecode manipulation libraries that would have to be changed. What
> might encourage adoption would be a benchmark showing this saves a lot
> of time.
>
> Personally I'm expecting it won't make much of a difference for real
> programs since almost always the cost of creating the function is
> dwarfed by the (total) cost of running it. But Python does create a
> lot of functions, and there's also lambdas.

This change alone is very unlikely to have a measurable performance impact.
The intention is to clean up ceval.c/compile.c a bit, nothing more.
If many other opcodes were somehow slimmed down in the similar fashion,
then we might (or might not) see perf gains.

For example, most slot dispatch opcodes can be compressed into a single
opcode+slot index with inlined dispatch logic, instead of each one individually
calling C API functions...

> There's also talk of switching to wordcode, in a different thread.
> Maybe the idea would be easier to introduce there? (Bytecode libraries
> would have to change anyways, so the additional concern for this
> change would be minimal.)

Wordcode can benefit from this change, because it guarantees
single-byte MAKE_FUNCTION oparg.

I think that Python should make bytecode explicitly unstable and subject
to change with any major release. The potential for a faster Python
interpreter (or simple JIT) is huge; requiring bytecode compatibility
will slow down any progress in this area.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Nikita Nemkin]

On Thu, Apr 14, 2016 at 8:32 PM, Victor Stinner
 wrote:
>
> Would you like to work on a patch to implement that change?

I'll work on a patch. Should I post it to bugs.python.org?

> Since Python 3.6 may get a new bytecode format  (wordcode, see the
> other thread on this mlailing list), I think that it's ok to change
> MAKE_FUNCTION in the same release.

Wordcode looks like pure win from (projected) 25% bytecode size
reduction alone.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Victor Stinner]

2016-04-14 17:27 GMT+02:00 Guido van Rossum :
> Great analysis! What might stand in the way of adoption is concern for
> bytecode manipulation libraries that would have to be changed.
> (...)
> There's also talk of switching to wordcode, in a different thread.

I agree that breaking backward compatibility just for MAKE_FUNCTION is
not worth. But if we accept the wordcode change, IMHO it's ok to take
this as an opportunity to also modify MAKE_FUNCTION.

> Maybe the idea would be easier to introduce there? (Bytecode libraries
> would have to change anyways, so the additional concern for this
> change would be minimal.)

Exactly ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Victor Stinner]

2016-04-14 11:04 GMT+02:00 Nikita Nemkin :
> MAKE_FUNCTION opcode is complex due to the way it receives
> input arguments: (...)

Yeah, I was always disturbed how this opcode gets parameters.

> My suggestion is to pre-package 1-4 before calling MAKE_FUNCTION,
> i.e. explicitly emit BUILD_TUPLE for defaults args and BUILD_MAPs
> for keyword defaults and annotations.

I read the code. I fact, I don't understand why it wasn't done like
that since the beginning :-p

> Then, MAKE_FUNCTION will become a dramatically simpler
> 5 argument opcode, taking

Would you like to work on a patch to implement that change?

Since Python 3.6 may get a new bytecode format  (wordcode, see the
other thread on this mlailing list), I think that it's ok to change
MAKE_FUNCTION in the same release.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Guido van Rossum]

Great analysis! What might stand in the way of adoption is concern for
bytecode manipulation libraries that would have to be changed. What
might encourage adoption would be a benchmark showing this saves a lot
of time.

Personally I'm expecting it won't make much of a difference for real
programs since almost always the cost of creating the function is
dwarfed by the (total) cost of running it. But Python does create a
lot of functions, and there's also lambdas.

There's also talk of switching to wordcode, in a different thread.
Maybe the idea would be easier to introduce there? (Bytecode libraries
would have to change anyways, so the additional concern for this
change would be minimal.)

On Thu, Apr 14, 2016 at 2:04 AM, Nikita Nemkin  wrote:
> MAKE_FUNCTION opcode is complex due to the way it receives
> input arguments:
>
>  1) default args, individually;
>  2) default kwonly args, individual name-value pairs;
>  3) a tuple of parameter names (single constant);
>  4) annotation values, individually;
>  5) code object;
>  6) qualname.
>
> The counts for 1,2,4 are packed into oparg bitfields, making oparg large.
>
> My suggestion is to pre-package 1-4 before calling MAKE_FUNCTION,
> i.e. explicitly emit BUILD_TUPLE for defaults args and BUILD_MAPs
> for keyword defaults and annotations.
>
> Then, MAKE_FUNCTION will become a dramatically simpler
> 5 argument opcode, taking
>
>  1) default args tuple (optional);
>  2) default keyword only args dict (optional);
>  3) annotations dict (optional);
>  4) code object;
>  5) qualname.
>
> These arguments correspond exactly to __annotations__, __kwdefaults__,
> __defaults__, __code__ and __qualname__ attributes.
>
> For optional args, oparg bits should indicate individual arg presence.
> (This also saves None checks in opcode implementation.)
>
> If we add another optional argument (and oparg bit) for __closure__
> attribute, then separate MAKE_CLOSURE opcode becomes unnecessary.
>
> Default args tuple is likely to be a constant and can be packaged whole,
> compensating for the extra size of explicit BUILD_* instructions.
>
> Compare the current implementation:
>
> https://github.com/python/cpython/blob/master/Python/ceval.c#L3262
>
> with this provisional implementation (untested):
>
> TARGET(MAKE_FUNCTION) {
> PyObject *qualname = POP();
> PyObject *codeobj = POP();
> PyFunctionObject *func;
> func = (PyFunctionObject *)PyFunction_NewWithQualName(
>codeobj, f->f_globals, qualname);
> Py_DECREF(codeobj);
> Py_DECREF(qualname);
> if (func == NULL)
> goto error;
>
> /* NB: Py_None is not an acceptable value for these. */
> if (oparg & 0x08)
> func->func_closure = POP();
> if (oparg & 0x04)
> func->func_annotations = POP();
> if (oparg & 0x02)
> func->func_kwdefaults = POP();
> if (oparg & 0x01)
> func->func_defaults = POP();
>
> PUSH((PyObject *)func);
> DISPATCH();
> }
>
> compile.c also gets a bit simpler, but not much.
>
> What do you think?
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] MAKE_FUNCTION simplification

[Nikita Nemkin]

MAKE_FUNCTION opcode is complex due to the way it receives
input arguments:

 1) default args, individually;
 2) default kwonly args, individual name-value pairs;
 3) a tuple of parameter names (single constant);
 4) annotation values, individually;
 5) code object;
 6) qualname.

The counts for 1,2,4 are packed into oparg bitfields, making oparg large.

My suggestion is to pre-package 1-4 before calling MAKE_FUNCTION,
i.e. explicitly emit BUILD_TUPLE for defaults args and BUILD_MAPs
for keyword defaults and annotations.

Then, MAKE_FUNCTION will become a dramatically simpler
5 argument opcode, taking

 1) default args tuple (optional);
 2) default keyword only args dict (optional);
 3) annotations dict (optional);
 4) code object;
 5) qualname.

These arguments correspond exactly to __annotations__, __kwdefaults__,
__defaults__, __code__ and __qualname__ attributes.

For optional args, oparg bits should indicate individual arg presence.
(This also saves None checks in opcode implementation.)

If we add another optional argument (and oparg bit) for __closure__
attribute, then separate MAKE_CLOSURE opcode becomes unnecessary.

Default args tuple is likely to be a constant and can be packaged whole,
compensating for the extra size of explicit BUILD_* instructions.

Compare the current implementation:

https://github.com/python/cpython/blob/master/Python/ceval.c#L3262

with this provisional implementation (untested):

TARGET(MAKE_FUNCTION) {
PyObject *qualname = POP();
PyObject *codeobj = POP();
PyFunctionObject *func;
func = (PyFunctionObject *)PyFunction_NewWithQualName(
   codeobj, f->f_globals, qualname);
Py_DECREF(codeobj);
Py_DECREF(qualname);
if (func == NULL)
goto error;

/* NB: Py_None is not an acceptable value for these. */
if (oparg & 0x08)
func->func_closure = POP();
if (oparg & 0x04)
func->func_annotations = POP();
if (oparg & 0x02)
func->func_kwdefaults = POP();
if (oparg & 0x01)
func->func_defaults = POP();

PUSH((PyObject *)func);
DISPATCH();
}

compile.c also gets a bit simpler, but not much.

What do you think?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com