[issue13607] Move generator specific sections out of ceval.

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> out of date
stage: patch review -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The WHY_* codes are gone in 3.8 (see issue17611). Exception state was moved 
from frame to generator in 3.7 (see issue25612).

Can some parts of the patch be applied to the current code? Or it is completely 
outdated?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
versions: +Python 3.4 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Mark Shannon]

Mark Shannon m...@hotpy.org added the comment:

Why do the codes used internally by ceval have to be the same as those made 
public?

Have you considered returning the exit code directly, rather than
adding another field to frameobject?

yieldfrom can be handled in a similar way to yield by returning the iterator 
(instead of the yielded value).

See attached file for example code.

--
nosy: +Mark.Shannon
Added file: http://bugs.python.org/file24509/exit_code_example.c

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Changes by Ron Adam ron3...@gmail.com:


Removed file: http://bugs.python.org/file24047/f_why1.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Ron Adam ron3...@gmail.com added the comment:

Updated patch with suggested changes.

It also has a cleaned up fast_block_end section.

Concerning speed. What happens is (as Tim and Raymond have pointed out) that we 
can make some things a little faster, in exchange for other things being a 
little slower.  You can play around with the order of the why cases in the 
fast_block_end section and see that effect.

By using a switch instead of if-else's, that should result in more consistent 
balance between the block exit cases.  The order I currently have gives a 
little more priority for exceptions and that seems to help a tiny bit with the 
ccbench scores.  I think that is a better bench mark than the small micro tests 
like pybench does.  The problem with pybench is, it doesn't test deeper nesting 
where these particular changes will have a greater effect.

--
Added file: http://bugs.python.org/file24087/f_why2.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Ron Adam ron3...@gmail.com added the comment:

I think the time benefits I saw are dependent on how the C code is compiled.  
So it may be different on different compilers or the same compiler with only a 
very minor change.

Some of the things I've noticed...

A switch is sometimes slower if it has a default: block.

Moving why = tstate-why_exit; to just before the if helps a tiny bit.

why = tstate-why_exit;
if (why != WHY_EXCEPTION) {

Returning directly out of the WHY_YIELD case.

case WHY_YIELD:
return result;

These will be mostly compiler dependent, but they won't slow things down any 
either.

What I was trying to do is clean up things a bit in ceval.c.  Having the why 
available on the frame can help by allowing some things to be moved out of 
ceval.c where it makes sense to do that.

I'll post a new diff tonight with the why_exit moved back to the frame object 
and the why's back to enums.  Yes, I think the frame object is a good place for 
it.

One of the odd issues is the opcode and why values sometimes need to be pushed 
on the value stack.  Which means it needs to be converted to a pyobject first, 
then converted back after it's pulled off the stack.  I'm looking for a way to 
avoid that.

I also was able to make fast_block_end section simpler and more understandable 
without making it slower.  I think it was trying to be too clever in order to 
save some lines of code.  That makes it very hard to figure out by someone else.

But first I need to finish my Christmas shopping, I'll post a new patch tonight 
when I get a chance. :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


--
nosy: +eric.snow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Meador Inge]

Changes by Meador Inge mead...@gmail.com:


--
nosy: +meador.inge

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Meador Inge]

Meador Inge mead...@gmail.com added the comment:

With the new patch I see no benefits on the same micro-benchmarks you posted 
(it is even slower for the smaller case) on a quad-core 64-bit F15 box:

BEFORE:

$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(10))
100 loops, best of 3: 1.33 usec per loop
$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(100))
10 loops, best of 3: 66 msec per loop

AFTER:

$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(10))
100 loops, best of 3: 1.45 usec per loop
$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(100))
10 loops, best of 3: 65.8 msec per loop

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Nick Coghlan]

Nick Coghlan ncogh...@gmail.com added the comment:

The thing that most appeals to me with this concept is moving closer to making 
it possible to experiment with generator-style functionality in *extension* 
modules (albeit extension modules that are coupled to private CPython APIs). 
So, for me, not worse than the status quo would be the main thing I'd be 
looking for out of any micro-benchmarks.

However, that also makes me question the movement of the why_exit from the 
frame to the tstate - having it on the thread state is significantly less 
flexible when it comes to experimenting with execution models.

The move from orthogonal bit flags in a dedicated enum to int fields and macro 
definitions also seems like a completely unnecessary pessimisation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Changes by Ron Adam ron3...@gmail.com:


Removed file: http://bugs.python.org/file23969/f_why.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Ron Adam ron3...@gmail.com added the comment:

New diff file.

The main difference is I moved the saved why value to the tstate object instead 
of the frame object as why_exit.

I'm not seeing the time savings now for some reason.  Maybe the previous 
increase was a case of coincidental noise. (?)  Still looking for other reasons 
though.  ;-)

Moving the generator code from the eval loop to the generator object may still 
be a good thing to do.

--
Added file: http://bugs.python.org/file24047/f_why1.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Jesús Cea Avión]

Changes by Jesús Cea Avión j...@jcea.es:


--
nosy: +jcea

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
nosy: +terry.reedy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Benjamin Peterson]

Benjamin Peterson benja...@python.org added the comment:

Seems mostly fine to me.

--
assignee:  - benjamin.peterson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

New submission from Ron Adam ron3...@gmail.com:

The following changes cleanup the eval loop and result in a pretty solid 2 to 
3% improvement in pybench for me.

And it is about 5% faster for long generators.

* Change why enum type to int and #defines.  And moved the why defines to 
opcode.h so that they can be seen by the genrator objects after a yield, 
return, or exception.

* Added an int f_why to frames so the generator can see why it returned from 
a send.

* Refactored generator obj so it can use the f-f_why to determine what to do 
without having to do several checks first.

* Moved the generator specific execption save/swap/and clear out of the cevel 
main loop.  No need to check for those on every function call.


The only test that fails is the frame size is test_sys.  I left that in for now 
so someone could check that, and tell me if it's ok to fix it, or if I need to 
do something else.

I also considered putting the why on the tstate object.  It might save some 
memory as there wouldn't be as many of those.

--
components: Interpreter Core
files: f_why.diff
keywords: patch
messages: 149583
nosy: ron_adam
priority: normal
severity: normal
status: open
title: Move generator specific sections out of ceval.
type: performance
versions: Python 3.3
Added file: http://bugs.python.org/file23969/f_why.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Ron Adam]

Ron Adam ron3...@gmail.com added the comment:

A simple test to show the difference.

BEFORE:

$ python3 -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(10))
10 loops, best of 3: 3.87 usec per loop
$ python3 -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(100))
10 loops, best of 3: 186 msec per loop

AFTER:
$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(10))
10 loops, best of 3: 3.81 usec per loop
$ ./python -mtimeit def y(n):   for x in range(n): yield  x 
sum(y(100))
10 loops, best of 3: 168 msec per loop

   before   after
y(10) usec's   3.873.81 - 1.55%   
y(100)msec's   186 168  - 9.67%

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13607] Move generator specific sections out of ceval.

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +benjamin.peterson, ncoghlan
stage:  - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13607
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com