subject:"\[Python\-Dev\] FAT Python \(lack of\) performance"

On Jan 25, 2016, at 19:32, INADA Naoki  wrote:
> 
>> On Tue, Jan 26, 2016 at 12:02 PM, Andrew Barnert  wrote:
>> On Jan 25, 2016, at 18:21, INADA Naoki  wrote:
>> >
>> > I'm very interested in it.
>> >
>> > Ruby 2.2 and PHP 7 are faster than Python 2.
>> > Python 3 is slower than Python 2.
>> 
>> Says who?
> 
> For example, http://benchmarksgame.alioth.debian.org/u64q/php.html
> In Japanese, many people compares language performance by microbench like 
> fibbonacci.

"In Japan, the hand is sharper than a knife [man splits board with karate 
chop], but the same doesn't work with a tomato [man splatters tomato all over 
himself with karate chop]."

A cheap knife really is better than a karate master at chopping tomatoes. And 
Python 2 really is better than Python 3 at doing integer arithmetic on the edge 
of what can fit into a machine word. But so what? Without seeing any of your 
Japanese web code, much less running a profiler, I'm willing to bet that your 
code is rarely CPU-bound, and, when it is, it spends a lot more time doing 
things like processing Unicode strings that are almost always UCS-2 (about 110% 
slower on Python 2) than doing this kind of arithmetic (9% faster on Python 2), 
or cutting tomatoes (TypeError on both versions).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

Do you say I and many people are so fool?
People use same algorithm on every language when compares base language
performance [1].

[1] There are no solid definition about "Base language performance".
   But it includes function call, method lookup, GC.  It may include basic
string and arithmetic operations.


See here for example:
http://d.hatena.ne.jp/satosystems/20121228/1356655565

This article is written in 2012.
In this article, php 5.3 takes 85sec, Python 2.7 takes 53sec and CRuby 1.8
takes 213sec. (!!)

For now:

$ python2 -V
Python 2.7.11
$ time python2 -S fib.py
39088169

real 0m17.133s
user 0m16.970s
sys 0m0.055s

$ python3 -V
Python 3.5.1
$ time python3 -S fib.py
39088169

real 0m21.380s
user 0m21.337s
sys 0m0.028s

$ php -v
PHP 7.0.2 (cli) (built: Jan  7 2016 10:40:21) ( NTS )
Copyright (c) 1997-2015 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2015 Zend Technologies
$ time php fib.php
39088169

real 0m7.706s
user 0m7.654s
sys 0m0.027s

$ ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin14]
$ time ruby fib.rb
39088169

real 0m6.195s
user 0m6.124s
sys 0m0.032s


Fibonacci microbench measures performance of function call.
When I said "Base language performance", I meant performance of
function call, attribute lookup, GC, etc...

PHP and Ruby made grate effort to improve base language performance.
While I'm fan of Python,  I respect people made PHP and Ruby faster.

Of course, I respect people making Python faster too.
But I wonder if CPython is more faster, especially about global lookup and
function call.

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Stephen J. Turnbull

INADA Naoki writes:

 > For example, http://benchmarksgame.alioth.debian.org/u64q/php.html
 > In Japanese, many people compares language performance by
 > microbench like fibbonacci.

True enough.  But as a teacher in a Japanese engineering school, I am
ashamed to see that posted to a public list.  We Japanese ;-) need to
get over that tendency (and not just in programming, my own biggest
battle is against those who worship goodness-of-fit statistics).  Our
universities are doing an awful job at getting "big picture thinking"
across to our students.

I see that you respond that you're talking about "base language
performance" which is a big improvement over blind faith in micro-
benchmarks, but still it's a matter of pulling your nose back from the
bark so you can see the tree.  At least from the point of view of the
computations the people around me are doing with these languages, you
still miss the forest.

 > While performance is not a matter when choosing first language,
 > slowest of three makes bad impression and people feel less
 > attractive about Python.

My school is the one that Matz graduated from (different department),
and as far as I can see (which is pretty local, I admit), Python is a
clear first and growing as a teaching language and as a glue language
for research computations.  Java is second for both[1], Ruby is rare,
and PHP just isn't on the map.  I see some lab home pages using
Ruby-on-Rails, and a couple of Mediawiki installations and the like
based on PHP.

For me personally, PHP and Ruby are both unusable for *performance*
reasons: I need something like Pandas, which is based on NumPy, and at
least the last time I checked a couple years ago Ruby had no
equivalent to NumPy, let alone Pandas.  (For PHP I asked the nearest
PHP fanboy web programmer, so I could be way wrong about no efficient
computation for PHP.)

At least as used at this university, I just don't see a performance
issue that Python needs to be specifically worried about.

That said, optimization is important, and IMO anything that allows 1%
of people currently writing (and debugging ...) modules in C for
performance reasons to write them in pure Python instead is a win.
But to me, relieving programmer pain, not microbenchmarks, should be
the performance measure we care about (even if others do care about
those benchmarks).  Measure them, yes, but focus on them only when
we're pretty sure that poor benchmark numbers are associated with
programmer pain.

Footnotes: 
[1]  Java was clearly first as a teaching language until 2010 or so, and
since most of our grad students and assistant profs were students here
in that era, I suspect its persistence is just the inertia that goes
with academic nepotism.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Chris Angelico

On Tue, Jan 26, 2016 at 2:32 PM, INADA Naoki  wrote:
>
> I know.
> But people compares language speed by simple microbench like fibbonacci.
> They doesn't use listcomp or libraries to compare *language* speed.
>

Well, that's a stupid way to decide on a language. Here, look: Python
is faster than C. Proof!

rosuav@sikorsky:~$ time python3 fib.py
2880067194370816120

real 0m0.033s
user 0m0.032s
sys 0m0.000s
rosuav@sikorsky:~$ cat fib.py
import functools

@functools.lru_cache()
def fib(n):
if n < 2: return n
return fib(n-2) + fib(n-1)

print(fib(90))

rosuav@sikorsky:~$ gcc fib.c && time ./a.out
1134903170

real 0m9.104s
user 0m9.064s
sys 0m0.000s
rosuav@sikorsky:~$ cat fib.c
#include 

unsigned long fib(unsigned long n)
{
if (n < 2) return n;
return fib(n-2) + fib(n-1);
}

int main()
{
printf("%lu\n",fib(45));
}

Algorithmic differences - even subtle ones - can easily outdo choice
of language for run-time performance. And if you try to write a true C
equivalent of that Python code, good luck - I'll have the Python one
written and running while you're still trying to figure out how to
write a cache, much less how to keep the syntax clean as you add a
cache to an existing function.

Of course, rewriting the whole thing to work iteratively instead of
double-recursively will make a dramatic difference to both programs.
That's an unsubtle algorithmic difference, and if you're coding like
that, you probably can't see the difference in performance between any
two languages at anything up to a machine word (about the 90th or so
Fibonacci number, on a 64-bit system) - all you'll see is the startup
performance. As soon as you go beyond a machine word, Python massively
trumps C, because its default integer type is a bignum. Going beyond a
machine word in C is a hassle. Going beyond a machine word in Python 2
is almost insignificant - hey look, now your repr has an 'L' on the
end, and performance is immeasurably worse. In Python 3, there's no
machine word whatsoever.

So, yeah... Python beats C for Fibonacci calculation, too. You just
have to not be stupid with your benchmarking.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

On Tue, Jan 26, 2016 at 12:02 PM, Andrew Barnert  wrote:

> On Jan 25, 2016, at 18:21, INADA Naoki  wrote:
> >
> > I'm very interested in it.
> >
> > Ruby 2.2 and PHP 7 are faster than Python 2.
> > Python 3 is slower than Python 2.
>
> Says who?
>

For example, http://benchmarksgame.alioth.debian.org/u64q/php.html
In Japanese, many people compares language performance by microbench like
fibbonacci.

>
> That was certainly true in the 3.2 days, but nowadays, most things that
> differ seem to be faster in 3.x.

Python is little faster than ever in these years.
But PHP and Ruby are much more faster than these years.

Matz announced Ruby 3x3. Ruby hackers will make more effort to optimize
ruby.
http://engineering.appfolio.com/appfolio-engineering/2015/11/18/ruby-3x3

> Maybe it's just the kinds of programs I write, but speedup in decoding
> UTF-8 that's usually ASCII (and then processing the decoded unicode when
> it's usually 1/4th the size), faster listcomps, and faster datetime seem to
> matter more than slower logging or slower imports. And that's just when
> running the same code; when you actually use new features, yield from is
> much faster than looping over yield; scandir blows away listdir; asyncio
> blows away asyncore or threading even harder; etc.
>

I know.
But people compares language speed by simple microbench like fibbonacci.
They doesn't use listcomp or libraries to compare *language* speed.

> Maybe if you do different things, you have a different experience. But if
> you have a specific problem, you'd do a lot better to file specific bugs
> for that problem than to just hope that everything magically gets so much
> faster that your bottleneck no longer matters.
>

I did it sometimes.
But I'd like to base language performance like function call more faster.

>
> > Performance is a attractive feature.  Python 3 lacks it.
>
> When performance matters, people don't use Python 2, Ruby, or PHP, any
> more than they use Python 3. Or, rather, they use _any_ of those languages
> for the 95% of their code that doesn't matter, and C (often through
> existing libraries like NumPy--and try to find a good equivalent of that
> for Ruby or PHP) for the 5% that does.

In the case of Web devs, many people choose main language from PHP, Ruby
and Python.
When peformance matters, they choose sub language from node.js, Go and
Scala.

While performance is not a matter when choosing first language, slowest of
three makes bad impression
and people feel less attractive about Python.

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Ryan Gonzalez



On January 25, 2016 9:32:07 PM CST, INADA Naoki  wrote:
>On Tue, Jan 26, 2016 at 12:02 PM, Andrew Barnert 
>wrote:
>
>> On Jan 25, 2016, at 18:21, INADA Naoki 
>wrote:
>> >
>> > I'm very interested in it.
>> >
>> > Ruby 2.2 and PHP 7 are faster than Python 2.
>> > Python 3 is slower than Python 2.
>>
>> Says who?
>>
>
>For example, http://benchmarksgame.alioth.debian.org/u64q/php.html
>In Japanese, many people compares language performance by microbench
>like
>fibbonacci.
>

...does writing Fibonacci in a foreign language make a performance difference? 
Or did you mean "In Japan?"

>
>>
>> That was certainly true in the 3.2 days, but nowadays, most things
>that
>> differ seem to be faster in 3.x.
>
>
>Python is little faster than ever in these years.
>But PHP and Ruby are much more faster than these years.
>
>Matz announced Ruby 3x3. Ruby hackers will make more effort to optimize
>ruby.
>http://engineering.appfolio.com/appfolio-engineering/2015/11/18/ruby-3x3
>
>
>
>> Maybe it's just the kinds of programs I write, but speedup in
>decoding
>> UTF-8 that's usually ASCII (and then processing the decoded unicode
>when
>> it's usually 1/4th the size), faster listcomps, and faster datetime
>seem to
>> matter more than slower logging or slower imports. And that's just
>when
>> running the same code; when you actually use new features, yield from
>is
>> much faster than looping over yield; scandir blows away listdir;
>asyncio
>> blows away asyncore or threading even harder; etc.
>>
>
>I know.
>But people compares language speed by simple microbench like
>fibbonacci.
>They doesn't use listcomp or libraries to compare *language* speed.
>
>
>> Maybe if you do different things, you have a different experience.
>But if
>> you have a specific problem, you'd do a lot better to file specific
>bugs
>> for that problem than to just hope that everything magically gets so
>much
>> faster that your bottleneck no longer matters.
>>
>
>I did it sometimes.
>But I'd like to base language performance like function call more
>faster.
>
>
>>
>> > Performance is a attractive feature.  Python 3 lacks it.
>>
>> When performance matters, people don't use Python 2, Ruby, or PHP,
>any
>> more than they use Python 3. Or, rather, they use _any_ of those
>languages
>> for the 95% of their code that doesn't matter, and C (often through
>> existing libraries like NumPy--and try to find a good equivalent of
>that
>> for Ruby or PHP) for the 5% that does.
>
>
>In the case of Web devs, many people choose main language from PHP,
>Ruby
>and Python.
>When peformance matters, they choose sub language from node.js, Go and
>Scala.
>
>While performance is not a matter when choosing first language, slowest
>of
>three makes bad impression
>and people feel less attractive about Python.

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

On Tue, Jan 26, 2016 at 2:44 PM, Andrew Barnert  wrote:

> On Jan 25, 2016, at 19:32, INADA Naoki  wrote:
>
> On Tue, Jan 26, 2016 at 12:02 PM, Andrew Barnert 
> wrote:
>
>> On Jan 25, 2016, at 18:21, INADA Naoki  wrote:
>> >
>> > I'm very interested in it.
>> >
>> > Ruby 2.2 and PHP 7 are faster than Python 2.
>> > Python 3 is slower than Python 2.
>>
>> Says who?
>>
>
> For example, http://benchmarksgame.alioth.debian.org/u64q/php.html
> In Japanese, many people compares language performance by microbench like
> fibbonacci.
>
>
> "In Japan, the hand is sharper than a knife [man splits board with karate
> chop], but the same doesn't work with a tomato [man splatters tomato all
> over himself with karate chop]."
>
> A cheap knife really is better than a karate master at chopping tomatoes.
> And Python 2 really is better than Python 3 at doing integer arithmetic on
> the edge of what can fit into a machine word. But so what? Without seeing
> any of your Japanese web code, much less running a profiler, I'm willing to
> bet that your code is rarely CPU-bound, and, when it is, it spends a lot
> more time doing things like processing Unicode strings that are almost
> always UCS-2 (about 110% slower on Python 2) than doing this kind of
> arithmetic (9% faster on Python 2), or cutting tomatoes (TypeError on both
> versions).
>
>
Calm down, please.
I didn't say "microbench is more important than macrobench".

While editor is not a main problem of software development, people likes
comparing vim and emacs.
Like that, Japanese dev people likes comparing speed.

While it's not a real problem of typical application, new people should
choose first (and probably main)
editor and language.
Slowest on such a basic microbench gives bad impression for them.

Additionally, some application (e.g. traversing DOM) makes much function
calls.
Faster function call may makes some *real* application faster.

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] FAT Python (lack of) performance

Hi,

Summary: FAT Python is not faster, but it will be ;-)

--

When I started the FAT Python as a fork of CPython 3.6, I put
everything in the same repository. Last weeks, I focused on splitting
my giant patch (10k lines) into small reviewable patches. I wrote 3
PEP (509 dict version, 510 function specialziation, 511 code
tranformers) and I enhanced the API to make it usable for more use
cases than just FAT Python. I also created fatoptimizer (the AST
optimizer) and fat (runtime dependency of the optimizer) projects on
GitHub to separate clearly what should be outside Python core. For all
links, see:

   http://faster-cpython.readthedocs.org/fat_python.html

For the fatoptimizer project, my constraint is to be able to run the
full Python test suite unmodified. In practice, I have to disable some
optimizations by putting a "__fatoptimizer__= {...}" configuration to
some test files. For example, I have to disable constant folding on
test_bool because it tests that False+2 gives 2 at runtime, whereas
the optimizer replaces directly False+2 with 2 during the compilation.
Well, test_bool.py is not the best example because all tests pass with
the constant folding optimization (if I comment my
"__fatoptimizer__={...}" change).

This constraint ensures that the optimizer "works" and doesn't break
(too much ;-)) the Python semantics, but it's more difficult to
implement powerful optimizations.

I also found and fixed various kinds of bugs. In my code obviously,
but also in the Python core, in various places. Some bugs only concern
AST transformers which is a new feature, but I had to fix them. For
example, Python didn't support negative line number delta in
co_lntotab of code objects, and so line number were all wrong on
optimized code. I merged my enhancement in the default branch of
CPython (issue #26107).

In short, I focused on having something working (respecting the Python
semantics), rather than spending time on writing optimizations.

--

When I asked explicitly "Is someone opposed to this PEP 509 [dict
verion] ?", Barry Warsaw answered that a performance analysis is
required. Extract of his mail:

   "I still think this is maintenance and potential performance
overhead we don't want to commit to long term unless it enables
significant optimization.  Since you probably can't prove that without
some experimentation, this API should be provisional."

Last week, I ran some benchmarks and I have to admin that I was
disappointed. Not only fatoptimizer doesn't make Python faster, but it
makes it much slower on some tests!

   http://fatoptimizer.readthedocs.org/en/latest/benchmarks.html

Quickly, I identified a major performance issue when nested functions
are specialized, especially in Lib/json/encoder.py (tested by
bm_json_v2.py benchmark). I fixed my optimizer to not specialize
nested functions anymore. This simple change fixed the main
performance issue. Reminder: in performance critical code, don't use
nested functions! I will maybe propose patches for Lib/json/encoder.py
to stop using nested functions.

I only ran benchmarks with the optimizer enabled. I now have to
measure the overhead of my patches (PEP 509, 510 and 511) adding the
API fat AST optimizers. The overhead must be negligible. For me, it's
a requirement of the whole project. Changes must not make Python
slower when the optimizer is not used.

fatoptimizer is faster on microbenchmarks, but I had to write manually
some optimizations:

   http://fatoptimizer.readthedocs.org/en/latest/microbenchmarks.html

IMHO fatoptimizer is not faster on macro benchmarks because it is not
smart enough (yet) to generate the most interesting optimizations,
like function inlining and specialization for argument types. You can
estimate the speedup if you specialize manually your functions.

--

Barry also wrote: "Did you address my suggestion on python-ideas to
make the new C API optionally compiled in?"

Well, it is an option, but I would prefer to have the API for AST
optimizer directly built in Python.

The first beta version of Python 3.6 is scheduled in September 2016
(deadline for new features in Python 3.6), so I still have a few
months to implement more powerful optimizations and prove that it can
be faster ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Sven R. Kunze

Hi Victor,

I encourage you to proceed here. I would love to see your PEPs (509-511)
incorporated into CPython. It's not that I consider Python slow
(although some folks claim so), but performance improvements are always
welcome; especially when I glance over diagrams like those:
http://blog.carlesmateo.com/wp-content/uploads/2014/10/blog-carlesmateo-com-performance-several-languages-php7-phantomjs-nodejs-java-bash-go-perl-luajit-hhvm3_9-scale_mod5.png

So, I join Barry when he says, we want more benchmarking and definite
results, however, I might be less strict than he is and say:

- even if FAT might not optimize significantly (whatever definition we
apply), PEP 509 to 511 are a great win for CPython
- they provide a great infrastructure for optimizing CPython AND
extending/experimenting Python as an ecosystem
- FAT provides interesting insights into the field of optimizing a
dynamic language

So, keep up the good work. I am eager to see where this goes.

If there's anything I can do, let me know. :)

Best,
Sven

On 25.01.2016 19:16, Victor Stinner wrote:

Hi,

Summary: FAT Python is not faster, but it will be ;-)

When I started the FAT Python as a fork of CPython 3.6, I put
everything in the same repository. Last weeks, I focused on splitting
my giant patch (10k lines) into small reviewable patches. I wrote 3
PEP (509 dict version, 510 function specialziation, 511 code
tranformers) and I enhanced the API to make it usable for more use
cases than just FAT Python. I also created fatoptimizer (the AST
optimizer) and fat (runtime dependency of the optimizer) projects on
GitHub to separate clearly what should be outside Python core. For all
links, see:

http://faster-cpython.readthedocs.org/fat_python.html

For the fatoptimizer project, my constraint is to be able to run the
full Python test suite unmodified. In practice, I have to disable some
optimizations by putting a "__fatoptimizer__= {...}" configuration to
some test files. For example, I have to disable constant folding on
test_bool because it tests that False+2 gives 2 at runtime, whereas
the optimizer replaces directly False+2 with 2 during the compilation.
Well, test_bool.py is not the best example because all tests pass with
the constant folding optimization (if I comment my
"__fatoptimizer__={...}" change).

This constraint ensures that the optimizer "works" and doesn't break
(too much ;-)) the Python semantics, but it's more difficult to
implement powerful optimizations.

I also found and fixed various kinds of bugs. In my code obviously,
but also in the Python core, in various places. Some bugs only concern
AST transformers which is a new feature, but I had to fix them. For
example, Python didn't support negative line number delta in
co_lntotab of code objects, and so line number were all wrong on
optimized code. I merged my enhancement in the default branch of
CPython (issue #26107).

In short, I focused on having something working (respecting the Python
semantics), rather than spending time on writing optimizations.

When I asked explicitly "Is someone opposed to this PEP 509 [dict
verion] ?", Barry Warsaw answered that a performance analysis is
required. Extract of his mail:

"I still think this is maintenance and potential performance
overhead we don't want to commit to long term unless it enables
significant optimization. Since you probably can't prove that without
some experimentation, this API should be provisional."

Last week, I ran some benchmarks and I have to admin that I was
disappointed. Not only fatoptimizer doesn't make Python faster, but it
makes it much slower on some tests!

http://fatoptimizer.readthedocs.org/en/latest/benchmarks.html

Quickly, I identified a major performance issue when nested functions
are specialized, especially in Lib/json/encoder.py (tested by
bm_json_v2.py benchmark). I fixed my optimizer to not specialize
nested functions anymore. This simple change fixed the main
performance issue. Reminder: in performance critical code, don't use
nested functions! I will maybe propose patches for Lib/json/encoder.py
to stop using nested functions.

I only ran benchmarks with the optimizer enabled. I now have to
measure the overhead of my patches (PEP 509, 510 and 511) adding the
API fat AST optimizers. The overhead must be negligible. For me, it's
a requirement of the whole project. Changes must not make Python
slower when the optimizer is not used.

fatoptimizer is faster on microbenchmarks, but I had to write manually
some optimizations:

http://fatoptimizer.readthedocs.org/en/latest/microbenchmarks.html

IMHO fatoptimizer is not faster on macro benchmarks because it is not
smart enough (yet) to generate the most interesting optimizations,
like function inlining and specialization for argument types. You can
estimate the speedup if you specialize manually your functions.

Barry also wrote: "Did you address my suggestion on python-ideas to
make the new C API optionally

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Ludovic Gasc

Hi,

Just thanks for this big contribution.
And maybe this project could give new ideas to optimize Python, who knows ?

At least, you've win a beer for the FOSDEM event this week-end ;-)

Have a nice week.


--
Ludovic Gasc (GMLudo)
http://www.gmludo.eu/

2016-01-25 19:16 GMT+01:00 Victor Stinner :

> Hi,
>
> Summary: FAT Python is not faster, but it will be ;-)
>
> --
>
> When I started the FAT Python as a fork of CPython 3.6, I put
> everything in the same repository. Last weeks, I focused on splitting
> my giant patch (10k lines) into small reviewable patches. I wrote 3
> PEP (509 dict version, 510 function specialziation, 511 code
> tranformers) and I enhanced the API to make it usable for more use
> cases than just FAT Python. I also created fatoptimizer (the AST
> optimizer) and fat (runtime dependency of the optimizer) projects on
> GitHub to separate clearly what should be outside Python core. For all
> links, see:
>
>http://faster-cpython.readthedocs.org/fat_python.html
>
> For the fatoptimizer project, my constraint is to be able to run the
> full Python test suite unmodified. In practice, I have to disable some
> optimizations by putting a "__fatoptimizer__= {...}" configuration to
> some test files. For example, I have to disable constant folding on
> test_bool because it tests that False+2 gives 2 at runtime, whereas
> the optimizer replaces directly False+2 with 2 during the compilation.
> Well, test_bool.py is not the best example because all tests pass with
> the constant folding optimization (if I comment my
> "__fatoptimizer__={...}" change).
>
> This constraint ensures that the optimizer "works" and doesn't break
> (too much ;-)) the Python semantics, but it's more difficult to
> implement powerful optimizations.
>
> I also found and fixed various kinds of bugs. In my code obviously,
> but also in the Python core, in various places. Some bugs only concern
> AST transformers which is a new feature, but I had to fix them. For
> example, Python didn't support negative line number delta in
> co_lntotab of code objects, and so line number were all wrong on
> optimized code. I merged my enhancement in the default branch of
> CPython (issue #26107).
>
> In short, I focused on having something working (respecting the Python
> semantics), rather than spending time on writing optimizations.
>
> --
>
> When I asked explicitly "Is someone opposed to this PEP 509 [dict
> verion] ?", Barry Warsaw answered that a performance analysis is
> required. Extract of his mail:
>
>"I still think this is maintenance and potential performance
> overhead we don't want to commit to long term unless it enables
> significant optimization.  Since you probably can't prove that without
> some experimentation, this API should be provisional."
>
> Last week, I ran some benchmarks and I have to admin that I was
> disappointed. Not only fatoptimizer doesn't make Python faster, but it
> makes it much slower on some tests!
>
>http://fatoptimizer.readthedocs.org/en/latest/benchmarks.html
>
> Quickly, I identified a major performance issue when nested functions
> are specialized, especially in Lib/json/encoder.py (tested by
> bm_json_v2.py benchmark). I fixed my optimizer to not specialize
> nested functions anymore. This simple change fixed the main
> performance issue. Reminder: in performance critical code, don't use
> nested functions! I will maybe propose patches for Lib/json/encoder.py
> to stop using nested functions.
>
> I only ran benchmarks with the optimizer enabled. I now have to
> measure the overhead of my patches (PEP 509, 510 and 511) adding the
> API fat AST optimizers. The overhead must be negligible. For me, it's
> a requirement of the whole project. Changes must not make Python
> slower when the optimizer is not used.
>
> fatoptimizer is faster on microbenchmarks, but I had to write manually
> some optimizations:
>
>http://fatoptimizer.readthedocs.org/en/latest/microbenchmarks.html
>
> IMHO fatoptimizer is not faster on macro benchmarks because it is not
> smart enough (yet) to generate the most interesting optimizations,
> like function inlining and specialization for argument types. You can
> estimate the speedup if you specialize manually your functions.
>
> --
>
> Barry also wrote: "Did you address my suggestion on python-ideas to
> make the new C API optionally compiled in?"
>
> Well, it is an option, but I would prefer to have the API for AST
> optimizer directly built in Python.
>
> The first beta version of Python 3.6 is scheduled in September 2016
> (deadline for new features in Python 3.6), so I still have a few
> months to implement more powerful optimizations and prove that it can
> be faster ;-)
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/gmludo%40gmail.com
>

Re: [Python-Dev] FAT Python (lack of) performance

Hi,

2016-01-25 22:20 GMT+01:00 Ludovic Gasc :
> Just thanks for this big contribution.
> And maybe this project could give new ideas to optimize Python, who knows ?

Sorry for my long email. I should try to summarize next time :-) In
short: FAT Python is not fast today, but it will be faster if you give
me a few more months to implement optimizations which will unlock the
real power of AST optimizers ;-)

I have a long list of ideas of optimizations:
https://fatoptimizer.readthedocs.org/en/latest/todo.html

According to microbenchmarks, the most promising optimizations are
functions inlining (Python function calls are slow :-/) and specialize
the code for the type of arguments.

And I agree to wait until fatoptimizer is proven to be faster than the
regular CPython before taking a decision of my 3 PEPs (509, 510, 511).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

On Jan 25, 2016, at 13:43, Victor Stinner  wrote:
> 
> According to microbenchmarks, the most promising optimizations are
> functions inlining (Python function calls are slow :-/) and specialize
> the code for the type of arguments.

Can you specialize a function with a C API function, or only with bytecode? I'm 
not sure how much benefit you'd get out of specializing list vs. generic 
iterable or int vs. whatever from an AST transform, but substituting raw C 
code, on the other hand...
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

On Jan 25, 2016, at 14:46, Victor Stinner  wrote:
> 
> You can design an AST optimizer to compile some functions to C and
> then register them as specialized code at runtime. I have a side
> project to use Cython and/or pythran to specialize some functions
> using type annotation on parameters.

That last part is exactly what I was thinking of. One way in which cythonizing 
your code isn't 100% compatible is that if you, say, shadow or replace int or 
range, the cython code is now wrong. Which is exactly the kind of thing FAT can 
guard against. Which is very cool. Glad to see you already thought of that 
before me. :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Barry Warsaw

On Jan 25, 2016, at 07:16 PM, Victor Stinner wrote:

>Barry also wrote: "Did you address my suggestion on python-ideas to
>make the new C API optionally compiled in?"
>
>Well, it is an option, but I would prefer to have the API for AST
>optimizer directly built in Python.

In my plan, it would be, but it would have to be enabled with a configure
switch, and provisionally protected by an #ifdef.

>And I agree to wait until fatoptimizer is proven to be faster than the
>regular CPython before taking a decision of my 3 PEPs (509, 510, 511).

+1 - and just to be clear, I hope you succeed beyond your wildest
imagination. :)

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Brett Cannon

On Mon, 25 Jan 2016 at 14:30 Andrew Barnert via Python-Dev <
python-dev@python.org> wrote:

> On Jan 25, 2016, at 13:43, Victor Stinner 
> wrote:
> >
> > According to microbenchmarks, the most promising optimizations are
> > functions inlining (Python function calls are slow :-/) and specialize
> > the code for the type of arguments.
>
> Can you specialize a function with a C API function, or only with
> bytecode? I'm not sure how much benefit you'd get out of specializing list
> vs. generic iterable or int vs. whatever from an AST transform, but
> substituting raw C code, on the other hand...
>

Victor's work is only manipulation of ASTs and bytecode. If you want
something that low-level you need to either reach for Cython or hope a
project like Pyjion pays off.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

Hi,

2016-01-25 23:28 GMT+01:00 Andrew Barnert :
> On Jan 25, 2016, at 13:43, Victor Stinner  wrote:
>>
>> According to microbenchmarks, the most promising optimizations are
>> functions inlining (Python function calls are slow :-/) and specialize
>> the code for the type of arguments.
>
> Can you specialize a function with a C API function, or only with bytecode? 
> I'm not sure how much benefit you'd get out of specializing list vs. generic 
> iterable or int vs. whatever from an AST transform, but substituting raw C 
> code, on the other hand...

As I wrote in the first part of my email, I redesigned to API to make
it more generic. One of my change was to change
PyFunction_Specialize() to not only accept code objects, but any
callable object.

The PEP 510 even contains an example using a builtin function as the
specialized code:
https://www.python.org/dev/peps/pep-0510/#using-builtin-function

"On a microbenchmark, calling the C builtin takes 95 ns, whereas the
original bytecode takes 155 ns (+60 ns): 1.6 times as fast. Calling
directly chr(65) takes 76 ns."

You can design an AST optimizer to compile some functions to C and
then register them as specialized code at runtime. I have a side
project to use Cython and/or pythran to specialize some functions
using type annotation on parameters.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 22:51 GMT+01:00 Sven R. Kunze :
> - they provide a great infrastructure for optimizing CPython AND
> extending/experimenting Python as an ecosystem

I hope that these API will create more optimizer projects than just
fatoptimizer.

For example, I expect more specialized optimizers like numba or
pythran which are very efficient but more specific (ex: numeric
computations) than fatoptimizer. Maybe not new optimizers, but just
glue to existing static compilers (numba, pythran, cython, etc.).


> If there's anything I can do, let me know. :)

Oh, they are a lot of things to do! My patches for PEP 509, 510 and
511 still need some love (reviews):

http://bugs.python.org/issue26058
http://bugs.python.org/issue26098
http://bugs.python.org/issue26145

I'm finishing my patch adding ast.Constant. This one is less
controversal, it has no impact on performance nor the Python
semantics:

http://bugs.python.org/issue26146


But these patches are boring C code. You may prefer to work on the
funny fatoptimizer project which is written in pure Python:

https://fatoptimizer.readthedocs.org/en/latest/

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

I'm very interested in it.

Ruby 2.2 and PHP 7 are faster than Python 2.
Python 3 is slower than Python 2.
Performance is a attractive feature.  Python 3 lacks it.

How can I help your work?

On Tue, Jan 26, 2016 at 7:58 AM, Victor Stinner 
wrote:

> 2016-01-25 22:51 GMT+01:00 Sven R. Kunze :
> > - they provide a great infrastructure for optimizing CPython AND
> > extending/experimenting Python as an ecosystem
>
> I hope that these API will create more optimizer projects than just
> fatoptimizer.
>
> For example, I expect more specialized optimizers like numba or
> pythran which are very efficient but more specific (ex: numeric
> computations) than fatoptimizer. Maybe not new optimizers, but just
> glue to existing static compilers (numba, pythran, cython, etc.).
>
>
> > If there's anything I can do, let me know. :)
>
> Oh, they are a lot of things to do! My patches for PEP 509, 510 and
> 511 still need some love (reviews):
>
> http://bugs.python.org/issue26058
> http://bugs.python.org/issue26098
> http://bugs.python.org/issue26145
>
> I'm finishing my patch adding ast.Constant. This one is less
> controversal, it has no impact on performance nor the Python
> semantics:
>
> http://bugs.python.org/issue26146
>
>
> But these patches are boring C code. You may prefer to work on the
> funny fatoptimizer project which is written in pure Python:
>
> https://fatoptimizer.readthedocs.org/en/latest/
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
>



-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance

2016-01-25 Thread Brett Cannon

On Mon, 25 Jan 2016 at 18:22 INADA Naoki  wrote:

> I'm very interested in it.
>
> Ruby 2.2 and PHP 7 are faster than Python 2.
> Python 3 is slower than Python 2.
> Performance is a attractive feature.  Python 3 lacks it.
>

That is not a fair statement to make about Python 3. It entirely depends on
your workload whether it is faster or slower.

https://gist.github.com/brettcannon/9d19cc184ea45b3e7ca0

-Brett


>
> How can I help your work?
>
> On Tue, Jan 26, 2016 at 7:58 AM, Victor Stinner 
> wrote:
>
>> 2016-01-25 22:51 GMT+01:00 Sven R. Kunze :
>> > - they provide a great infrastructure for optimizing CPython AND
>> > extending/experimenting Python as an ecosystem
>>
>> I hope that these API will create more optimizer projects than just
>> fatoptimizer.
>>
>> For example, I expect more specialized optimizers like numba or
>> pythran which are very efficient but more specific (ex: numeric
>> computations) than fatoptimizer. Maybe not new optimizers, but just
>> glue to existing static compilers (numba, pythran, cython, etc.).
>>
>>
>> > If there's anything I can do, let me know. :)
>>
>> Oh, they are a lot of things to do! My patches for PEP 509, 510 and
>> 511 still need some love (reviews):
>>
>> http://bugs.python.org/issue26058
>> http://bugs.python.org/issue26098
>> http://bugs.python.org/issue26145
>>
>> I'm finishing my patch adding ast.Constant. This one is less
>> controversal, it has no impact on performance nor the Python
>> semantics:
>>
>> http://bugs.python.org/issue26146
>>
>>
>> But these patches are boring C code. You may prefer to work on the
>> funny fatoptimizer project which is written in pure Python:
>>
>> https://fatoptimizer.readthedocs.org/en/latest/
>>
>> Victor
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>>
> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
>>
>
>
>
> --
> INADA Naoki  
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FAT Python (lack of) performance