Re: [Python-Dev] FAT Python (lack of) performance
On Mon, Jan 25, 2016 at 11:58:12PM +0100, Victor Stinner wrote: > ... > Oh, they are a lot of things to do! ... Just wondering, do you also need a set of (abusive) test-cases which check 100% conformity to the CPython semantics? I'm sure many of us would be able to whip up some ideas of things that are possible with Python and are of the kind "but you should not do that! That's bad programming!" which may or may not break the optimizations (especially specialized functions). I'm thinking of things like def override_length_function_test(): global len value = len("abc") len = lambda obj: ord(obj[0])) value += len("abc") assert value == 3 + 97, "Outdated `len` used." And also cases where `len` was changed not in the function itself, but in another function that was called indirectly (maybe 4 functions down, one of which was monkey-patched in after the compilation step): module_a.py def test_func(callback): value = len("abc") callback() value += len("abc") assert value == 3 + 97, "Outdated `len` used." module_b.py import module_a def override_length_function_test(): def callback(): module_a.len = lambda obj: ord(obj[0]) assert module_a.test_func(callback) == 3 + 97, "Outdated `len` used." (I'm sure some of the other readers of this list can be even more creative in trying to show that making optimizations like this can break semantics.) Other fun tricks I'd like to try is overriding the `len` method from a signal handler, what happens when you monkey-patch a dependent method, having `__getitem__` and `__getattr__` on some value override `len`. Basically: trying things that I normally should not try during my working hours, on account of wanting to still have a job the next day. Kind regards, Sjoerd Job ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Daily reference leaks (cbd4a6a2657e): sum=134
Looks like Victor's ast.Constant change introduced a refleak. On Tue, 26 Jan 2016 at 00:47 wrote: > results for cbd4a6a2657e on branch "default" > > > test_ast leaked [39, 39, 39] references, sum=117 > test_ast leaked [5, 5, 5] memory blocks, sum=15 > test_collections leaked [-2, 0, 0] references, sum=-2 > test_functools leaked [0, 2, 2] memory blocks, sum=4 > > > Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', > '3:3:/home/psf-users/antoine/refleaks/reflogqIZGVY', '--timeout', '7200'] > ___ > Python-checkins mailing list > python-check...@python.org > https://mail.python.org/mailman/listinfo/python-checkins > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Hoping to Find a Mentor
Dear Everyone, My name is Truong Nguyen and I like web development. I'm writing hoping to find a mentor (who likes to teach), and opportunity to contribute to the python.org website to gain skills in full stack development. Python Software Foundation interest me because the language is used in a wide variety of fields. My background is not in computer science, but I've earned a B.S. in Applied Mathematics and have an equivalent minor in computer science. I did front end project with HTML, CSS3 & Media Query, and JavaScript myself, but hope to work with someone to learn from. I'm a hard worker and responsible person and do my best to contribute to the organization. Have a nice day, Truong Nguyen ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
Terry Reedy writes: > On 1/26/2016 12:02 AM, INADA Naoki wrote: > > > People use same algorithm on every language when compares base language > > performance [1]. > > The python code is NOT using the same algorithm. The proof is that the > Python function will return the correct value for, say fib(50) while > most if not all the other versions will not. True, but that's not a reasonable criterion for "same algorithm" in this context. Naoki's application ("base language performance" benchmarking) requires fib(n) only for n < 40, and run it in a loop 100 times if you want 2 more decimal places of precision ("40" is appropriate for an implementation with 32-bit ints). On that restricted domain the algorithm *is* the same. If you want to argue that the bigger domain is a better one to use for evaluating programming languages, be our guest. But then you're comparing apples (speed) to oranges (domain), and Naoki (or the Japanese benchmarkers) can argue that a smaller, more risky, domain is covered by "consenting adults" -- if you know there's a risk, you need to write code to deal with it, but if you know there isn't, you shouldn't have to accept lower performance. Obviously, I don't think that's an appropriate tradeoff myself, but that's based on "IMHO" not "comparison is invalid because algorithms differ". ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
Nick Coghlan writes: > On 26 January 2016 at 17:16, Stephen J. Turnbull wrote: > > Our universities are doing an awful job at getting "big picture > > thinking" across to our students. > > That problem isn't specific to Japan - I'm not aware of *anywhere* > that does a particularly good job of teaching developers not to get > tribal about their programming language choice, But that's a different issue. The approach that Naoki describes isn't "tribal", in fact it's the exact opposite: it's an attempt to base such decisions on strictly objective measures. > and instead evaluate their options based on their specific problem, > the team that will be working on it, and the pre-existing problem > solving tools available in that ecosystem. One of which is the language itself, and the team's experience with it. "We're a C++/Java/Python/Ruby/Brainf!ck/assembler/COBOL shop" isn't a bad heuristic in most cases -- outside of research and/or "we also walk dogs" generalist consultancies -- especially when you're under pressure from the bean-counting department to reduce costs. That heuristic is hard to distinguish from "tribalism", though. AFAICT, in fact, the generalists (including massive entities like IBM and Google, as well as companies like Red Hat and cooperatives like Debian) are quite good at the kind of evaluation you describe. And Python has been quite good at targeting the kinds of improvements that make it appealing to people who can and do do that kind of evaluation, in more and more areas. > While improving those kinds of metrics isn't a *reason* to do > anything, it does count as an added bonus when it comes as a > beneficial side effect of working on something else (such as the > function specialisation infrastructure underpinning Victor's > optimiser project). Sure, but that's a "have your cake and eat it too" situation. Nobody's going to complain about that! If Naoki -- or anybody else -- wants to work on optimizations enabled by FAT Python, more power to them, I say. I just would like to see them reviewed with the goal of making Python a better language for solving a wide variety of problems, rather than strictly focusing on benchmarks. If the benchmarks can be improved without closing off potential syntax improvements or restricting the domain of algorithms (cf Terry's comment), wonderful! I thought Chris's point about efficient algorithms that would be hard to implement correctly in other languages but are easy to do in Python *because* of Python's carefully crafted, but not always maximally speedy, semantics was especially apropos here. Of course his claim that Python is faster than C is tongue in cheek, and a caching version of fibonacci wouldn't be that hard to write in C, and an iterative version even easier. But others have pointed out many other syntaxes (comprehensions, generators, yield from, and so on) that put together often make efficient computation TOOWTDI. That, to me, is The Python Advantage. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
I completely agree with INADA. It's like saying, because a specific crossroad features a higher accident rate, *people need to change their driving behavior*. *No!* People won't change and it's not necessary either. The crossroad needs to be changed to be safer. Same goes for Python. If it's slow using the very same piece of code (even superficially), you better make the language faster. Developers won't change and they won't change their code either. Just not necessary. Btw. it would be a great feature for Python 3 to be faster than Python 2. I've heard a lot of complaints of the scientific community that Python is slow. Would Python 3 be significantly faster than Python 2, that'll be a huge reason to upgrade (and would create pressure to upgrade libs as well). They are satisfied with Python so far, but would there be a language equally readable/maintainable and 10x faster (of course proven by some weird micro benchmarks - incomprehensible to most nervous subscribers to this list), they would readily switch over. I for one hope that *Python itself will be that language* in the foreseeable future. This is some sort of marketing but also requires hard facts indeed. Best, Sven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
Hi, will look into it soon. :) Best, Sven On 26.01.2016 16:32, Victor Stinner wrote: Hi, 2016-01-26 3:21 GMT+01:00 INADA Naoki : How can I help your work? I don't know exactly yet, but I started to write a documentation to explain how to contribute: http://faster-cpython.readthedocs.org/fat_python.html#how-can-you-contribute You may contact me directly ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
On Wed, Jan 27, 2016 at 2:28 AM, Ryan Gonzalez wrote: >>rosuav@sikorsky:~$ gcc fib.c && time ./a.out >>1134903170 >> >>real 0m9.104s >>user 0m9.064s >>sys 0m0.000s >>rosuav@sikorsky:~$ cat fib.c >>#include >> >>unsigned long fib(unsigned long n) >>{ >>if (n < 2) return n; >>return fib(n-2) + fib(n-1); >>} >> >>int main() >>{ >>printf("%lu\n",fib(45)); >>} >> > > *cough* -O3 *cough* > Oh, I'm sorry. Let's try it again. rosuav@sikorsky:~$ gcc -O3 fib.c && time ./a.out 1134903170 real 0m3.153s user 0m3.088s sys 0m0.052s Cool! Seems to be linear. From which we can deduce that Python on my system was compiled at about -O275. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
Hi, 2016-01-26 3:21 GMT+01:00 INADA Naoki : > How can I help your work? I don't know exactly yet, but I started to write a documentation to explain how to contribute: http://faster-cpython.readthedocs.org/fat_python.html#how-can-you-contribute You may contact me directly ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
On January 25, 2016 9:59:36 PM CST, Chris Angelico wrote: >On Tue, Jan 26, 2016 at 2:32 PM, INADA Naoki >wrote: >> >> I know. >> But people compares language speed by simple microbench like >fibbonacci. >> They doesn't use listcomp or libraries to compare *language* speed. >> > >Well, that's a stupid way to decide on a language. Here, look: Python >is faster than C. Proof! > >rosuav@sikorsky:~$ time python3 fib.py >2880067194370816120 > >real 0m0.033s >user 0m0.032s >sys 0m0.000s >rosuav@sikorsky:~$ cat fib.py >import functools > >@functools.lru_cache() >def fib(n): >if n < 2: return n >return fib(n-2) + fib(n-1) > >print(fib(90)) > >rosuav@sikorsky:~$ gcc fib.c && time ./a.out >1134903170 > >real 0m9.104s >user 0m9.064s >sys 0m0.000s >rosuav@sikorsky:~$ cat fib.c >#include > >unsigned long fib(unsigned long n) >{ >if (n < 2) return n; >return fib(n-2) + fib(n-1); >} > >int main() >{ >printf("%lu\n",fib(45)); >} > *cough* -O3 *cough* > >Algorithmic differences - even subtle ones - can easily outdo choice >of language for run-time performance. And if you try to write a true C >equivalent of that Python code, good luck - I'll have the Python one >written and running while you're still trying to figure out how to >write a cache, much less how to keep the syntax clean as you add a >cache to an existing function. > >Of course, rewriting the whole thing to work iteratively instead of >double-recursively will make a dramatic difference to both programs. >That's an unsubtle algorithmic difference, and if you're coding like >that, you probably can't see the difference in performance between any >two languages at anything up to a machine word (about the 90th or so >Fibonacci number, on a 64-bit system) - all you'll see is the startup >performance. As soon as you go beyond a machine word, Python massively >trumps C, because its default integer type is a bignum. Going beyond a >machine word in C is a hassle. Going beyond a machine word in Python 2 >is almost insignificant - hey look, now your repr has an 'L' on the >end, and performance is immeasurably worse. In Python 3, there's no >machine word whatsoever. > >So, yeah... Python beats C for Fibonacci calculation, too. You just >have to not be stupid with your benchmarking. > >ChrisA >___ >Python-Dev mailing list >Python-Dev@python.org >https://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
On 1/26/2016 12:02 AM, INADA Naoki wrote: People use same algorithm on every language when compares base language performance [1]. The python code is NOT using the same algorithm. The proof is that the Python function will return the correct value for, say fib(50) while most if not all the other versions will not. The domain of an algorithm is part of what characterizes an algorithm. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FAT Python (lack of) performance
On 26 January 2016 at 17:16, Stephen J. Turnbull wrote: > Our universities are doing an awful job at getting "big picture thinking" > across to our students. That problem isn't specific to Japan - I'm not aware of *anywhere* that does a particularly good job of teaching developers not to get tribal about their programming language choice, and instead evaluate their options based on their specific problem, the team that will be working on it, and the pre-existing problem solving tools available in that ecosystem. As a result, folks making programming language choices based on criteria that aren't actually relevant to the problem they're trying to solve is going to be a fact of life. While improving those kinds of metrics isn't a *reason* to do anything, it does count as an added bonus when it comes as a beneficial side effect of working on something else (such as the function specialisation infrastructure underpinning Victor's optimiser project). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com