Re: Future of Pypy?

2015-02-27 Thread Paul Rubin
Steven D'Aprano writes: > An interesting point of view: threading is harmful because it removes > determinism from your program. > http://radar.oreilly.com/2007/01/threads-considered-harmful.html Concurrent programs are inherently nondeterministic because they respond to i/o events that can happ

Re: Are threads bad? - was: Future of Pypy?

2015-02-26 Thread Paul Rubin
Ryan Stuart writes: > My point is malloc, something further up (down?) the stack, is making > modifications to shared state when threads are involved. Modifying > shared state makes it infinitely more difficult to reason about the > correctness of your software. If you're saying the libc malloc

Re: Are threads bad? - was: Future of Pypy?

2015-02-25 Thread Chris Angelico
On Thu, Feb 26, 2015 at 4:16 AM, Mark Lawrence wrote: > IIRC the underlying JET engine was replaced by SQL Server years ago. Maybe > not the best technlogy in the world, but you'd be hard pushed to do worse > than JET :) The way I understood it, MS Access could connect to a variety of database ba

Re: Are threads bad? - was: Future of Pypy?

2015-02-25 Thread Mark Lawrence
On 25/02/2015 17:00, Ian Kelly wrote: On Wed, Feb 25, 2015 at 9:37 AM, Mark Lawrence wrote: On 25/02/2015 06:02, Ian Kelly wrote: Is the name of that database program "Microsoft Access" perchance? Are you referring to the GUI, the underlying database engine, both, or what? The engine. I

Re: Are threads bad? - was: Future of Pypy?

2015-02-25 Thread Ian Kelly
On Wed, Feb 25, 2015 at 9:37 AM, Mark Lawrence wrote: > On 25/02/2015 06:02, Ian Kelly wrote: >> >> >> Is the name of that database program "Microsoft Access" perchance? >> > > Are you referring to the GUI, the underlying database engine, both, or what? The engine. In theory it supports concurren

Re: Are threads bad? - was: Future of Pypy?

2015-02-25 Thread Mark Lawrence
On 25/02/2015 06:02, Ian Kelly wrote: Is the name of that database program "Microsoft Access" perchance? Are you referring to the GUI, the underlying database engine, both, or what? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mar

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Ian Kelly
On Tue, Feb 24, 2015 at 10:54 PM, Chris Angelico wrote: > On Wed, Feb 25, 2015 at 4:46 PM, Marko Rauhamaa wrote: >> Marcos Almeida Azevedo : >> >>> Synchronized methods in Java really makes programming life simpler. >>> But I think it is standard practice to avoid this if there is a >>> lighter a

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Marcos Almeida Azevedo
On Wed, Feb 25, 2015 at 1:46 PM, Marko Rauhamaa wrote: > Marcos Almeida Azevedo : > > > Synchronized methods in Java really makes programming life simpler. > > But I think it is standard practice to avoid this if there is a > > lighter alternative as synchronized methods are slow. Worse case I >

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Chris Angelico
On Wed, Feb 25, 2015 at 4:46 PM, Marko Rauhamaa wrote: > Marcos Almeida Azevedo : > >> Synchronized methods in Java really makes programming life simpler. >> But I think it is standard practice to avoid this if there is a >> lighter alternative as synchronized methods are slow. Worse case I >> use

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Marko Rauhamaa
Marcos Almeida Azevedo : > Synchronized methods in Java really makes programming life simpler. > But I think it is standard practice to avoid this if there is a > lighter alternative as synchronized methods are slow. Worse case I > used double checked locking. I have yet to see code whose perform

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Ryan Stuart
On Tue Feb 24 2015 at 3:32:47 PM Paul Rubin wrote: > Ryan Stuart writes: > Sure, the shared memory introduces the possibility of some bad errors, > I'm just saying that I've found that by staying with a certain > straightforward style, it doesn't seem difficult in practice to avoid > those error

Re: Are threads bad? - was: Future of Pypy?

2015-02-24 Thread Marko Rauhamaa
Chris Angelico : > Actually, you can quite happily have multiple threads messing with the > underlying file descriptors, that's not a problem. (Though you will > tend to get interleaved output. But if you always produce output in > single blocks of text that each contain one line with a trailing >

Re: Future of Pypy?

2015-02-23 Thread Chris Angelico
On Tue, Feb 24, 2015 at 5:56 PM, Steven D'Aprano wrote: > Most people are not using the bleeding edge version of Python, and even > those who do, aren't usually using it in production. There are still plenty > of people using Python 2.3 in production, and even a few using 1.5. > > But as you say,

Re: Future of Pypy?

2015-02-23 Thread Steven D'Aprano
Paul Rubin wrote: >> With threads in a single process, this isn't a problem. They all >> access the same memory space, so they can all share state. As soon as >> you go to separate processes, these considerations become serious. > > Right, that's a limitation of processes compared to threads. >

Re: Future of Pypy?

2015-02-23 Thread Steven D'Aprano
Chris Angelico wrote: > On Mon, Feb 23, 2015 at 6:41 PM, Steven D'Aprano > wrote: - and you must be using libraries and tools which prevent you moving to Jython or IronPython or some other alternative. >>> >>> I don't get this at all. Why should I not want Python to have the same >>> c

Re: Are threads bad? - was: Future of Pypy?

2015-02-23 Thread Chris Angelico
On Tue, Feb 24, 2015 at 4:27 PM, Paul Rubin wrote: >> Sure, your code might not be making any mutations (that you know of), >> but malloc definitely is [1], and that's just the tip of the iceberg. >> Other things like buffers for stdin and stdout, DNS resolution etc. >> all have the same issue. >

Re: Are threads bad? - was: Future of Pypy?

2015-02-23 Thread Paul Rubin
Ryan Stuart writes: > I'm not sure what else to say really. It's just a fact of life that > Threads by definition run in the same memory space and hence always > have the possibility of nasty unforeseen problems. They are unforeseen > because it is extremely difficult (maybe impossible?) to try an

Re: Future of Pypy?

2015-02-23 Thread Paul Rubin
Chris Angelico writes: > So, you would have to pass code to the other process, probably. What > about this: > y = 4 > other_thread_queue.put(lambda x: x*y) the y in the lambda is a free variable that's a reference to the surrounding mutable context, so that's at best dubious. You could use:

Re: Future of Pypy?

2015-02-23 Thread Steven D'Aprano
Ethan Furman wrote: > On 02/22/2015 11:41 PM, Steven D'Aprano wrote: > >> If you want *CPython* to work without a GIL, well, are you volunteering >> to do the work? It is a massive job, and the core devs aren't terribly >> interested. Probably because they understand that the GIL is not often an

Re: Future of Pypy?

2015-02-23 Thread Laura Creighton
Arrgh! I forgot to warn you that you need a very recent version of virtualenv to work with PyPy. I am very sorry about that. Glad to see that things are working now. Laura -- https://mail.python.org/mailman/listinfo/python-list

Cython - was: Future of Pypy?

2015-02-23 Thread Stefan Behnel
Dave Farrance schrieb am 23.02.2015 um 15:13: > Dave Cook wrote: >> On 2015-02-22, Dave Farrance wrote: >> >>> It's still quicker to do a re-write in the more cumbersome C >> >> You should try Cython. > > I did try Cython when I was trying to figure out what to do about the slow > speed. My initi

Re: Future of Pypy?

2015-02-23 Thread Dave Farrance
Dave Cook wrote: >On 2015-02-22, Dave Farrance wrote: > >> It's still quicker to do a re-write in the more cumbersome C > >You should try Cython. I did try Cython when I was trying to figure out what to do about the slow speed. My initial attempt showed no speedup at all. The documentation to

Re: Future of Pypy?

2015-02-23 Thread Dave Farrance
Laura Creighton wrote: >Good news -- it seems to be working fine with PyPy. >https://travis-ci.org/hugovk/Pillow/builds > >for me, not extensively tested, it just seems to be working. > >I have several pypy's floating around here, each within its own >virtualenv. If you aren't familiar with vir

Re: Future of Pypy?

2015-02-23 Thread Dave Cook
On 2015-02-22, Dave Farrance wrote: > It's still quicker to do a re-write in the more cumbersome C You should try Cython. Dave -- https://mail.python.org/mailman/listinfo/python-list

Re: Future of Pypy?

2015-02-23 Thread Chris Angelico
On Mon, Feb 23, 2015 at 6:41 PM, Steven D'Aprano wrote: >>> - and you must be using libraries and tools which prevent you moving to >>> Jython or IronPython or some other alternative. >> >> I don't get this at all. Why should I not want Python to have the same >> capabilities? > > Python does hav

Re: Future of Pypy?

2015-02-23 Thread Marko Rauhamaa
Steven D'Aprano : > Yes, but my point is that if some other way solves the problem, then > you should *use that other technique* rather than complain about the > GIL. The GIL is not a bottleneck if you can bypass it. > > I can sympathize with somebody who says "I have this problem that can > only

Re: Future of Pypy?

2015-02-22 Thread Steven D'Aprano
Paul Rubin wrote: > Steven D'Aprano writes: >> I'm sorry, but the instant somebody says "eliminate the GIL", they lose >> credibility with me. Yes yes, I know that in *your* specific case you've >> done your research and (1) multi-threaded code is the best solution for >> your application and (2)

Re: Future of Pypy?

2015-02-22 Thread Ryan Stuart
On Mon Feb 23 2015 at 4:15:42 PM Paul Rubin wrote: > > What do you mean about Queues working with processes? I meant > Queue.Queue. There is multiprocessing.Queue but that's much less > capable, and it uses cumbersome IPC like pipes or sockets instead of a > lighter weight lock. Threads can als

Re: Future of Pypy?

2015-02-22 Thread Terry Reedy
On 2/22/2015 7:45 AM, Dave Farrance wrote: As an engineer, I can quickly knock together behavioural models of electronic circuits, complete units, and control systems in Python, then annoyingly in a few recent cases, have to re-write in C for speed. I've tried PyPy, the just-in-time compiler fo

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Ryan Stuart writes: > I think that is a pretty accurate summary. In fact, the article even > says that. So, just to iterate its point, if you are using > non-blocking Queues to communicate to these threads, then you just > have a communicating event loop. Given that Queues work perfectly with > wi

Re: Future of Pypy?

2015-02-22 Thread Ryan Stuart
On Mon Feb 23 2015 at 1:50:40 PM Paul Rubin wrote: > That article is about the hazards of mutable state shared between > threads. The key to using threads safely is to not do that. So the > "transfer" example in the article would instead be a message handler in > the thread holding the account

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Ryan Stuart writes: > Many people have written at length about why it's bad. The most recent > example I have come across is here -> > https://glyph.twistedmatrix.com/2014/02/unyielding.html That article is about the hazards of mutable state shared between threads. The key to using threads safel

Re: Future of Pypy?

2015-02-22 Thread Chris Angelico
On Mon, Feb 23, 2015 at 2:16 PM, Ryan Stuart wrote: > Many people have written at length about why it's bad. The most recent > example I have come across is here -> > https://glyph.twistedmatrix.com/2014/02/unyielding.html > > It's not a specific Python problem. I must be in the limited crowd that

Re: Future of Pypy?

2015-02-22 Thread Ryan Stuart
On Mon Feb 23 2015 at 12:05:46 PM Paul Rubin wrote: > I don't see what the big deal is. I hear tons of horror stories about > threads and I believe them, but the thing is, they almost always revolve > around acquiring and releasing locks in the wrong order, forgetting to > lock things, stuff lik

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Laura Creighton writes: > And given that Lennart is a friend, well really a good friend of my > lover and a something-better- than-an-acquaintance with me I > should make the effort to get these two under the same roof (mine, by > preference) for the fun of the experience. Oh cool, right, I

Re: Future of Pypy?

2015-02-22 Thread Chris Angelico
On Mon, Feb 23, 2015 at 1:04 PM, Paul Rubin wrote: >> if you're running single-thread code on a single-core machine and >> still complaining about the GIL, you have no clue. > > Even the Raspberry Pi has 4 cores now, and fancy smartphones have had > them for years. Single core cpu's are specializ

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Steven D'Aprano writes: > I'm sorry, but the instant somebody says "eliminate the GIL", they lose > credibility with me. Yes yes, I know that in *your* specific case you've > done your research and (1) multi-threaded code is the best solution for > your application and (2) alternatives aren't suit

Re: Future of Pypy?

2015-02-22 Thread Steven D'Aprano
Paul Rubin wrote: > Laura Creighton writes: >> Because one thing we do know is that people who are completely and >> utterly ignorant about whether having multiple cores will improve >> their code still want to use a language that lets them use the >> multiple processors. If the TM dream of havi

Re: Future of Pypy?

2015-02-22 Thread Laura Creighton
In a message of Sun, 22 Feb 2015 12:14:45 -0800, Paul Rubin writes: >Laura Creighton writes: >> The GIL isn't going away from PyPy any time real soon, alas. > >I thought the GIL's main purpose was to avoid having to lock all the >CPython refcount updates, so if PyPy has tracing GC, why is there st

Re: Future of Pypy?

2015-02-22 Thread Laura Creighton
Good news -- it seems to be working fine with PyPy. https://travis-ci.org/hugovk/Pillow/builds for me, not extensively tested, it just seems to be working. I have several pypy's floating around here, each within its own virtualenv. If you aren't familiar with virtualenv, read all about it here:

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Laura Creighton writes: > The GIL isn't going away from PyPy any time real soon, alas. I thought the GIL's main purpose was to avoid having to lock all the CPython refcount updates, so if PyPy has tracing GC, why is there still a GIL? And how is TM going to help with parallelism if the GIL is st

Re: Future of Pypy?

2015-02-22 Thread Laura Creighton
In a message of Sun, 22 Feb 2015 11:02:29 -0800, Paul Rubin writes: >Laura Creighton writes: >> Because one thing we do know is that people who are completely and >> utterly ignorant about whether having multiple cores will improve >> their code still want to use a language that lets them use the

Re: Future of Pypy?

2015-02-22 Thread Dave Farrance
Dave Farrance wrote: >Steven D'Aprano wrote: > >>I assume you're talking about drawing graphics rather than writing text. Can >>you tell us which specific library or libraries won't run under PyPy? > >Yes, mainly the graphics. I'm a hardware engineer, not a software >engineer, so I might well b

Re: Future of Pypy?

2015-02-22 Thread Paul Rubin
Laura Creighton writes: > Because one thing we do know is that people who are completely and > utterly ignorant about whether having multiple cores will improve > their code still want to use a language that lets them use the > multiple processors. If the TM dream of having that just happen, > se

Re: Future of Pypy?

2015-02-22 Thread Laura Creighton
In a message of Sun, 22 Feb 2015 15:36:42 +, Dave Farrance writes: >Laura Creighton wrote: > >>I don't understand 'an interpreter rather than a JIT'. PyPy has a >>JIT, that sort of is the whole point. > >Yes. I meant that from my end-user, non-software-engineer perspective, it >looked as tho

Re: Future of Pypy?

2015-02-22 Thread Dave Farrance
Steven D'Aprano wrote: >I assume you're talking about drawing graphics rather than writing text. Can >you tell us which specific library or libraries won't run under PyPy? Yes, mainly the graphics. I'm a hardware engineer, not a software engineer, so I might well be misunderstanding PyPy's curr

Re: Future of Pypy?

2015-02-22 Thread Dave Farrance
Laura Creighton wrote: >I don't understand 'an interpreter rather than a JIT'. PyPy has a >JIT, that sort of is the whole point. Yes. I meant that from my end-user, non-software-engineer perspective, it looked as though CPython was proceeding with leaps and bounds and that PyPy remained mostly

Re: Future of Pypy?

2015-02-22 Thread Dave Farrance
jkn wrote: > I'm curious what ...behavioural... models you are creating quickly in > Python that then need rewriting in C for speed. SPICE? some other CAD? > Might be interesting to learn more about what and how you are actually > doing. The convert-to-C cases were complex filtering functions.

Re: Future of Pypy?

2015-02-22 Thread Steven D'Aprano
Dave Farrance wrote: > As an engineer, I can quickly knock together behavioural models of > electronic circuits, complete units, and control systems in Python, then > annoyingly in a few recent cases, have to re-write in C for speed. > > I've tried PyPy, the just-in-time compiler for Python, and

Re: Future of Pypy?

2015-02-22 Thread Laura Creighton
In a message of Sun, 22 Feb 2015 12:45:03 +, Dave Farrance writes: >Maybe there's not enough people like me that have really felt the need for >the speed. Or maybe it's simply the accident of the historical >development path that's set-in-stone an interpreter rather than a JIT. >Anybody got a

Re: Future of Pypy?

2015-02-22 Thread jkn
On Sunday, 22 February 2015 12:45:15 UTC, Dave Farrance wrote: > As an engineer, I can quickly knock together behavioural models of > electronic circuits, complete units, and control systems in Python, then > annoyingly in a few recent cases, have to re-write in C for speed. > > I've tried PyPy,

Future of Pypy?

2015-02-22 Thread Dave Farrance
As an engineer, I can quickly knock together behavioural models of electronic circuits, complete units, and control systems in Python, then annoyingly in a few recent cases, have to re-write in C for speed. I've tried PyPy, the just-in-time compiler for Python, and that is impressively, hugely fa