Re: [pypy-dev] pypy-stm: not dead

2014-06-16 Thread Armin Rigo
Hi again, We have also been "delayed" by writing two papers about STM :-) Thanks to Remi for doing the main work. The first is a position paper already accepted at ICOOOLPS'14: http://bitbucket.org/pypy/extradoc/raw/extradoc/talk/icooolps2014/position-paper.pdf The second one is submitted for D

[pypy-dev] pypy-stm: not dead

2014-06-16 Thread Armin Rigo
Hi all, Just a quick note to tell you sorry: although the 2.3 release of PyPy said the release of PyPy-STM would occur very soon, it's still delayed. We have found some real use cases in which it is several times slower than PyPy (both with the JIT) and so we're busy setting up a few missing thin

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-25 Thread Andrew Francis
Hi Nathan: On Thursday, October 24, 2013 6:21 PM, Nathan Hurst wrote: >Thanks for the paper reference, very interesting.  It is a really cool paper. I have spoken to the authors. > Is it worth implementing the api first with a naive implementation with >some Not really. the Join Patterns A

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-25 Thread Andrew Francis
Hi Armin: On Friday, October 25, 2013 4:36 AM, Armin Rigo wrote: >Precisely my point.  You're coming with paper X describing Y and >thinking "pypy-stm could do this", whereas I don't see any obvious >connection between the two.  To repeat myself, pypy-stm has no GIL >internally, but exposes t

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-25 Thread Andrew Francis
Hi Armin: On Friday, October 25, 2013 4:36 AM, Armin Rigo wrote: >Precisely my point.  You're coming with paper X describing Y and >thinking "pypy-stm could do this", whereas I don't see any obvious >connection between the two.  To repeat myself, pypy-stm has no GIL >internally, but exposes t

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-25 Thread Armin Rigo
Hi Andrew, On Thu, Oct 24, 2013 at 8:27 PM, Andrew Francis wrote: > A bit more context - I have looked at the C code for the Go channel > implementation and said "wow, there is a lot of locking there"! I also have > been following conversations concerning race conditions and the race > detector (

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-24 Thread Nathan Hurst
On Thu, Oct 24, 2013 at 11:27:05AM -0700, Andrew Francis wrote: > The University of Chicago/Microsoft Research paper "Scalable Join > Patterns" (http://www.mpi-sws.org/~turon/scalable-joins.pdf) > outlines an approach for implementing join patterns using > essentially lock-free bags, optimistic lo

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-24 Thread Andrew Francis
Hi Armin: On Thursday, October 24, 2013 4:38 AM, Armin Rigo wrote: >  I can't answer your questions without >knowing exactly what you're planning.  Sorry if I forgot, I know you >mentioned it several times here.  I still have a hard time figuring >out the real goal, rather than implementatio

Re: [pypy-dev] pypy-stm and lock-free data structures

2013-10-24 Thread Armin Rigo
Hi Andrew, On Tue, Oct 22, 2013 at 11:10 PM, Andrew Francis wrote: > 1) Would I be using the pypy-stm branch? (because there is no GIL) > 2) Would I write some of this in RPython? (I am assuming yes) The pypy-stm branch has no GIL in the implementation, but its behavior is equivalent to the GIL.

[pypy-dev] pypy-stm and lock-free data structures

2013-10-22 Thread Andrew Francis
Hi Folks: I have been reading a few papers on lock-free data structures (and reading some posts on Golang Nuts). I was thinking about writing new a version of stackless.py with Go's select. However I would try to use lock free data structures and techniques in the channel implementation. My que

Re: [pypy-dev] pypy-stm - how do I know when transactions are turning inevitable?

2013-09-15 Thread Armin Rigo
Hi, On Sun, Sep 15, 2013 at 7:15 AM, Костя Лопухин wrote: > As far as I understand, pypy-stm reports only conflicts, but not > transactions that turn inevitable. But if transaction turns > inevitable, it means that other transactions can not proceed. So it is > hard to debug loss of parallelizati

[pypy-dev] pypy-stm - how do I know when transactions are turning inevitable?

2013-09-14 Thread Костя Лопухин
Hello! As far as I understand, pypy-stm reports only conflicts, but not transactions that turn inevitable. But if transaction turns inevitable, it means that other transactions can not proceed. So it is hard to debug loss of parallelization when some transactions turn inevitable but other transacti

Re: [pypy-dev] PyPy STM

2012-10-05 Thread Armin Rigo
Hi, On Thu, Oct 4, 2012 at 7:58 PM, Randall Leeds wrote: > If we're looking for benchmark problems which are not trivially > parallelized, STAMP[0] is probably a great place to start. > > [0] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.139.472 You're welcome to contribute by rewritin

Re: [pypy-dev] PyPy STM

2012-10-04 Thread Randall Leeds
On Thu, Oct 4, 2012 at 5:53 AM, Armin Rigo wrote: > Hi Bengt, > > On Tue, Oct 2, 2012 at 1:40 PM, Bengt Richter wrote: >> Just a triggered thought: I am wondering if Conway's Game of Life >> http://en.wikipedia.org/wiki/Conway%27s_Game_of_Life >> might be be an interesting/fun basis for exper

Re: [pypy-dev] PyPy STM

2012-10-04 Thread Ronny Pfannschmidt
On 10/04/2012 06:14 PM, Andrew Francis wrote: Hi Ronny: *From:* Ronny Pfannschmidt *To:* Andrew Francis *Cc:* Armin Rigo ; Sarah Mount ; PyPy_Developers *Sent:* Monday, October 1, 2012 2:51 PM *Subject:* Re: [pypy-dev

Re: [pypy-dev] PyPy STM

2012-10-04 Thread Andrew Francis
Hi Ronny: From: Ronny Pfannschmidt To: Andrew Francis Cc: Armin Rigo ; Sarah Mount ; PyPy_Developers Sent: Monday, October 1, 2012 2:51 PM Subject: Re: [pypy-dev] PyPy STM On 10/01/2012 08:32 PM, Andrew Francis wrote: AF> That is why I use Stackl

Re: [pypy-dev] PyPy STM

2012-10-04 Thread Armin Rigo
Hi Bengt, On Tue, Oct 2, 2012 at 1:40 PM, Bengt Richter wrote: > Just a triggered thought: I am wondering if Conway's Game of Life > http://en.wikipedia.org/wiki/Conway%27s_Game_of_Life > might be be an interesting/fun basis for experiments and maybe a benchmark > for STM use in the parallel

Re: [pypy-dev] PyPy STM

2012-10-02 Thread Bengt Richter
On 09/30/2012 04:22 PM Armin Rigo wrote: Hi Ronny, On Sun, Sep 30, 2012 at 3:50 PM, Ronny Pfannschmidt wrote: after my thesis i'll be experimenting with a relaxed csp-ish model based on python native generator based continuations as well as the new continulet-jit-3 based greenlets. my basic

Re: [pypy-dev] PyPy STM

2012-10-01 Thread Ronny Pfannschmidt
On 10/01/2012 08:32 PM, Andrew Francis wrote: Hi Ronny: *From:* Ronny Pfannschmidt *To:* Armin Rigo *Cc:* Sarah Mount ; PyPy_Developers *Sent:* Sunday, September 30, 2012 10:43 AM *Subject:* Re: [pypy-dev] PyPy STM

Re: [pypy-dev] PyPy STM

2012-10-01 Thread Andrew Francis
Hi Ronny: From: Ronny Pfannschmidt To: Armin Rigo Cc: Sarah Mount ; PyPy_Developers Sent: Sunday, September 30, 2012 10:43 AM Subject: Re: [pypy-dev] PyPy STM >the reason why i start with generators instead of green-lets is simply >cause they

Re: [pypy-dev] PyPy STM

2012-09-30 Thread Ronny Pfannschmidt
On 09/30/2012 04:22 PM, Armin Rigo wrote: Hi Ronny, On Sun, Sep 30, 2012 at 3:50 PM, Ronny Pfannschmidt wrote: after my thesis i'll be experimenting with a relaxed csp-ish model based on python native generator based continuations as well as the new continulet-jit-3 based greenlets. my basic

Re: [pypy-dev] PyPy STM

2012-09-30 Thread Armin Rigo
Hi Ronny, On Sun, Sep 30, 2012 at 3:50 PM, Ronny Pfannschmidt wrote: > after my thesis i'll be experimenting with a relaxed csp-ish model > based on python native generator based continuations as well as > the new continulet-jit-3 based greenlets. > > my basic assumption is that having limited am

Re: [pypy-dev] PyPy STM

2012-09-30 Thread Ronny Pfannschmidt
Hi, the following is a collection of unfinished thoughts. after my thesis i'll be experimenting with a relaxed csp-ish model based on python native generator based continuations as well as the new continulet-jit-3 based greenlets. my basic assumption is that having limited amount of shared memo

Re: [pypy-dev] PyPy STM

2012-09-30 Thread Armin Rigo
Hi Russel, On Sun, Sep 30, 2012 at 11:10 AM, Russel Winder wrote: > However we want to get > away from a reliance on multiprocessing since it is rather heavyweight > for the sort of parallelism we are after. STM as an infrastructure layer > in PyPy and CPython would get us away from the GIL and a

[pypy-dev] PyPy STM

2012-09-30 Thread Russel Winder
Armin, Sarah and I are restarting our work on CSP, and extending to creating actors and a dataflow library. It would be good to make this work on Jython, IronPython and PyPy as well as CPython. However we want to get away from a reliance on multiprocessing since it is rather heavyweight for the so

Re: [pypy-dev] pypy-stm

2012-05-11 Thread Armin Rigo
Hi again, An update of pypy-stm. The previous one was not working properly with any Python function call --- on any example of any non-trivial size, it would not actually run on multiple cores. This one works better. http://wyvern.cs.uni-duesseldorf.de/~arigo/pypy-c-r55013-stm.tar.bz2 I also d

Re: [pypy-dev] pypy-stm

2012-05-10 Thread Armin Rigo
Re-hi, I updated my fork of CPython 3.x to include a very similar functionality, _thread.atomic_enter() and _thread.atomic_exit(). (Of course without any multicore capabilities, but it gives you the atomicity.) A very small diff overall. http://bitbucket.org/arigo/cpython-withatomic A bie

Re: [pypy-dev] pypy-stm

2012-05-09 Thread gelin yan
On Wed, May 9, 2012 at 10:01 PM, Armin Rigo wrote: > Hi all, > > Here is the very first Python 2.7 interpreter to run existing > multithreaded programs on multiple cores. Linux 32 bits. It is > horribly slow right now, so consider this as nothing more than a demo > :-) > >http://wyvern.cs.u

[pypy-dev] pypy-stm

2012-05-09 Thread Armin Rigo
Hi all, Here is the very first Python 2.7 interpreter to run existing multithreaded programs on multiple cores. Linux 32 bits. It is horribly slow right now, so consider this as nothing more than a demo :-) http://wyvern.cs.uni-duesseldorf.de/~arigo/pypy-c-r54960-stm.tar.bz2 Documentation: