On Wed, Oct 19, 2011 at 10:38 -0400, Gary Robinson wrote: > > By the way, did you ever considered the possibility of running pypy and > > cpython side-by-side? > > You do your pure-python computation on pypy, then you pipe them (e.g. by > > using execnet) to a cpython process which does the processing using scipy. > > Depending on how big the data is, the overhead of passing the data around > > should not be too high > . > Absolutely -- I've thought about that general approach though this is the > first time I recall hearing about execnet. Of course I'm concerned that the > overhead would be too much in some cases, such as huge numbers of calls to > scipy.stats.stats.chisqprob. Such overhead seems like it might cancel all the > benefit of PyPy, depending on the script. But maybe it's not as much overhead > as I fear. For example I see that execnet does not do pickling. Hm.
You can add pickling on top of execnet by using dumps/loads and sending the bytes which is fast. I recommend to use pickling with great care and not everywhere though. holger > -- > > Gary Robinson > CTO > Emergent Discovery, LLC > personal email: gary...@me.com > work email: grobin...@emergentdiscovery.com > Company: http://www.emergentdiscovery.com > Blog: http://www.garyrobinson.net > > > > > On Oct 19, 2011, at 10:27 AM, Antonio Cuni wrote: > > > Hello Gary, > > > > On 19/10/11 15:38, Gary Robinson wrote: > >>>> You would like pypy+numpy+scipy so that you could write fast > >>>> python-only algorithms and still use the existing libraries. I > >>>> suppose this is a perfectly reasonable usecase, and indeed > >>>> the current plan does not focus on this. > >>> > >> > >> Yes. That is exactly what I want. > > [cut] > > > > thank you for the input: indeed, I agree that for your usecase the current > > plan is not the best. OTOH, there is probably someone else for which the > > current plan is better than others, we cannot make everyone happy at the > > same time, although we might do it eventually :-). > > > > By the way, did you ever considered the possibility of running pypy and > > cpython side-by-side? > > You do your pure-python computation on pypy, then you pipe them (e.g. by > > using execnet) to a cpython process which does the processing using scipy. > > Depending on how big the data is, the overhead of passing the data around > > should not be too high > > > > It's not ideal, but it might be worth of being tried. > > > > ciao, > > Anto > > _______________________________________________ > pypy-dev mailing list > pypy-dev@python.org > http://mail.python.org/mailman/listinfo/pypy-dev > _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
