Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Stefan Behnel, 15.02.2012 12:32: > The current state of the discussion seems to be that PyPy provides ways to > talk to C code, but nothing as complete as CPython's C-API in the sense > that it allows efficient two-way communication between C code and Python > objects. Thus, we need to either impro

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
Hi, 2012/2/18 Stefan Behnel > Stefan Behnel, 15.02.2012 12:32: > > The current state of the discussion seems to be that PyPy provides ways > to > > talk to C code, but nothing as complete as CPython's C-API in the sense > > that it allows efficient two-way communication between C code and Python

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 10:08: > 2012/2/18 Stefan Behnel >> Stefan Behnel, 15.02.2012 12:32: >>> http://wiki.cython.org/enhancements/pypy >> >> So, any volunteers or otherwise interested parties to help in getting this >> to work? Anyone in for financial support? > > Actually I spent seve

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Maciej Fijalkowski
On Sat, Feb 18, 2012 at 11:27 AM, Stefan Behnel wrote: > Amaury Forgeot d'Arc, 18.02.2012 10:08: >> 2012/2/18 Stefan Behnel >>> Stefan Behnel, 15.02.2012 12:32: http://wiki.cython.org/enhancements/pypy >>> >>> So, any volunteers or otherwise interested parties to help in getting this >>> to w

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Maciej Fijalkowski, 18.02.2012 10:35: > On Sat, Feb 18, 2012 at 11:27 AM, Stefan Behnel wrote: >> Given that XML processing is currently slower in PyPy than in CPython, I >> don't think that's all that bad. Users can still switch their imports to >> ElementTree if they only want to push XML out and

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Maciej Fijalkowski
On Sat, Feb 18, 2012 at 11:48 AM, Stefan Behnel wrote: > Maciej Fijalkowski, 18.02.2012 10:35: >> On Sat, Feb 18, 2012 at 11:27 AM, Stefan Behnel wrote: >>> Given that XML processing is currently slower in PyPy than in CPython, I >>> don't think that's all that bad. Users can still switch their im

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 10:08: > I made some modifications to pypy, cython and lxml, > and now I can compile and install cython, lxml, and they seem to work! > > For example:: > html = etree.Element("html") > body = etree.SubElement(html, "body") > body.text = "TEXT" > br

Re: [pypy-dev] STM status

2012-02-18 Thread Armin Rigo
Hi Chris, On Sat, Feb 18, 2012 at 03:51, Chris Dashiell wrote: > I just wanted to let you know the numbers I got on a 32 core server. > > 1 thread: 3280 ms > 4 threads: 2665 ms > 8 threads: 2976 ms > 16 threads: 2878 ms > 32 threads: 2714 ms We just checked on a 24-core server (2 physical proces

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Maciej Fijalkowski, 18.02.2012 10:56: > On Sat, Feb 18, 2012 at 11:48 AM, Stefan Behnel wrote: >> Maciej Fijalkowski, 18.02.2012 10:35: >>> On Sat, Feb 18, 2012 at 11:27 AM, Stefan Behnel wrote: Given that XML processing is currently slower in PyPy than in CPython, I don't think that's al

Re: [pypy-dev] Poor performance with custom bytecode

2012-02-18 Thread Armin Rigo
Hi Antoni, On Fri, Feb 17, 2012 at 18:49, Antoni Segura Puimedon wrote: > Any suggestion on where to start reading pypy code/material? In your case, I'd suggest that you should first write a Clojure interpreter in regular Python, and then worry about making it RPython. This is done mostly by fo

Re: [pypy-dev] Poor performance with custom bytecode

2012-02-18 Thread Antoni Segura Puimedon
Hi Armin, On Sat, Feb 18, 2012 at 12:54 PM, Armin Rigo wrote: > Hi Antoni, > > On Fri, Feb 17, 2012 at 18:49, Antoni Segura Puimedon > wrote: > > Any suggestion on where to start reading pypy code/material? > > In your case, I'd suggest that you should first write a Clojure > interpreter in reg

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Maciej Fijalkowski, 18.02.2012 10:56: > On Sat, Feb 18, 2012 at 11:48 AM, Stefan Behnel wrote: >> Maciej Fijalkowski, 18.02.2012 10:35: >>> On Sat, Feb 18, 2012 at 11:27 AM, Stefan Behnel wrote: Given that XML processing is currently slower in PyPy than in CPython, I don't think that's al

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 10:08: > I made some modifications to pypy, cython and lxml, > and now I can compile and install cython, lxml, and they seem to work! > > For example:: > html = etree.Element("html") > body = etree.SubElement(html, "body") > body.text = "TEXT" > br

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Martijn Faassen
Hi there, On Sat, Feb 18, 2012 at 10:56 AM, Maciej Fijalkowski wrote: > I somehow doubt it's possible to make this run fast using cpyext > (although there are definitely some ways). Maybe speeding up > ElementTree would be the way if all we want to get is a fast XML > processor? I doubt this is

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Martijn Faassen, 18.02.2012 14:11: > For some insight of why people want would lxml, there's an interesting > discussion on google app engine's bug tracker about it. > > http://code.google.com/p/googleappengine/issues/detail?id=18 > > This type of discussion is instructive as PyPy's barriers to p

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 10:08: > 2012/2/18 Stefan Behnel > I made some modifications to pypy, cython and lxml, > and now I can compile and install cython, lxml, and they seem to work! > > Here are the changes I made, some parts are really hacks and should be > polished: > lxml: http://pas

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > The weakref changes are really unfortunate as they appear in one of the > most performance critical spots of lxml's API: on-the-fly proxy creation. > > I can understand why the original code won't work as is, but could you > elaborate on why the weak references are neede

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > The exception handling code that you deleted in __Pyx_GetException(), that > which accesses exc_type and friends, is actually needed for correct > semantics of Cython code and Python code. Basically, it implements the part > of the except clause that moves the hot except

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 14:52: > 2012/2/18 Stefan Behnel > >> The exception handling code that you deleted in __Pyx_GetException(), that >> which accesses exc_type and friends, is actually needed for correct >> semantics of Cython code and Python code. Basically, it implements the part >>

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > I couldn't find the PyWeakref_LockObject() function anywhere. That's a > PyPy-only thing, right? I aliased it to (NULL) when compiling for CPython. > Yes, this function is PyPy-only, to fix a flaw of PyWeakref_GetObject: it returns a borrowed reference, which is very da

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 15:18: > 2012/2/18 Stefan Behnel > >> I couldn't find the PyWeakref_LockObject() function anywhere. That's a >> PyPy-only thing, right? I aliased it to (NULL) when compiling for CPython. >> > > Yes, this function is PyPy-only, to fix a flaw of PyWeakref_GetObject:

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > Here's an example. > > Python code: > > def print_excinfo(): > print(sys.exc_info()) > > Cython code: > > from stuff import print_excinfo > > try: > raise TypeError > except TypeError: > print_excinfo() > > With the code removed, Cython will not store

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Stefan Behnel, 18.02.2012 09:48: > Once we have the test suite runnable, we can set up a PyPy instance on our > CI server to get feed-back on any advances. > > https://sage.math.washington.edu:8091/hudson/ I've set up a build job for my development branch here: https://sage.math.washington.edu:8

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Stefan Behnel, 18.02.2012 16:29: > Stefan Behnel, 18.02.2012 09:48: >> Once we have the test suite runnable, we can set up a PyPy instance on our >> CI server to get feed-back on any advances. >> >> https://sage.math.washington.edu:8091/hudson/ > > I've set up a build job for my development branch

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Martijn Faassen
Hey, On Sat, Feb 18, 2012 at 11:20 AM, Stefan Behnel wrote: > Amaury Forgeot d'Arc, 18.02.2012 10:08: >> I made some modifications to pypy, cython and lxml, >> and now I can compile and install cython, lxml, and they seem to work! >> >> For example:: >>     html = etree.Element("html") >>     bod

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > And now the question is: how do I debug into PyPy? From the nightly build, > I don't get any debugging symbols in gdb, just a useless list of call > addresses (running the ref-counting related "arg_incref" test here): > > """ > #0 0x00ef93ef in ?? () > #1 0x000

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Stefan Behnel
Amaury Forgeot d'Arc, 18.02.2012 21:20: > 2012/2/18 Stefan Behnel > >> And now the question is: how do I debug into PyPy? From the nightly build, >> I don't get any debugging symbols in gdb, just a useless list of call >> addresses (running the ref-counting related "arg_incref" test here): >> >> "

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Alex Gaynor
On Sat, Feb 18, 2012 at 3:24 PM, Stefan Behnel wrote: > Amaury Forgeot d'Arc, 18.02.2012 21:20: > > 2012/2/18 Stefan Behnel > > > >> And now the question is: how do I debug into PyPy? From the nightly > build, > >> I don't get any debugging symbols in gdb, just a useless list of call > >> address

Re: [pypy-dev] Bringing Cython and PyPy closer together

2012-02-18 Thread Amaury Forgeot d'Arc
2012/2/18 Stefan Behnel > And now the question is: how do I debug into PyPy? Part of the answer: I never debug pypy. Even with debug symbols, the (generated) code is so complex that most of the time you cannot get anything interesting beyond the function names. But pypy is written in RPython,