Okay, with respect to the SWIG + pypy, works for me, and about 4x slower then
cppyy or cpython ( which show same results more or less).
time cppyy ./cppyy_timings.py
real 0m0.265s
user 0m0.225s
sys 0m0.040s
time pypy ./pypy_timings.py
real 0m2.424s
user 0m2.338s
sys 0m0.086s
time python2 ./pypy_timings.py
real 0m0.414s
user 0m0.390s
sys 0m0.024s
So apparently pypy is dramatically bad with SWIG. cppyy seems to work nice. I
have, however, encountered an interesting problem. Say I have a dictionary
built. I'd like to know which symbols/classes are actually available (if any),
but I can not do that. The implication is that in case your library does not
have any symbols in it after SUCCESSFUL build, there is no way to figure out
why and how that happened. Please advise. Also I can bench the entire thing on
a real, quite large app that makes intensive use of pypy and also runs on
python2 if needed, but for that we would need to figure why no classes are
visible in the dictionary=)
BR,
Alex
sunnuntai 03 kesäkuu 2012 16:44:19 wlavrij...@lbl.gov kirjoitti:
> Hi Alex,
>
> >> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
> >
> > Performance was originally the point, right? Or am I missing something
> > (lol)?
> well, remember that there are two problems here: 1) since there are no
> PyObject's as such in pypy-c (they have different internal structure), one
> has the create them, to have a PyObject* pointer to them; and keep track of
> them to mix the reference counting scheme with the garbage collection scheme
> (this is all done in/by cpyext). And 2) now that the object is created,
> pinning down a variable in the .py code, the JIT can't unbox it.
>
> The example I used, which was like so:
>
> for i in range(N):
> a.add_data(i)
>
> and is rather pathological, of course. But then, all micro-benches are.
>
> Conversely, doing:
>
> l = []
> for i in range(N):
> l.append(i)
> a.add_data(l)
>
> is very speedy, presumably (I don't know, guessing here as it may also be
> that there is another optimization for the list that I can't prevent even
> with the append() calls) because now the cpyext code is in a loop in and
> of itself, so e.g. lookup of the type of the objects in the container (all
> the same here) can become constant.
>
> > I am giving you my benchmark code
>
> Which clearly is also pathological. :) I can't manage to run the SWIG test
> from pypy, though? I tried by adding cpyext, but no dice?
>
> Anyway, you'll be happy to know that:
>
> CPPYY
> Created container
> No more items
>
> real 0m2.236s
> user 0m2.156s
> sys 0m0.071s
>
> Cpython
> Created container
> No more items
>
> real 0m2.758s
> user 0m2.574s
> sys 0m0.174s
>
> (I verified with your debug=1 that the objects appear.)
>
> But that improvement is (I'm sure) only b/c all the other stuff you do
> around it with manipulating the dict on the python side. That does get
> JIT'ed just fine.
>
> Btw., I had to jack up N by 10x, otherwise all numbers were just noise. :}
>
> > right now it uses dictionary to map
> > pyobject id into objects when they leave c++ domain. Obviously, you add
> > about O(2*log(N)) overhead to the code, but that should not be too much
> > anyway. If that can be done in rpython, it could probably be made to work
> > massively faster. Like a module that does object caching for the storage
> > of objects that would be sent to c++ domain. Then in c++ one can do as
> > simple as have an long int index that can be used to resolve back to
> > byobject when the job is done.
>
> From a quick read of the cpyext code, that is rather similar to what is
> there.
>
> But as per above, the problem isn't with the extra overhead, the problem is
> with the JIT being prevented to do any work (and with pypy-c being slower
> than CPython in pure interpreted mode).
>
> > mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have
> > older gcc in a different slot?
>
> I do, but those should not be accessible from my pypy work environment. I
> did rebuild it. Not sure whether that matters. I'll update the docs showing
> how a different set of headers from another compiler can be selected.
>
> > The point is that if the header is included, and it confuses the parser,
> > then the patterns have no effect. So it is better not to have extra
> > headers like Qt library=)
>
> Qt is a story in and of itself, of course. :) Actually, what folks here do,
> is to link any code from external C++ through python to Qt (C++) by making
> use of the fact that both ends can handle opaque pointers, represented as
> long values. (The combination of addressof and bind_object in cppyy.)
>
> Best regards,
> Wim
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
