Re: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
On Wed, Dec 1, 2010 at 9:48 AM, Stefan Behnel stefan...@behnel.de wrote: Paolo Giarrusso, 01.12.2010 00:34: Anyway, this does not interact with benchmarks above - Stefan, I still don't get why you complained that pyexpat is slow by showing benchmarks for another module, I guess I do not understand your email, but it asks reasonable? after Amaury talks about pyexpat. Well, in CPython, I can see little to no reason why anyone would go as low-level as pyexpat when you can use cElementTree. So IMHO the level to compare is what people would normally use rather than what people could potentially use if they were beaten hard enough. Stefan Hey. Sure, makes sense :-) I think one of the reasons for some slowdown is that calls from C are not jitted if they don't contain loops themselves. This doesn't explain the whole thing obviously, because there is something really wrong going on looking at numbers. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] a possible leak in the object namespace...
On Tue, Nov 30, 2010 at 20:33, Carl Friedrich Bolz cfb...@gmx.de wrote: Hi Alex, On 11/29/2010 09:02 PM, Alex A. Naanou wrote: On Mon, Nov 29, 2010 at 21:46, Carl Friedrich Bolzcfb...@gmx.de wrote: [snip] a) lots of attributes, which we expect to be rare b) access them with setattr, which is a lot slower than a fixed attribute c) access a different attribute every loop iteration, which means the compiler has to produce one bit of code for every attribute This is intentional (all three points), I did not want the jit to factor out the loop -- I wanted to time the initial attribute creation... Yes, I fear initial attribute creation is never going to be very efficient. [snip] But quadratic behaviour + a three orders of magnitude increase in time it takes to create an attr is scarry... but you are right, how often does that usecase happen? :) I improved some things so that setattr/getattr is a bit faster. Should now only be one or two orders of magnitude :-). If you run it tomorrow with the nightly build from tonight, you should see an improvement. That's good news! It is not uncommon, at least in my work, to write generic code that does not know of attribute names it uses, thus getattr/setattr functions are unavoidable. The quadraticness is not easily fixable without giving up all optimizations with instances of more than X (maybe 1000?) attributes. Again, I don't think this is common. And I don't want to chose an X. I'd even argue that most of the cases were the number of attributes exceeds a certain sane N would and should be better implemented via a container... (in spite of the how I wrote the code in my first mail ;) ) I've been planning of moving most of my commercial projects to PyPy as soon as it is stable enough -- I'm tired of fighting the CPython implementation for most of the time -- we will see how it goes... too bad some of the code will be PyPy specific due to the way CPython works. Thanks! Cheers, Carl Friedrich -- Alex. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] (no subject)
hi, i'm trying to make some pyopengl [0] -based code [1] run on top of pypy. this is partially successful, but i need some advice to progress further. pyopengl 3.x makes use of ctypes to provide the opengl binding, and it works out of the box for simple functions (those that do not takes c-pointer to buffers of data as arguments). for the rest, the first issue is that pyopengl use two functions from the ctypes.pythonapi lib, namely PyString_AsString and PyBuffer_FromMemory. any advice on how to replace those functions to make them compatible with pypy? Mike Fletcher (pyopengl author) gave me some hints about that point: For the first issue, those are going to require some reworking, in essence those are C implemented code that happens to use Python/ctypes as the implementation language and makes assumptions about the data-storage for the objects (e.g. that a string is internally a contiguous series of bytes, which is *not necessarily* true in PyPy). We'd need to find a mechanism in PyPy that would give us that direct memory-pointer access to be able to use it. Note: a compacting garbage collector (or anything else that can move memory locations) will cause problems there, so we may need to find a way to signal PyPy not to move a given object, and to use contiguous data-arrays for their storage. thanks a lot for any advice, renaud 0. PyOpenGL 3.x / The Python OpenGL® Binding http://pyopengl.sourceforge.net/ 1. opengl-programmable / a short step by step tutorial to OpenGL programmable pipeline http://bitbucket.org/rndblnch/opengl-programmable/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] pyopengl on pypy
sorry, i forgot to put a subject. renaud ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] (no subject)
Hey. I would suggest using array module. It allocates memory in a non-moving location and it's address can be found using buffer_info method. Cheers, fijal On Wed, Dec 1, 2010 at 3:18 PM, renaud blanch rndbl...@gmail.com wrote: hi, i'm trying to make some pyopengl [0] -based code [1] run on top of pypy. this is partially successful, but i need some advice to progress further. pyopengl 3.x makes use of ctypes to provide the opengl binding, and it works out of the box for simple functions (those that do not takes c-pointer to buffers of data as arguments). for the rest, the first issue is that pyopengl use two functions from the ctypes.pythonapi lib, namely PyString_AsString and PyBuffer_FromMemory. any advice on how to replace those functions to make them compatible with pypy? Mike Fletcher (pyopengl author) gave me some hints about that point: For the first issue, those are going to require some reworking, in essence those are C implemented code that happens to use Python/ctypes as the implementation language and makes assumptions about the data-storage for the objects (e.g. that a string is internally a contiguous series of bytes, which is *not necessarily* true in PyPy). We'd need to find a mechanism in PyPy that would give us that direct memory-pointer access to be able to use it. Note: a compacting garbage collector (or anything else that can move memory locations) will cause problems there, so we may need to find a way to signal PyPy not to move a given object, and to use contiguous data-arrays for their storage. thanks a lot for any advice, renaud 0. PyOpenGL 3.x / The Python OpenGL® Binding http://pyopengl.sourceforge.net/ 1. opengl-programmable / a short step by step tutorial to OpenGL programmable pipeline http://bitbucket.org/rndblnch/opengl-programmable/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] pyopengl on pypy
Hi Renaud, On Wed, Dec 1, 2010 at 2:21 PM, renaud blanch rndbl...@gmail.com wrote: We'd need to find a mechanism in PyPy that would give us that direct memory-pointer access to be able to use it. Note: a compacting garbage collector (or anything else that can move memory locations) will cause problems there, so we may need to find a way to signal PyPy not to move a given object, and to use contiguous data-arrays for their storage. Indeed, this part from Mike Fletcher got the cause right, but the solution we recommend is different. It's kind of impossible in PyPy to mark objects as non-moving (pin them, as the garbage collection language goes). Instead you need to just use the official ctypes API, create_string_buffer(). It's equivalent, but requires one extra copy of the data. A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] pyopengl on pypy
Hi, 2010/12/1 renaud blanch rndbl...@gmail.com i'm trying to make some pyopengl [0] -based code [1] run on top of pypy. this is partially successful, but i need some advice to progress further. pyopengl 3.x makes use of ctypes to provide the opengl binding, and it works out of the box for simple functions (those that do not takes c-pointer to buffers of data as arguments). for the rest, the first issue is that pyopengl use two functions from the ctypes.pythonapi lib, namely PyString_AsString and PyBuffer_FromMemory. any advice on how to replace those functions to make them compatible with pypy? Mike Fletcher (pyopengl author) gave me some hints about that point: For the first issue, those are going to require some reworking, in essence those are C implemented code that happens to use Python/ctypes as the implementation language and makes assumptions about the data-storage for the objects (e.g. that a string is internally a contiguous series of bytes, which is *not necessarily* true in PyPy). We'd need to find a mechanism in PyPy that would give us that direct memory-pointer access to be able to use it. Note: a compacting garbage collector (or anything else that can move memory locations) will cause problems there, so we may need to find a way to signal PyPy not to move a given object, and to use contiguous data-arrays for their storage. Don't worry, with PyPy's ctypes you'll never get the address of a moving object. c_char_p makes a non-moving copy of the string. But IMO the call to pythonapi.PyString_AsString could be removed in OpenGL/arrays/strings.py: def dataPointer(value): return ctypes.cast(ctypes.c_char_p(value), ctypes.c_void_p).value I don't know about the other function, though. -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] pyopengl on pypy
hi, Amaury Forgeot d'Arc amauryfa at gmail.com writes: But IMO the call to pythonapi.PyString_AsString could be removed in OpenGL/arrays/strings.py: def dataPointer(value): return ctypes.cast(ctypes.c_char_p(value), ctypes.c_void_p).value great, this replacement (and also the one suggested by Armin with create_string_buffer instead of c_char_p) seams to works (i.e., runs fine on cpython). I don't know about the other function, though. this one is only in experimental code of pyopengl, so i won't bother much for now. the next issue is now that the array data types used by pyopengl are not compatible with pypy ctypes: they are lacking (at least) _CData_input as shown in the stack trace below. any advice on that one? should i try to fix PyOpenGL ArrayDatatype, or try to patch pypy's ctypes function.py to work around that one? thanks again, renaud % pypy 01-direct.py Traceback (most recent call last): File app_main.py, line 33, in run_toplevel File 01-direct.py, line 268, in module sys.exit(main()) File 01-direct.py, line 262, in main init_texture() File 01-direct.py, line 48, in init_texture .join(pixel(i, j, k) for i in range(width) File /Users/Shared/src/pypy-1.4-osx/site-packages/OpenGL/latebind.py, line 45, in __call__ return self._finalCall( *args, **named ) File /Users/Shared/src/pypy-1.4-osx/site-packages/OpenGL/wrapper.py, line 784, in wrapperCall result = self.wrappedOperation( *cArguments ) File /Users/Shared/src/pypy-1.4-osx/site-packages/OpenGL/platform/baseplatform.py, line 335, in __call__ return self( *args, **named ) File /Users/Shared/src/pypy-1.4-osx/lib_pypy/_ctypes/function.py, line 166, in __call__ argtypes, argsandobjs = self._wrap_args(argtypes, args) File /Users/Shared/src/pypy-1.4-osx/lib_pypy/_ctypes/function.py, line 281, in _wrap_args wrapped = argtype._CData_input(arg) AttributeError: type object 'ArrayDatatype' has no attribute '_CData_input' ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [PyPy Status Blog] New comment on PyPy 1.4: Ouroboros in practice.
On 01/12/10 23:21, Amaury Forgeot d'Arc wrote: [cut] There is already an ongoing effort to port PyPy to Python 2.7. But we need some help! It's a good way to become a PyPy developer. And no, you don't have to be a JIT expert to implement itertools.combinations or asian codecs. Nice comment Amaury :-). What about writing a blog post explicitly asking for help? I also agree that the fast-forward branch is a good way to enter pypy, moreover we should exploit the hype we seem to have at the moment :-). ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
