Thank you both, for your encouragement. So I have made progress. I now have some unit tests, and they can compile and run. I have imported the memmap.py from numpy, and modified it so it gets the needed items from micronumpy. I can run the unit tests and they fail for the correct reason -- that reason being the buffer attribute is not supported in interp_numarray.descr_new_array() . So that's great.
Here are my next questions: 1) The way the numpy mmap module works, it calls ndarray.__new__(), and monkey-patches the return value to add _mmap, offset, and mode attributes. This, for example, ensures the mmap object is kept around until the array is deleted. However, I can't monkey-patch a numpy ndarray object. I presume this is because it is an interpreter level object rather than an app level one? Anyway -- not sure how to deal with this situation. 2) Secondly, the mmap object itself doesn't really provide a usable buffer implementation. The implementation of buffer(mmap) is currently W_MMap.descr_buffer(), (found in interp_mmap.py), which returns a StringLikeBuffer object. This object (implemented in pypy/interpreter/buffer.py) is a subclass of Buffer, which does not implement get_raw_address(). Our current plan clearly requires the buffer object to implement get_raw_address so it can be used by ndarray.from_shape_and_storage(). Interestingly, it seems as if the interp_mmap author anticipated this shortcoming -- there is a comment: "improve to work directly on low-level address" right in the descr_buffer method. So -- am I on the wrong path? Should I not even bother trying to use the mmap? (since I can't monkey patch it and it doesn't do what I want?) This would mean perhaps using the underlying rffi mmap to build my own memmap module. Alternatively, can I fix the monkey-patching problem some other way, and then take the advice of interp_mmap's author to "improve to work directly on low-level address" by returning something better than a StringLikeBuffer object. Thoughts? Mike On Sun, Jun 16, 2013 at 3:29 PM, Amaury Forgeot d'Arc <[email protected]>wrote: > > 2013/6/16 Mike Beller <[email protected]> > >> Ok -- made some progress. Built a unit test -- it's similar to the >> doctest from memmap.py, but rewritten in py.test format (uses >> BaseNumpyAppTest from pypy.module.micronumpy.test.test_base, just like >> test_numeric.py does). >> >> Here is the next problem I face: To run my unit test, I added in an >> import of my new memmap.py, which I have placed in >> lib_pypy/numpypy/core/memmap.py, and linked into core/__init__.py. This >> module imports mmap.py for obvious reasons (it wants to use mmap.py to >> create the mmap object). That import fails when I run the test: python2.7 >> pytest.py pypy/module/test_lib_pypy/numpypy/core/test_memmap.py ("Import >> error: no module named mmap") >> > > great progress anyway! > > >> It fails because it can not import the module mmap. Whereas if I just >> fire up a normal interpreter-level pypy I have no problem importing mmap. >> Clearly it does exist in pypy. >> > >> I don't understand which modules will be loadable and which modules will >> not be loadable when I run in pytest.py. Somehow, clearly, the >> BaseNumpyAppTest inheritance mechanism has allowed all the numpypy stuff to >> show up (but only inside the class definition, not at module scope), but >> other pypy application-level modules are not available. I need to somehow >> invoke the same magic for mmap if I want to be able to use the mmap module >> to implement the numpypy memmap functionality. I read coding-guide.rst >> but it's still not obvious to me. (Alternatively I suppose the answer >> could be full translation, but testing this way would take a full 90-minute >> translation every time I wanted to change a line of code.) >> > > That's because most built-in modules are not made available by default. > You need to pass the equivalent of "--withmod-mmap" to the interpreter. In > a test, this is done in a statement like: > spaceconfig = dict(usemodules=["mmap"]) > at the top of the test class. There certainly is already one (in order to > import numpy), you could add "mmap" to the existing list. > >> > >> >> If you feel this detailed emailing is wasting your time, just let me know >> and I can drop it. (Or if you want to take the conversation off pypy-dev >> that's ok too.) >> > > You are welcome. PyPy is a wonderful platform with powerful tools, but > they are often not documented enough. > Pypy-dev traffic is not that high these days; to get immediate answers > there many of us are hanging on IRC: #pypy at freenode.net. > > -- > Amaury Forgeot d'Arc >
_______________________________________________ pypy-dev mailing list [email protected] http://mail.python.org/mailman/listinfo/pypy-dev
