Ok -- made some progress. Built a unit test -- it's similar to the doctest
from memmap.py, but rewritten in py.test format (uses BaseNumpyAppTest from
pypy.module.micronumpy.test.test_base, just like test_numeric.py does).
Here is the next problem I face: To run my unit test, I added in an import
of my new memmap.py, which I have placed in
lib_pypy/numpypy/core/memmap.py, and linked into core/__init__.py. This
module imports mmap.py for obvious reasons (it wants to use mmap.py to
create the mmap object). That import fails when I run the test: python2.7
pytest.py pypy/module/test_lib_pypy/numpypy/core/test_memmap.py ("Import
error: no module named mmap")
It fails because it can not import the module mmap. Whereas if I just fire
up a normal interpreter-level pypy I have no problem importing mmap.
Clearly it does exist in pypy.
I don't understand which modules will be loadable and which modules will
not be loadable when I run in pytest.py. Somehow, clearly, the
BaseNumpyAppTest inheritance mechanism has allowed all the numpypy stuff to
show up (but only inside the class definition, not at module scope), but
other pypy application-level modules are not available. I need to somehow
invoke the same magic for mmap if I want to be able to use the mmap module
to implement the numpypy memmap functionality. I read coding-guide.rst
but it's still not obvious to me. (Alternatively I suppose the answer
could be full translation, but testing this way would take a full 90-minute
translation every time I wanted to change a line of code.)
If you feel this detailed emailing is wasting your time, just let me know
and I can drop it. (Or if you want to take the conversation off pypy-dev
that's ok too.)
Regards
Mike
On Fri, Jun 14, 2013 at 5:43 PM, Amaury Forgeot d'Arc <[email protected]>wrote:
> 2013/6/14 Mike Beller <[email protected]>
>
>> It seems to me one needs to modify ndarray.__new__ so it can take a named
>> buffer= argument, where that argument can be an mmap object. Currently
>> the python ndarray.__new__ can accept such an argument, but the numpypy one
>> does not appear to.
>
>
> Correct.
>
>
>> When I try to figure out the changes to interp_numarray.py to make it
>> work, it goes way over my head.
>
>
> Hey, you need to get used to our RPython language. At least you opened the
> correct file, that's a great first step :-). In interp_numarray.py:
>
> - the block starting with "W_NDimArray.typedef" describe the type and the
> methods. There is a "__new__" entry, which describes the constructor. The
> implementation is descr_new_array().
>
> - in descr_new_array(), the variables that start with "w_" are Wrapped
> objects, i.e. visible to Python (similar to PyObject* in the CPython
> implementation). At the moment there is a check for
> "space.is_none(w_buffer)", we'll have to remove it of course.
>
> - The goal is to use the memory allocated for w_buffer memory instead of
> allocating a new one.
> So don't call W_NDimArray.from_shape(),
> call W_NDimArray.from_shape_and_storage() instead, it takes the raw address
> of the buffer. That's it!
>
> Of course I skipped over all the details, so the first thing to do is to
> write a unit test to be sure that everything works correctly before
> building a new PyPy.
>
>
> --
> Amaury Forgeot d'Arc
>
_______________________________________________
pypy-dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-dev
