On Sat, Sep 22, 2012 at 12:19 PM, Charles R Harris < [email protected]> wrote:
> > > On Sat, Sep 22, 2012 at 12:06 PM, Olivier Grisel <[email protected] > > wrote: > >> 2012/9/22 Charles R Harris <[email protected]>: >> > >> > >> > On Sat, Sep 22, 2012 at 11:52 AM, Charles R Harris >> > <[email protected]> wrote: >> >> >> >> >> >> >> >> On Sat, Sep 22, 2012 at 11:31 AM, Gael Varoquaux >> >> <[email protected]> wrote: >> >>> >> >>> On Sat, Sep 22, 2012 at 11:16:27AM -0600, Charles R Harris wrote: >> >>> > I think this is a bug, taking a view should probably update the >> >>> > offset. >> >>> >> >>> OK, we can include a fix for that alongside with the patch to keep >> track >> >>> of the filename. >> >> >> >> >> >> It already tracks the file name >> >> >> >> In [1]: a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, >> mode='w+', >> >> offset=4) >> >> >> >> In [2]: b = a[10:] >> >> >> >> In [3]: b.filename >> >> Out[3]: '/home/charris/tmp.mmap' >> >> >> >> or did you mean something else? I was guessing the fix could be mad in >> the >> >> same place that copied over the filename. >> >> >> > >> > You can also tell it is a memmap >> > >> > In [4]: b._mmap >> > Out[4]: <mmap.mmap at 0x2312570> >> >> The problem is with: >> >> >>> c = np.asarray(b) >> >>> c.base >> <mmap.mmap at 0x2312570> >> >> But you loose the pointer to the filename and the offset. In previous >> versions of numpy c.base used to be the np.memmap instance from which >> c is an array view. That allowed to make efficient pickling without >> any memory copy when doing single machine multiprocessing stuff by >> introspecting the base ancestry. >> >> This is no longer possible with the current base collapsing that is >> happening in numpy master. The only way would be to replace the >> mmap.mmap instance of a numpy.memmap object by a buffer implementation >> that would wrap or derive from mmap.mmap but also preserve the >> original filename and offset. >> > > Pickling was left as an unresolved problem after to offset updates to > memmap. It would be nice to get all those issues fixed up. > > As to the 1.7 release, I've been thinking we are violating the release > early, release often maxim. Bugs trickle in at a constant rate and if we > wait to fix them all we wait forever. So while it would be nice to have > this in 1.7.0, I think we should also plan on a 1.7.1 bug fix release a few > months after the 1.7.0 release. > > Previous work was at http://projects.scipy.org/numpy/ticket/1452, which is probabably worth a read. Chuck
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
