On Fri, Sep 4, 2009 at 1:36 PM, David Warde-Farley <d...@cs.toronto.edu>wrote:
> A friend of mine is trying to save approx 2GB of float32s with
> np.save, and it's been failing. I traced it to PyArray_ToFile in core/
> src/convert.c:
>
> Traceback (most recent call last):
> File "preprocessTIMIT.py", line 302, in <module>
> main()
> File "preprocessTIMIT.py", line 299, in main
> num.save("/ais/gobi1/gdahl/speech/%s" % k, d[k])
> File "/nobackup/murray/bin/pylab-02/x86_64_GenuineIntel_6.06/
> Python-2.5.4/lib/python2.5/site-packages/numpy/lib/io.py", line 241,
> in save
> format.write_array(fid, arr)
> File "/nobackup/murray/bin/pylab-02/x86_64_GenuineIntel_6.06/
> Python-2.5.4/lib/python2.5/site-packages/numpy/lib/format.py", line
> 322, in write_array
> array.tofile(fp)
> ValueError: 541110272 requested and 2028 written
>
> This is writing to a local ext3 filesystem. Apparently fwrite() is
> giving back values in the low thousands instead of the correct amount.
>
> Has anyone encountered this, or know why it would happen? The values
> are 1004 and 2028 that we seem to keep seeing, which as far as I can
> tell are not error codes.
>
>
The odd values might be from the format code in the error message:
PyErr_Format(PyExc_ValueError,
"%ld requested and %ld written",
(long) size, (long) n);
The code that is immediately responsible for the write is in lines 79-92 of
convert.c. You could do a bit of poking around in there to find out what is
happening.
Chuck
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
