[Numpy-discussion] Numpy large array bug
Hello, I have downloaded numpy 1.3rc2 sources and compiled it on Ubuntu Hardy Linux x86_64. numpy.test() seems to run ok as well. Here is the bug I can reproduce import numpy as np a=np.zeros((2*1024*1024*1024 + 1), dtype=uint8) a[:]=1 # returns immediately a.mean() 0.0 print a [0 0 0 ..., 0 0 0] The bug only happens when the nElements 2G (2^31). So for dtype=uint16/32, the bug happens when size is greater thatn 2^31 as well. Can someone please tell me if I can find a patch for this? I checked the mailing list and trac and I cannot find any related bug. Thanks, Ashwin ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
A Monday 21 September 2009 19:45:27 Kashyap Ashwin escrigué: Hello, I have downloaded numpy 1.3rc2 sources and compiled it on Ubuntu Hardy Linux x86_64. numpy.test() seems to run ok as well. Here is the bug I can reproduce import numpy as np a=np.zeros((2*1024*1024*1024 + 1), dtype=uint8) a[:]=1 # returns immediately a.mean() 0.0 print a [0 0 0 ..., 0 0 0] The bug only happens when the nElements 2G (2^31). So for dtype=uint16/32, the bug happens when size is greater thatn 2^31 as well. Yup. I can reproduce your problem with NumPy 1.3.0 (final) and a 64-bit platform. I suppose that you should file a bug better. -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
On Mon, Sep 21, 2009 at 12:30 PM, Francesc Alted fal...@pytables.orgwrote: A Monday 21 September 2009 19:45:27 Kashyap Ashwin escrigué: Hello, I have downloaded numpy 1.3rc2 sources and compiled it on Ubuntu Hardy Linux x86_64. numpy.test() seems to run ok as well. Here is the bug I can reproduce import numpy as np a=np.zeros((2*1024*1024*1024 + 1), dtype=uint8) a[:]=1 # returns immediately a.mean() 0.0 print a [0 0 0 ..., 0 0 0] The bug only happens when the nElements 2G (2^31). So for dtype=uint16/32, the bug happens when size is greater thatn 2^31 as well. Yup. I can reproduce your problem with NumPy 1.3.0 (final) and a 64-bit platform. I suppose that you should file a bug better. Does is persist for svn? IIRC, there is another ticket for a slicing bug for large arrays. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
Yes, it happens for the trunk as well. import numpy as np a=np.zeros((2*1024*1024*1024 + 1), dtype=uint8) a[:]=1 # returns immediately a.mean() 0.0 print a [0 0 0 ..., 0 0 0] The bug only happens when the nElements 2G (2^31). So for dtype=uint16/32, the bug happens when size is greater thatn 2^31 as well. Yup. I can reproduce your problem with NumPy 1.3.0 (final) and a 64-bit platform. I suppose that you should file a bug better. Does is persist for svn? IIRC, there is another ticket for a slicing bug for large arrays. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
I can confirm this bug for the last svn. Also: a.put([2*1024*1024*1024 + 100,], 8) IndexError: index out of range for array in this case, I think the error is that in numpy/core/src/multiarray/item_selection.c in PyArray_PutTo line 209 should be: intp i, chunk, ni, max_item, nv, tmp; instead of: int i, chunk, ni, max_item, nv, tmp; fixing it as suggested: a.put([2*1024*1024*1024 + 100,], 8) a.max() 8 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Numpy large array bug
Also, what about PyArray_PutMask() That function also has a line like int i, chunk, ni, max_item, nv, tmp; Should that be changed as well? (Your patch does not fix my original issue.) BTW, in numpy 1.3, that is present in numpy/core/src/multiarraymodule.c. Can someone please give me a temporary patch to test? I am not familiar with numpy codebase! -Ashwin I can confirm this bug for the last svn. Also: a.put([2*1024*1024*1024 + 100,], 8) IndexError: index out of range for array in this case, I think the error is that in numpy/core/src/multiarray/item_selection.c in PyArray_PutTo line 209 should be: intp i, chunk, ni, max_item, nv, tmp; instead of: int i, chunk, ni, max_item, nv, tmp; fixing it as suggested: a.put([2*1024*1024*1024 + 100,], 8) a.max() 8 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
I think the original bug is due to line 535 of numpy/core/src/multiarray/ctors.c (svn) that should be: intp numcopies, nbytes; instead of: int numcopies, nbytes; To resume: in line 535 of numpy/core/src/multiarray/ctors.c and in line 209 of numpy/core/src/multiarray/item_selection.c int should be replaced with intp. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
Hi, Luca, On Mon, Sep 21, 2009 at 4:52 PM, Citi, Luca lc...@essex.ac.uk wrote: I think the original bug is due to line 535 of numpy/core/src/multiarray/ctors.c (svn) that should be: intp numcopies, nbytes; instead of: int numcopies, nbytes; To resume: in line 535 of numpy/core/src/multiarray/ctors.c and in line 209 of numpy/core/src/multiarray/item_selection.c int should be replaced with intp. Please open a ticket for this. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy large array bug
Here it is... http://projects.scipy.org/numpy/ticket/1229 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
