[Numpy-discussion] memory address of array data?
Is there a function to get the memory address (int) of (contigious) ndarray data on Python level - like array.array.buffer_info() ? I'd need it to pass it to a camera function. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memory address of array data?
2009/8/16 Robert kxrobe...@googlemail.com: Is there a function to get the memory address (int) of (contigious) ndarray data on Python level - like array.array.buffer_info() ? I'd need it to pass it to a camera function. Have a look at the array interface: x.__array_interface__['data'] Regards Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a better way to do this arrayrepeat?
As you stress on repeat the array ... rather than repeat each element, you may want to consider tile as well: np.tile(a, [10,1]) array([[0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2]]) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memory address of array data?
On Sun, Aug 16, 2009 at 11:50:44PM -0700, Stéfan van der Walt wrote: 2009/8/16 Robert kxrobe...@googlemail.com: Is there a function to get the memory address (int) of (contigious) ndarray data on Python level - like array.array.buffer_info() ? I'd need it to pass it to a camera function. Have a look at the array interface: x.__array_interface__['data'] Also, it might be interesting to have a look at the ctypes support: http://www.scipy.org/Cookbook/Ctypes Gaël ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Specifying Index Programmatically
There is also numpy.s_: inds = np.s_[...,2,:] z[inds] (Though there are some problems with negative indices: see for example http://www.mail-archive.com/numpy-discussion@scipy.org/msg18245.html) On 8 Aug 2009, at 10:02 PM, T J wrote: On Sat, Aug 8, 2009 at 8:54 PM, Neil Martinsen-Burrelln...@wartburg.edu wrote: The ellipsis is a built-in python constant called Ellipsis. The colon is a slice object, again a python built-in, called with None as an argument. So, z[...,2,:] == z[Ellipsis,2,slice(None)]. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IndexExpression bug?
Submitted as ticket 1196 http://projects.scipy.org/numpy/ticket/1196 On 5 Jun 2009, at 4:12 PM, Robert Kern wrote: On Fri, Jun 5, 2009 at 16:14, Michael McNeil Forbes mfor...@physics.ubc.ca wrote: np.array([0,1,2,3])[1:-1] array([1, 2]) but np.array([0,1,2,3])[np.s_[1:-1]] array([1, 2, 3]) np.array([0,1,2,3])[np.index_exp[1:-1]] array([1, 2, 3]) ... I think that getting rid of __getslice__ and __len__ should work better. I don't really understand what the logic was behind including them in the first place, though. I might be missing something. ... ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a better way to do this arrayrepeat?
That's exactly it. Thanks! On Mon, Aug 17, 2009 at 5:24 AM, Citi, Lucalc...@essex.ac.uk wrote: As you stress on repeat the array ... rather than repeat each element, you may want to consider tile as well: np.tile(a, [10,1]) array([[0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2]]) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Strange crash in numpy.linalg.lstsq.
Hi, I am getting a strange crash in numpy.linalg.lstsq. I have put the code that causes the crash along with two data files on my website at: http://www.cs.toronto.edu/~jtaylor/crash/ I would be interested to know if this bug can be duplicated and/or if anyone has any suggestions as to why: import numpy as np A = np.load('A.npy') b = np.load('b.npy') rc = np.linalg.lstsq(A,b) produces: *** glibc detected *** /usr/bin/python: free(): invalid next size (normal): 0x091793c0 *** === Backtrace: = /lib/tls/i686/cmov/libc.so.6[0xb7dc7a85] /lib/tls/i686/cmov/libc.so.6(cfree+0x90)[0xb7dcb4f0] /u/jtaylor/lib/python2.5/site-packages/numpy/core/multiarray.so[0xb795403e] /usr/bin/python[0x811247a] /usr/bin/python(PyEval_EvalCodeEx+0x323)[0x80cae33] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalCode+0x57)[0x80cb347] /usr/bin/python(PyRun_FileExFlags+0xf8)[0x80ea818] /usr/bin/python[0x80c1f5a] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x4064)[0x80c7e04] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python[0x8113696] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x4064)[0x80c7e04] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python[0x8113696] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python[0x8062bfb] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x3d07)[0x80c7aa7] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalFrameEx+0x5945)[0x80c96e5] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x6d09)[0x80caaa9] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalFrameEx+0x5945)[0x80c96e5] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalCode+0x57)[0x80cb347] /usr/bin/python(PyRun_FileExFlags+0xf8)[0x80ea818] /usr/bin/python(PyRun_SimpleFileExFlags+0x199)[0x80eaab9] /usr/bin/python(Py_Main+0xa35)[0x8059335] /usr/bin/python(main+0x22)[0x80587f2] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe0)[0xb7d72450] /usr/bin/python[0x8058761] === Memory map: 08048000-0814 r-xp 08:06 83501 /usr/bin/python2.5 0814-08165000 rw-p 000f7000 08:06 83501 /usr/bin/python2.5 08165000-0919a000 rw-p 08165000 00:00 0 [heap] b520-b5221000 rw-p b520 00:00 0 b5221000-b530 ---p b5221000 00:00 0 b53fc000-b5499000 r-xp 00:1a 552170 /h/44/jtaylor/lib/python2.5/site-packages/Cython/Compiler/Parsing.so b5499000-b54a2000 rw-p 0009d000 00:1a 552170 /h/44/jtaylor/lib/python2.5/site-packages/Cython/Compiler/Parsing.so b54a2000-b5624000 rw-p b54a2000 00:00 0 b5624000-b568f000 r-xp 00:1a 553542 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_backend_agg.so b568f000-b5691000 rw-p 0006a000 00:1a 553542 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_backend_agg.so b5691000-b56f6000 r-xp 08:06 90831 /usr/lib/python2.5/lib-dynload/unicodedata.so b56f6000-b5705000 rw-p 00065000 08:06 90831 /usr/lib/python2.5/lib-dynload/unicodedata.so b5705000-b5725000 r-xp 00:1a 553545 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_tkagg.so b5725000-b5726000 rw-p 0002 00:1a 553545 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_tkagg.so b5726000-b5727000 ---p b5726000 00:00 0 b5727000-b5f27000 rwxp b5727000 00:00 0 b5f27000-b5f3e000 r-xp 08:06 85532 /usr/lib/libxcb.so.1.0.0 b5f3e000-b5f3f000 rw-p 00016000 08:06 85532 /usr/lib/libxcb.so.1.0.0 b5f3f000-b5f53000 r-xp 08:06 1187870/lib/tls/i686/cmov/ libnsl-2.7.so b5f53000-b5f55000 rw-p 00013000 08:06 1187870/lib/tls/i686/cmov/ libnsl-2.7.so b5f55000-b5f57000 rw-p b5f55000 00:00 0 b5f57000-b603b000 r-xp 08:06 85536 /usr/lib/libX11.so.6.2.0 b603b000-b603e000 rw-p 000e4000 08:06 85536 /usr/lib/libX11.so.6.2.0 b603e000-b60e7000 r-xp 08:06 85098 /usr/lib/libtcl8.4.so.0 b60e7000-b60f1000 rw-p 000a8000 08:06 85098 /usr/lib/libtcl8.4.so.0 b60f1000-b60f2000 rw-p b60f1000 00:00 0 b60f2000-b61c4000 r-xp 08:06 85102 /usr/lib/libtk8.4.so.0 b61c4000-b61cf000 rw-p 000d2000 08:06 85102 /usr/lib/libtk8.4.so.0
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 1:42 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: Hi, I am getting a strange crash in numpy.linalg.lstsq. I have put the code that causes the crash along with two data files on my website at: http://www.cs.toronto.edu/~jtaylor/crash/ I would be interested to know if this bug can be duplicated and/or if anyone has any suggestions as to why: import numpy as np A = np.load('A.npy') b = np.load('b.npy') rc = np.linalg.lstsq(A,b) produces: *** glibc detected *** /usr/bin/python: free(): invalid next size (normal): 0x091793c0 *** === Backtrace: = /lib/tls/i686/cmov/libc.so.6[0xb7dc7a85] /lib/tls/i686/cmov/libc.so.6(cfree+0x90)[0xb7dcb4f0] /u/jtaylor/lib/python2.5/site-packages/numpy/core/multiarray.so[0xb795403e] /usr/bin/python[0x811247a] /usr/bin/python(PyEval_EvalCodeEx+0x323)[0x80cae33] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalCode+0x57)[0x80cb347] /usr/bin/python(PyRun_FileExFlags+0xf8)[0x80ea818] /usr/bin/python[0x80c1f5a] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x4064)[0x80c7e04] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python[0x8113696] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x4064)[0x80c7e04] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python[0x8113696] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python[0x8062bfb] /usr/bin/python(PyObject_Call+0x27)[0x805cb97] /usr/bin/python(PyEval_EvalFrameEx+0x3d07)[0x80c7aa7] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalFrameEx+0x5945)[0x80c96e5] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x6d09)[0x80caaa9] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalFrameEx+0x5945)[0x80c96e5] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalFrameEx+0x565e)[0x80c93fe] /usr/bin/python(PyEval_EvalCodeEx+0x6e7)[0x80cb1f7] /usr/bin/python(PyEval_EvalCode+0x57)[0x80cb347] /usr/bin/python(PyRun_FileExFlags+0xf8)[0x80ea818] /usr/bin/python(PyRun_SimpleFileExFlags+0x199)[0x80eaab9] /usr/bin/python(Py_Main+0xa35)[0x8059335] /usr/bin/python(main+0x22)[0x80587f2] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe0)[0xb7d72450] /usr/bin/python[0x8058761] === Memory map: 08048000-0814 r-xp 08:06 83501 /usr/bin/python2.5 0814-08165000 rw-p 000f7000 08:06 83501 /usr/bin/python2.5 08165000-0919a000 rw-p 08165000 00:00 0 [heap] b520-b5221000 rw-p b520 00:00 0 b5221000-b530 ---p b5221000 00:00 0 b53fc000-b5499000 r-xp 00:1a 552170 /h/44/jtaylor/lib/python2.5/site-packages/Cython/Compiler/Parsing.so b5499000-b54a2000 rw-p 0009d000 00:1a 552170 /h/44/jtaylor/lib/python2.5/site-packages/Cython/Compiler/Parsing.so b54a2000-b5624000 rw-p b54a2000 00:00 0 b5624000-b568f000 r-xp 00:1a 553542 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_backend_agg.so b568f000-b5691000 rw-p 0006a000 00:1a 553542 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_backend_agg.so b5691000-b56f6000 r-xp 08:06 90831 /usr/lib/python2.5/lib-dynload/unicodedata.so b56f6000-b5705000 rw-p 00065000 08:06 90831 /usr/lib/python2.5/lib-dynload/unicodedata.so b5705000-b5725000 r-xp 00:1a 553545 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_tkagg.so b5725000-b5726000 rw-p 0002 00:1a 553545 /h/44/jtaylor/build/matplotlib/lib/matplotlib/backends/_tkagg.so b5726000-b5727000 ---p b5726000 00:00 0 b5727000-b5f27000 rwxp b5727000 00:00 0 b5f27000-b5f3e000 r-xp 08:06 85532 /usr/lib/libxcb.so.1.0.0 b5f3e000-b5f3f000 rw-p 00016000 08:06 85532 /usr/lib/libxcb.so.1.0.0 b5f3f000-b5f53000 r-xp 08:06 1187870 /lib/tls/i686/cmov/libnsl-2.7.so b5f53000-b5f55000 rw-p 00013000 08:06 1187870 /lib/tls/i686/cmov/libnsl-2.7.so b5f55000-b5f57000 rw-p b5f55000 00:00 0 b5f57000-b603b000 r-xp 08:06 85536 /usr/lib/libX11.so.6.2.0 b603b000-b603e000 rw-p 000e4000 08:06 85536 /usr/lib/libX11.so.6.2.0 b603e000-b60e7000 r-xp 08:06 85098 /usr/lib/libtcl8.4.so.0 b60e7000-b60f1000 rw-p 000a8000 08:06 85098 /usr/lib/libtcl8.4.so.0 b60f1000-b60f2000 rw-p
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
Hi Jonathan, http://www.cs.toronto.edu/~jtaylor/crash/ I would be interested to know if this bug can be duplicated and/or if anyone has any suggestions as to why: import numpy as np A = np.load('A.npy') b = np.load('b.npy') rc = np.linalg.lstsq(A,b) produces: *** glibc detected *** /usr/bin/python: free(): invalid next size (normal): 0x091793c0 *** I just tried it on 4 ubuntu machines, and one Fedora 11 machine, in various states of numpy-ness (including recent SVN) with no crash. What versions of stuff do you have over there? See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 11:42 AM, Jonathan Taylor jonathan.tay...@utoronto.ca wrote: Hi, I am getting a strange crash in numpy.linalg.lstsq. I have put the code that causes the crash along with two data files on my website at: http://www.cs.toronto.edu/~jtaylor/crash/http://www.cs.toronto.edu/%7Ejtaylor/crash/ I would be interested to know if this bug can be duplicated and/or if anyone has any suggestions as to why: Usually these problems are due to ATLAS. If you are using ATLAS, what is your OS/distribution? What hardware are you running on? Did you build ATLAS yourself? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
Hi, I am using a computer that is administered. It is an intel Ubuntu box and came with an ATLAS compiled. I thus compiled my own numpy1.3.0 against that ATLAS. I was thinking about recompiling ATLAS myself. This machine only has g77 and not gfortran on it. Will that still work? Thanks, Jonathan. On Mon, Aug 17, 2009 at 3:12 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Aug 17, 2009 at 11:42 AM, Jonathan Taylor jonathan.tay...@utoronto.ca wrote: Hi, I am getting a strange crash in numpy.linalg.lstsq. I have put the code that causes the crash along with two data files on my website at: http://www.cs.toronto.edu/~jtaylor/crash/ I would be interested to know if this bug can be duplicated and/or if anyone has any suggestions as to why: Usually these problems are due to ATLAS. If you are using ATLAS, what is your OS/distribution? What hardware are you running on? Did you build ATLAS yourself? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ImportError: No module named multiarray
Hi, I created a pickled file on my Windows PC, uploaded to a Linux machine and then received the following error: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import cPickle cPickle.load(open('tst.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray Obviously, the pickled file loads fine on the Windows PC. the following is the result of numpy.test() numpy.test() Running unit tests for numpy NumPy version 1.2.1 NumPy is installed in /nfs/02/reckoner/Starburst/lib/python2.5/site-packages/numpy Python version 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] nose version 0.10.3 ..FK.. .. == FAIL: test_umath.TestComplexFunctions.test_against_cmath -- Traceback (most recent call last): File /nfs/02/reckoner/Starburst/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg/nose/case.py, line 182, in runTest self.test(*self.arg) File /nfs/02/reckoner/Starburst/lib/python2.5/site-packages/numpy/core/tests/test_umath.py, line 268, in test_against_cmath assert abs(a - b) atol, %s %s: %s; cmath: %s%(fname,p,a,b) AssertionError: arcsinh -2j: (-1.31695789692-1.57079632679j); cmath: (1.31695789692-1.57079632679j) -- Ran 1740 tests in 10.493s FAILED (KNOWNFAIL=1, failures=1) nose.result.TextTestResult run=1740 errors=0 failures=1 Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 1:43 PM, Jonathan Taylor jonathan.tay...@utoronto.ca wrote: Hi, I am using a computer that is administered. It is an intel Ubuntu box and came with an ATLAS compiled. I thus compiled my own numpy1.3.0 against that ATLAS. I was thinking about recompiling ATLAS myself. This machine only has g77 and not gfortran on it. Will that still work? As long as everything is consistent it should. Ubuntu has had some issues with ATLAS and I suspect that is what you are seeing. David Cournapeau could tell you more. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
I compiled lapack and atlas from scratch using g77 but now numpy.test() hangs when I try to use any numpy functionality. I think I saw someone else write about this. Is this a common problem? Thanks, Jonathan. On Mon, Aug 17, 2009 at 4:38 PM, Charles R Harrischarlesr.har...@gmail.com wrote: On Mon, Aug 17, 2009 at 1:43 PM, Jonathan Taylor jonathan.tay...@utoronto.ca wrote: Hi, I am using a computer that is administered. It is an intel Ubuntu box and came with an ATLAS compiled. I thus compiled my own numpy1.3.0 against that ATLAS. I was thinking about recompiling ATLAS myself. This machine only has g77 and not gfortran on it. Will that still work? As long as everything is consistent it should. Ubuntu has had some issues with ATLAS and I suspect that is what you are seeing. David Cournapeau could tell you more. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 1:50 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: I compiled lapack and atlas from scratch using g77 but now numpy.test() hangs when I try to use any numpy functionality. I think I saw someone else write about this. Is this a common problem? Yes, it seems common. I know of 4 recent ATLAS builds (including mine but not yours) that have failed on 32 bit systems. The recent successes I have seen (including mine) have been on 64 bit systems. But maybe 32/64 bit has nothing to do with it. I am sure there are many 32 bit systems running a self-compiled ATLAS just fine. Oh, you crash on any numpy functionality? I only crashed on ATLAS type problems. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
Yes... ATLAS type problems like matrix multiplication. Is there some alternative to get a working numpy going? How might I go about compiling numpy without ATLAS? I really got to get at least something working temporarily. Thanks, Jon. On Mon, Aug 17, 2009 at 5:03 PM, Keith Goodmankwgood...@gmail.com wrote: On Mon, Aug 17, 2009 at 1:50 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: I compiled lapack and atlas from scratch using g77 but now numpy.test() hangs when I try to use any numpy functionality. I think I saw someone else write about this. Is this a common problem? Yes, it seems common. I know of 4 recent ATLAS builds (including mine but not yours) that have failed on 32 bit systems. The recent successes I have seen (including mine) have been on 64 bit systems. But maybe 32/64 bit has nothing to do with it. I am sure there are many 32 bit systems running a self-compiled ATLAS just fine. Oh, you crash on any numpy functionality? I only crashed on ATLAS type problems. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 2:13 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: Is there some alternative to get a working numpy going? How might I go about compiling numpy without ATLAS? I really got to get at least something working temporarily. Just build numpy again but skip the ATLAS steps. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
It seems to automatically detect it though. Specifically lapack_lite.so always seems to reference libatlas. On Mon, Aug 17, 2009 at 5:21 PM, Keith Goodmankwgood...@gmail.com wrote: On Mon, Aug 17, 2009 at 2:13 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: Is there some alternative to get a working numpy going? How might I go about compiling numpy without ATLAS? I really got to get at least something working temporarily. Just build numpy again but skip the ATLAS steps. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange crash in numpy.linalg.lstsq.
On Mon, Aug 17, 2009 at 2:27 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: It seems to automatically detect it though. Specifically lapack_lite.so always seems to reference libatlas. On Mon, Aug 17, 2009 at 5:21 PM, Keith Goodmankwgood...@gmail.com wrote: On Mon, Aug 17, 2009 at 2:13 PM, Jonathan Taylorjonathan.tay...@utoronto.ca wrote: Is there some alternative to get a working numpy going? How might I go about compiling numpy without ATLAS? I really got to get at least something working temporarily. Just build numpy again but skip the ATLAS steps. Yes, sorry. The only way I've tried doing it is uninstalling the unbuntu ATLAS binary. I don't know how to ignore it if it is installed. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] How to compile numpy without ATLAS support?
I am wondering how I might be able to compile numpy without ATLAS on a ubuntu machine that has an atlas deb installed. It seems that the numpy build routine automatically detects it. Thanks for any help, Jonathan. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Indexing empty array with empty boolean array causes IndexError: invalid index exception
On Aug 12, 2009, at 9:25 AM, Robert Kern wrote: On Mon, Aug 10, 2009 at 14:19, Maria Liukisliu...@usc.edu wrote: Hello everybody, I'm using following versions of Scipy and Numpy packages: scipy.__version__ '0.7.1' np.__version__ '1.3.0' My code uses boolean array to filter 2-dimensional array which sometimes happens to be an empty array. It seems like I have to take special care when dimension I'm filtering is zero, otherwise I'm getting an IndexError: invalid index exception: import numpy as np a = np.zeros((2,10)) a array([[ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], [ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]]) If that were actually your output from zeros(), that would definitely be a bug. :-) Sorry, I realized my copy-and-paste mistake a minute after I posted the message. Obviously, it was too late :) filter_array = np.zeros(2,) filter_array array([False, False], dtype=bool) a[filter_array,:] array([], shape=(0, 10), dtype=float64) Now if filtered dimension is zero: a = np.ones((0,10)) a array([], shape=(0, 10), dtype=float64) filter_array = np.zeros((0,), dtype=bool) filter_array array([], dtype=bool) filter_array.shape (0,) a.shape (0, 10) a[filter_array,:] Traceback (most recent call last): File stdin, line 1, in module IndexError: invalid index Would somebody know if it's an expected behavior, a package bug or am I doing something wrong? I would call it a bug. It's a corner case that we should probably handle gracefully rather than raising an exception. Thanks, Robert! -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to compile numpy without ATLAS support?
On Mon, Aug 17, 2009 at 4:25 PM, Jonathan Taylor jonathan.tay...@utoronto.ca wrote: I am wondering how I might be able to compile numpy without ATLAS on a ubuntu machine that has an atlas deb installed. It seems that the numpy build routine automatically detects it. Try BLAS=None LAPACK=None* ATLAS*=None python setup.py I haven't tried it myself but it is rumored to work ;) Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] unique rows of array
Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique (array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: ## A SNIPPET FROM THE DISCUSSION [Numpy-discussion] Finding unique rows in an array [Was: Finding a row match within a numpy array] A Tuesday 21 August 2007, Mark.Miller escrigué: A slightly related question on this topic... Is there a good loopless way to identify all of the unique rows in an array? Something like numpy.unique() is ideal, but capable of extracting unique subarrays along an axis. You can always do a view of the rows as strings and then use unique(). Here is an example: In [1]: import numpy In [2]: a=numpy.arange(12).reshape(4,3) In [3]: a[2]=(3,4,5) In [4]: a Out[4]: array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) now, create the view and select the unique rows: In [5]: b=numpy.unique(a.view('S%d'%a.itemsize*a.shape[0])).view('i4') and finally restore the shape: In [6]: b.reshape((len(b)/a.shape[1], a.shape[1])) Out[6]: array([[ 0, 1, 2], [ 3, 4, 5], [ 9, 10, 11]]) If you want to find unique columns instead of rows, do a tranpose first on the initial array. END OF DISCUSSION Provided example works only because array elements are row-sorted. Changing tested array to (in my case, it's 'c'): c array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) c[0] = (11, 10, 0) c array([[11, 10, 0], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) b = np.unique(c.view('S%s' %c.itemsize*c.shape[0])) b array(['', '\x03', '\x04', '\x05', '\t', '\n', '\x0b'], dtype='|S4') b.view('i4') array([ 0, 3, 4, 5, 9, 10, 11]) b.reshape((len(b)/c.shape[1], c.shape[1])).view('i4') Traceback (most recent call last): File stdin, line 1, in module ValueError: total size of new array must be unchanged Since len(b) = 7. Suggested approach would work if the whole row would be converted to a single string, I guess. But from what I could gather, numpy.array.view() only changes display element-wise. Before I start re-inventing the wheel, I was just wondering if using existing numpy functionality one could find unique rows in an array. Many thanks in advance! Masha liu...@usc.edu ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Tue, Aug 18, 2009 at 12:30 AM, Maria Liukisliu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: ## A SNIPPET FROM THE DISCUSSION [Numpy-discussion] Finding unique rows in an array [Was: Finding a row match within a numpy array] A Tuesday 21 August 2007, Mark.Miller escrigué: A slightly related question on this topic... Is there a good loopless way to identify all of the unique rows in an array? Something like numpy.unique() is ideal, but capable of extracting unique subarrays along an axis. You can always do a view of the rows as strings and then use unique(). Here is an example: In [1]: import numpy In [2]: a=numpy.arange(12).reshape(4,3) In [3]: a[2]=(3,4,5) In [4]: a Out[4]: array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) now, create the view and select the unique rows: In [5]: b=numpy.unique(a.view('S%d'%a.itemsize*a.shape[0])).view('i4') and finally restore the shape: In [6]: b.reshape((len(b)/a.shape[1], a.shape[1])) Out[6]: array([[ 0, 1, 2], [ 3, 4, 5], [ 9, 10, 11]]) If you want to find unique columns instead of rows, do a tranpose first on the initial array. END OF DISCUSSION Provided example works only because array elements are row-sorted. Changing tested array to (in my case, it's 'c'): c array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) c[0] = (11, 10, 0) c array([[11, 10, 0], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) b = np.unique(c.view('S%s' %c.itemsize*c.shape[0])) b array(['', '\x03', '\x04', '\x05', '\t', '\n', '\x0b'], dtype='|S4') b.view('i4') array([ 0, 3, 4, 5, 9, 10, 11]) b.reshape((len(b)/c.shape[1], c.shape[1])).view('i4') Traceback (most recent call last): File stdin, line 1, in module ValueError: total size of new array must be unchanged Since len(b) = 7. Suggested approach would work if the whole row would be converted to a single string, I guess. But from what I could gather, numpy.array.view() only changes display element-wise. Before I start re-inventing the wheel, I was just wondering if using existing numpy functionality one could find unique rows in an array. Many thanks in advance! Masha liu...@usc.edu one way is to convert to structured array c = np.array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view(c.dtype).reshape(-1,c.shape[1]) array([[ 0, 1, 2], [ 3, 4, 5], [ 9, 10, 11]]) for explanation, I asked a similar question last december about sortrows. (I never remember, when I need the last reshape and when not) Josef ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Mon, Aug 17, 2009 at 10:30 PM, Maria Liukis liu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: Just to be clear, do you mean finding all rows that only occur once in the array? snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
Josef, Thanks, I'll try that and will search for your question from last december :) Masha liu...@usc.edu On Aug 17, 2009, at 9:44 PM, josef.p...@gmail.com wrote: On Tue, Aug 18, 2009 at 12:30 AM, Maria Liukisliu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: ## A SNIPPET FROM THE DISCUSSION [Numpy-discussion] Finding unique rows in an array [Was: Finding a row match within a numpy array] A Tuesday 21 August 2007, Mark.Miller escrigué: A slightly related question on this topic... Is there a good loopless way to identify all of the unique rows in an array? Something like numpy.unique() is ideal, but capable of extracting unique subarrays along an axis. You can always do a view of the rows as strings and then use unique (). Here is an example: In [1]: import numpy In [2]: a=numpy.arange(12).reshape(4,3) In [3]: a[2]=(3,4,5) In [4]: a Out[4]: array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) now, create the view and select the unique rows: In [5]: b=numpy.unique(a.view('S%d'%a.itemsize*a.shape[0])).view ('i4') and finally restore the shape: In [6]: b.reshape((len(b)/a.shape[1], a.shape[1])) Out[6]: array([[ 0, 1, 2], [ 3, 4, 5], [ 9, 10, 11]]) If you want to find unique columns instead of rows, do a tranpose first on the initial array. END OF DISCUSSION Provided example works only because array elements are row-sorted. Changing tested array to (in my case, it's 'c'): c array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) c[0] = (11, 10, 0) c array([[11, 10, 0], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) b = np.unique(c.view('S%s' %c.itemsize*c.shape[0])) b array(['', '\x03', '\x04', '\x05', '\t', '\n', '\x0b'], dtype='|S4') b.view('i4') array([ 0, 3, 4, 5, 9, 10, 11]) b.reshape((len(b)/c.shape[1], c.shape[1])).view('i4') Traceback (most recent call last): File stdin, line 1, in module ValueError: total size of new array must be unchanged Since len(b) = 7. Suggested approach would work if the whole row would be converted to a single string, I guess. But from what I could gather, numpy.array.view() only changes display element-wise. Before I start re-inventing the wheel, I was just wondering if using existing numpy functionality one could find unique rows in an array. Many thanks in advance! Masha liu...@usc.edu one way is to convert to structured array c = np.array([[ 0, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view (c.dtype).reshape(-1,c.shape[1]) array([[ 0, 1, 2], [ 3, 4, 5], [ 9, 10, 11]]) for explanation, I asked a similar question last december about sortrows. (I never remember, when I need the last reshape and when not) Josef ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Aug 17, 2009, at 9:51 PM, Charles R Harris wrote: On Mon, Aug 17, 2009 at 10:30 PM, Maria Liukis liu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: Just to be clear, do you mean finding all rows that only occur once in the array? Yes. snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Tue, Aug 18, 2009 at 12:59 AM, Maria Liukisliu...@usc.edu wrote: On Aug 17, 2009, at 9:51 PM, Charles R Harris wrote: On Mon, Aug 17, 2009 at 10:30 PM, Maria Liukis liu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: Just to be clear, do you mean finding all rows that only occur once in the array? Yes. I interpreted your question as removing duplicates. It keeps rows that occur more than once. That's what my example is intended to do. Josef snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Tue, Aug 18, 2009 at 1:03 AM, josef.p...@gmail.com wrote: On Tue, Aug 18, 2009 at 12:59 AM, Maria Liukisliu...@usc.edu wrote: On Aug 17, 2009, at 9:51 PM, Charles R Harris wrote: On Mon, Aug 17, 2009 at 10:30 PM, Maria Liukis liu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: Just to be clear, do you mean finding all rows that only occur once in the array? Yes. I interpreted your question as removing duplicates. It keeps rows that occur more than once. That's what my example is intended to do. Josef snip Chuck Just a reminder about views on views, I don't think the recommendation to take the transpose to get unique columns works. We had the discussion some time ago, that views work on the original array data and not on the view, and in this case the transpose creates a view. example below Also, unique does a sort and doesn't preserve order. Josef c=np.array([[ 10, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) cc = c.copy() #backup c = cc.T cc array([[10, 1, 2], [ 3, 4, 5], [ 3, 4, 5], [ 9, 10, 11]]) np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view(c.dtype).reshape(-1,c.shape[1]) Traceback (most recent call last): File pyshell#46, line 1, in module np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view(c.dtype).reshape(-1,c.shape[1]) ValueError: new type not compatible with array. c = cc.T.copy() c array([[10, 3, 3, 9], [ 1, 4, 4, 10], [ 2, 5, 5, 11]]) np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view(c.dtype).reshape(-1,c.shape[1]) array([[ 1, 4, 4, 10], [ 2, 5, 5, 11], [10, 3, 3, 9]]) c = np.ascontiguousarray(cc.T) np.unique1d(c.view([('',c.dtype)]*c.shape[1])).view(c.dtype).reshape(-1,c.shape[1]) array([[ 1, 4, 4, 10], [ 2, 5, 5, 11], [10, 3, 3, 9]]) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique rows of array
On Aug 17, 2009, at 10:03 PM, josef.p...@gmail.com wrote: On Tue, Aug 18, 2009 at 12:59 AM, Maria Liukisliu...@usc.edu wrote: On Aug 17, 2009, at 9:51 PM, Charles R Harris wrote: On Mon, Aug 17, 2009 at 10:30 PM, Maria Liukis liu...@usc.edu wrote: Hello everybody, While re-implementing some Matlab code in Python, I've run into a problem of finding a NumPy function analogous to the Matlab's unique(array, 'rows') to get unique rows of an array. Searching the web, I've found a similar discussion from couple of years ago with an example: Just to be clear, do you mean finding all rows that only occur once in the array? Sorry, I think it shows that I should stop working pass 10pm :) Yes. I interpreted your question as removing duplicates. It keeps rows that occur more than once. Yes, I meant keeping only unique (without duplicates) rows. That's what my example is intended to do. Josef snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion