[Numpy-discussion] Treatment of the Matrix by Numpy
To: Discussion of Numerical Python Message-ID: Content-Type: text/plain; charset=ISO-8859-1 On Wed, Jul 24, 2013 at 8:53 AM, St?fan van der Walt wrote: On Wed, Jul 24, 2013 at 2:15 AM, Chris Barker - NOAA Federal wrote: On Tue, Jul 23, 2013 at 6:09 AM, Pauli Virtanen wrote: The .H property has been implemented in Numpy matrices and Scipy's sparse matrices for many years. Then we're done. Numpy is an array package, NOT a matrix package, and while you can implement matrix math with arrays (and we do), having quick and easy mnemonics for common matrix math operations (but uncommon general purpose array operations) is not eh job of numpy. That's what the matrix object is for. I would argue that the ship sailed when we added .T already. Most users see no difference between the addition of .T and .H. I agree. During the Numarray period, I developed a Matrix sub-class which provided: # Properties A= property(fget= toArray, doc= 'Deliver the data as an array.') Adj= property(fget= getAdjoint, doc= 'Deliver the adjoint matrix.') Conj= property(fget= getConjugate, doc= 'Deliver the conjugate of the matrix.') Diag= property(fget= getDiagonal, doc= 'Extract the diagonal as a row vector.') lTri= property(fget= getLTri, doc= 'Extract the lower triangular matrix, ' + 'ie. elements on and below the diagonal.') Cond= property(fget= getCond, doc= 'Deliver the 2-norm Condition number.') Det= property(fget=getDeterminant, doc= 'Deliver the determinant.') EValues= property(fget= getEigenvalues, doc= 'Deliver the eigenvalues.') EVectors= property(fget= getEigenvectors, doc= 'Deliver the eigenvectors.') I= property(fget= getInverse, doc= 'Deliver the inverse.') Imag= property(fget= getImag, doc= 'Return the imaginary part of the matrix.') Norm= property(fget= getNorm, doc= 'To calculate the 2-norm of the matrix.') Real= property(fget= getReal, doc= 'Return the real part of the matrix.') Sqr= property(fget= getSqr, doc= 'Return the square of each element.') SVD= property(fget= getSVD, doc= 'Return S, V, D (Singular Value Decomposition.') T= property(fget= getTranspose, doc= 'Deliver the transpose.') uTri= property(fget= getUTri, doc= 'Extract the upper triangular matrix, ' + 'ie. elements on and above the diagonal.' I think H was in there too. All of this was lost when Travis came along with numpy. No thought was given to sparse matrices at that time. The matrix class should probably be deprecated and removed from NumPy in the long run--being a second class citizen not used by the developers themselves is not sustainable. And, now that we have "dot" as a method, there's very little advantage to it. I would argue that, in some sense, it should be promoted. Perhaps it's better as a separate module. St?fan Colin W. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
24.07.2013 18:33, Pauli Virtanen kirjoitti: > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? Done: https://github.com/numpy/numpydoc https://github.com/numpy/numpy/pull/3547 https://github.com/scipy/scipy/pull/2657 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
On 24.07.2013 17:33, Pauli Virtanen wrote: > Hi, > > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? +1 -- Andreas ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
On Jul 24, 2013, at 10:33 AM, Pauli Virtanen wrote: > Hi, > > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? > > It's a separate Python package, after all. Moreover, this would make it > easier to use it as a git submodule (e.g. in Scipy). Moreover, its > release cycle is not in any way tied to that of Numpy. > > Pauli > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion I'm a big +1 on this idea. I've used the numpydoc sphinx extensions in a number of package I've worked on, having them as a separate git repo would make these even easier to use. - Jonathan Helmus ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
On Wed, 24 Jul 2013, Pauli Virtanen wrote: > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? +1 > It's a separate Python package, after all. Moreover, this would make it > easier to use it as a git submodule (e.g. in Scipy). Moreover, its > release cycle is not in any way tied to that of Numpy. yeap -- it has a life of its own -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Senior Research Associate, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
On Wed, Jul 24, 2013 at 4:33 PM, Pauli Virtanen wrote: > > Hi, > > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? > > It's a separate Python package, after all. Moreover, this would make it > easier to use it as a git submodule (e.g. in Scipy). Moreover, its > release cycle is not in any way tied to that of Numpy. Works for me. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Casting Errors in AIX
Hello Numpy Discussion List, So I'm trying to get numpy working on an AIX 6.1 system. Initially I had a lot of problems trying to compile the package because the xlc compiler weren't installed on this machine, but apparently the Python package we installed had been built with them. Once we got xlc installed the process seemed to work pretty well until we got to compiling heapsort.c. At this point I began to get a huge number of errors in the form: compile options: '-Inumpy/core/include -Ibuild/src.aix-6.1-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/freeware/include/python2.7 -Ibuild/src.aix-6.1-2.7/numpy/core/src/multiarray -Ibuild/src.aix-6.1-2.7/numpy/core/src/umath -c' xlc_r: build/src.aix-6.1-2.7/numpy/core/src/npysort/heapsort.c "/usr/include/stdio.h", line 528.12: 1506-343 (S) Redeclaration of fgetpos64 differs from previous declaration on line 323 of "/usr/include/stdio.h". "/usr/include/stdio.h", line 528.12: 1506-377 (I) The type "long long*" of parameter 2 differs from the previous type "long* restrict". "/usr/include/stdio.h", line 531.12: 1506-343 (S) Redeclaration of fseeko64 differs from previous declaration on line 471 of "/usr/include/stdio.h". "/usr/include/stdio.h", line 531.12: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/stdio.h", line 532.12: 1506-343 (S) Redeclaration of fsetpos64 differs from previous declaration on line 325 of "/usr/include/stdio.h". "/usr/include/stdio.h", line 532.12: 1506-377 (I) The type "const long long*" of parameter 2 differs from the previous type "const long*". "/usr/include/stdio.h", line 533.16: 1506-343 (S) Redeclaration of ftello64 differs from previous declaration on line 472 of "/usr/include/stdio.h". "/usr/include/stdio.h", line 533.16: 1506-050 (I) Return type "long long" in redeclaration is not compatible with the previous return type "long". "/usr/include/unistd.h", line 171.17: 1506-343 (S) Redeclaration of lseek64 differs from previous declaration on line 169 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 171.17: 1506-050 (I) Return type "long long" in redeclaration is not compatible with the previous return type "long". "/usr/include/unistd.h", line 171.17: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/sys/lockf.h", line 64.20: 1506-343 (S) Redeclaration of lockf64 differs from previous declaration on line 62 of "/usr/include/sys/lockf.h". ... "/usr/include/unistd.h", line 942.25: 1506-050 (I) Return type "long long" in redeclaration is not compatible with the previous return type "long". "/usr/include/unistd.h", line 942.25: 1506-377 (I) The type "long long" of parameter 2 differs from the previous type "long". "/usr/include/unistd.h", line 943.25: 1506-343 (S) Redeclaration of fsync_range64 differs from previous declaration on line 940 of "/usr/include/unistd.h". "/usr/include/unistd.h", line 943.25: 1506-377 (I) The type "long long" of parameter 3 differs from the previous type "long". error: Command "/usr/vac/bin/xlc_r -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -q32 -qbitfields=signed -qmaxmem=7 -qalloca -bmaxdata:0x8000 -Wl,-brtl -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DNDEBUG -O2 -Inumpy/core/include -Ibuild/src.aix-6.1-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/freeware/include/python2.7 -Ibuild/src.aix-6.1-2.7/numpy/core/src/multiarray -Ibuild/src.aix-6.1-2.7/numpy/core/src/umath -c build/src.aix-6.1-2.7/numpy/core/src/npysort/heapsort.c -o build/temp.aix-6.1-2.7/build/src.aix-6.1-2.7/numpy/core/src/npysort/heapsort.o" failed with exit status 1 There are a lot more than this. Probably in neighborhood of 40 lines all told. I spent some time doing research and this appears to be something not terribly uncommon when compiling F/OSS on AIX. Most of the instances appeared to involve either sshd or smb. Unfortunately the most commonly cited solution (using the --disable-largefile to configure) won't work for compiling a Python module. One suggestion I found that did help was to explicitly include some of the standard libraries in the .c file. So I added: #include #include To heapsort.c. That dramatically reduced the error messages. Now I get: compile options: '-Inumpy/core/include -Ibuild/src.aix-6.1-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/freeware/include/p
Re: [Numpy-discussion] Splitting numpydoc to a separate repo
On Wed, Jul 24, 2013 at 5:33 PM, Pauli Virtanen wrote: > How about splitting doc/sphinxext out from the main Numpy repository to > a separate `numpydoc` repo under Numpy project? That would be great, also for scikits that rely on these extensions. Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Splitting numpydoc to a separate repo
Hi, How about splitting doc/sphinxext out from the main Numpy repository to a separate `numpydoc` repo under Numpy project? It's a separate Python package, after all. Moreover, this would make it easier to use it as a git submodule (e.g. in Scipy). Moreover, its release cycle is not in any way tied to that of Numpy. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
>> >> plan >> would be to remove the Matrix class from numpy over two or three >> releases, and publish it as a separate package on PyPi. Anyone willing to take ownership of it? Maybe we should still do it of not-- at least it will make it clear that it is orphaned. Though one plus to having matrix in numpy is that it was a testbed for ndarray subclassing... -Chris > Please do! There are some sticky issues to work through (e.g. how to > deprecate the "matrix" entry in the numpy namespace, what to do with > scipy.sparse), and I don't know whether we'll decide to go through > with it in the end, but the way to figure that out is to, you know, > work through them :-). > > -n > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Jul 23, 2013, at 11:54 PM, "Stéfan van der Walt" wrote: >>> The .H property has been implemented in Numpy matrices and Scipy's >>> sparse matrices for many years. >> >> Then we're done. Numpy is an array package, NOT a matrix package, and >> while you can implement matrix math with arrays (and we do), having >> quick and easy mnemonics for common matrix math operations (but >> uncommon general purpose array operations) is not eh job of numpy. >> That's what the matrix object is for. > > I would argue that the ship sailed when we added .T already. Most > users see no difference between the addition of .T and .H. I don't know who can speak for "most users", but I see them quite differently. Transposing is a common operation outside of linear algebra--I, for one, use it to work with image arrays, which are often stored in a way by image libraries that is the transpose of the "natural" numpy way. But anyway, just because we have one domain-specific convenience attribute, doesn't mean we should add them all. > The matrix class should probably be deprecated and removed from NumPy > in the long run--being a second class citizen not used by the > developers themselves is not sustainable. I agree, but the fact that no one has stepped up to maintain and improve it tells me that there is not a very large community that wants a clean linear algebra interface, not that we should try to build such an interface directly into numpy. Is there really a point to a clean interface to the Hermetian transpose, but not plain old matrix multiply? > And, now that we have "dot" > as a method, Agh, but "dot" is a method--so we still don't have a clean relationship with the math in text books: AB => A.dot(B) Anyway, adding .H is clearly not a big deal, I just don't think it's going to satisfy anyone anyway. -Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 12:54 PM, Nathaniel Smith wrote: > The point isn't that there's code that relies specifically on .T > returning a view. It's that to be a good programmer, you need to *know > whether* it returns a view -- exactly as you say in the second > paragraph. And a library should not hide these kinds of details. After listening to the arguments by yourself and Dag, I think I buy into the idea that we should hold off on this until we have ufunc views or something similar implemented. Also, if we split off the matrix package, we can give other people who really care about that (perhaps Alan is interested?) ownership, and let them run with it (I mainly use ndarrays myself). Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fresh performance boosts and elderly hits e.g. identity, ones
Added some basic constructors benchmarks: http://www.onerussian.com/tmp/numpy-vbench/vb_vb_core.html quite a bit of fresh enhancements are present (cool) but also some freshly discovered elderly hits, e.g. http://www.onerussian.com/tmp/numpy-vbench/vb_vb_core.html#numpy-identity-100 http://www.onerussian.com/tmp/numpy-vbench/vb_vb_core.html#numpy-ones-100 Cheers, On Fri, 19 Jul 2013, Yaroslav Halchenko wrote: > I have just added a few more benchmarks, and here they come > http://www.onerussian.com/tmp/numpy-vbench/vb_vb_linalg.html#numpy-linalg-pinv-a-float32 > it seems to be very recent so my only check based on 10 commits > didn't pick it up yet so they are not present in the summary table. > could well be related to 80% faster det()? ;) > norm was hit as well a bit earlier, might well be within these commits: > https://github.com/numpy/numpy/compare/24a0aa5...29dcc54 > I will rerun now benchmarking for the rest of commits (was running last > in the day iirc) > Cheers, -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Senior Research Associate, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question regarding documentation of structured arrays
Hallo Hanno On Wed, Jul 24, 2013 at 1:29 PM, Hanno Klemm wrote: > I would be happy to file a pull request against the docs if you (or > somebody) could point me to a document on how and where to do that. The file you want to edit is here: https://github.com/numpy/numpy/blob/master/numpy/doc/structured_arrays.py#L194 You can click on the "edit" button, then GitHub will help you to make a pull request. Thanks! Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 8:47 AM, Daπid wrote: > An idea: > > If .H is ideally going to be a view, and we want to keep it this way, > we could have a .h() method with the present implementation. This > would preserve the name .H for the conjugate view --when someone finds > the way to do it. > > This way we would increase the readability, simplify some matrix > algebra code, and keep the API consistency. > > I could get behind a .h() method until .H attribute is ready. +1 Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On 7/24/2013 3:15 AM, Sebastian Haase wrote: > I feel that adding a method > .H() > would be the compromise ! > > Alan, could you live with that ? I feel .H() now would get in the way of a .H attribute later, which some have indicated could be added as an iterative view in a future numpy. I'd rather wait for that. My assessment of the conversation so far: there is not adequate support for a .H attribute until it can be an iterative view. I believe that almost everyone (possibly not Josef) would accept or want a .H attribute if it could provide an iterative view. (Is that correct?) So I'll drop out of the conversation, but I hope the user interest that has been displayed stimulates interest in that feature request. Thanks to everyone who shared their perspective on this issue. And my apologies to those (e.g., Dag) whom I annoyed by being too bullheaded. Cheers, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
An idea: If .H is ideally going to be a view, and we want to keep it this way, we could have a .h() method with the present implementation. This would preserve the name .H for the conjugate view --when someone finds the way to do it. This way we would increase the readability, simplify some matrix algebra code, and keep the API consistency. On 24 July 2013 13:08, Dave Hirschfeld wrote: > Nathaniel Smith pobox.com> writes: > >> >> >> As soon as you talk about attributes "returning" things you've already >> broken Python's mental model... attributes are things that sit there, >> not things that execute arbitrary code. Of course this is not how the >> actual implementation works, attribute access *can* in fact execute >> arbitrary code, but the mental model is important, so we should >> preserve it where-ever we can. Just mentioning an attribute should not >> cause unbounded memory allocations. >> > > Yep, sorry - sloppy use of terminology which I agree is important in helping > understand what's happening. > > -Dave > > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question regarding documentation of structured arrays
Hi Stefan, I would be happy to file a pull request against the docs if you (or somebody) could point me to a document on how and where to do that. Hanno On 24.07.2013 12:31, Stéfan van der Walt wrote: > Hi Hanno > > On Wed, Jul 24, 2013 at 11:46 AM, Hanno Klemm > wrote: >> I found the following inconsistency between the advertised and the >> actual behviour of structured arrays: >> >> on http://docs.scipy.org/doc/numpy/user/basics.rec.html it says in the >> section >> >> "Accessing multiple fields at once" >> Notice that the fields are always returned in the same order >> regardless >> of the sequence they are asked for. > > I can confirm the behavior you see under the latest development > version. Would you mind filing a pull request against the docs? > > Stéfan > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Hanno Klemm kl...@phys.ethz.ch ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
Nathaniel Smith pobox.com> writes: > > > As soon as you talk about attributes "returning" things you've already > broken Python's mental model... attributes are things that sit there, > not things that execute arbitrary code. Of course this is not how the > actual implementation works, attribute access *can* in fact execute > arbitrary code, but the mental model is important, so we should > preserve it where-ever we can. Just mentioning an attribute should not > cause unbounded memory allocations. > Yep, sorry - sloppy use of terminology which I agree is important in helping understand what's happening. -Dave ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 8:30 AM, Stéfan van der Walt wrote: > I am willing to write up a NEP if there's any interest. The plan > would be to remove the Matrix class from numpy over two or three > releases, and publish it as a separate package on PyPi. Please do! There are some sticky issues to work through (e.g. how to deprecate the "matrix" entry in the numpy namespace, what to do with scipy.sparse), and I don't know whether we'll decide to go through with it in the end, but the way to figure that out is to, you know, work through them :-). -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 9:23 AM, Dave Hirschfeld wrote: > If we're voting my vote goes to add the .H attribute for all the reasons > Alan has specified. Document that it returns a copy but that it may in > future return a view so it it not future proof to operate on the result > inplace. As soon as you talk about attributes "returning" things you've already broken Python's mental model... attributes are things that sit there, not things that execute arbitrary code. Of course this is not how the actual implementation works, attribute access *can* in fact execute arbitrary code, but the mental model is important, so we should preserve it where-ever we can. Just mentioning an attribute should not cause unbounded memory allocations. Consider these two expressions: x = solve(dot(arr, arr.T), arr.T) x = solve(dot(arr, arr.H), arr.H) Mathematically, they're very similar, and the mathematics-like notation does a good job of expressing that similarity while hiding mathematically irrelevant details. Which is what mathematical notation is for. But numpy isn't a toolkit for writing mathematical formula, it's a toolkit for writing computational algorithms that implement mathematical formula, and algorithmically, those two expressions are radically different. The first one allocates one temporary (the result from 'dot'); the second one allocates 3 temporaries. The second one is gratuitously inefficient, since two of those temporaries are identical, but they're being computed twice anyway. > I'm sceptical that there's much code out there actually relying on the fact > that a transpose is a view with the specified intention of altering the > original array inplace. > > I work with a lot of beginners and whenever I've seen them operate inplace > on a transpose it has been a bug in the code, leading to a discussion of > how, for performance reasons, numpy will return a view where possible, > leading to yet further discussion of when it is and isn't possible to return > a view. The point isn't that there's code that relies specifically on .T returning a view. It's that to be a good programmer, you need to *know whether* it returns a view -- exactly as you say in the second paragraph. And a library should not hide these kinds of details. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question regarding documentation of structured arrays
Hi Hanno On Wed, Jul 24, 2013 at 11:46 AM, Hanno Klemm wrote: > I found the following inconsistency between the advertised and the > actual behviour of structured arrays: > > on http://docs.scipy.org/doc/numpy/user/basics.rec.html it says in the > section > > "Accessing multiple fields at once" > Notice that the fields are always returned in the same order regardless > of the sequence they are asked for. I can confirm the behavior you see under the latest development version. Would you mind filing a pull request against the docs? Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Question regarding documentation of structured arrays
Hi,
I found the following inconsistency between the advertised and the
actual behviour of structured arrays:
on http://docs.scipy.org/doc/numpy/user/basics.rec.html it says in the
section
"Accessing multiple fields at once"
Notice that the fields are always returned in the same order regardless
of the sequence they are asked for.
Fortunately that does not seem to be the case in my simple test (see
below). Is that a change in behaviour I can rely on or am I somehow
lucky in this particular example?
Thanks,
Hanno
In [596]: test_array = np.ones((10),dtype=[('a', float), ('b',float)])
In [597]: test_array
Out[597]:
array([(1.0, 1.0), (1.0, 1.0), (1.0, 1.0), (1.0, 1.0), (1.0, 1.0),
(1.0, 1.0), (1.0, 1.0), (1.0, 1.0), (1.0, 1.0), (1.0, 1.0)],
dtype=[('a', 'http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
gmail.com> writes:
>
> I think a H is feature creep and too specialized
>
> What's .H of a int a str a bool ?
>
> It's just .T and a view, so you cannot rely that conj() makes a copy
> if you don't work with complex.
>
> .T is just a reshape function and has **nothing** to do with matrix
algebra.
>
It seems to me that that ship has already sailed - i.e. conj doesn't make
much sense for str arrays, but it still works in the sense that it's a nop
In [16]: A = asarray(list('abcdefghi')).reshape(3,3)
...: np.all(A.T == A.conj().T)
...:
Out[16]: True
If we're voting my vote goes to add the .H attribute for all the reasons
Alan has specified. Document that it returns a copy but that it may in
future return a view so it it not future proof to operate on the result
inplace.
I'm -1 on .H() as it will require code changes if it ever changes to a
property and it will simply result in questions about why .T is a property
and .H is a function (and why it's a property for (sparse) matrices)
Regarding Dag's example:
xh = x.H
x *= 2
assert np.all(2 * xh == x.H)
I'm sceptical that there's much code out there actually relying on the fact
that a transpose is a view with the specified intention of altering the
original array inplace.
I work with a lot of beginners and whenever I've seen them operate inplace
on a transpose it has been a bug in the code, leading to a discussion of
how, for performance reasons, numpy will return a view where possible,
leading to yet further discussion of when it is and isn't possible to return
a view.
The third option of .H returning a view would probably be agreeable to
everyone but I don't think we should punt on this decision for something
that if it does happen is likely years away. It seems that work on this
front is happening in different projects to numpy. Even if for example
sometime in the future numpy's internals were replaced with libdynd or other
expression graph engine surely this would result in more breaking changes
than .H returning a view rather than a copy?!
IANAD so I'm happy with whatever the consensus is I just thought I'd put
forward the view from a (specific type of) user perspective.
Regards,
Dave
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Tue, Jul 23, 2013 at 4:35 AM, Dag Sverre Seljebotn wrote: ... > There's lots of uses for A.H to be a conjugating-view, e.g., np.dot(A.H, > A) can be done on-the-fly by BLAS at no extra cost, and so on. These are > currently not possible with pure NumPy without a copy, which is a pretty > big defect IMO (and one reason I'd call BLAS myself using Cython rather > than use np.dot...) Wouldn't the simpler way not just be to expose those linalg functions? hdot(X, Y) == dot(X.T, Y) (if not complex) == dot(X.H, Y)(if complex) Josef ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
I think a H is feature creep and too specialized
What's .H of a int a str a bool ?
It's just .T and a view, so you cannot rely that conj() makes a copy
if you don't work with complex.
.T is just a reshape function and has **nothing** to do with matrix algebra.
>>> x = np.arange(12).reshape(3,4)
>>> x
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> np.may_share_memory(x, x.T)
True
>>> np.may_share_memory(x, x.conj())
True
>>> y = x + 1j
>>> np.may_share_memory(y, y.conj())
False
>>> y.dtype
dtype('complex128')
>>> x.conj().dtype
dtype('int32')
Josef
On Wed, Jul 24, 2013 at 3:30 AM, Stéfan van der Walt wrote:
> On Wed, Jul 24, 2013 at 9:15 AM, Sebastian Haase wrote:
>> I feel that adding a method
>> .H()
>> would be the compromise !
>
> Thinking about this more, I think it would just confuse most users...
> why .T and not .H; then you have to start explaining the underlying
> implementation detail. For users who already understand the
> implementation detail, finding .T.conj() would not be too hard.
>
>> (( I would not open the discussion about ndarray vs. matrix -- it gets
>> far to involving and we would be talking about far-future directions
>> instead of "a single letter addition", which abvious already has big
>> enough support and had so years ago))
>
> I am willing to write up a NEP if there's any interest. The plan
> would be to remove the Matrix class from numpy over two or three
> releases, and publish it as a separate package on PyPi.
>
> Stéfan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 9:15 AM, Sebastian Haase wrote: > I feel that adding a method > .H() > would be the compromise ! Thinking about this more, I think it would just confuse most users... why .T and not .H; then you have to start explaining the underlying implementation detail. For users who already understand the implementation detail, finding .T.conj() would not be too hard. > (( I would not open the discussion about ndarray vs. matrix -- it gets > far to involving and we would be talking about far-future directions > instead of "a single letter addition", which abvious already has big > enough support and had so years ago)) I am willing to write up a NEP if there's any interest. The plan would be to remove the Matrix class from numpy over two or three releases, and publish it as a separate package on PyPi. Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] add .H attribute?
On Wed, Jul 24, 2013 at 8:53 AM, Stéfan van der Walt wrote: > On Wed, Jul 24, 2013 at 2:15 AM, Chris Barker - NOAA Federal > wrote: >> >> On Tue, Jul 23, 2013 at 6:09 AM, Pauli Virtanen wrote: >> >> > The .H property has been implemented in Numpy matrices and Scipy's >> > sparse matrices for many years. >> >> Then we're done. Numpy is an array package, NOT a matrix package, and >> while you can implement matrix math with arrays (and we do), having >> quick and easy mnemonics for common matrix math operations (but >> uncommon general purpose array operations) is not eh job of numpy. >> That's what the matrix object is for. > > I would argue that the ship sailed when we added .T already. Most > users see no difference between the addition of .T and .H. > > The matrix class should probably be deprecated and removed from NumPy > in the long run--being a second class citizen not used by the > developers themselves is not sustainable. And, now that we have "dot" > as a method, there's very little advantage to it. > > Stéfan Maybe this is the point where one just needs to do a poll. And finally someone has to make the decision. I feel that adding a method .H() would be the compromise ! Alan, could you live with that ? It is short enough and still emphasises the fact that it is NOT a view and therefore behaves sensitively different in certain scenarios as .T . It also leaves the door open to adding an iterator .H attribute later on without introducing the above mentioned code breaks. Who could make (i.e. is willing to make) the decision ? (( I would not open the discussion about ndarray vs. matrix -- it gets far to involving and we would be talking about far-future directions instead of "a single letter addition", which abvious already has big enough support and had so years ago)) Regards, Sebastian Haase ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
