[Numpy-discussion] Advanced indexing: fancy vs. orthogonal
There are two different interpretations in common use of how to handle multi-valued (array/sequence) indexes. The numpy style is to consider all multi-valued indices together which allows arbitrary points to be extracted. The orthogonal style (e.g. as provided by netcdf4-python) is to consider each multi-valued index independently. For example: type(v)type 'netCDF4.Variable' v.shape (240, 37, 49) v[(0, 1), (0, 2, 3)].shape (2, 3, 49) np.array(v)[(0, 1), (0, 2, 3)].shape Traceback (most recent call last): File stdin, line 1, in moduleIndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (2,) (3,) In a netcdf4-python GitHub issue https://github.com/Unidata/netcdf4-python/issues/385 the authors of various orthogonal indexing packages have been discussing how to distinguish the two behaviours and have currently settled on a boolean __orthogonal_indexing__ attribute. 1. Is there any appetite for adding that attribute (with the value `False`) to ndarray? 2. As suggested by shoyer https://github.com/Unidata/netcdf4-python/issues/385#issuecomment-87775034, is there any appetite for adding an alternative indexer to ndarray where __orthogonal_indexing__ = True? For example: myarray.ix_[(0,1), (0, 2, 3)] Richard ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Advanced indexing: fancy vs. orthogonal
On Wed, Apr 1, 2015 at 2:17 AM, R Hattersley rhatters...@gmail.com wrote: There are two different interpretations in common use of how to handle multi-valued (array/sequence) indexes. The numpy style is to consider all multi-valued indices together which allows arbitrary points to be extracted. The orthogonal style (e.g. as provided by netcdf4-python) is to consider each multi-valued index independently. For example: type(v)type 'netCDF4.Variable' v.shape (240, 37, 49) v[(0, 1), (0, 2, 3)].shape (2, 3, 49) np.array(v)[(0, 1), (0, 2, 3)].shape Traceback (most recent call last): File stdin, line 1, in moduleIndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (2,) (3,) In a netcdf4-python GitHub issue https://github.com/Unidata/netcdf4-python/issues/385 the authors of various orthogonal indexing packages have been discussing how to distinguish the two behaviours and have currently settled on a boolean __orthogonal_indexing__ attribute. 1. Is there any appetite for adding that attribute (with the value `False`) to ndarray? 2. As suggested by shoyer https://github.com/Unidata/netcdf4-python/issues/385#issuecomment-87775034, is there any appetite for adding an alternative indexer to ndarray where __orthogonal_indexing__ = True? For example: myarray.ix_[(0,1), (0, 2, 3)] Is there any other package implementing non-orthogonal indexing aside from numpy? I understand that it would be nice to do: if x.__orthogonal_indexing__: return x[idx] else: return x.ix_[idx] But I think you would get the exact same result doing: if isinstance(x, np.ndarray): return x[np.ix_(*idx)] else: return x[idx] If `not x.__orthogonal_indexing__` is going to be a proxy for `isinstance(x, ndarray)` I don't really see the point of disguising it, explicit is better than implicit and all that. If the functionality is lacking, e,g, use of slices in `np.ix_`, I'm all for improving that to provide the full functionality of orthogonal indexing. I just need a little more convincing that those new attributes/indexers are going to ever see any real use. Jaime Richard ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- (\__/) ( O.o) ( ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] IDE's for numpy development?
Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio, https://pytools.codeplex.com/, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
mixed C and python development? I would just wait for the Jupyter folks to create IC and maybe even IC++! On Wed, Apr 1, 2015 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio, https://pytools.codeplex.com/, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
That would really be hilarious - and IFortran probably! :) Shawn On Wed, Apr 1, 2015 at 12:07 PM, Benjamin Root ben.r...@ou.edu wrote: mixed C and python development? I would just wait for the Jupyter folks to create IC and maybe even IC++! On Wed, Apr 1, 2015 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio,, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Yuxiang Shawn Wang Gerling Research Lab University of Virginia yw...@virginia.edu +1 (434) 284-0836 https://sites.google.com/a/virginia.edu/yw5aj/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
On Wed, Apr 1, 2015 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio,, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? I have no experience with the C/C++ part, but I'm using the C/C++ version of Eclipse with PyDev. It should have all the extra features available, but I don't use them and don't have compiler, debugger and so on for C/C++ connected to Eclipse. It looks like it supports Visual C++ and MingW GCC toolchain. (I'm not sure the same project can be a C/C++ and a PyDev project at the same time.) Josef Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
The PTVS can debug into native code. On Wed, Apr 1, 2015 at 2:21 PM, josef.p...@gmail.com wrote: On Wed, Apr 1, 2015 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio,, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? I have no experience with the C/C++ part, but I'm using the C/C++ version of Eclipse with PyDev. It should have all the extra features available, but I don't use them and don't have compiler, debugger and so on for C/C++ connected to Eclipse. It looks like it supports Visual C++ and MingW GCC toolchain. (I'm not sure the same project can be a C/C++ and a PyDev project at the same time.) Josef Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
Charles R Harris charlesr.har...@gmail.com wrote: I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? I guess we could include project files for Visual Studio (and perhaps Eclipse?), like Python does. But then we would need to make sure the different build systems are kept in sync, and it will be a PITA for those who do not use Windows and Visual Studio. It is already bad enough with Distutils and Bento. I, for one, would really prefer if there only was one build process to care about. One should also note that a Visual Studio project is the only supported build process for Python on Windows. So they are not using this in addition to something else. Eclipse is better than Visual Studio for mixed Python and C development. It is also cross-platform. cmake needs to be mentioned too. It is not fully integrated with Visual Studio, but better than having multiple build processes. But still, there is nothing that prevents the use of Visual Studio as a glorified text editor. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Adding 'where' to ufunc methods?
This question on StackOverflow: http://stackoverflow.com/questions/29394377/minimum-of-numpy-array-ignoring-diagonal Got me thinking that I had finally found a use for the 'where' kwarg of ufuncs. Unfortunately it is only provided for the ufunc itself, but not for any of its methods. Is there any fundamental reason these were not implemented back in the day? Any frontal opposition to having them now? Jaime -- (\__/) ( O.o) ( ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
Sorry for the OT and top-posting but, It reminds me of ITex (https://www.youtube.com/watch?v=eKaI78K_rgA) ... On Wed, Apr 1, 2015 at 6:43 PM, Yuxiang Wang yw...@virginia.edu wrote: That would really be hilarious - and IFortran probably! :) Shawn On Wed, Apr 1, 2015 at 12:07 PM, Benjamin Root ben.r...@ou.edu wrote: mixed C and python development? I would just wait for the Jupyter folks to create IC and maybe even IC++! On Wed, Apr 1, 2015 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, In a recent exchange Mark Wiebe suggested that the lack of support for numpy development in Visual Studio might limit the number of developers attracted to the project. I'm a vim/console developer myself and make no claim of familiarity with modern development tools, but I wonder if such tools might now be available for Numpy. A quick google search turns up a beta plugin for Visual Studio,, and there is an xcode IDE for the mac that apparently offers some Python support. The two things that I think are required are: 1) support for mixed C, python developement and 2) support for building and testing numpy. I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Yuxiang Shawn Wang Gerling Research Lab University of Virginia yw...@virginia.edu +1 (434) 284-0836 https://sites.google.com/a/virginia.edu/yw5aj/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IDE's for numpy development?
On Wed, Apr 1, 2015 at 11:55 AM, Sturla Molden sturla.mol...@gmail.com wrote: Charles R Harris charlesr.har...@gmail.com wrote: I'd be interested in information from anyone with experience in using such an IDE and ideas of how Numpy might make using some of the common IDEs easier. Thoughts? I guess we could include project files for Visual Studio (and perhaps Eclipse?), like Python does. But then we would need to make sure the different build systems are kept in sync, and it will be a PITA for those who do not use Windows and Visual Studio. It is already bad enough with Distutils and Bento. I, for one, would really prefer if there only was one build process to care about. One should also note that a Visual Studio project is the only supported build process for Python on Windows. So they are not using this in addition to something else. Eclipse is better than Visual Studio for mixed Python and C development. It is also cross-platform. cmake needs to be mentioned too. It is not fully integrated with Visual Studio, but better than having multiple build processes. Mark chose cmake for DyND because it supported Visual Studio projects. OTOH, he said it was a PITA to program. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Adding 'where' to ufunc methods?
Another usecase would be for MaskedArrays. ma.masked_array.min() wouldn't have to make a copy anymore (there is a github issue about that). It could just pass its mask into the where= argument of min() and be done with it. Problem would be generalizing situations where where= effectively results in nowhere. Cheers! Ben Root On Wed, Apr 1, 2015 at 2:34 PM, Jaime Fernández del Río jaime.f...@gmail.com wrote: This question on StackOverflow: http://stackoverflow.com/questions/29394377/minimum-of-numpy-array-ignoring-diagonal Got me thinking that I had finally found a use for the 'where' kwarg of ufuncs. Unfortunately it is only provided for the ufunc itself, but not for any of its methods. Is there any fundamental reason these were not implemented back in the day? Any frontal opposition to having them now? Jaime -- (\__/) ( O.o) ( ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Adding 'where' to ufunc methods?
On Wed, Apr 1, 2015 at 11:34 AM, Jaime Fernández del Río jaime.f...@gmail.com wrote: This question on StackOverflow: http://stackoverflow.com/questions/29394377/minimum-of-numpy-array-ignoring-diagonal Got me thinking that I had finally found a use for the 'where' kwarg of ufuncs. Unfortunately it is only provided for the ufunc itself, but not for any of its methods. Is there any fundamental reason these were not implemented back in the day? Any frontal opposition to having them now? The where= argument stuff was rescued from the last aborted attempt to add missing value support to numpy. The only reason they aren't implemented for the ufunc methods is that Mark didn't get that far. +1 to adding them now. -n -- Nathaniel J. Smith -- http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to Force Storage Order
Klemm, Michael michael.kl...@intel.com wrote: I have found that the numpy.linalg.svd algorithm creates the resulting U, sigma, and V matrixes with Fortran storage. Is there any way to force these kind of algorithms to not change the storage order? That would make passing the matrixes to the native dgemm operation much easier. NumPy's dot function will call cblas_dgemm in the most efficient way regardless of storage. It knows what to do with C and Fortran arrays. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Adding 'where' to ufunc methods?
On Wed, Apr 1, 2015 at 3:47 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Apr 1, 2015 at 11:34 AM, Jaime Fernández del Río jaime.f...@gmail.com wrote: This question on StackOverflow: http://stackoverflow.com/questions/29394377/minimum-of-numpy-array-ignoring-diagonal Got me thinking that I had finally found a use for the 'where' kwarg of ufuncs. Unfortunately it is only provided for the ufunc itself, but not for any of its methods. Is there any fundamental reason these were not implemented back in the day? Any frontal opposition to having them now? The where= argument stuff was rescued from the last aborted attempt to add missing value support to numpy. The only reason they aren't implemented for the ufunc methods is that Mark didn't get that far. +1 to adding them now. can you get `where` in ufuncs without missing value support? what's the result for ufuncs that are not reduce operations? what's the result for reduce operations along an axis if there is nothing there (in a row or column or ...)? Josef -n -- Nathaniel J. Smith -- http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Adding 'where' to ufunc methods?
On Apr 1, 2015 12:55 PM, josef.p...@gmail.com wrote: On Wed, Apr 1, 2015 at 3:47 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Apr 1, 2015 at 11:34 AM, Jaime Fernández del Río jaime.f...@gmail.com wrote: This question on StackOverflow: http://stackoverflow.com/questions/29394377/minimum-of-numpy-array-ignoring-diagonal Got me thinking that I had finally found a use for the 'where' kwarg of ufuncs. Unfortunately it is only provided for the ufunc itself, but not for any of its methods. Is there any fundamental reason these were not implemented back in the day? Any frontal opposition to having them now? The where= argument stuff was rescued from the last aborted attempt to add missing value support to numpy. The only reason they aren't implemented for the ufunc methods is that Mark didn't get that far. +1 to adding them now. can you get `where` in ufuncs without missing value support? where= is implemented since 1.7 iirc, for regular ufunc calls. I.e. you can currently do np.add(a, b, where=mask), but not np.add.reduce(a, b, where=mask). what's the result for ufuncs that are not reduce operations? The operation skips over any entries where the mask is false. So if you pass an out= array, the masked out entries will remain unchanged from before the call; if you don't pass an out= array then one will be allocated for you as if by calling np.empty, and then the masked out entries will remain uninitialized. what's the result for reduce operations along an axis if there is nothing there (in a row or column or ...)? The same as a reduce operation on a zero length axis: the identity if the ufunc has one, and an error otherwise. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
