Re: [Numpy-discussion] timezones and datetime64
On 04/04/2013 15:34, Jonathan T. Niehof wrote: Keeping a leap second database up to date is annoying but not as bad as it could be: they're not arbitrary. Although they can occur monthly, they've only ever happened at June 30 and Dec 31, announced in January and July, respectively. So it would be easy to check the date of a leapsecond database and warn if 1) date we're processing is after June 30 of a year AND 2) LSDB older than January same year (similar checks for the Dec. 31 opportunity.) As far as I know there is no official standardization of such rules, furthermore, how do you plan to process datetimes far in the future? Cheers, Daniele ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] einsum and broadcasting
On Thu, 2013-04-04 at 16:56 +0300, Jaakko Luttinen wrote:
I don't quite understand how einsum handles broadcasting. I get the
following error, but I don't understand why:
In [8]: import numpy as np
In [9]: A = np.arange(12).reshape((4,3))
In [10]: B = np.arange(6).reshape((3,2))
In [11]: np.einsum('ik,k...-i...', A, B)
---
ValueError: operand 0 did not have enough dimensions to match the
broadcasting, and couldn't be extended because einstein sum subscripts
were specified at both the start and end
However, if I use explicit indexing, it works:
In [12]: np.einsum('ik,kj-ij', A, B)
Out[12]:
array([[10, 13],
[28, 40],
[46, 67],
[64, 94]])
It seems that it also works if I add '...' to the first operand:
In [12]: np.einsum('ik...,k...-i...', A, B)
Out[12]:
array([[10, 13],
[28, 40],
[46, 67],
[64, 94]])
However, as far as I understand, the syntax
np.einsum('ik,k...-i...', A, B)
should work. Have I misunderstood something or is there a bug?
My guess is, it is by design because the purpose of the ellipsis is more
to allow extra dimensions that are not important to the problem itself.
A vector product is np.einsum('i,i-i') and if I write
np.einsum('...i,...i-...i') I allow generalizing that arrays of 1-d
arrays (like the new gufunc linalg stuff).
I did not check the code though, so maybe thats not the case. But in any
case I don't see a reason why it should not be possible to only allow
extra dims for some inputs (I guess it can also make sense to not
give ... for the output).
So I would say, if you want to generalize it, go ahead ;).
- Sebastian
Thanks for your help!
Jaakko
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Import error while freezing with cxfreeze
Hi Anand, On Friday, April 5, 2013, Anand Gadiyar wrote: Hi all, I have a small program that uses numpy and scipy. I ran into a couple of errors while trying to use cxfreeze to create a windows executable. I'm running Windows 7 x64, Python 2.7.3 64-bit, Numpy 1.7.1rc1 64-bit, Scipy-0.11.0 64-bit, all binary installs from http://www.lfd.uci.edu/~gohlke/pythonlibs/ I was able to replicate this with scipy-0.12.0c1 as well. 1) from scipy import constants triggers the below: Traceback (most recent call last): File D:\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py, line 27, in module exec_code in m.__dict__ File mSimpleGui.py, line 10, in module File mSystem.py, line 7, in module File D:\Python27\lib\site-packages\scipy\__init__.py, line 64, in module from numpy import show_config as show_numpy_config File D:\Python27\lib\site-packages\numpy\__init__.py, line 165, in module from core import * AttributeError: 'module' object has no attribute 'sys' It's a bug in cx_freeze that has been fixed in the development branch. See https://bitbucket.org/anthony_tuininga/cx_freeze/pull-request/17/avoid-polluting-extension-module-namespace/diff 2) from scipy import interpolate triggers the below: Traceback (most recent call last): File D:\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py, line 27, in module exec_code in m.__dict__ File mSimpleGui.py, line 10, in module File mSystem.py, line 9, in module File mSensor.py, line 10, in module File D:\Python27\lib\site-packages\scipy\interpolate\__init__.py, line 154, in module from rbf import Rbf File D:\Python27\lib\site-packages\scipy\interpolate\rbf.py, line 50, in module from scipy import linalg ImportError: cannot import name linalg You might want to try the dev branch of cxfreeze to see if this has been fixed as well. Brad ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Indexing bug
On Sun, Mar 31, 2013 at 12:14 AM, Ivan Oseledets ivan.oseled...@gmail.comwrote: snip Oh! So it is not a bug, it is a feature, which is completely incompatible with other array based languages (MATLAB and Fortran). To me, I can not find a single explanation why it is so in numpy. Taking submatrices from a matrix is a common operation and the syntax above is very natural to take submatrices, not a weird diagonal stuff. i.e., c = np.random.randn(100,100) d = c[[0,3],[2,3]] should NOT produce two numbers! (and you can not do it using slices!) In MATLAB and Fortran c(indi,indj) will produce a 2 x 2 matrix. How it can be done in numpy (and why the complications?) So, please consider this message as a feature request. There is already a function, ix, in the index_tricks that does this (I think it is essentially implementing the broadcasting trick that Nathaniel mentions. For me the index trick is easier, as I often forget the broadcasting details). Example: In [14]: c = np.random.randn(100,100) In [15]: c[[0,3],[2,3]] Out[15]: array([ 0.99141998, -0.88928917]) In [16]: c[np.ix_([0,3],[2,3])] Out[16]: array([[ 0.99141998, -1.98406295], [ 0.0729076 , -0.88928917]]) So for me, I think this is superior to MATLAB, as I have often had the case of wanting the result from the second option. In MATLAB you would need to extract the 2x2 matrix, and then take its diagonal. This can be wasteful when the index arrays become large. Cheers, Aronne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] dynamically choosing atlas libraries
Hello List,
We're standing up a new computational cluster and going through the
steps of building Numpy etc. We would like to be able to install
multiple versions of ATLAS (with different build settings / tunings /
etc) and have Numpy load the shared Atlas libraries dynamically based on
the LD_LIBRARY_PATH.
It's not clear to me how to do this given that the library path is hard
coded in Numpy's site.cfg. It seems like other people have gotten this
working though (i.e. RHEL/Centos with their collection of Atlas versions
{atlas, atlas-sse3, etc}). Does anyone have any pointers on this?
Thanks much.
-Ed Walter
Carnegie Mellon University
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? I think that is short-sighted and I think it will damage numpy. Believe me, I have as much investment in backward compatibility as you do. All the three libraries that I spend a long time maintaining need to test against old numpy versions - but - for heaven's sake - only back to numpy 1.2 or numpy 1.3. We don't support Python 2.5 any more, and I don't think we need to maintain compatibility with Numeric either. If you are saying that we need to maintain compatibility for 10 years at a stretch, then we will have to accept that numpy will gradually decay into a legacy library, because it is certain that, if we stay static, someone else with more ambition will do a better job. There is a cost to being averse to any change at all, no matter how gradually it is managed. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. Believe me, I have as much investment in backward compatibility as you do. All the three libraries that I spend a long time maintaining need to test against old numpy versions - but - for heaven's sake - only back to numpy 1.2 or numpy 1.3. We don't support Python 2.5 any more, and I don't think we need to maintain compatibility with Numeric either. Really? This is from 3 months ago: http://article.gmane.org/gmane.comp.python.numeric.general/52632. It's now 2013, we are probably dropping numarray compat in 1.8. Not exactly 10 years, but of the same order. If you are saying that we need to maintain compatibility for 10 years at a stretch, then we will have to accept that numpy will gradually decay into a legacy library, because it is certain that, if we stay static, someone else with more ambition will do a better job. There is a cost to being averse to any change at all, no matter how gradually it
[Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. Believe me, I have as much investment in backward compatibility as you do. All the three libraries that I spend a long time maintaining need to test against old numpy versions - but - for heaven's sake - only back to numpy 1.2 or numpy 1.3. We don't support Python 2.5 any more, and I don't think we need to maintain compatibility with Numeric either. Really? This is from 3 months ago:
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 3:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: Hi, On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: snip Maybe we should go through and rename order to something more descriptive in each case, so we'd have a.reshape(..., index_order=C) a.copy(memory_order=F) etc.? I'd like to propose this instead: a.reshape(..., order=C) a.copy(layout=F) I actually like this, makes the point clearer that it has to do with memory layout and implies contiguity, plus it is short and from the numpy perspective copy, etc. are the ones that add additional info to order and not reshape (because IMO memory order is something new users should not worry about at first). A and K orders will still have their quirks with np.array and copy=True/False, but for many functions they are esoteric anyway. It will be one hell of a deprecation though, but I am +0.5 for adding an alias for now (maybe someone knows an even better name?), but I think that in this case, it probably really is better to wait with actual deprecation warnings for a few versions, since it touches a *lot* of code. Plus I think at the point of starting deprecation warnings (and best earlier) numpy should provide an automatic fixer script... The only counter point that remains for me is the difficulty of deprecation, since I think the new name idea is very clean. And this is unfortunately even more invasive then the index_order proposal. I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. Believe me, I have as much investment in backward compatibility as you do. All the three libraries that I spend a long time maintaining need to test against old numpy versions - but - for heaven's sake - only back to numpy 1.2 or numpy 1.3. We don't support Python 2.5 any more, and I don't think we need to maintain
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 4:27 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey snip I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. So far the consensus is that the documentation needs improvement. The only thing all of the No camp agree with is documentation improvement, I think that's fair. After that ??? Well I think we have: Flat-no - the change not important, almost any cost is too high You Ralf Bradley Mid-no - maybe something could work, but not sure we've seen it yet. Chris Middle - current situation can be confusing, maybe one of the proposed solutions would be acceptable Sebastian Nathaniel Mid-yes - previous apparent vote for argument name change Éric Depagne Andrew Jaffe (sorry if I misrepresent you) And then me. I am trying to be balanced. Unlike others, I think better names would have a significant impact on how coherent numpy is to explain and use. It seems to me that a change would be beneficial in the long term, and I'm confident we can agree on a schedule for that change that would be acceptable. But you know that. So - as I understand our 'model' - our job is to try and come to some shared agreement, if we possibly can. It has been good and encouraging for me at least to see that we have developed our ideas over the course of this thread. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
On Fri, Apr 5, 2013 at 9:50 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 4:27 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey snip I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. So far the consensus is that the documentation needs improvement. The only thing all of the No camp agree with is documentation improvement, I think that's fair. After that ??? Well I think we have: Flat-no - the change not important, almost any cost is too high It's not *any* cost, this goes deep and wide, it's one of the basic concepts of numpy that you want to rename. Note, I'm just a user of numpy My main objection was to N and Z, which would have affected me (and statsmodels developers) I don't really care about the layout change. I have no or almost no code depending on it. And, I don't have to implement it, nor do I have to struggle with the low level numpy behavior that would be affected by this. (And renaming doesn't change the concept.) Josef You Ralf Bradley Mid-no - maybe something could work, but not sure we've seen it yet. Chris Middle - current situation can be confusing, maybe one of the proposed solutions would be acceptable Sebastian Nathaniel Mid-yes - previous apparent vote for argument name change Éric Depagne Andrew Jaffe (sorry if I misrepresent you) And then me. I am trying to be balanced. Unlike others, I think better names would have a significant impact on how coherent numpy is to explain and use. It seems to me that a change would be beneficial in the long term, and I'm confident we can agree on a schedule for that change that would be acceptable. But you know that. So - as I understand our 'model' - our job is to try and come to some shared agreement, if we possibly can. It has been good and encouraging for me at least to see that we have developed our ideas over the course of this thread. Cheers, Matthew
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 7:39 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:50 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 4:27 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey snip I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. So far the consensus is that the documentation needs improvement. The only thing all of the No camp agree with is documentation improvement, I think that's fair. After that ??? Well I think we have: Flat-no - the change not important, almost any cost is too high It's not *any* cost, this goes deep and wide, it's one of the basic concepts of numpy that you want to rename. The proposal I last made was to change the default name to 'layout' after some period to be agreed - say - P - with suitable warning in the docstring up until that time, and after, and leave 'order' as an alias forever. The only problem I can see with this, is that if someone, after period P, does not read the docstring, and uses 'layout' instead of 'order', then they will find that their code is not backwards compatible with versions of numpy of greater age than P. They can fix this, forever, by reverting to 'order'. That's certainly not zero cost, but it's not much cost either, and the cost will depend on P. Note, I'm just a user of numpy My main objection was to N and Z, which would have affected me (and statsmodels developers) Right. I don't really care about the layout change. I have no or almost no code depending on it. And, I don't have to implement it, nor do I have to struggle with the low level numpy behavior that would be affected by this. (And renaming doesn't change the concept.) No, right, the renaming is to clarify and distinguish the concepts. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
On Fri, Apr 5, 2013 at 10:47 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 7:39 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:50 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 4:27 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey snip I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. So far the consensus is that the documentation needs improvement. The only thing all of the No camp agree with is documentation improvement, I think that's fair. After that ??? Well I think we have: Flat-no - the change not important, almost any cost is too high It's not *any* cost, this goes deep and wide, it's one of the basic concepts of numpy that you want to rename. The proposal I last made was to change the default name to 'layout' after some period to be agreed - say - P - with suitable warning in the docstring up until that time, and after, and leave 'order' as an alias forever. The only problem I can see with this, is that if someone, after period P, does not read the docstring, and uses 'layout' instead of 'order', then they will find that their code is not backwards compatible with versions of numpy of greater age than P. They can fix this, forever, by reverting to 'order'. That's certainly not zero cost, but it's not much cost either, and the cost will depend on P. You edit large parts of the numpy tutorial and explanations, you add a second keyword to (rough guess) 10 functions and a similar number of methods (even wilder guess), the methods are in C, so you have to change it both on the c and the python level. Two keywords will confuse users for a long time (and which one is in the tutorial documentation) I'm just guessing and I have no idea about the c-level. Josef Note, I'm just a user of numpy My main objection was to N and Z, which would have affected me (and statsmodels developers) Right.
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi, On Fri, Apr 5, 2013 at 8:31 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 10:47 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 7:39 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:50 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 4:27 PM, josef.p...@gmail.com wrote: On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey snip I completely agree that we'd have to be gentle with the change. The problem we'd want to avoid is people innocently using 'layout' and finding to their annoyance that the code doesn't work with other people's numpy. How about: Step 1: 'order' remains as named keyword, layout added as alias, comment on the lines of layout will become the default keyword for this option in later versions of numpy; please consider updating any code that does not need to remain backwards compatible'. Step 2: default keyword becomes 'layout' with 'order' as alias, comment like order is an alias for 'layout' to maintain backwards compatibility with numpy = 1.7.1', please update any code that does not need to maintain backwards compatibility with these numpy versions' Step 3: Add deprecation warning for 'order', order will be removed as an alias in future versions of numpy Step 4: (distant future) Remove alias ? A very strong -1 from me. Now we're talking about deprecation warnings and a backwards compatibility break after all. I thought we agreed that this was a very bad idea, so why are you proposing it now? Here's how I see it: deprecation of order is a no go. Therefore we have two choices here: 1. Simply document the current order keyword better and leave it at that. 2. Add a layout (or index_order) keyword, and live with both order and layout keywords forever. (2) is at least as confusing as (1), more work and poor design. Therefore I propose to go with (1). You are saying that deprecation of 'order' at any stage in the next 10 years of numpy's lifetime is a no go? For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. I think that is short-sighted and I think it will damage numpy. It will damage numpy to be conservative and not change a name for a little bit of clarity for some people that avoids reading the docs maybe a little more carefully? There's a lot of things that can damage numpy, but this isn't even close in my book. Too few developers, continuous backwards compatibility issues, faster alternative libraries surpassing numpy - that's the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. So far the consensus is that the documentation needs improvement. The only thing all of the No camp agree with is documentation improvement, I think that's fair. After that ??? Well I think we have: Flat-no - the change not important, almost any cost is too high It's not *any* cost, this goes deep and wide, it's one of the basic concepts of numpy that you want to rename. The proposal I last made was to change the default name to 'layout' after some period to be agreed - say - P - with suitable warning in the docstring up until that time, and after, and leave 'order' as an alias forever. The only problem I can see with this, is that if someone, after period P, does not read the docstring, and uses 'layout' instead of 'order', then they will find that their code is not backwards compatible with versions of numpy of greater age than P. They can fix this, forever, by reverting to 'order'. That's certainly not zero cost, but it's not much cost either, and the cost will depend on P. You edit large parts of the numpy tutorial and explanations, We agree that these concepts need to be clarified in the explanations. For the docs, we would first add the keyword as an alias and note it so. you add a second keyword to (rough guess) 10 functions and a similar number of methods (even wilder guess), the methods are in C, so you have to change it both on the c and the python level. I'm OK to do the code changes, I don't think that's a concern at the
