Re: [Numpy-discussion] 1.9.x branch
On Mi, 2014-04-23 at 10:11 -0400, josef.p...@gmail.com wrote: On Wed, Apr 23, 2014 at 5:32 AM, Sebastian Berg sebast...@sipsolutions.net wrote: On Di, 2014-04-22 at 15:35 -0600, Charles R Harris wrote: Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) Sounds good. There are currently a number of 1.9.0 blockers, which can be seen here. Datetime timezone handling broken in 1.7.x I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. Return multiple field selection as ro view I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. I am a bit worried here that comparisons might make trouble. Object array creation new conversion to int This one needs a decision. Julian, Sebastian, thoughts? Maybe for all to consider this is about what happens for object arrays if you do things like: # Array cast to object array (np.array(arr) would be identical): a = np.arange(10).astype(object) # Array passed into new array creation (not just *one* array): b = np.array([np.arange(10)], dtype=object) # Numerical array is assigned to object array: c = np.empty(10, dtype=object) c[...] = np.arange(10) Before this change, the result was: type(a[0]) is int type(b[0,0]) is np.int_ # Note the numpy type type(c[0]) is int After this change, they are all `int`. Though note that the numpy type is preserved for example for long double. On the one hand preserving the numpy type might be nice, but on the other hand we don't care much about the dtypes of scalars and in practice the python types are probably more often wanted. what if I don't like python? Fair point :). I also think it is more consistent if we use the numpy types (and the user has to cast to the python type explicitly). Could argue that if someone casts to object, they might like python objects, but if you don't want them that is tricky, too. There is the option that the numerical array - object array cast always returns an array of numpy scalars. Making it consistent (opposite to what is currently in numpy master). This would mean that `numerical_arr.astype(object)` would give an array of numpy scalars always. Getting python scalars would only be possible using `arr.item()` (or `tolist`). I guess that change is less likely to cause problems, because the numpy types can do more things normally though they are slower. So a (still a bit unsure) +1 from me for making numeric - object casts return arrays of numpy scalars unless we have reason to expect that to cause problems. Not sure how easy that would be to change, it wasn't a one line change when I tried, so there is something slightly tricky to clear out, but probably not too hard either. - Sebastian np.int_(0)**(-1) inf 0**-1 Traceback (most recent call last): File pyshell#28, line 1, in module 0**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power type(np.arange(5)[0]) type 'numpy.int32' np.arange(5)[0]**-1 inf type(np.arange(5)[0].item()) type 'int' np.arange(5)[0].item()**-1 Traceback (most recent call last): File pyshell#40, line 1, in module np.arange(5)[0].item()**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power np.__version__ '1.6.1' I remember struggling through this (avoiding python operations) quite a bit in my early bugfixes to scipy.stats.distributions. (IIRC I ended up avoiding most ints.) Josef Since I just realized that things are safe (float128 does not cast to float after all), I changed my mind and am tempted to keep the new behaviour. That is, if it does not create any problems (there was some issue in scipy, not sure how bad). - Sebastian Median of np.matrix is broken( Not sure what the status of this one is. 1.8 deprecations: Follow-up ticket Things that might should be removed. ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3 I believe this one was fixed. For general problems reading/writing big files on OS X, I believe they were fixed in Maverick and I'm inclined to recommend an OS upgrade rather than work to chunk all the io. Deprecate NPY_CHAR This one is waiting on a fix from Pearu to make f2py use numpy strings. I think we will need to do this ourselves if we want to carry through the deprecation. In any case it probably needs to be pushed off to 1.10. 1.7 deprecations: Follow-up ticket Some of these changes have been made, ro diagonal view for instance, some still remain.
Re: [Numpy-discussion] 1.9.x branch
On Sat, Apr 26, 2014 at 1:12 PM, Sebastian Berg sebast...@sipsolutions.netwrote: On Mi, 2014-04-23 at 10:11 -0400, josef.p...@gmail.com wrote: On Wed, Apr 23, 2014 at 5:32 AM, Sebastian Berg sebast...@sipsolutions.net wrote: On Di, 2014-04-22 at 15:35 -0600, Charles R Harris wrote: Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) Sounds good. There are currently a number of 1.9.0 blockers, which can be seen here. Datetime timezone handling broken in 1.7.x I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. Return multiple field selection as ro view I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. I am a bit worried here that comparisons might make trouble. Object array creation new conversion to int This one needs a decision. Julian, Sebastian, thoughts? Maybe for all to consider this is about what happens for object arrays if you do things like: # Array cast to object array (np.array(arr) would be identical): a = np.arange(10).astype(object) # Array passed into new array creation (not just *one* array): b = np.array([np.arange(10)], dtype=object) # Numerical array is assigned to object array: c = np.empty(10, dtype=object) c[...] = np.arange(10) Before this change, the result was: type(a[0]) is int type(b[0,0]) is np.int_ # Note the numpy type type(c[0]) is int After this change, they are all `int`. Though note that the numpy type is preserved for example for long double. On the one hand preserving the numpy type might be nice, but on the other hand we don't care much about the dtypes of scalars and in practice the python types are probably more often wanted. what if I don't like python? Fair point :). I also think it is more consistent if we use the numpy types (and the user has to cast to the python type explicitly). Could argue that if someone casts to object, they might like python objects, but if you don't want them that is tricky, too. There is the option that the numerical array - object array cast always returns an array of numpy scalars. Making it consistent (opposite to what is currently in numpy master). This would mean that `numerical_arr.astype(object)` would give an array of numpy scalars always. Getting python scalars would only be possible using `arr.item()` (or `tolist`). I guess that change is less likely to cause problems, because the numpy types can do more things normally though they are slower. So a (still a bit unsure) +1 from me for making numeric - object casts return arrays of numpy scalars unless we have reason to expect that to cause problems. Not sure how easy that would be to change, it wasn't a one line change when I tried, so there is something slightly tricky to clear out, but probably not too hard either. More general background question. Why is there casting involved in object arrays? I thought object arrays will just take and return whatever you put in. If I use python ints, I might want python ints. If I use numpy int_s, I might want numpy ints. b1 = np.array([np.arange(10)], dtype=object) versus b2 = np.array([list(range(10))], dtype=object) b1 = np.array([np.arange(10)], dtype=object) type(b1[0,2]) type 'numpy.int32' type(b1[0][2]) type 'numpy.int32' b2 = np.array([list(range(10))], dtype=object) type(b2[0,2]) type 'int' type(b2[0][2]) type 'int' another version type(np.array(np.arange(10).tolist(), dtype=object)[2]) type 'int' type(np.array(list(np.arange(10)), dtype=object)[2]) type 'numpy.int32' Josef - Sebastian np.int_(0)**(-1) inf 0**-1 Traceback (most recent call last): File pyshell#28, line 1, in module 0**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power type(np.arange(5)[0]) type 'numpy.int32' np.arange(5)[0]**-1 inf type(np.arange(5)[0].item()) type 'int' np.arange(5)[0].item()**-1 Traceback (most recent call last): File pyshell#40, line 1, in module np.arange(5)[0].item()**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power np.__version__ '1.6.1' I remember struggling through this (avoiding python operations) quite a bit in my early bugfixes to scipy.stats.distributions. (IIRC I ended up avoiding most ints.) Josef Since I just realized that things are safe (float128 does not cast to float after all), I changed my mind and am tempted to keep the new behaviour. That is, if it does
Re: [Numpy-discussion] 1.9.x branch
On Di, 2014-04-22 at 15:35 -0600, Charles R Harris wrote: Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) Sounds good. There are currently a number of 1.9.0 blockers, which can be seen here. Datetime timezone handling broken in 1.7.x I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. Return multiple field selection as ro view I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. I am a bit worried here that comparisons might make trouble. Object array creation new conversion to int This one needs a decision. Julian, Sebastian, thoughts? Maybe for all to consider this is about what happens for object arrays if you do things like: # Array cast to object array (np.array(arr) would be identical): a = np.arange(10).astype(object) # Array passed into new array creation (not just *one* array): b = np.array([np.arange(10)], dtype=object) # Numerical array is assigned to object array: c = np.empty(10, dtype=object) c[...] = np.arange(10) Before this change, the result was: type(a[0]) is int type(b[0,0]) is np.int_ # Note the numpy type type(c[0]) is int After this change, they are all `int`. Though note that the numpy type is preserved for example for long double. On the one hand preserving the numpy type might be nice, but on the other hand we don't care much about the dtypes of scalars and in practice the python types are probably more often wanted. Since I just realized that things are safe (float128 does not cast to float after all), I changed my mind and am tempted to keep the new behaviour. That is, if it does not create any problems (there was some issue in scipy, not sure how bad). - Sebastian Median of np.matrix is broken( Not sure what the status of this one is. 1.8 deprecations: Follow-up ticket Things that might should be removed. ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3 I believe this one was fixed. For general problems reading/writing big files on OS X, I believe they were fixed in Maverick and I'm inclined to recommend an OS upgrade rather than work to chunk all the io. Deprecate NPY_CHAR This one is waiting on a fix from Pearu to make f2py use numpy strings. I think we will need to do this ourselves if we want to carry through the deprecation. In any case it probably needs to be pushed off to 1.10. 1.7 deprecations: Follow-up ticket Some of these changes have been made, ro diagonal view for instance, some still remain. Additions, updates, and thoughts welcome. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.9.x branch
On Wed, Apr 23, 2014 at 5:32 AM, Sebastian Berg sebast...@sipsolutions.net wrote: On Di, 2014-04-22 at 15:35 -0600, Charles R Harris wrote: Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) Sounds good. There are currently a number of 1.9.0 blockers, which can be seen here. Datetime timezone handling broken in 1.7.x I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. Return multiple field selection as ro view I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. I am a bit worried here that comparisons might make trouble. Object array creation new conversion to int This one needs a decision. Julian, Sebastian, thoughts? Maybe for all to consider this is about what happens for object arrays if you do things like: # Array cast to object array (np.array(arr) would be identical): a = np.arange(10).astype(object) # Array passed into new array creation (not just *one* array): b = np.array([np.arange(10)], dtype=object) # Numerical array is assigned to object array: c = np.empty(10, dtype=object) c[...] = np.arange(10) Before this change, the result was: type(a[0]) is int type(b[0,0]) is np.int_ # Note the numpy type type(c[0]) is int After this change, they are all `int`. Though note that the numpy type is preserved for example for long double. On the one hand preserving the numpy type might be nice, but on the other hand we don't care much about the dtypes of scalars and in practice the python types are probably more often wanted. what if I don't like python? np.int_(0)**(-1) inf 0**-1 Traceback (most recent call last): File pyshell#28, line 1, in module 0**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power type(np.arange(5)[0]) type 'numpy.int32' np.arange(5)[0]**-1 inf type(np.arange(5)[0].item()) type 'int' np.arange(5)[0].item()**-1 Traceback (most recent call last): File pyshell#40, line 1, in module np.arange(5)[0].item()**-1 ZeroDivisionError: 0.0 cannot be raised to a negative power np.__version__ '1.6.1' I remember struggling through this (avoiding python operations) quite a bit in my early bugfixes to scipy.stats.distributions. (IIRC I ended up avoiding most ints.) Josef Since I just realized that things are safe (float128 does not cast to float after all), I changed my mind and am tempted to keep the new behaviour. That is, if it does not create any problems (there was some issue in scipy, not sure how bad). - Sebastian Median of np.matrix is broken( Not sure what the status of this one is. 1.8 deprecations: Follow-up ticket Things that might should be removed. ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3 I believe this one was fixed. For general problems reading/writing big files on OS X, I believe they were fixed in Maverick and I'm inclined to recommend an OS upgrade rather than work to chunk all the io. Deprecate NPY_CHAR This one is waiting on a fix from Pearu to make f2py use numpy strings. I think we will need to do this ourselves if we want to carry through the deprecation. In any case it probably needs to be pushed off to 1.10. 1.7 deprecations: Follow-up ticket Some of these changes have been made, ro diagonal view for instance, some still remain. Additions, updates, and thoughts welcome. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.9.x branch
On Tue, Apr 22, 2014 at 2:35 PM, Charles R Harris charlesr.har...@gmail.com wrote: *Datetime timezone handling broken in 1.7.x*https://github.com/numpy/numpy/issues/3388 I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. * https://github.com/numpy/numpy/issues/3804* Darn! that's what we said for 1.8. However, Sankarshan Mudkavi has written up a NEP, and this is really a matter of ripping code out, rather than adding much. Someone familiar with the code should be able to whip this out pretty easily (I think we've abandoned for now any hope of proper TZ handling) ( https://github.com/Sankarshan-Mudkavi/numpy/blob/Enhance-datetime64/doc/neps/datetime-improvement-proposal.rst ) Datetime64 is really pretty broken this way -- the sooner it gets cleaned up the better. https://github.com/Sankarshan-Mudkavi/numpy/blob/Enhance-datetime64/doc/neps/datetime-improvement-proposal.rst The trick is that as simple as it may be, someone still needs to do it. I, for one, am not familiar with the code, and have pathetic C skills anyway, so it would not be an efficient use of my time to try to do it. but I will go and edit the NEP and write test cases! -Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] 1.9.x branch
Hi All, I'd like to branch 1.9.x at the end of the month. There are a couple of reasons for the timing. First, we have a lot of new stuff in the development branch. Second, there is work ongoing in masked arrays that I'd like to keep out of the release so that it has more time to settle. Third, it's past time ;) There are currently a number of 1.9.0 blockers, which can be seen herehttps://github.com/numpy/numpy/issues?direction=descmilestone=29page=1sort=updatedstate=open. *Datetime timezone handling broken in 1.7.x*https://github.com/numpy/numpy/issues/3388 I don't think there is time to get this done for 1.9.0 and it needs to be pushed off to 1.10.0. *Return multiple field selection as ro view https://github.com/numpy/numpy/issues/3804* I have a branch for this, but because the returned type differs from a copy by alignment spacing there was a test failure. Merging that branch might cause some incompatibilities. *Object array creation new conversion to int*https://github.com/numpy/numpy/issues/3804 This one needs a decision. Julian, Sebastian, thoughts? *Median of np.matrix is broken* https://github.com/numpy/numpy/issues/4301 Not sure what the status of this one is. *1.8 deprecations: Follow-up ticket https://github.com/numpy/numpy/issues/4131* Things that might should be removed. *ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3 https://github.com/numpy/numpy/issues/3858* I believe this one was fixed. For general problems reading/writing big files on OS X, I believe they were fixed in Maverick and I'm inclined to recommend an OS upgrade rather than work to chunk all the io. Deprecate NPY_CHAR https://github.com/numpy/numpy/issues/2801This one is waiting on a fix from Pearu to make f2py use numpy strings. I think we will need to do this ourselves if we want to carry through the deprecation. In any case it probably needs to be pushed off to 1.10. 1.7 deprecations: Follow-up tickethttps://github.com/numpy/numpy/issues/456Some of these changes have been made, ro diagonal view for instance, some still remain. Additions, updates, and thoughts welcome. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
