Re: [Numpy-discussion] tighten up ufunc casting rule
On Tue, Jun 7, 2011 at 2:08 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Tue, Jun 7, 2011 at 8:59 PM, Mark Wiebe mwwi...@gmail.com wrote: On Tue, Jun 7, 2011 at 1:41 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe mwwi...@gmail.com wrote: On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.comwrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. There's no reason to set the minimum required numpy to 1.6 AFAIK, and it's definitely not desirable. Ok, I think there are two ways to resolve this kind of error. One would be to add a condition testing for a version = 1.6, and setting casting='unsafe' when that occurs, and the other would be to insert a call to .astype() to force the type. The former is probably preferable, to avoid the unnecessary copy. Does numpy provide the version in tuple form, so a version comparison like this can be done easily? numpy.version doesn't seem to have one in it. No, just as a string. I think it's fine to do: In [4]: np.version.short_version Out[4]: '2.0.0' In [5]: np.version.short_version '1.5.1' Out[5]: True For a very convoluted version that produces a tuple see parse_numpy_version() in scipy.pavement.py I've submitted a pull request to scipy which fixes up the ndimage errors, as well as a netcdf bug. I think it would be a good idea to add something like np.version.short_version_tuple, at least before we get to numpy 10.0.0 or 1.10.0, where this comparison will break down. -Mark Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe mwwi...@gmail.com wrote: On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. There's no reason to set the minimum required numpy to 1.6 AFAIK, and it's definitely not desirable. Ralf I've updated the tighten_casting branch so it now passes all tests. For masked arrays, this required changing some tests to not assume float - int casts are fine by default, but otherwise I fixed things by relaxing the rules just where necessary. It now depends on the datetime-fixes branch, which I would like to merge at its current point. -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Tue, Jun 7, 2011 at 1:41 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe mwwi...@gmail.com wrote: On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. There's no reason to set the minimum required numpy to 1.6 AFAIK, and it's definitely not desirable. Ok, I think there are two ways to resolve this kind of error. One would be to add a condition testing for a version = 1.6, and setting casting='unsafe' when that occurs, and the other would be to insert a call to .astype() to force the type. The former is probably preferable, to avoid the unnecessary copy. Does numpy provide the version in tuple form, so a version comparison like this can be done easily? numpy.version doesn't seem to have one in it. -Mark Ralf I've updated the tighten_casting branch so it now passes all tests. For masked arrays, this required changing some tests to not assume float - int casts are fine by default, but otherwise I fixed things by relaxing the rules just where necessary. It now depends on the datetime-fixes branch, which I would like to merge at its current point. -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Tue, Jun 7, 2011 at 8:59 PM, Mark Wiebe mwwi...@gmail.com wrote: On Tue, Jun 7, 2011 at 1:41 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Mon, Jun 6, 2011 at 6:56 PM, Mark Wiebe mwwi...@gmail.com wrote: On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. There's no reason to set the minimum required numpy to 1.6 AFAIK, and it's definitely not desirable. Ok, I think there are two ways to resolve this kind of error. One would be to add a condition testing for a version = 1.6, and setting casting='unsafe' when that occurs, and the other would be to insert a call to .astype() to force the type. The former is probably preferable, to avoid the unnecessary copy. Does numpy provide the version in tuple form, so a version comparison like this can be done easily? numpy.version doesn't seem to have one in it. No, just as a string. I think it's fine to do: In [4]: np.version.short_version Out[4]: '2.0.0' In [5]: np.version.short_version '1.5.1' Out[5]: True For a very convoluted version that produces a tuple see parse_numpy_version() in scipy.pavement.py Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. -Mark Cheers, Ralf NUMPY == ERROR: test_ones_like (test_numeric.TestLikeFuncs) -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py, line 1257, in test_ones_like self.check_like_function(np.ones_like, 1) File /Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py, line 1200, in check_like_function dz = like_function(d, dtype=dtype) TypeError: found a loop for ufunc 'ones_like' matching the type-tuple, but the inputs and/or outputs could not be cast according to the casting rule == ERROR: test_methods_with_output (test_core.TestMaskedArrayArithmetic) -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1109, in test_methods_with_output result = xmmeth(axis=0, out=output) File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 4774, in std out **= 0.5 File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3765, in __ipow__ ndarray.__ipow__(self._data, np.where(self._mask, 1, other_data)) TypeError: ufunc 'power' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') according to the casting rule 'same_kind' == ERROR: Check that we don't shrink a mask when not wanted -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1044, in test_noshrinking a /= 1. File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3725, in __idiv__ ndarray.__idiv__(self._data, np.where(self._mask, 1, other_data)) TypeError: ufunc 'divide' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') according to the casting rule 'same_kind' == ERROR: Test of inplace additions -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1651, in test_inplace_addition_array x += a File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3686, in __iadd__
Re: [Numpy-discussion] tighten up ufunc casting rule
On Mon, Jun 6, 2011 at 10:30 AM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Jun 5, 2011 at 3:43 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cool, thanks for running those. I already took a chunk out of the NumPy failures. The ones_like function shouldn't really be a ufunc, but rather be like zeros_like and empty_like, but that's probably not something to change right now. The datetime-fixes type resolution change provides a mechanism to fix that up pretty easily. For Scipy, what do you think is the best way to resolve it? If NumPy 1.6 is the minimum version for the next scipy, I would add casting='unsafe' to the failing sqrt call. I've updated the tighten_casting branch so it now passes all tests. For masked arrays, this required changing some tests to not assume float - int casts are fine by default, but otherwise I fixed things by relaxing the rules just where necessary. It now depends on the datetime-fixes branch, which I would like to merge at its current point. -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. Here are some results of testing your tighten_casting branch on a few projects - no need to first put it in master first to do that. Four failures in numpy, two in scipy, four in scikit-learn (plus two that don't look related), none in scikits.statsmodels. I didn't check how many of them are actual bugs. I'm not against trying out your change, but it would probably be good to do some more testing first and fix the issues found before putting it in. Then at least if people run into issues with the already tested packages, you can just tell them to update those to latest master. Cheers, Ralf NUMPY == ERROR: test_ones_like (test_numeric.TestLikeFuncs) -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py, line 1257, in test_ones_like self.check_like_function(np.ones_like, 1) File /Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py, line 1200, in check_like_function dz = like_function(d, dtype=dtype) TypeError: found a loop for ufunc 'ones_like' matching the type-tuple, but the inputs and/or outputs could not be cast according to the casting rule == ERROR: test_methods_with_output (test_core.TestMaskedArrayArithmetic) -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1109, in test_methods_with_output result = xmmeth(axis=0, out=output) File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 4774, in std out **= 0.5 File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3765, in __ipow__ ndarray.__ipow__(self._data, np.where(self._mask, 1, other_data)) TypeError: ufunc 'power' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') according to the casting rule 'same_kind' == ERROR: Check that we don't shrink a mask when not wanted -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1044, in test_noshrinking a /= 1. File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3725, in __idiv__ ndarray.__idiv__(self._data, np.where(self._mask, 1, other_data)) TypeError: ufunc 'divide' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') according to the casting rule 'same_kind' == ERROR: Test of inplace additions -- Traceback (most recent call last): File /Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py, line 1651, in test_inplace_addition_array x += a File /Users/rgommers/Code/numpy/numpy/ma/core.py, line 3686, in __iadd__ ndarray.__iadd__(self._data, np.where(self._mask, 0, getdata(other))) TypeError: ufunc 'add' output (typecode 'd') could not be coerced to provided output parameter (typecode 'l') according to the casting rule 'same_kind' SCIPY == ERROR: gaussian gradient magnitude filter 1 -- Traceback (most recent call last): File /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/case.py, line 187, in runTest self.test(*self.arg) File
Re: [Numpy-discussion] tighten up ufunc casting rule
On Sun, Jun 05, 2011 at 10:43:45PM +0200, Ralf Gommers wrote: four in scikit-learn (plus two that don't look related), Yeah, some of these failures are due to numerical unstabilities in tests (nasty ones, still fighting) and some simply to crappy code (HMMs are hopeless :( ). Now, with regards to the actual failures induced by the new branch, it took me a while to understand why they where happening, and now I realise that we probably should have explicit coercions at these locations. Gaël ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Sun, Jun 5, 2011 at 3:15 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Sun, Jun 05, 2011 at 10:43:45PM +0200, Ralf Gommers wrote: four in scikit-learn (plus two that don't look related), Yeah, some of these failures are due to numerical unstabilities in tests (nasty ones, still fighting) and some simply to crappy code (HMMs are hopeless :( ). Now, with regards to the actual failures induced by the new branch, it took me a while to understand why they where happening, and now I realise that we probably should have explicit coercions at these locations. So the failures in those cases is a good thing? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Sun, Jun 05, 2011 at 04:41:26PM -0600, Charles R Harris wrote: Now, with regards to the actual failures induced by the new branch, it took me a while to understand why they where happening, and now I realise that we probably should have explicit coercions at these locations. So the failures in those cases is a good thing? Yes. Gael ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Sun, Jun 5, 2011 at 6:51 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Sun, Jun 05, 2011 at 04:41:26PM -0600, Charles R Harris wrote: Now, with regards to the actual failures induced by the new branch, it took me a while to understand why they where happening, and now I realise that we probably should have explicit coercions at these locations. So the failures in those cases is a good thing? Yes. I remember in stats distributions, there were casting bugs like this and they are nasty to find. assigning a nan to an int has been changed to raise an exception, but inline operation still produces silent results a = np.arange(3, dtype=np.int32) a[0] = np.nan Traceback (most recent call last): File pyshell#9, line 1, in module a[0] = np.nan ValueError: cannot convert float NaN to integer a += np.nan a array([-2147483648, -2147483648, -2147483648]) a = np.arange(3, dtype=np.int32) a *= np.array([1.,1., np.nan]) a array([ 0, 1, -2147483648]) definitely wrong Josef Gael ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Gael ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] tighten up ufunc casting rule
On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote: Would anyone object to, at least temporarily, tightening up the default ufunc casting rule to 'same_kind' in NumPy master? It's a one line change, so would be easy to undo, but such a change is very desirable in my opinion. This would raise an exception, since it's np.add(a, 1.9, out=a), converting a float to an int: a = np.arange(3, dtype=np.int32) a += 1.9 That's probably going to break a huge amount of code which relies on the current behavior. Am I right in believing that this should only be considered for a major release of numpy, say numpy 2.0? Absolutely, and that's why I'm proposing to do it in master now, fairly early in a development cycle, so we can evaluate its effects. If the next version is 1.7, we probably would roll it back for release (a 1 line change), and if the next version is 2.0, we probably would keep it in. I suspect at least some of the code relying on the current behavior may have bugs, and tightening this up is a way to reveal them. -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
