On Thu, Jun 2, 2011 at 10:12 PM, Mark Wiebe <mwwi...@gmail.com> wrote:
> On Thu, Jun 2, 2011 at 3:09 PM, Gael Varoquaux <
> gael.varoqu...@normalesup.org> wrote:
>
>> On Thu, Jun 02, 2011 at 03:06:58PM -0500, Mark Wiebe wrote:
>> > Would anyone object to, at least temporarily, tightening up the
>> default
>> > ufunc casting rule to 'same_kind' in NumPy master? It's a one line
>> change,
>> > so would be easy to undo, but such a change is very desirable in my
>> > opinion.
>> > This would raise an exception, since it's np.add(a, 1.9, out=a),
>> > converting a float to an int:
>>
>> > >>> a = np.arange(3, dtype=np.int32)
>>
>> > >>> a += 1.9
>>
>> That's probably going to break a huge amount of code which relies on the
>> current behavior.
>>
>> Am I right in believing that this should only be considered for a major
>> release of numpy, say numpy 2.0?
>
>
> Absolutely, and that's why I'm proposing to do it in master now, fairly
> early in a development cycle, so we can evaluate its effects. If the next
> version is 1.7, we probably would roll it back for release (a 1 line
> change), and if the next version is 2.0, we probably would keep it in.
>
> I suspect at least some of the code relying on the current behavior may
> have bugs, and tightening this up is a way to reveal them.
>
>
Here are some results of testing your tighten_casting branch on a few
projects - no need to first put it in master first to do that. Four failures
in numpy, two in scipy, four in scikit-learn (plus two that don't look
related), none in scikits.statsmodels. I didn't check how many of them are
actual bugs.
I'm not against trying out your change, but it would probably be good to do
some more testing first and fix the issues found before putting it in. Then
at least if people run into issues with the already tested packages, you can
just tell them to update those to latest master.
Cheers,
Ralf
NUMPY
======================================================================
ERROR: test_ones_like (test_numeric.TestLikeFuncs)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py", line
1257, in test_ones_like
self.check_like_function(np.ones_like, 1)
File "/Users/rgommers/Code/numpy/numpy/core/tests/test_numeric.py", line
1200, in check_like_function
dz = like_function(d, dtype=dtype)
TypeError: found a loop for ufunc 'ones_like' matching the type-tuple, but
the inputs and/or outputs could not be cast according to the casting rule
======================================================================
ERROR: test_methods_with_output (test_core.TestMaskedArrayArithmetic)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py", line 1109,
in test_methods_with_output
result = xmmeth(axis=0, out=output)
File "/Users/rgommers/Code/numpy/numpy/ma/core.py", line 4774, in std
out **= 0.5
File "/Users/rgommers/Code/numpy/numpy/ma/core.py", line 3765, in __ipow__
ndarray.__ipow__(self._data, np.where(self._mask, 1, other_data))
TypeError: ufunc 'power' output (typecode 'd') could not be coerced to
provided output parameter (typecode 'l') according to the casting rule
'same_kind'
======================================================================
ERROR: Check that we don't shrink a mask when not wanted
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py", line 1044,
in test_noshrinking
a /= 1.
File "/Users/rgommers/Code/numpy/numpy/ma/core.py", line 3725, in __idiv__
ndarray.__idiv__(self._data, np.where(self._mask, 1, other_data))
TypeError: ufunc 'divide' output (typecode 'd') could not be coerced to
provided output parameter (typecode 'l') according to the casting rule
'same_kind'
======================================================================
ERROR: Test of inplace additions
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/rgommers/Code/numpy/numpy/ma/tests/test_core.py", line 1651,
in test_inplace_addition_array
x += a
File "/Users/rgommers/Code/numpy/numpy/ma/core.py", line 3686, in __iadd__
ndarray.__iadd__(self._data, np.where(self._mask, 0, getdata(other)))
TypeError: ufunc 'add' output (typecode 'd') could not be coerced to
provided output parameter (typecode 'l') according to the casting rule
'same_kind'
SCIPY
======================================================================
ERROR: gaussian gradient magnitude filter 1
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/case.py",
line 187, in runTest
self.test(*self.arg)
File "/Users/rgommers/Code/scipy/scipy/ndimage/tests/test_ndimage.py",
line 670, in test_gaussian_gradient_magnitude01
1.0)
File "/Users/rgommers/Code/scipy/scipy/ndimage/filters.py", line 479, in
gaussian_gradient_magnitude
cval, extra_arguments = (sigma,))
File "/Users/rgommers/Code/scipy/scipy/ndimage/filters.py", line 450, in
generic_gradient_magnitude
numpy.sqrt(output, output)
TypeError: ufunc 'sqrt' output (typecode 'd') could not be coerced to
provided output parameter (typecode 'l') according to the casting rule
'same_kind'
======================================================================
ERROR: gaussian gradient magnitude filter 2
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/case.py",
line 187, in runTest
self.test(*self.arg)
File "/Users/rgommers/Code/scipy/scipy/ndimage/tests/test_ndimage.py",
line 685, in test_gaussian_gradient_magnitude02
output)
File "/Users/rgommers/Code/scipy/scipy/ndimage/filters.py", line 479, in
gaussian_gradient_magnitude
cval, extra_arguments = (sigma,))
File "/Users/rgommers/Code/scipy/scipy/ndimage/filters.py", line 450, in
generic_gradient_magnitude
numpy.sqrt(output, output)
TypeError: ufunc 'sqrt' output (typecode 'd') could not be coerced to
provided output parameter (typecode 'l') according to the casting rule
'same_kind'
SCIKIT-LEARN
======================================================================
ERROR: Test decision_function
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/case.py",
line 187, in runTest
self.test(*self.arg)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/svm/tests/test_svm.py",
line 254, in test_decision_function
assert_array_almost_equal(-dec, np.ravel(clf.decision_function(data)))
File "/Users/rgommers/Code/scikit-learn/scikits/learn/svm/base.py", line
276, in decision_function
**self._get_params())
File "libsvm.pyx", line 385, in scikits.learn.svm.libsvm.decision_function
(scikits/learn/svm/libsvm.c:4748)
ValueError: ndarray is not C-contiguous
======================================================================
ERROR: Test weights on individual samples
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/case.py",
line 187, in runTest
self.test(*self.arg)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/svm/tests/test_svm.py",
line 282, in test_sample_weights
assert_array_equal(clf.predict(X[2]), [1.])
File "/Users/rgommers/Code/scikit-learn/scikits/learn/svm/base.py", line
186, in predict
svm_type=svm_type, **self._get_params())
File "libsvm.pyx", line 224, in scikits.learn.svm.libsvm.predict
(scikits/learn/svm/libsvm.c:3133)
ValueError: ndarray is not C-contiguous
======================================================================
FAIL: Doctest: scikits.learn.decomposition.pca.PCA
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 2145, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for scikits.learn.decomposition.pca.PCA
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
94, in PCA
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 161, in scikits.learn.decomposition.pca.PCA
Failed example:
pca.fit(X)
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.PCA[4]>", line 1, in
<module>
pca.fit(X)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
191, in fit
self._fit(X, **params)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
227, in _fit
X -= self.mean_
TypeError: ufunc 'subtract' output (typecode 'd') could not be coerced
to provided output parameter (typecode 'l') according to the casting rule
'same_kind'
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 163, in scikits.learn.decomposition.pca.PCA
Failed example:
print pca.explained_variance_ratio_
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.PCA[5]>", line 1, in
<module>
print pca.explained_variance_ratio_
AttributeError: 'PCA' object has no attribute
'explained_variance_ratio_'
>> raise self.failureException(self.format_failure(<StringIO.StringIO
instance at 0xfe05648>.getvalue()))
======================================================================
FAIL: Doctest: scikits.learn.decomposition.pca.ProbabilisticPCA
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 2145, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for
scikits.learn.decomposition.pca.ProbabilisticPCA
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
273, in ProbabilisticPCA
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 342, in scikits.learn.decomposition.pca.ProbabilisticPCA
Failed example:
pca.fit(X)
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.ProbabilisticPCA[4]>",
line 1, in <module>
pca.fit(X)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
191, in fit
self._fit(X, **params)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
227, in _fit
X -= self.mean_
TypeError: ufunc 'subtract' output (typecode 'd') could not be coerced
to provided output parameter (typecode 'l') according to the casting rule
'same_kind'
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 344, in scikits.learn.decomposition.pca.ProbabilisticPCA
Failed example:
print pca.explained_variance_ratio_
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.ProbabilisticPCA[5]>",
line 1, in <module>
print pca.explained_variance_ratio_
AttributeError: 'PCA' object has no attribute
'explained_variance_ratio_'
>> raise self.failureException(self.format_failure(<StringIO.StringIO
instance at 0xfd8d1c0>.getvalue()))
======================================================================
FAIL: Doctest: scikits.learn.decomposition.pca.RandomizedPCA
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 2145, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for
scikits.learn.decomposition.pca.RandomizedPCA
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
330, in RandomizedPCA
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 377, in scikits.learn.decomposition.pca.RandomizedPCA
Failed example:
pca.fit(X)
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.RandomizedPCA[4]>",
line 1, in <module>
pca.fit(X)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py", line
438, in fit
X -= self.mean_
TypeError: ufunc 'subtract' output (typecode 'd') could not be coerced
to provided output parameter (typecode 'l') according to the casting rule
'same_kind'
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/decomposition/pca.py",
line 379, in scikits.learn.decomposition.pca.RandomizedPCA
Failed example:
print pca.explained_variance_ratio_
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.decomposition.pca.RandomizedPCA[5]>",
line 1, in <module>
print pca.explained_variance_ratio_
AttributeError: 'RandomizedPCA' object has no attribute
'explained_variance_ratio_'
>> raise self.failureException(self.format_failure(<StringIO.StringIO
instance at 0xfe05670>.getvalue()))
======================================================================
FAIL: Doctest: scikits.learn.mixture.GMM
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 2145, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for scikits.learn.mixture.GMM
File "/Users/rgommers/Code/scikit-learn/scikits/learn/mixture.py", line
127, in GMM
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/mixture.py", line 226,
in scikits.learn.mixture.GMM
Failed example:
g.fit(20 * [[0]] + 20 * [[10]])
Exception raised:
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py",
line 1241, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.mixture.GMM[14]>", line 1, in <module>
g.fit(20 * [[0]] + 20 * [[10]])
File "/Users/rgommers/Code/scikit-learn/scikits/learn/mixture.py",
line 480, in fit
k=self._n_states).fit(X).cluster_centers_
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/cluster/k_means_.py", line
432, in fit
tol=self.tol, random_state=self.random_state, copy_x=self.copy_x)
File
"/Users/rgommers/Code/scikit-learn/scikits/learn/cluster/k_means_.py", line
199, in k_means
X -= Xmean
TypeError: ufunc 'subtract' output (typecode 'd') could not be coerced
to provided output parameter (typecode 'l') according to the casting rule
'same_kind'
----------------------------------------------------------------------
File "/Users/rgommers/Code/scikit-learn/scikits/learn/mixture.py", line 228,
in scikits.learn.mixture.GMM
Failed example:
np.round(g.weights, 2)
Expected:
array([ 0.5, 0.5])
Got:
array([ 0.25, 0.75])
>> raise self.failureException(self.format_failure(<StringIO.StringIO
instance at 0xfde5238>.getvalue()))
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
