Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sun, Jul 20, 2008 at 2:18 AM, Nils Wagner [EMAIL PROTECTED] wrote: On Sat, 19 Jul 2008 13:29:45 -0500 Travis E. Oliphant [EMAIL PROTECTED] wrote: Pauli Virtanen wrote: Hi all, Re: Ticket 854. I wrote tests for the branch cuts for all complex arc* functions in umathmodule. It turns out that all except arccosh were OK. The formula for arcsinh was written in a non-standard form with an unnecessary nc_neg, but this didn't affect the results. I also wrote tests for checking values of the functions at infs and nans. Thanks for looking into these. These functions were contributed by Konrad Hinsen (IIRC) many years ago and I don't think they've really been reviewed since then. I'm all for using C99 when it is available and improving these functions with help from cmathmodule. IIRC, the cmathmodule was contributed by Konrad originally also. So +1 on C99 standardization. -Travis == ERROR: test_umath.TestC99.test_catanh(ufunc 'arctanh', (nan, nan), (nan, nan), '') -- Traceback (most recent call last): File /usr/lib/python2.4/site-packages/nose-0.10.3-py2.4.egg/nose/case.py, line 182, in runTest self.test(*self.arg) File /usr/lib/python2.4/site-packages/numpy/core/tests/test_umath.py, line 405, in _check func(point) FloatingPointError: invalid value encountered in arctanh snip What architecture and OS? Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sun, Jul 20, 2008 at 03:34, Charles R Harris
[EMAIL PROTECTED] wrote:
What architecture and OS?
I get the following on OS X 10.5.3 Intel Core 2 Duo:
==
FAIL: test_umath.TestC99.test_clog(ufunc 'log', (-0.0, -0.0), (-inf,
-0.0), 'divide')
--
Traceback (most recent call last):
File
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg/nose/case.py,
line 182, in runTest
self.test(*self.arg)
File /Users/rkern/svn/numpy/numpy/core/tests/test_umath.py, line
394, in _check
assert got == expected, (got, expected)
AssertionError: ('(-inf, 3.1415926535897931)', '(-inf, 0.0)')
==
FAIL: test_umath.TestC99.test_csqrt(ufunc 'sqrt', (-inf, 1.0),
(-0.0, inf), '')
--
Traceback (most recent call last):
File
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg/nose/case.py,
line 182, in runTest
self.test(*self.arg)
File /Users/rkern/svn/numpy/numpy/core/tests/test_umath.py, line
394, in _check
assert got == expected, (got, expected)
AssertionError: ('(0.0, inf)', '(-0.0, inf)')
--
Ran 1887 tests in 7.716s
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sun, 20 Jul 2008 02:34:48 -0600 Charles R Harris [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 2:18 AM, Nils Wagner [EMAIL PROTECTED] wrote: On Sat, 19 Jul 2008 13:29:45 -0500 Travis E. Oliphant [EMAIL PROTECTED] wrote: Pauli Virtanen wrote: Hi all, Re: Ticket 854. I wrote tests for the branch cuts for all complex arc* functions in umathmodule. It turns out that all except arccosh were OK. The formula for arcsinh was written in a non-standard form with an unnecessary nc_neg, but this didn't affect the results. I also wrote tests for checking values of the functions at infs and nans. Thanks for looking into these. These functions were contributed by Konrad Hinsen (IIRC) many years ago and I don't think they've really been reviewed since then. I'm all for using C99 when it is available and improving these functions with help from cmathmodule. IIRC, the cmathmodule was contributed by Konrad originally also. So +1 on C99 standardization. -Travis == ERROR: test_umath.TestC99.test_catanh(ufunc 'arctanh', (nan, nan), (nan, nan), '') -- Traceback (most recent call last): File /usr/lib/python2.4/site-packages/nose-0.10.3-py2.4.egg/nose/case.py, line 182, in runTest self.test(*self.arg) File /usr/lib/python2.4/site-packages/numpy/core/tests/test_umath.py, line 405, in _check func(point) FloatingPointError: invalid value encountered in arctanh snip What architecture and OS? Linux linux 2.6.11.4-21.17-default #1 Fri Apr 6 08:42:34 UTC 2007 i686 athlon i386 GNU/Linux SuSe Linux 9.3 gcc --version gcc (GCC) 3.3.5 20050117 (prerelease) (SUSE Linux) Copyright (C) 2003 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. /usr/bin/python Python 2.4 (#1, Oct 13 2006, 17:13:31) [GCC 3.3.5 20050117 (prerelease) (SUSE Linux)] on linux2 Type help, copyright, credits or license for more information. Nils ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
Travis, thank you for your encouraging words. On Sat, 19 Jul 2008, Travis E. Oliphant wrote: Really this code grew from a simple thing into a complicated thing as more features were added. This is a common issue that happens all over the place. Aye. The reason I say I'm not sure is that I don't remember seeing a DECREF after the PyArray_Scalar in the obj = NULL part of the code in my looking at your patches.But, I could have just missed it. There wasn't -- instead, I was trying to retain the typecode (and paying the price of releasing it on all the early returns!) From a generic reference counting point of view you did correctly emphasize the problem of having a reference count creation occur in an if-statement but a DECREF occur all the time in the finish: section of the code. Yah -- I think the core idea I was trying to get over is that of an invariant property at each point in the code to capture what needs to be true for the code to be correct. It was really that statement: the fantasy that PyArray_Scalar steals a reference that tipped me off to what I consider one of the real problems to be.The fact that it was masked at the end of a long discussion about other reference counting and a stealing discussion that were not the core problem was distracting and ultimately not very helpful. That was the really hard bit for me. To me the issue was actually very obvious (though I didn't realise that typecode could be regenerated, which simplifies things enormously), so the problem was trying to figure out what you and Charles were not seeing. I think that's why I ended up throwing everything into the pot! I'm very impressed with your ability to follow these reference count issues. Especially given that you only started learning about the Python C-API a few months ago (if I remember correctly). Alas no. I'm a bit of an old lag really, I did dabble with the Python C API quite a few years ago (2001ish maybe?). Myy roots are in computer science and then assembler (graduated 1980) before Pascal (seriously) then C, then C++ (which I now regard as a serious mistake) and finally shell script plus Python, all largely on embedded applications. I'd love the opportunity to learn and use Haskell now. I'm also very glad you are checking these corner cases which have not received the testing that they deserve. I hope we have not discouraged you too much from continuing to help. Your input is highly valued. Maybe I'll have a further play. The memory leak issue was a direct consequence of using numpy in an embedded application, and that issue's closed now, but I ought to see if this painful code can be revisited. I'm learning my way around git and have just used `git svn` to grab (and update) the numpy repository. I'm hugely impressed by it, though it is very expensive to run the first time -- it fetches every single svn revision! Hopefully that didn't overload the web server... This will make working on patches much easier. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
2008/7/20 Michael Abbott [EMAIL PROTECTED]: I'm very impressed with your ability to follow these reference count issues. Especially given that you only started learning about the Python C-API a few months ago (if I remember correctly). Alas no. I'm a bit of an old lag really, I did dabble with the Python C API quite a few years ago (2001ish maybe?). Myy roots are in computer science and then assembler (graduated 1980) before Pascal (seriously) then C, then C++ (which I now regard as a serious mistake) and finally shell It's scary how many of us were scarred for life by C++. If you have ten minutes free sometime, would you please consider writing up your reference counting explanation on the wiki (even if you just copy and paste that part out of your email and annotate it)? Having more eyes on the NumPy code is imperative; we need to teach more people to understand how to find these sorts of problems. I'm learning my way around git and have just used `git svn` to grab (and update) the numpy repository. I'm hugely impressed by it, though it is very expensive to run the first time -- it fetches every single svn revision! Hopefully that didn't overload the web server... This will make working on patches much easier. I hope that we can move over to a distributed revision control system sometime in the foreseeable future. From what I've seen, its model strongly encourages community interaction. Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
Hi, Sorry, Sun, 20 Jul 2008 10:18:47 +0200, Nils Wagner wrote: ERROR: test_umath.TestC99.test_catanh(ufunc 'arctanh', (nan, nan), (nan, nan), '') FloatingPointError: invalid value encountered in arctanh Skipped now. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (1.0, nan), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, 0.0), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, 1.0), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, nan), (nan, nan), '') FloatingPointError: invalid value encountered in log Skipped. Sun, 20 Jul 2008 03:42:45 -0500, Robert Kern wrote: FAIL: test_umath.TestC99.test_clog(ufunc 'log', (-0.0, -0.0), (-inf, -0.0), 'divide') Interesting, there's no test like this in there, only one with positive zeros. Where does this one come from? FAIL: test_umath.TestC99.test_csqrt(ufunc 'sqrt', (-inf, 1.0), (-0.0, inf), '') Fails on your platform, skipped. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sun, 20 Jul 2008 12:10:23 + (UTC) Pauli Virtanen [EMAIL PROTECTED] wrote: Hi, Sorry, Sun, 20 Jul 2008 10:18:47 +0200, Nils Wagner wrote: ERROR: test_umath.TestC99.test_catanh(ufunc 'arctanh', (nan, nan), (nan, nan), '') FloatingPointError: invalid value encountered in arctanh Skipped now. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (1.0, nan), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, 0.0), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, 1.0), (nan, nan), '') FloatingPointError: invalid value encountered in log Bug in tests, fixed. ERROR: test_umath.TestC99.test_clog(ufunc 'log', (nan, nan), (nan, nan), '') FloatingPointError: invalid value encountered in log Skipped. Sun, 20 Jul 2008 03:42:45 -0500, Robert Kern wrote: FAIL: test_umath.TestC99.test_clog(ufunc 'log', (-0.0, -0.0), (-inf, -0.0), 'divide') Interesting, there's no test like this in there, only one with positive zeros. Where does this one come from? FAIL: test_umath.TestC99.test_csqrt(ufunc 'sqrt', (-inf, 1.0), (-0.0, inf), '') Fails on your platform, skipped. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion Ran 2069 tests in 30.300s OK (SKIP=44) nose.result.TextTestResult run=2069 errors=0 failures=0 Thank you. Cheers, Nils ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
On Sun, 20 Jul 2008, Stéfan van der Walt wrote: 2008/7/20 Michael Abbott [EMAIL PROTECTED]: C, then C++ (which I now regard as a serious mistake) and finally shell It's scary how many of us were scarred for life by C++. What's really annoying for me is that my most recent big project (http://sourceforge.net/projects/libera-epics) was written in C++, entirely at my own choice. It seemed like a good idea at the time. If you have ten minutes free sometime, would you please consider writing up your reference counting explanation on the wiki Good idea. Do you mean at http://scipy.org/scipy/numpy ? Somewhere under CodingStyleGuidelines?___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
2008/7/20 Michael Abbott [EMAIL PROTECTED]: On Sun, 20 Jul 2008, Stéfan van der Walt wrote: 2008/7/20 Michael Abbott [EMAIL PROTECTED]: C, then C++ (which I now regard as a serious mistake) and finally shell It's scary how many of us were scarred for life by C++. What's really annoying for me is that my most recent big project (http://sourceforge.net/projects/libera-epics) was written in C++, entirely at my own choice. It seemed like a good idea at the time. Other are scarred for life with C and are more than happy with C++... Matthieu, who really hopes he will not ever write one more line of C code -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
2008/7/20 Michael Abbott [EMAIL PROTECTED]: On Sun, 20 Jul 2008, Stéfan van der Walt wrote: 2008/7/20 Michael Abbott [EMAIL PROTECTED]: C, then C++ (which I now regard as a serious mistake) and finally shell It's scary how many of us were scarred for life by C++. What's really annoying for me is that my most recent big project (http://sourceforge.net/projects/libera-epics) was written in C++, entirely at my own choice. It seemed like a good idea at the time. If you have ten minutes free sometime, would you please consider writing up your reference counting explanation on the wiki Good idea. Do you mean at http://scipy.org/scipy/numpy ? Somewhere under CodingStyleGuidelines? Or here: http://projects.scipy.org/scipy/numpy/ under `guidelines`. Thanks! Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
On Sun, Jul 20, 2008 at 5:00 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/7/20 Michael Abbott [EMAIL PROTECTED]: I'm very impressed with your ability to follow these reference count issues. Especially given that you only started learning about the Python C-API a few months ago (if I remember correctly). Alas no. I'm a bit of an old lag really, I did dabble with the Python C API quite a few years ago (2001ish maybe?). Myy roots are in computer science and then assembler (graduated 1980) before Pascal (seriously) then C, then C++ (which I now regard as a serious mistake) and finally shell It's scary how many of us were scarred for life by C++. I rather like C++, especially the templates and (BOOST) smart pointers. One just has to avoid using the more exotic features, think ten or twenty times before using inheritance, and be very suspicious of operator overloading. And cstdio is your friend. But If you need to ration memory and worry about dynamic allocation, forget it. I wouldn't use it for drivers. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] 1.1.1rc1 to be tagged tonight
Hello, This is a reminder that 1.1.1rc1 will be tagged tonight. Chuck is planning to spend some time today fixing a few final bugs on the 1.1.x branch. If anyone else is planning to commit anything to the 1.1.x branch today, please let me know immediately. Obviously now is not the time to commit anything to the branch that could break anything, so please be extremely careful if you have to touch the branch. Once the release is tagged, Chris and David will create binary installers for both Windows and Mac. Hopefully, this will give us an opportunity to have much more widespread testing before releasing 1.1.1 final at the end of the month. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.1.1rc1 to be tagged tonight
Jarrod Millman wrote:
Hello,
This is a reminder that 1.1.1rc1 will be tagged tonight. Chuck is
planning to spend some time today fixing a few final bugs on the 1.1.x
branch. If anyone else is planning to commit anything to the 1.1.x
branch today, please let me know immediately. Obviously now is not
the time to commit anything to the branch that could break anything,
so please be extremely careful if you have to touch the branch.
Once the release is tagged, Chris and David will create binary
installers for both Windows and Mac. Hopefully, this will give us an
opportunity to have much more widespread testing before releasing
1.1.1 final at the end of the month.
Can I get anyone to look at this patch for loadtext()?
I was trying to use loadtxt() today to read in some text data, and I had
a problem when I specified a dtype that only contained as many elements
as in columns in usecols. The example below shows the problem:
import numpy as np
import StringIO
data = '''STID RELH TAIR
JOE 70.1 25.3
BOB 60.5 27.9
'''
f = StringIO.StringIO(data)
names = ['stid', 'temp']
dtypes = ['S4', 'f8']
arr = np.loadtxt(f, usecols=(0,2),dtype=zip(names,dtypes), skiprows=1)
With current 1.1 (and SVN head), this yields:
IndexErrorTraceback (most recent call last)
/home/rmay/ipython console in module()
/usr/lib64/python2.5/site-packages/numpy/lib/io.pyc in loadtxt(fname,
dtype, comments, delimiter, converters, skiprows, usecols, unpack)
309 for j in xrange(len(vals))]
310 if usecols is not None:
-- 311 row = [converterseq[j](vals[j]) for j in usecols]
312 else:
313 row = [converterseq[j](val) for j,val in
enumerate(vals)]
IndexError: list index out of range
-
I've added a patch that checks for usecols, and if present, correctly
creates the converters dictionary to map each specified column with
converter for the corresponding field in the dtype. With the attached
patch, this works fine:
arr
array([('JOE', 25.301), ('BOB', 27.899)],
dtype=[('stid', '|S4'), ('temp', 'f8')])
Thanks,
Ryan
--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
--- io.py.bak 2008-07-18 18:12:17.0 -0400
+++ io.py 2008-07-16 22:49:13.0 -0400
@@ -292,8 +292,13 @@
if converters is None:
converters = {}
if dtype.names is not None:
-converterseq = [_getconv(dtype.fields[name][0]) \
-for name in dtype.names]
+if usecols is None:
+converterseq = [_getconv(dtype.fields[name][0]) \
+for name in dtype.names]
+else:
+converters.update([(col,_getconv(dtype.fields[name][0])) \
+for col,name in zip(usecols, dtype.names)])
+
for i,line in enumerate(fh):
if iskiprows: continue
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] MSVC warnings.
The Windows_XP buildbot shows several warnings about npy_intp - int conversions. Two of them look OK, but probably explicit casts should be made. The third is: numpy\core\src\ufuncobject.c(2422) : warning C4244: '=' : conversion from 'npy_intp' to 'int', possible loss of data Which looks possibly legitimate to me. Unfortunately, it looks like fixing it involves a change in the loop type. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Dealing with NAN.
The NAN macro isn't defined on all platforms and we have various workarounds scattered here and there. Is it reasonable to determine a suitable value of NAN for the various float types as part of the initial setup and configuration and have the results appended to one of the include files? This shouldn't be difficult for the gnu compilers, but might be a hassle for MSVC and some of the others. It can be hardwired into MSVC if we assume the Intel architecture. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Windows_XP buildbot error.
The log file shows: File c:\numpy-buildbot\numpy\b11\install\Lib\site-packages\numpy\lib\tests\test_format.py, line 429, in test_memmap_roundtrip fp = open(nfn, 'wb') IOError: [Errno 2] No such file or directory: 'c:\\docume~1\\thomas\\locals~1\\temp\\tmp_yrykj\\normal.npy' Is this some sort of permissions error or something specific to Thomas' machine? I don't want this to show up in the 1.1.1 release and I'm wondering if there is an easy fix besides disabling the test. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Ticket #794 and can o' worms.
Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is herehttp://thread.gmane.org/gmane.comp.python.numeric.general/21526/focus=21537. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. 2) The way NaN is converted to int types. I think it should be 0. These problems show up in failing tests. I'm reverting the fix for now, but I wonder what we should do about these things. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
Alan, Stefan Not raising errors seems ok for examples, but some of the unit tests are also implemented as doctests and the failures are hidden in the logs. I'm not sure what to do about this, but thought it worth pointing out. Also, it would be nice if skipped tests didn't generate large bits of printout, it makes it hard to find relevant failures. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 4:47 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L So we should shoot for: nan - bool : True nan - integer kind : 0 nan - complex : Nan+0j nan - string kind : ?, currently it is any one of 'n', 'na', 'nan', depending on string length. nan - object: float object nan. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 20:32, Charles R Harris [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 4:47 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L So we should shoot for: nan - bool : True nan - integer kind : 0 nan - complex : Nan+0j nan - string kind : ?, currently it is any one of 'n', 'na', 'nan', depending on string length. nan - object: float object nan. Sounds right. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 3:47 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. DELURK I agree that this is a good goal. However, in the past, Python's treatment of NaNs has been rather platform dependent and add hock. In this case, I suspect that you are OK since the section Truth Value Testing in the Python docs is pretty clear that any non-zero value of a numerical type is True. However... 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L This is GvR in http://mail.python.org/pipermail/python-dev/2008-January/075865.html: *If long(nan) or int(nan) returns 0 on most platforms in 2.5, we should* *fix them to always return 0 in 2.5 *and* 2.6. In 3.0 they should raise* *ValueError.* This implies that in version 2.4 and earlier, the Python behaviour is platform dependent. And that 3.0 this is going to change to raise a ValueError. Whether it's more important to match current behaviour (return 0) or future behaviour (raise ValueError), I'm not certain. I would lean towards a ValueError since it's less long term pain and it's IMO more correct. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- . __ . |-\ . . [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 9:17 PM, Alan McIntyre [EMAIL PROTECTED] wrote: The skipped test verbosity is annoying; I'll see if there's a way to make that a bit cleaner-looking for some low verbosity level. The latest release version of nose from easy_install (0.10.3) doesn't generate that verbose output for skipped tests. Should we move up to requiring 0.10.3 for tests? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 21:47, Alan McIntyre [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 9:17 PM, Alan McIntyre [EMAIL PROTECTED] wrote: The skipped test verbosity is annoying; I'll see if there's a way to make that a bit cleaner-looking for some low verbosity level. The latest release version of nose from easy_install (0.10.3) doesn't generate that verbose output for skipped tests. Should we move up to requiring 0.10.3 for tests? I don't think aesthetics are worth requiring a particular version. numpy doesn't need it; the users can decide whether they want it or not. We should try to have it installed on the buildbots, though, since we *are* the users in that case. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 10:56 PM, Robert Kern [EMAIL PROTECTED] wrote: I don't think aesthetics are worth requiring a particular version. numpy doesn't need it; the users can decide whether they want it or not. We should try to have it installed on the buildbots, though, since we *are* the users in that case. Actually I was considering asking to move the minimum nose version up to 0.10.3 just because it's the current version before this aesthetic issue came up. There's about 30 bug fixes between 0.10.0 and 0.10.3, including one that fixed some situations in which exceptions were being hidden and one that makes the coverage reporting more accurate. It's not a big deal, though. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 11:09:04PM -0400, Alan McIntyre wrote: Actually I was considering asking to move the minimum nose version up to 0.10.3 just because it's the current version before this aesthetic issue came up. There's about 30 bug fixes between 0.10.0 and 0.10.3, including one that fixed some situations in which exceptions were being hidden and one that makes the coverage reporting more accurate. It's not a big deal, though. There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 11:17 PM, Gael Varoquaux [EMAIL PROTECTED] wrote: There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers. Fair enough; does anybody have any idea which version of nose is generally available from distributions like the ones you mentioned? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 11:19:57PM -0400, Alan McIntyre wrote: On Sun, Jul 20, 2008 at 11:17 PM, Gael Varoquaux [EMAIL PROTECTED] wrote: There might be a case to move to 10.3, considering the large amount of bug fixes, but in general I think it is a bad idea to require leading edge packages. The reason being that you would like people to be able to rely on packaged version of the different tools to build an test a package. By packaged versions, I mean versions in the repositories of the main linux distributions, and macport and fink. Each time we require something outside a repository, we loose testers. Fair enough; does anybody have any idea which version of nose is generally available from distributions like the ones you mentioned? Ubuntu hardy (current): 10.0 (http://packages.ubuntu.com) Ubuntu intrepid (next): 10.3 (http://packages.ubuntu.com) Debian unstable:10.3 (http://packages.dbian.com) Fedora 8: 10.0 (https://admin.fedoraproject.org/pkgdb/) For the rest I can't figure out how to get the information. I suspect we can standardise on things around six month old. Debian unstable tracks closely upstream, Ubuntu and Fedora have a release cycle of 6 months, I don't know about SUSE, but I think it is similar, and macports, fink, or Gentoo trac closely upstream. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Testing: Failed examples don't raise errors on buildbot.
On Sun, Jul 20, 2008 at 11:34 PM, Gael Varoquaux [EMAIL PROTECTED] wrote: For the rest I can't figure out how to get the information. I suspect we can standardise on things around six month old. Debian unstable tracks closely upstream, Ubuntu and Fedora have a release cycle of 6 months, I don't know about SUSE, but I think it is similar, and macports, fink, or Gentoo trac closely upstream. It looks like Macports is at 0.10.1: http://py-nose.darwinports.com/ So it looks like 0.10.0 should still be a safe bet for being generally available. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 8:32 PM, Timothy Hochberg [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 3:47 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. DELURK I agree that this is a good goal. However, in the past, Python's treatment of NaNs has been rather platform dependent and add hock. In this case, I suspect that you are OK since the section Truth Value Testing in the Python docs is pretty clear that any non-zero value of a numerical type is True. However... 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L This is GvR in http://mail.python.org/pipermail/python-dev/2008-January/075865.html: Well, now, that opens a whole other bag of toasted scorpions. It looks like long(inf) and int(inf) already raise OverflowError and that should stay. In [3]: (ones(2)*float(inf)).astype(int8) Out[3]: array([0, 0], dtype=int8) In [4]: (ones(2)*float(inf)).astype(int32) Out[4]: array([-2147483648, -2147483648]) In [5]: (ones(2)*float(inf)).astype(int64) Out[5]: array([-9223372036854775808, -9223372036854775808], dtype=int64) Hmmm, Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket #794 and can o' worms.
On Sun, Jul 20, 2008 at 10:41 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 8:32 PM, Timothy Hochberg [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 3:47 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Jul 20, 2008 at 17:42, Charles R Harris [EMAIL PROTECTED] wrote: Hi All, I fixed ticket #754, but it leads to a ton of problems. The original discussion is here. The problems that arise come from conversion to different types. In [26]: a Out[26]: array([ Inf, -Inf, NaN, 0., 3., -3.]) In [27]: sign(a).astype(int) Out[27]: array([ 1, -1, -2147483648, 0, 1, -1]) In [28]: sign(a).astype(bool) Out[28]: array([ True, True, True, False, True, True], dtype=bool) In [29]: sign(a) Out[29]: array([ 1., -1., NaN, 0., 1., -1.]) In [30]: bool(NaN) Out[30]: True So there are problems with at minimum the following. 1) The way NaN is converted to bool. I think it should be False. It's not really our choice. That's Python's bool(). For the things that are our choice (e.g. array([nan]).astype(bool)) I think we should stay consistent with Python. DELURK I agree that this is a good goal. However, in the past, Python's treatment of NaNs has been rather platform dependent and add hock. In this case, I suspect that you are OK since the section Truth Value Testing in the Python docs is pretty clear that any non-zero value of a numerical type is True. However... 2) The way NaN is converted to int types. I think it should be 0. I agree. That's what int(nan) gives: int(nan) 0L This is GvR in http://mail.python.org/pipermail/python-dev/2008-January/075865.html: Well, now, that opens a whole other bag of toasted scorpions. It looks like long(inf) and int(inf) already raise OverflowError and that should stay. In [3]: (ones(2)*float(inf)).astype(int8) Out[3]: array([0, 0], dtype=int8) In [4]: (ones(2)*float(inf)).astype(int32) Out[4]: array([-2147483648, -2147483648]) In [5]: (ones(2)*float(inf)).astype(int64) Out[5]: array([-9223372036854775808, -9223372036854775808], dtype=int64) Also, In [8]: (ones(2)*float(inf)).astype(uint8) Out[8]: array([0, 0], dtype=uint8) In [9]: (ones(2)*float(inf)).astype(uint16) Out[9]: array([0, 0], dtype=uint16) In [10]: (ones(2)*float(inf)).astype(uint32) Out[10]: array([0, 0], dtype=uint32) In [11]: (ones(2)*float(inf)).astype(uint64) Out[11]: array([0, 0], dtype=uint64) Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
