Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
FYI, I can't reproduce David's failures on my machine (intel core2 duo w/ 10.5.5) * python 2.6 from macports * numpy svn 6098 * GCC 4.0.1 (Apple Inc. build 5488) I have only 1 failure: FAIL: test_umath.TestComplexFunctions.test_against_cmath -- Traceback (most recent call last): File /opt/local/lib/python2.6/site-packages/nose-0.10.4-py2.6.egg/ nose/case.py, line 182, in runTest self.test(*self.arg) File /Users/pierregm/Computing/.pythonenvs/default26/lib/python2.6/ site-packages/numpy/core/tests/test_umath.py, line 423, in test_against_cmath assert abs(a - b) atol, %s %s: %s; cmath: %s%(fname,p,a,b) AssertionError: arcsin 2: (1.57079632679-1.31695789692j); cmath: (1.57079632679+1.31695789692j) -- (Well, there's another one in numpy.ma.min, but that's a different matter). On Nov 25, 2008, at 2:19 AM, David Cournapeau wrote: On Mon, 2008-11-24 at 22:06 -0700, Charles R Harris wrote: Well, it may not be that easy to figure. The (generated) pyconfig-32.h has /* Define to 1 if your processor stores words with the most significant byte first (like Motorola and SPARC, unlike Intel and VAX). The block below does compile-time checking for endianness on platforms that use GCC and therefore allows compiling fat binaries on OSX by using '-arch ppc -arch i386' as the compile flags. The phrasing was choosen such that the configure-result is used on systems that don't use GCC. */ #ifdef __BIG_ENDIAN__ #define WORDS_BIGENDIAN 1 #else #ifndef __LITTLE_ENDIAN__ /* #undef WORDS_BIGENDIAN */ #endif #endif Hm, interesting: just by grepping, I do have WORDS_BIGENDIAN defined to 1 on *both* python 2.5 and python 2.6 on Mac OS X (running Intel). Looking closer, I do have the above code (conditional) in 2.5, but not in 2.6: it is inconditionally defined to BIGENDIAN on 2.6 !! That's actually part of something I have wondered for quite some time about fat binaries: how do you handle config headers, since they are generated only once for every fat binary, but they should really be generated for each arch. And I guess that __BIG_ENDIAN__ is a compiler flag, it isn't in any of the include files. In any case, this looks like a Python bug or the Python folks have switched their API on us. Hm, actually, it is a bug in numpy as much as in python: python should NOT include any config.h in their public namespace, and we should not rely on it. But with this info, it should be relatively easy to fix (by setting the correct endianness by ourselves with some detection code) David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
Pierre GM wrote: FYI, I can't reproduce David's failures on my machine (intel core2 duo w/ 10.5.5) * python 2.6 from macports I think that's the main difference. I feel more and more that the problem is linked to fat binaries (more exactly multi arch build in one autoconf run: since only one pyconfig.h is generated for all archs, only one value is defined for CPU specific configurations). On my machine, pyconfig.h has WORDS_BIGENDIAN defined to one, which I can only explain by the binary being built on ppc (unfortunately, I can't find this information from python itself - maybe in the release notes). And that cannot work on Intel. The general solution would be to generate different arch specific config files, and import them conditionally in the main config file. But doing so in a platform-neutral manner is not trivial, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Tue, Nov 25, 2008 at 10:55 PM, David Cournapeau [EMAIL PROTECTED] wrote: I used the path of least resistance: instead of using the WORDS_BIGENDIAN macro, I added a numpy header which gives the endianness every time it is included. IOW, instead of the endianness to be fixed at numpy build time (which would fail for universal builds), it is set everytime the numpy headers are included (which is the only way to make it work). A better solution IMO would be to avoid any endianness dependency at all in the headers, but that does not seem possible without breaking the API (because the endianness-related macro PyArray_NBO and co would need to be set as functions instead). Hm, for reference, I came across this: http://www.mail-archive.com/[EMAIL PROTECTED]/msg14382.html So some people thought about the same problem. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Tue, Nov 25, 2008 at 8:03 AM, David Cournapeau [EMAIL PROTECTED]wrote: On Tue, Nov 25, 2008 at 10:55 PM, David Cournapeau [EMAIL PROTECTED] wrote: I used the path of least resistance: instead of using the WORDS_BIGENDIAN macro, I added a numpy header which gives the endianness every time it is included. IOW, instead of the endianness to be fixed at numpy build time (which would fail for universal builds), it is set everytime the numpy headers are included (which is the only way to make it work). A better solution IMO would be to avoid any endianness dependency at all in the headers, but that does not seem possible without breaking the API (because the endianness-related macro PyArray_NBO and co would need to be set as functions instead). Hm, for reference, I came across this: http://www.mail-archive.com/[EMAIL PROTECTED]/msg14382.html So some people thought about the same problem. Apart from the Mac, the ppc can be configured to run either bigendian or littleendian, so the hardware encompasses more than just the cpu, it's the whole darn board. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Wed, Nov 26, 2008 at 12:59 AM, Charles R Harris [EMAIL PROTECTED] wrote: Apart from the Mac, the ppc can be configured to run either bigendian or littleendian, so the hardware encompasses more than just the cpu, it's the whole darn board. Yep, many CPU families have double endian support (MIPS, ARM, PA-RISC, ALPHA. There is also mixed endian. Honestly, I think it is safe to assume that we don't need to care so much about those configurations for the time being. If it is a problem, we can then discuss about our headers being endian-free (which is really the best approach). David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Numpy on Mac OS X python 2.6
Hi,
Following the discussion on python 2.6 support for numpy, I tried last
svn on mac os X, and I get a number of failures which I don't
understand, which seem to be linked to dtype code, more exactly to
endianness:
==
FAIL: test_basic (test_multiarray.TestClip)
--
Traceback (most recent call last):
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 677, in test_basic
self._clip_type('float',1024,-12.8,100.2, inplace=inplace)
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 671, in _clip_type
assert_equal(x.dtype.byteorder,byteorder)
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 183, in assert_equal
raise AssertionError(msg)
AssertionError:
Items are not equal:
ACTUAL: ''
DESIRED: '='
==
FAIL: test_binary (test_multiarray.TestFromstring)
--
Traceback (most recent call last):
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 120, in test_binary
assert_array_equal(a, array([1,2,3,4]))
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 303, in assert_array_equal
verbose=verbose, header='Arrays are not equal')
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 295, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not equal
(mismatch 100.0%)
x: array([ 4.60060299e-41, 8.96831017e-44, 2.30485571e-41,
4.60074312e-41], dtype=float32)
y: array([1, 2, 3, 4])
...
http://scipy.org/scipy/numpy/ticket/958
Does anyone have a clue about where to look at ? I am wondering why it
only appears on Mac OS X,
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Mon, Nov 24, 2008 at 8:18 PM, David Cournapeau [EMAIL PROTECTED]wrote:
Hi,
Following the discussion on python 2.6 support for numpy, I tried last
svn on mac os X, and I get a number of failures which I don't
understand, which seem to be linked to dtype code, more exactly to
endianness:
==
FAIL: test_basic (test_multiarray.TestClip)
--
Traceback (most recent call last):
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 677, in test_basic
self._clip_type('float',1024,-12.8,100.2, inplace=inplace)
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 671, in _clip_type
assert_equal(x.dtype.byteorder,byteorder)
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 183, in assert_equal
raise AssertionError(msg)
AssertionError:
Items are not equal:
ACTUAL: ''
DESIRED: '='
==
FAIL: test_binary (test_multiarray.TestFromstring)
--
Traceback (most recent call last):
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py,
line 120, in test_binary
assert_array_equal(a, array([1,2,3,4]))
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 303, in assert_array_equal
verbose=verbose, header='Arrays are not equal')
File
/Users/david/pylocal/lib/python2.6/site-packages/numpy/testing/utils.py,
line 295, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not equal
(mismatch 100.0%)
x: array([ 4.60060299e-41, 8.96831017e-44, 2.30485571e-41,
4.60074312e-41], dtype=float32)
y: array([1, 2, 3, 4])
Sure enough, it's byteswapped. Is it correct that:
1) This problem is specific to 2.6 and 2.5 works.
2) It's on Intel hardware?
What about normal doubles, do they work?
Does python work?
Is it possible there are two versions of (python, lib,...), one for ppc and
the other for Intel that are getting confused?
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Tue, Nov 25, 2008 at 12:41 PM, Charles R Harris
[EMAIL PROTECTED] wrote:
1) This problem is specific to 2.6 and 2.5 works.
Yes
2) It's on Intel hardware?
Yes.
Here is a minimal test which shows the problem:
import numpy as np
assert np.dtype('f4').isnative
This fails on python 2.6, but works on 2.5. In both cases,
sys.byteorder returns little. So for some reasons, dtype description
(see other failures) is not working on 2.6.
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Mon, Nov 24, 2008 at 9:02 PM, David Cournapeau [EMAIL PROTECTED]wrote:
On Tue, Nov 25, 2008 at 12:41 PM, Charles R Harris
[EMAIL PROTECTED] wrote:
1) This problem is specific to 2.6 and 2.5 works.
Yes
2) It's on Intel hardware?
Yes.
Here is a minimal test which shows the problem:
import numpy as np
assert np.dtype('f4').isnative
So what does dtype(float32).descr and dtype(float32).byteorder show?
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Mon, Nov 24, 2008 at 9:38 PM, Charles R Harris [EMAIL PROTECTED]
wrote:
On Mon, Nov 24, 2008 at 9:02 PM, David Cournapeau [EMAIL PROTECTED]wrote:
On Tue, Nov 25, 2008 at 12:41 PM, Charles R Harris
[EMAIL PROTECTED] wrote:
1) This problem is specific to 2.6 and 2.5 works.
Yes
2) It's on Intel hardware?
Yes.
Here is a minimal test which shows the problem:
import numpy as np
assert np.dtype('f4').isnative
So what does dtype(float32).descr and dtype(float32).byteorder show?
Numpy gets it's byte order from the macro WORDS_BIGENDIAN defined by Python.
Try
[EMAIL PROTECTED] numpy.git]$ grep -r -n WORDS_BIGENDIAN
/usr/include/python2.5/*
/usr/include/python2.5/pyconfig-32.h:902:#define WORDS_BIGENDIAN 1
/usr/include/python2.5/pyconfig-32.h:905:/* #undef WORDS_BIGENDIAN */
or the OS X equivalent.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Mon, Nov 24, 2008 at 9:58 PM, Charles R Harris [EMAIL PROTECTED]
wrote:
On Mon, Nov 24, 2008 at 9:38 PM, Charles R Harris
[EMAIL PROTECTED] wrote:
On Mon, Nov 24, 2008 at 9:02 PM, David Cournapeau [EMAIL PROTECTED]wrote:
On Tue, Nov 25, 2008 at 12:41 PM, Charles R Harris
[EMAIL PROTECTED] wrote:
1) This problem is specific to 2.6 and 2.5 works.
Yes
2) It's on Intel hardware?
Yes.
Here is a minimal test which shows the problem:
import numpy as np
assert np.dtype('f4').isnative
So what does dtype(float32).descr and dtype(float32).byteorder show?
Numpy gets it's byte order from the macro WORDS_BIGENDIAN defined by
Python. Try
[EMAIL PROTECTED] numpy.git]$ grep -r -n WORDS_BIGENDIAN
/usr/include/python2.5/*
/usr/include/python2.5/pyconfig-32.h:902:#define WORDS_BIGENDIAN 1
/usr/include/python2.5/pyconfig-32.h:905:/* #undef WORDS_BIGENDIAN */
or the OS X equivalent.
Well, it may not be that easy to figure. The (generated) pyconfig-32.h has
/* Define to 1 if your processor stores words with the most significant byte
first (like Motorola and SPARC, unlike Intel and VAX).
The block below does compile-time checking for endianness on platforms
that use GCC and therefore allows compiling fat binaries on OSX by using
'-arch ppc -arch i386' as the compile flags. The phrasing was choosen
such that the configure-result is used on systems that don't use GCC.
*/
#ifdef __BIG_ENDIAN__
#define WORDS_BIGENDIAN 1
#else
#ifndef __LITTLE_ENDIAN__
/* #undef WORDS_BIGENDIAN */
#endif
#endif
And I guess that __BIG_ENDIAN__ is a compiler flag, it isn't in any of the
include files. In any case, this looks like a Python bug or the Python folks
have switched their API on us.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Mac OS X python 2.6
On Mon, 2008-11-24 at 22:06 -0700, Charles R Harris wrote: Well, it may not be that easy to figure. The (generated) pyconfig-32.h has /* Define to 1 if your processor stores words with the most significant byte first (like Motorola and SPARC, unlike Intel and VAX). The block below does compile-time checking for endianness on platforms that use GCC and therefore allows compiling fat binaries on OSX by using '-arch ppc -arch i386' as the compile flags. The phrasing was choosen such that the configure-result is used on systems that don't use GCC. */ #ifdef __BIG_ENDIAN__ #define WORDS_BIGENDIAN 1 #else #ifndef __LITTLE_ENDIAN__ /* #undef WORDS_BIGENDIAN */ #endif #endif Hm, interesting: just by grepping, I do have WORDS_BIGENDIAN defined to 1 on *both* python 2.5 and python 2.6 on Mac OS X (running Intel). Looking closer, I do have the above code (conditional) in 2.5, but not in 2.6: it is inconditionally defined to BIGENDIAN on 2.6 !! That's actually part of something I have wondered for quite some time about fat binaries: how do you handle config headers, since they are generated only once for every fat binary, but they should really be generated for each arch. And I guess that __BIG_ENDIAN__ is a compiler flag, it isn't in any of the include files. In any case, this looks like a Python bug or the Python folks have switched their API on us. Hm, actually, it is a bug in numpy as much as in python: python should NOT include any config.h in their public namespace, and we should not rely on it. But with this info, it should be relatively easy to fix (by setting the correct endianness by ourselves with some detection code) David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
