Re: [Numpy-discussion] Is this a bug in repr ?
In that case, would you agree that it is a bug for assert_array_almost_equal to use repr() to display the arrays, since it is printing identical values and saying they are different? Or is there also a reason to do that? It should probably use np.array_repr(x, precision=16) Ok, thanks - I see the issue. I'll enter a ticket for an enhancement request for assert_array_almost_equal ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Is this a bug in repr ?
The usual expectation is that (when possible) repr() returns a value
that you can eval() to get the original data back. But,
from numpy import *
a = array( [ 16.5069863163822 ] )
b = eval(repr(a))
a-b
array([ -3.6111e-09])
import numpy.testing
numpy.testing.assert_array_almost_equal(a,b,decimal=15)
Traceback (most recent call last):
File stdin, line 1, in module
File /usr/stsci/pyssgdev/2.7/numpy/testing/utils.py, line 775, in
assert_array_almost_equal
header=('Arrays are not almost equal to %d decimals' % decimal))
File /usr/stsci/pyssgdev/2.7/numpy/testing/utils.py, line 618, in
assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 15 decimals
(mismatch 100.0%)
x: array([ 16.50698632])
y: array([ 16.50698632])
I noticed this because a bunch of tests failed exactly this way. Of
course, the problem is that assert_array_almost_equal does not print
with the precision that it compared, which in turn happens because it
just uses repr() to convert the array.
I would expect that repr would print the values at least to the
resolution that they are stored, so I think this is a bug.
This happens with the current trunk of numpy in python 2.7 on Red Hat
Enterprise linux in 32 and 64 bits, and on Macintosh Leopard in 32
bits. I did not try any other configuration.
Mark
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is this a bug in repr ?
Robert Kern wrote: On Tue, Mar 15, 2011 at 12:39, Charles R Harris charlesr.har...@gmail.com wrote: Yes, I think it is a bug. IIRC, it also shows up for object arrays. It's extremely long-standing, documented, intentional behavior dating back to Numeric. [~] |1 import Numeric [~] |2 a = Numeric.array( [ 16.5069863163822 ] ) [~] |3 print repr(a) array([ 16.50698632]) You can disagree with the feature, but it's not a bug. So it is needed to maintain backward compatibility? (Still?) In that case, would you agree that it is a bug for assert_array_almost_equal to use repr() to display the arrays, since it is printing identical values and saying they are different? Or is there also a reason to do that? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy docs dependency problem in Ubuntu
We can't put python-matplotlib in main because of *its* dependencies. As a digression, I think the python-matplotlib dependencies could be significantly reduced. For a number of use cases (this is one of them, but there are others), you don't need any GUI backend. Independent of this issue, it would be great to be able to install python-matplotlib in a headless server environment without pulling in all of those GUI bits. Looking at the list of the hard dependencies, I don't understand why half of them are there. http://packages.ubuntu.com/natty/python/python-matplotlib This web page lists several dependencies that are optional. Just flipping through the list, I can see several packages that I know that I do not have, and several more that I have never heard of. Never heard of could mean that it is in my linux distro and I don't know it, but I am certain that I have machines that do not have cairo or gtk+ or qt. A matplotlib application selects one of the available backends to draw the graphics. If I remember correctly _all_ backends are optional in matplotlib, but there is at least one (agg) that is available everywhere. When you install matplotlib, it builds support for any backends that it can. A backend that depends on a missing library does not get a C extension built. BUT the python code is still installed. The only way to know that a backend is not supported in this installation is to try to use it. For example, import matplotlib.backends.backend_qt Traceback (most recent call last): File stdin, line 1, in module File /usr/stsci/pyssgdev/2.7/matplotlib/backends/backend_qt.py, line 19, in module raise ImportError(Qt backend requires pyqt to be installed.) ImportError: Qt backend requires pyqt to be installed. Ok - I don't have qt on this machine. So, you can try this: Get a build machine that has all the packages required by the various backends. Build a binary distribution of matplotlib. Install it on a machine that has only some of the libraries required by the backends. The result is a copy of matplotlib with _some_ working backends and some that fail because of missing libraries. As you install other supporting packages, additional backends can start working because their shared library is now present. So, if I install matplotlib and pyqt is not there I get a working matplotlib without QT support. If I use a package installer to install pyqt, presumably it will also install the QT libraries, resulting in matplotlib that does have qt support. I'm not saying you want to do this, but it is an option. If you want to experiment in this direction, there is a list that breaks out requirements for the core and requirements for each of the backends at http://matplotlib.sourceforge.net/users/installing.html . Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] current numpy in python 2.5 ?
Hi, I looked on the web site, but I couldn't find a list of python versions that numpy expects to work on. Is the trunk still expected to work in Python 2.5 ? I'm asking because it doesn't, but I don't want to file a bug report if it is not expected to. The problem shows up when I run numpy.test(): File /usr/stsci/pyssgdev/2.5.4/numpy/testing/utils.py, line 619 except ValueError as e: ^ SyntaxError: invalid syntax Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy distutils breaks scipy install on mac
Robert Kern wrote: On Tue, Dec 8, 2009 at 15:36, Mark Sienkiewicz sienk...@stsci.edu wrote: ( Presumably, some other version of gfortan does accept -arch, or this code wouldn't be here, right? ) Right. The -arch flag was added by Apple to GCC and their patch really should be applied to all builds of GCC compilers for the Mac. It is deeply disappointing that Fink ignored this. So, you're saying that an un-patched GCC doesn't know -arch ? In that case, isn't it a mistake to see gfortran on the path, and then assume that you can say -arch to it? The only Mac gfortran build that I can recommend is here: http://r.research.att.com/tools/ I saw the should note about that in the installation instructions. If I were doing this for personal use, I would have just installed that compiler and been done with it. Unfortunately, I am supporting many users who are already going to have the fink gfortran installed, and therefore I have to build scipy to use those libraries. It would be a tremendous amount of work just to convince my IT department to uninstall the fink gfortran and install the ATT gfortran on every mac at the institute. _can_target() should be fixed to be more accurate, though, so if you find a patch that works for you, please let us know. Here is an idea: I have a Mac Tiger machine that I believe has the ATT gfortran installed. It can make universal binaries. It says: % gfortran -arch bananapc6000 -v gfortran: Invalid arch name : bananapc6000 % But the Mac Leopard machine with the Fink: % gfortran -arch bananapc6000 -v Using built-in specs. Target: i686-apple-darwin9 Configured with: ../gcc-4.3.0/configure --prefix=/sw --prefix=/sw/lib/gcc4.3 --mandir=/sw/share/man --infodir=/sw/share/info --enable-languages=c,c++,fortran,objc,java --with-arch=nocona --with-tune=generic --build=i686-apple-darwin9 --with-gmp=/sw --with-libiconv-prefix=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --disable-libjava-multilib Thread model: posix gcc version 4.3.0 (GCC) % So, if you ask gfortran to use an obviously bogus architecture and it objects with the message Invalid arch name, then it knows what -arch means. If it says anything else, then it doesn't. It should work until somebody ports gfortran to make bananapc6000 binaries. :) I think this is an ugly hack, but that's how it is when you use auto-detection. (I wish there were some way that distutils could autodetect everything, write it into a file, let me edit that file, then next time I run setup.py it would use the values in the file, but I expect that would require a near complete re-write of distutils, and I just don't have time.) Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy distutils breaks scipy install on mac
When I compile scipy on a mac, the build fails with: ... gfortran:f77: scipy/fftpack/src/dfftpack/dcosqb.f f951: error: unrecognized command line option -arch f951: error: unrecognized command line option -arch f951: error: unrecognized command line option -arch f951: error: unrecognized command line option -arch error: Command /sw/bin/gfortran -Wall -ffixed-form -fno-second-underscore -arch i686 -arch x86_64 -fPIC -O3 -funroll-loops -I/usr/stsci/pyssgdev/2.5.4/numpy/core/include -c -c scipy/fftpack/src/dfftpack/dcosqb.f -o build/temp.macosx-10.3-i386-2.5/scipy/fftpack/src/dfftpack/dcosqb.o failed with exit status 1 I have % gfortran --version GNU Fortran (GCC) 4.3.0 Copyright (C) 2008 Free Software Foundation, Inc. % which gfortran /sw/bin/gfortran (This /sw/bin apparently means it was installed by fink. My IT department did this. This is not the recommended compiler from ATT, but it seems a likely configuration to encounter in the wild, and I didn't expect a problem. ) I traced the problem to numpy/distutils/fcompiler/gnu.py in the class Gnu94FCompiler. The function _universal_flags() tries to detect which processor types are recognized by the compiler, presumably in an attempt to make a macintosh universal binary. It adds -arch whatever for each architecture that it thinks it detected. Since gfortran does not recognize -arch, the compile fails. ( Presumably, some other version of gfortan does accept -arch, or this code wouldn't be here, right? ) The function _can_target() attempts to recognize what architectures the compiler is capable of by passing in -arch parameters with various known values, but gfortran does not properly indicate a problem in a way that _can_target() can detect: % gfortran -arch i386 hello_world.f f951: error: unrecognized command line option -arch % gfortran -arch i386 -v Using built-in specs. Target: i686-apple-darwin9 Configured with: ../gcc-4.3.0/configure --prefix=/sw --prefix=/sw/lib/gcc4.3 --mandir=/sw/share/man --infodir=/sw/share/info --enable-languages=c,c++,fortran,objc,java --with-arch=nocona --with-tune=generic --build=i686-apple-darwin9 --with-gmp=/sw --with-libiconv-prefix=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --disable-libjava-multilib Thread model: posix gcc version 4.3.0 (GCC) % echo $status 0 % That is, when you say -v, it gives no indication that it doesn't understand the -arch flag. I didn't ask for a universal binary and I don't need one, so I'm surprised that it is trying to make one for me. I think the correct solution is that _universal_flag() should not add -arch flags unless the user specifically requests one. Unfortunately, I can't write a patch, because I don't have the time it would take to reverse engineer distutils well enough to know how to do it. As is usual when a setup.py auto-detects the wrong compiler flags, the easiest solution is to create a shell script that looks like the compiler, but add/removes flags as necessary: % cat /eng/ssb/auto/prog/binhacks/scipy.osx/gfortran #!/bin/sh args= for x in $* do case $x in -arch) shift shift ;; *) args=$args $x ;; esac done /sw/bin/gfortran $args Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy install script changes permissions?
In the source build folder, all numpy test scripts have the correct permissions and are not marked as executable, but in the install directory (/usr/local/lib/python2.6/dist-packages/numpy/), the test test scripts have completely different permissions, and are all marked as executable. Thus, nose wont run the tests. It works ok for me with python 2.5.1 on a mac and python 2.6.1 on linux. That doesn't help you, but it may be a clue at some point. Is it only the test scripts that are executable, or is it everything that gets installed? Does it affect packages other than numpy / scipy? If so, we can suspect distutils and/or ubuntu, rather than numpy. Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Numpy depends on OpenSSL ???
I have discovered the hard way that numpy depends on openssl. I am building a 64 bit python environment for the macintosh. I currently do not have a 64 bit openssl library installed, so the python interpreter does not have hashlib. (hashlib gets its md5 function from the openssl library.) The problem is in numpy/core/code_generators/genapi.py, where it appears to be trying to make an md5 hash of the declarations of some of the C functions. What is this hash used for? Is there a particular reason that it needs to be cryptographically strong? Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy depends on OpenSSL ???
Robert Kern wrote: On Wed, Sep 23, 2009 at 10:52, Mark Sienkiewicz sienk...@stsci.edu wrote: I have discovered the hard way that numpy depends on openssl. I am building a 64 bit python environment for the macintosh. I currently do not have a 64 bit openssl library installed, so the python interpreter does not have hashlib. (hashlib gets its md5 function from the openssl library.) There are builtin implementations that do not depend on OpenSSL. hashlib should be using them for MD5 and the standard SHA variants when OpenSSL is not available. This is the clue that I needed. Here is where it led: setup.py tries to detect the presence of openssl by looking for the library and the include files. It detects the library that Apple provided in /usr/lib/libssl.dylib and tries to build the openssl version of hashlib. But when it actually builds the module, the link fails because that library file is not for the correct architecture. I am building for x86_64, but the library contains only ppc and i386. The result is that hashlib cannot be imported, so the python installer decides not to install it at all. That certainly appears to indicate that the python developers consider hashlib to be optional, but it _should_ work in most any python installation. So, the problem is really about the python install automatically detecting libraries. If I hack the setup.py that builds all the C modules so that it can't find the openssl library, then it uses the fallbacks that are distributed with python. That gets me a as far as EnvironmentError: math library missing; rerun setup.py after setting the MATHLIB env variable, which is a big improvement. (The math library is not missing, but this is a different problem entirely.) Thanks, and sorry for the false alarm. Mark S. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] 1.3.0 rc1 MATHLIB env variable / bad compiler flags
I have this configuration: numpy 1.3.0 rc1 Solaris 10 Python 2.5.4 compiled as a 64 bit executable When I try to install numpy, it says: C compiler: cc -DNDEBUG -O -xarch=native64 -xcode=pic32 compile options: '-Inumpy/core/src -Inumpy/core/include -I/usr/stsci/Python-2.5.4/include/python2.5 -c' cc: _configtest.c cc _configtest.o -lm -o _configtest ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest failure. ... mathlibs = check_mathlib(config_cmd) File numpy/core/setup.py, line 253, in check_mathlib raise EnvironmentError(math library missing; rerun EnvironmentError: math library missing; rerun setup.py after setting the MATHLIB env variable Of course, the problem is that it is using the wrong compiler flags during the link phase, so nothing I set MATHLIB to can possibly work. I found that I can get it to compile by creating these shell scripts: % cat cc #!/bin/sh /opt/SUNWspro-6u2/bin/cc -xarch=native64 -xcode=pic32 $* % cat f90 #!/bin/sh /opt/SUNWspro-6u2/bin/f90 -xarch=native64 -xcode=pic32 $* I think this looks like a bug. I thought I might try to make a patch (since this is all about installing, so, in principle, you don't need to know much about numpy), but I did not get very far in figuring out how the install works. The good news is that once you get it to build, it seems to work. (IIRC, rc1 fails the same test that it had problems with in my other email. The fix on the trunk also works in 64 bit solaris.) Mark S. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.3.0 rc1 fails find_duplicates on Solaris
Pauli Virtanen wrote: Probably they are both related to unspecified sort order for the duplicates. There were some sort-order ignoring missing in the test. I think the test is now fixed in trunk: http://projects.scipy.org/numpy/changeset/6827 The test passes in 1.4.0.dev6827. Tested on Solaris 8, Mac OSX 10.4 (Tiger) on x86 and ppc, and both 32 and 64 bit Red Hat Enterprise, all with Python 2.5.1. Thanks for fixing this. Mark S. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy 1.3.0 rc1 fails find_duplicates on Solaris
==
FAIL: Test find_duplicates
--
...
AssertionError:
Arrays are not equal
(mismatch 50.0%)
x: array([(1, (2.0, 'B')), (2, (2.0, 'B')), (2, (2.0, 'B')), (1, (2.0,
'B'))],
dtype=[('A', 'i4'), ('B', [('BA', 'f8'), ('BB', '|S1')])])
y: array([(2, (2.0, 'B')), (1, (2.0, 'B')), (2, (2.0, 'B')), (1, (2.0,
'B'))],
dtype=[('A', 'i4'), ('B', [('BA', 'f8'), ('BB', '|S1')])])
--
These are new (two months old) tests. Hmm, they are also marked as known
failures on win32. I wonder why they fail there and not on linux? I think
you should open a ticket for this.
I'm not sure how old the test is, but I see that it has been failing
since Feb 1. (That is the earliest report I have online at the moment.)
The ticket is http://projects.scipy.org/numpy/ticket/1039 . I added
this specific failure mode to the ticket today.
It does not surprise me at all when the trunk is broken on solaris. I'm
mentioning it on the list because I see it is still broken in the
release candidate. I assume somebody would want to either fix the
problem or remove the non-working feature from the release.
Mark S.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
