[Numpy-discussion] compile process of numpy-1.5.0 fails in Itanium cluster with icc
hi,
I'm trying to install numpy-1.5.0 in our Itanium (IA-64) supercomputer
[http://www.cesga.es/content/view/917/115/lang,en/ ].
This is my site.cfg (I have to install umfpack but I think this is not the
problem):
$ cat site.cfg
[DEFAULT]
library_dirs =
/opt/cesga/fftw-3.2.2/lib:/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64
include_dirs =
/opt/cesga/fftw-3.2.2/include:/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include
#[umfpack]
#umfpack_libs = umfpack
[fftw]
libraries = fftw3
[mkl]
library_dirs = /opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64
lapack_libs = mkl_lapack
mkl_libs = mkl_core
The only hacks that I made:
$ diff numpy/distutils/system_info.py ../numpy-
scipy/numpy-1.5.0/numpy/distutils/system_info.py
818c818
_lib_mkl = ['mkl','mkl_lapack','guide']
---
_lib_mkl = ['mkl','vml','guide']
$ diff numpy/distutils/intelccompiler.py ../numpy-
scipy/numpy-1.5.0/numpy/distutils/intelccompiler.py
11c11
cc_exe = 'icc -O3 -openmp -lpthread -fPIC'
---
cc_exe = 'icc'
And I have used this script to launch the compilation process:
$ cat compilar.sh
#!/bin/bash
module load nose/0.10.4 fftw/3.2.2 ifort/11.1.056 icc/11.1.056 mkl/10.2.2
python setup.py config --compiler=intele --fcompiler=intele build install --
prefix=/home/cesga/mpan/numpy-1.5.0
exit 0
blas and lapack are detected properly:
$ head -n 40 compilar.log
Running from numpy source directory.F2PY Version 1
blas_opt_info:
blas_mkl_info:
FOUND:
libraries = ['mkl_core', 'pthread']
library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/cesga/fftw-3.2.2/include',
'/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include']
FOUND:
libraries = ['mkl_core', 'pthread']
library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/cesga/fftw-3.2.2/include',
'/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include']
lapack_opt_info:
lapack_mkl_info:
mkl_info:
FOUND:
libraries = ['mkl_core', 'pthread']
library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/cesga/fftw-3.2.2/include',
'/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include']
FOUND:
libraries = ['mkl_lapack', 'mkl_core', 'pthread']
library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/cesga/fftw-3.2.2/include',
'/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include']
FOUND:
libraries = ['mkl_lapack', 'mkl_core', 'pthread']
library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/cesga/fftw-3.2.2/include',
'/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include']
running config
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler
options
but, unfortunately, the compilation process ends with an unexpected
realocation problem:
$ tail -n 11 compilar.log
/opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -shared
build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o
build/lib.linux-ia64-2.4/numpy/core/umath_tests.so
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol inner1d_signature
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol innerwt_signature
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol matrix_multiply_signature
ld: final link failed: Nonrepresentable section on output
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol inner1d_signature
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol innerwt_signature
ld: build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic
symbol matrix_multiply_signature
ld: final link failed: Nonrepresentable section on output
error: Command /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -
shared build/temp.linux-ia64-2.4/build/src.linux-
ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o
build/lib.linux-ia64-2.4/numpy/core/umath_tests.so failed with exit status 1
I have tried to link this files manually but I haven't any success:
$ /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -shared
build/temp.linux-ia64-2.4/build/src.linux-
[Numpy-discussion] [ANN] SciPy India 2010 Call for Presentations
== SciPy 2010 Call for Papers == The second `SciPy India Conference http://scipy.in`_ will be held from December 13th to 18th, 2010 at `IIIT-Hyderabad http://www.iiit.ac.in/`_. At this conference, novel applications and breakthroughs made in the pursuit of science using Python are presented. Attended by leading figures from both academia and industry, it is an excellent opportunity to experience the cutting edge of scientific software development. The conference is followed by two days of tutorials and a code sprint, during which community experts provide training on several scientific Python packages. We invite you to take part by submitting a talk abstract on the conference website at: http://scipy.in Talk/Paper Submission == We solicit talks and accompanying papers (either formal academic or magazine-style articles) that discuss topics regarding scientific computing using Python, including applications, teaching, development and research. Papers are included in the peer-reviewed conference proceedings, published online. Please note that submissions primarily aimed at the promotion of a commercial product or service will not be considered. Important Dates == Monday, Oct. 11: Abstracts Due Saturday, Oct. 30: Schedule announced Tuesday, Nov. 30: Proceedings paper submission due Monday-Tuesday, Dec. 13-14: Conference Wednesday-Friday, Dec. 15-17: Tutorials/Sprints Saturday, Dec. 18: Sprints Organizers == * Jarrod Millman, Neuroscience Institute, UC Berkeley, USA (Conference Co-Chair) * Prabhu Ramachandran, Department of Aerospace Engineering, IIT Bombay, India (Conference Co-Chair) * FOSSEE Team ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] read ascii file from complex fortran format() -- genfromtxt
Hi all, I've got an ascii file with a relatively complicated structure, originally written by fortran with the format: 135format(a12,1x,2(f10.5,1x),i3,1x,4(f9.3,1x),4(i2,1x),3x, 1 16(f7.2,1x),i3,3x,f13.5,1x,f10.5,1x,f10.6,1x,i3,1x, 2 4(f10.6,1x), 2 i2,1x,f5.2,1x,f10.3,1x,i3,1x,f7.2,1x,f7.2,3x,4(f7.4,1x), 3 4(f7.2,1x),3x,f7.2,1x,i4,3x,f10.3,1x,14(f6.2,1x),i3,1x, 1 3x,2f10.5,8f11.2,2f10.5,f12.3,3x, 4 2(a6,1x),a23,1x,a22,1x,a22) Note, in particular, that many of the strings contain white space. Is there a relatively straightforward way to translate this into dtype (and delimiter?) arguments for use with genfromtxt or do I just have to do it by hand? Andrew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique 2d arrays
On Tue, Sep 21, 2010 at 2:55 AM, Peter Schmidtke pschmid...@mmb.pcb.ub.es wrote: Dear all, I'd like to know if there is a pythonic / numpy way of retrieving unique lines of a 2d numpy array. In a way I have this : [[409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [426 193] [431 129]] And I'd like to get this : [[409 152] [426 193] [431 129]] How can I do this without workarounds like string concatenation or such things? Numpy.unique flattens the whole array so it's not really of use here. One possibility see thread at http://mail.scipy.org/pipermail/numpy-discussion/2009-August/044664.html Josef Cheers. -- Peter Schmidtke PhD Student Dept. Physical Chemistry Faculty of Pharmacy University of Barcelona ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] help from OS X 10.5 users wanted
Hi, There have been many reports from OS X 10.5(.8) users that the provided binaries on Sourceforge for numpy versions = 1.4.0 do not work, see http://projects.scipy.org/numpy/ticket/1322. If you're an OS X 10.5 user I would like your help to test if the same issue occurs for the following binary: http://sourceforge.net/projects/numpy/files/Temporary/numpy-2.0.0.dev-py2.6-osx10.5-python.org.dmg/download Please comment on the above ticket. Thanks, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique 2d arrays
Hey Josef, I didn't stumble upon these posts. Thanks for the hint...it doesn't look very pythonic or matlab like still. This would be a nice thing to have a unique function that is able to take an axis argument. Cheers. Peter josef.p...@gmail.com wrote: On Tue, Sep 21, 2010 at 2:55 AM, Peter Schmidtke pschmid...@mmb.pcb.ub.es wrote: Dear all, I'd like to know if there is a pythonic / numpy way of retrieving unique lines of a 2d numpy array. In a way I have this : [[409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [426 193] [431 129]] And I'd like to get this : [[409 152] [426 193] [431 129]] How can I do this without workarounds like string concatenation or such things? Numpy.unique flattens the whole array so it's not really of use here. One possibility see thread at http://mail.scipy.org/pipermail/numpy-discussion/2009-August/044664.html Josef Cheers. -- Peter Schmidtke PhD Student Dept. Physical Chemistry Faculty of Pharmacy University of Barcelona ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Peter Schmidtke PhD Student Dept. Physical Chemistry Faculty of Pharmacy University of Barcelona ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique 2d arrays
On Tue, Sep 21, 2010 at 1:55 AM, Peter Schmidtke
pschmid...@mmb.pcb.ub.eswrote:
Dear all,
I'd like to know if there is a pythonic / numpy way of retrieving unique
lines of a 2d numpy array.
In a way I have this :
[[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[426 193]
[431 129]]
And I'd like to get this :
[[409 152]
[426 193]
[431 129]]
How can I do this without workarounds like string concatenation or such
things? Numpy.unique flattens the whole array so it's not really of use
here.
Here is one alternative:
I[15]: a = np.array([[409, 152], [409, 152], [426, 193], [431, 129]])
I[16]: np.array(list(set(tuple(i) for i in a.tolist(
O[16]:
array([[409, 152],
[426, 193],
[431, 129]])
I[6]: %timeit
np.unique(a.view([('',a.dtype)]*a.shape[1])).view(a.dtype).reshape(-1,a.shape[1])
1 loops, best of 3: 51 us per loop
I[8]: %timeit np.array(list(set(tuple(i) for i in a.tolist(
1 loops, best of 3: 31.4 us per loop
# Try with a bigger array
I[9]: k = np.array((a.tolist()*5))
I[10]: %timeit np.array(list(set(tuple(i) for i in k.tolist(
1 loops, best of 3: 324 ms per loop
I[11]: %timeit
np.unique(k.view([('',k.dtype)]*k.shape[1])).view(k.dtype).reshape(-1,k.shape[1])
1 loops, best of 3: 790 ms per loop
Seems like faster on these tests comparing to the unique method. Also it is
more readable. Still not uber Pythonic. Haskell has nub to remove
duplicate list elements.
http://www.haskell.org/ghc/docs/6.12.2/html/libraries/base-4.2.0.1/Data-List.html#v%3Anub
--
Gökhan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] read ascii file from complex fortran format() -- genfromtxt
Have you tried http://code.google.com/p/python-fortranformat/ It's not officially released yet but it's probably worth a try. David H. On Tue, Sep 21, 2010 at 8:25 AM, Andrew Jaffe a.h.ja...@gmail.com wrote: Hi all, I've got an ascii file with a relatively complicated structure, originally written by fortran with the format: 135format(a12,1x,2(f10.5,1x),i3,1x,4(f9.3,1x),4(i2,1x),3x, 1 16(f7.2,1x),i3,3x,f13.5,1x,f10.5,1x,f10.6,1x,i3,1x, 2 4(f10.6,1x), 2 i2,1x,f5.2,1x,f10.3,1x,i3,1x,f7.2,1x,f7.2,3x,4(f7.4,1x), 3 4(f7.2,1x),3x,f7.2,1x,i4,3x,f10.3,1x,14(f6.2,1x),i3,1x, 1 3x,2f10.5,8f11.2,2f10.5,f12.3,3x, 4 2(a6,1x),a23,1x,a22,1x,a22) Note, in particular, that many of the strings contain white space. Is there a relatively straightforward way to translate this into dtype (and delimiter?) arguments for use with genfromtxt or do I just have to do it by hand? Andrew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique 2d arrays
On Tue, Sep 21, 2010 at 1:29 PM, Gökhan Sever gokhanse...@gmail.com wrote:
On Tue, Sep 21, 2010 at 1:55 AM, Peter Schmidtke pschmid...@mmb.pcb.ub.es
wrote:
Dear all,
I'd like to know if there is a pythonic / numpy way of retrieving unique
lines of a 2d numpy array.
In a way I have this :
[[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[409 152]
[426 193]
[431 129]]
And I'd like to get this :
[[409 152]
[426 193]
[431 129]]
How can I do this without workarounds like string concatenation or such
things? Numpy.unique flattens the whole array so it's not really of use
here.
Here is one alternative:
I[15]: a = np.array([[409, 152], [409, 152], [426, 193], [431, 129]])
I[16]: np.array(list(set(tuple(i) for i in a.tolist(
O[16]:
array([[409, 152],
[426, 193],
[431, 129]])
I[6]: %timeit
np.unique(a.view([('',a.dtype)]*a.shape[1])).view(a.dtype).reshape(-1,a.shape[1])
1 loops, best of 3: 51 us per loop
I[8]: %timeit np.array(list(set(tuple(i) for i in a.tolist(
1 loops, best of 3: 31.4 us per loop
# Try with a bigger array
I[9]: k = np.array((a.tolist()*5))
I[10]: %timeit np.array(list(set(tuple(i) for i in k.tolist(
1 loops, best of 3: 324 ms per loop
I[11]: %timeit
np.unique(k.view([('',k.dtype)]*k.shape[1])).view(k.dtype).reshape(-1,k.shape[1])
1 loops, best of 3: 790 ms per loop
I'm a bit surprised, I think np.unique does some extra work to
maintain the order.
The tolist() might not be necessary if you iterate over rows.
Josef
Seems like faster on these tests comparing to the unique method. Also it is
more readable. Still not uber Pythonic. Haskell has nub to remove
duplicate list
elements. http://www.haskell.org/ghc/docs/6.12.2/html/libraries/base-4.2.0.1/Data-List.html#v%3Anub
--
Gökhan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique 2d arrays
21/09/10 @ 12:55 (-0500), thus spake Gökhan Sever:
On Tue, Sep 21, 2010 at 12:43 PM, josef.p...@gmail.com wrote:
I'm a bit surprised, I think np.unique does some extra work to
maintain the order.
The tolist() might not be necessary if you iterate over rows.
Testing again with a smaller k array and more repeats
I[25]: k = np.array((a.tolist()*5000))
I[27]: %timeit -r 100 np.array(list(set(tuple(i) for i in k.tolist(
10 loops, best of 100: 31.3 ms per loop
I[28]: %timeit -r 100 np.array(list(set(tuple(i) for i in k)))
10 loops, best of 100: 55.4 ms per loop
I[30]: %timeit -r 100
np.unique(k.view([('',k.dtype)]*k.shape[1])).view(k.dtype).reshape(-1,k.shape[1])
10 loops, best of 100: 60.5 ms per loop
.tolist version is faster. Can you also verify this?
I get the same results:
In [14]: x=np.random.poisson(1.3, size=10).reshape(-1,2)
In [19]: %timeit np.array(tuple(set(map(tuple, x.tolist()
10 loops, best of 3: 86.5 ms per loop
In [20]: %timeit np.array(tuple(set(map(tuple, x
10 loops, best of 3: 125 ms per loop
Bye.
Ernest
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] A proposed change to rollaxis() behavior for negative 'start' values
Hi Numpy Folks, A while back, I filed this ticket: http://projects.scipy.org/numpy/ticket/1441 suggesting a change to rollaxis() and some fixes to the doc and error reporting. Ralf Gommers suggested I float the behavior change here, so that's what I'm doing. The motivation for the change comes because it seems like there should be a simpler way to get some axis into the last position than to do this: a = np.ones((3,4,5,6)) b = np.rollaxis(a, axis=0, start=len(a.shape)) b.shape (4, 5, 6, 3) But currently it seems there isn't because when you specify -1 as the 'start' argument, the axis is moved into the second-to-last position. My proposed change, which you can see on the ticket, would change that so that using -1 referred to the end position. Note that the use of negative 'start' arguments isn't currently documented and, in its current form, doesn't seem very useful. My proposal wouldn't change the behavior for positive 'start' values at all, and the interpretation of 'axis' arguments is also unaffected. If that's going to break too much code, here's a pathway that might be acceptable: Add a new function moveaxis() which works the way rollaxis() does for positive arguments but in the new way for negative arguments. Eventually, rollaxis could be deprecated to keep things tidy. This has the added advantage of using a name that seems to fit what the function does better - 'rollaxis' suggests a behavior like the roll() function which affects other axes, which isn't what happens. Thanks for listening; I'm a big fan of Numpy. Best, Ken Basye ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A proposed change to rollaxis() behavior for negative 'start' values
Hi Ken, This is a tricky one. The current behaviour of rollaxis is to remove the requested axis from the list of axes and then insert it before the axis specified. This is exactly how python's list insertion works: In [1]: a = range(10) In [3]: a.insert(-1,'a') In [4]: a Out[4]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 'a', 9] And indeed, there's no clean way to add something to the end of a list using insert (apart from the obvious a.insert(len(a),'b') ). For this you have .append(). Unfortunately numpy's rollaxis, while it agrees with insert in its behaviour, doesn't have a move_axis_to_end. The situation is also somewhat muddied by the fact that rollaxis also removes the axis from the original list of axes, so that the interpretation of index numbers is a little more subtle. But I think your suggested behaviour would be confusing because of the conflict with python's insert. How about allowing the string end as an argument to rollaxis to specify that the axis should go at the end? Anne On 21 September 2010 15:48, Ken Basye kbas...@jhu.edu wrote: Hi Numpy Folks, A while back, I filed this ticket: http://projects.scipy.org/numpy/ticket/1441 suggesting a change to rollaxis() and some fixes to the doc and error reporting. Ralf Gommers suggested I float the behavior change here, so that's what I'm doing. The motivation for the change comes because it seems like there should be a simpler way to get some axis into the last position than to do this: a = np.ones((3,4,5,6)) b = np.rollaxis(a, axis=0, start=len(a.shape)) b.shape (4, 5, 6, 3) But currently it seems there isn't because when you specify -1 as the 'start' argument, the axis is moved into the second-to-last position. My proposed change, which you can see on the ticket, would change that so that using -1 referred to the end position. Note that the use of negative 'start' arguments isn't currently documented and, in its current form, doesn't seem very useful. My proposal wouldn't change the behavior for positive 'start' values at all, and the interpretation of 'axis' arguments is also unaffected. If that's going to break too much code, here's a pathway that might be acceptable: Add a new function moveaxis() which works the way rollaxis() does for positive arguments but in the new way for negative arguments. Eventually, rollaxis could be deprecated to keep things tidy. This has the added advantage of using a name that seems to fit what the function does better - 'rollaxis' suggests a behavior like the roll() function which affects other axes, which isn't what happens. Thanks for listening; I'm a big fan of Numpy. Best, Ken Basye ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] real and imag functions should produce errors for object arrays
Hi, The following example demonstrates a rather unexpected result: import numpy x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) print x.real (1+1j) print x.imag 0 Shouldn't real and imag return an error in such a situation? Thanks, Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
On Tue, Sep 21, 2010 at 4:31 PM, Michael Gilbert michael.s.gilb...@gmail.com wrote: Hi, The following example demonstrates a rather unexpected result: import numpy x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) print x.real (1+1j) print x.imag 0 Shouldn't real and imag return an error in such a situation? Thanks, Mike Don't use 'numpy.object'. Because complex is a numerical type, numpy can handle it just fine. By setting dtype to numpy.object, numpy then treats it like an object rather than a numerical. x = numpy.array( complex(1.0, 1.0) ) should work just fine. I hope that helps! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
Hi,
On Tue, Sep 21, 2010 at 2:44 PM, Benjamin Root ben.r...@ou.edu wrote:
On Tue, Sep 21, 2010 at 4:31 PM, Michael Gilbert
michael.s.gilb...@gmail.com wrote:
Hi,
The following example demonstrates a rather unexpected result:
import numpy
x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object )
print x.real
(1+1j)
print x.imag
0
Shouldn't real and imag return an error in such a situation?
It looks like there was a decision to let 'real' and 'imag' pass
quietly for non-numerical types:
In [2]: a = np.array('hello', dtype=object)
In [3]: a.real
Out[3]: array('hello', dtype=object)
In [4]: a.imag
Out[4]: array(0, dtype=object)
and
In [6]: a = np.array('hello', dtype='S5')
In [7]: a.real
Out[7]:
array('hello',
dtype='|S5')
In [8]: a.imag
Out[8]:
array('',
dtype='|S5')
I can see that that could be confusing. I suppose the alternative
would be to raise an error for real and imag for non-numerical types
at least.
Best,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
Tue, 21 Sep 2010 17:31:55 -0400, Michael Gilbert wrote: The following example demonstrates a rather unexpected result: import numpy x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) print x.real (1+1j) print x.imag 0 Shouldn't real and imag return an error in such a situation? It probably shouldn't do *that* at the least. http://projects.scipy.org/numpy/ticket/1618 -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
On Tue, Sep 21, 2010 at 17:17, Pauli Virtanen p...@iki.fi wrote: Tue, 21 Sep 2010 21:50:08 +, Pauli Virtanen wrote: Tue, 21 Sep 2010 17:31:55 -0400, Michael Gilbert wrote: The following example demonstrates a rather unexpected result: import numpy x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) print x.real (1+1j) print x.imag 0 Shouldn't real and imag return an error in such a situation? It probably shouldn't do *that* at the least. *that* == return a complex number from .real What is the alternative? I'm personally happy with saying that many of the operations we define on numpy arrays can be done because we know the types and that object arrays subvert this. numpy can't, without excessive amounts of magic, always know a sensible thing to do with object arrays, so we implement the fast thing to do. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
Hi, On Tue, Sep 21, 2010 at 3:28 PM, Robert Kern robert.k...@gmail.com wrote: On Tue, Sep 21, 2010 at 17:17, Pauli Virtanen p...@iki.fi wrote: Tue, 21 Sep 2010 21:50:08 +, Pauli Virtanen wrote: Tue, 21 Sep 2010 17:31:55 -0400, Michael Gilbert wrote: The following example demonstrates a rather unexpected result: import numpy x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) print x.real (1+1j) print x.imag 0 Shouldn't real and imag return an error in such a situation? It probably shouldn't do *that* at the least. *that* == return a complex number from .real What is the alternative? I'm personally happy with saying that many of the operations we define on numpy arrays can be done because we know the types and that object arrays subvert this. numpy can't, without excessive amounts of magic, always know a sensible thing to do with object arrays, so we implement the fast thing to do. I agree that special-casing object array .real to detect complex contents seems a bit messy, but it does make sense I think to raise an error from .real and .imag for non-numerical types. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
Tue, 21 Sep 2010 17:28:08 -0500, Robert Kern wrote: [clip] *that* == return a complex number from .real What is the alternative? I'm personally happy with saying that many of the operations we define on numpy arrays can be done because we know the types and that object arrays subvert this. numpy can't, without excessive amounts of magic, always know a sensible thing to do with object arrays, so we implement the fast thing to do. As I see it, the alternatives are 1) Not to define .real and .imag for object arrays 2) Define them as elementwise .real and .imag I don't clearly see the reason for x.real is x True x.imag array([0], dtype=object) But it is a minor corner case, and there may be backward compatibility issues in changing it. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
On Tue, Sep 21, 2010 at 17:40, Pauli Virtanen p...@iki.fi wrote: Tue, 21 Sep 2010 17:28:08 -0500, Robert Kern wrote: [clip] *that* == return a complex number from .real What is the alternative? I'm personally happy with saying that many of the operations we define on numpy arrays can be done because we know the types and that object arrays subvert this. numpy can't, without excessive amounts of magic, always know a sensible thing to do with object arrays, so we implement the fast thing to do. As I see it, the alternatives are 1) Not to define .real and .imag for object arrays 2) Define them as elementwise .real and .imag I don't clearly see the reason for x.real is x True x.imag array([0], dtype=object) The reason it is that way is probably because object and str dtypes were an afterthought when we extended .real and .imag to all dtypes. We used to only have those attributes on complex arrays. We added the behavior described above to all of the other dtypes to allow one to use the .real and .imag attributes regardless of whether the array was complex or real. I don't think we gave much thought to what the right thing to do with object arrays or str arrays. Perhaps we did think about it and decided that consistency with the other non-complex dtypes was more important than trying to expensively inspect the individual objects for .real and .imag attributes. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Removing numpy numpy-1.1.0 in Python 2.5
I may have bounced a similar problem around here a few months ago, but this one is a bit more important to get an answer for. I'm about to distribute some report programs to three people who work together and should be using the same version of Python and libraries that I use, and a common application to us. However, to create two new report programs they do not have, I need to make sure we do have the same versions, and in particular need to use the numpy 1.2.0 I used for the report programs. To replace the old numpy and make sure we are all on on the same wave length, I wrote the description below. I want to know if what I say about removing 1.1.0 and installing 1.2.0 is correct. I think deleting the folder as below is the right way to go. numpy is not in Control Panel Add/Remove. Comments? = Required Libraries for Sentinel Report Programs for Win XP ... You will need to replace your numpy-1.1.0 library. This is accomplished by first removing numpy-1.1.0 from the Python library located at C:\Python2.5\Lib\site-packages. Delete the numpy folder. Next execute the numpy-1.2.0-win32-superpack-python2.5.exe to install numpy 1.2.0. Try sentuser.py to make sure it runs properly. If not, contact me. == -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] slicing / indexing question
Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim -- Timothy W. Hilton PhD Candidate, Department of Meteorology The Pennsylvania State University 503 Walker Building, University Park, PA 16802 hil...@meteo.psu.edu ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing numpy numpy-1.1.0 in Python 2.5
On Tue, Sep 21, 2010 at 7:01 PM, Wayne Watson sierra_mtnv...@sbcglobal.net wrote: I may have bounced a similar problem around here a few months ago, but this one is a bit more important to get an answer for. I'm about to distribute some report programs to three people who work together and should be using the same version of Python and libraries that I use, and a common application to us. However, to create two new report programs they do not have, I need to make sure we do have the same versions, and in particular need to use the numpy 1.2.0 I used for the report programs. To replace the old numpy and make sure we are all on on the same wave length, I wrote the description below. I want to know if what I say about removing 1.1.0 and installing 1.2.0 is correct. I think deleting the folder as below is the right way to go. numpy is not in Control Panel Add/Remove. I have some numpy versions in there I would recommend to first check Add/Remove, and delete from there and then delete the numpy folder in site-packages if it is still there. Why don't you upgrade to the latest release. If you are already upgrading, jumping to the latest might be better. Josef Comments? = Required Libraries for Sentinel Report Programs for Win XP ... You will need to replace your numpy-1.1.0 library. This is accomplished by first removing numpy-1.1.0 from the Python library located at C:\Python2.5\Lib\site-packages. Delete the numpy folder. Next execute the numpy-1.2.0-win32-superpack-python2.5.exe to install numpy 1.2.0. Try sentuser.py to make sure it runs properly. If not, contact me. == -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] slicing / indexing question
On Tue, Sep 21, 2010 at 6:20 PM, Timothy W. Hilton hil...@meteo.psu.edu wrote: Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim This works, although I imagine it could be streamlined. In [1]: this_par = N.ones((2,4,4)) In [2]: idx = N.random.random((4,4)) 0.5 In [3]: pars = N.arange(2) - 10 In [4]: this_par[:,idx] = N.tile(pars, (idx.sum(), 1)).transpose() In [5]: idx Out[5] array([[ True, False, True, False], [False, False, True, True], [False, False, False, False], [False, False, False, False]], dtype=bool) In [6]: this_par Out[6]: array([[[-10., 1., -10., 1.], [ 1., 1., -10., -10.], [ 1., 1.,1., 1.], [ 1., 1.,1., 1.]], [[ -9., 1., -9., 1.], [ 1., 1., -9., -9.], [ 1., 1.,1., 1.], [ 1., 1.,1., 1.]]]) Brett ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] slicing / indexing question
On Tue, Sep 21, 2010 at 7:54 PM, Brett Olsen brett.ol...@gmail.com wrote: On Tue, Sep 21, 2010 at 6:20 PM, Timothy W. Hilton hil...@meteo.psu.edu wrote: Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim This works, although I imagine it could be streamlined. In [1]: this_par = N.ones((2,4,4)) In [2]: idx = N.random.random((4,4)) 0.5 In [3]: pars = N.arange(2) - 10 In [4]: this_par[:,idx] = N.tile(pars, (idx.sum(), 1)).transpose() In [5]: idx Out[5] array([[ True, False, True, False], [False, False, True, True], [False, False, False, False], [False, False, False, False]], dtype=bool) In [6]: this_par Out[6]: array([[[-10., 1., -10., 1.], [ 1., 1., -10., -10.], [ 1., 1., 1., 1.], [ 1., 1., 1., 1.]], [[ -9., 1., -9., 1.], [ 1., 1., -9., -9.], [ 1., 1., 1., 1.], [ 1., 1., 1., 1.]]]) introspection works easier with an example indexing with slice and boolean seems to flatten the boolean part, then we need only one newaxis ? seems to work this_par[:,idx].shape # = N.tile(pars, (idx.sum(), 1)).transpose() (2, 8) idx array([[ True, False, False, False], [ True, True, True, False], [ True, True, False, False], [ True, False, True, False]], dtype=bool) this_par[:,idx] = pars[:,N.newaxis] this_par array([[[-10., 1., 1., 1.], [-10., -10., -10., 1.], [-10., -10., 1., 1.], [-10., 1., -10., 1.]], [[ -9., 1., 1., 1.], [ -9., -9., -9., 1.], [ -9., -9., 1., 1.], [ -9., 1., -9., 1.]]]) Josef Brett ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] real and imag functions should produce errors for object arrays
Hi, I see that I have interpreted this thread as Doctor, it hurts when I do this... Well, don't do that! Sorry for the noise. It's all good - a reply is almost always more friendly and helpful than no reply ;) See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing numpy numpy-1.1.0 in Python 2.5
I don't see it in Add/Remove. Whoops. It is on this Win7 machine. I need to check my XP machine. I'll be back when I figure out if that's right. My not always recollection was that it was not on Add/Remove under XP. I'm dealing with people who barely understand software, so keeping this simple is the best thing to do. This version of the application is the last, and we might as well treat it as frozen. I really don't want to try to correct people's install problems or attempts to move ahead with the various libraries we use over the internet, so simple is it. On 9/21/2010 4:38 PM, josef.p...@gmail.com wrote: On Tue, Sep 21, 2010 at 7:01 PM, Wayne Watson sierra_mtnv...@sbcglobal.net wrote: I may have bounced a similar problem around here a few months ago, but this one is a bit more important to get an answer for. I'm about to distribute some report programs to three people who work together and should be using the same version of Python and libraries that I use, and a common application to us. However, to create two new report programs they do not have, I need to make sure we do have the same versions, and in particular need to use the numpy 1.2.0 I used for the report programs. To replace the old numpy and make sure we are all on on the same wave length, I wrote the description below. I want to know if what I say about removing 1.1.0 and installing 1.2.0 is correct. I think deleting the folder as below is the right way to go. numpy is not in Control Panel Add/Remove. I have some numpy versions in there I would recommend to first check Add/Remove, and delete from there and then delete the numpy folder in site-packages if it is still there. Why don't you upgrade to the latest release. If you are already upgrading, jumping to the latest might be better. Josef Comments? = Required Libraries for Sentinel Report Programs for Win XP ... You will need to replace your numpy-1.1.0 library. This is accomplished by first removing numpy-1.1.0 from the Python library located at C:\Python2.5\Lib\site-packages. Delete the numpy folder. Next execute the numpy-1.2.0-win32-superpack-python2.5.exe to install numpy 1.2.0. Try sentuser.py to make sure it runs properly. If not, contact me. == -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing numpy numpy-1.1.0 in Python 2.5
On Tue, Sep 21, 2010 at 10:32 PM, Wayne Watson sierra_mtnv...@sbcglobal.net wrote: I don't see it in Add/Remove. Whoops. It is on this Win7 machine. I need to check my XP machine. I'll be back when I figure out if that's right. My not always recollection was that it was not on Add/Remove under XP. I'm dealing with people who barely understand software, so keeping this simple is the best thing to do. This version of the application is the last, and we might as well treat it as frozen. I really don't want to try to correct people's install problems or attempts to move ahead with the various libraries we use over the internet, so simple is it. If you want to have a frozen application, the best might be to use py2exe, but maybe not worth the effort, if they don't mess with the installed python packages. Josef On 9/21/2010 4:38 PM, josef.p...@gmail.com wrote: On Tue, Sep 21, 2010 at 7:01 PM, Wayne Watson sierra_mtnv...@sbcglobal.net wrote: I may have bounced a similar problem around here a few months ago, but this one is a bit more important to get an answer for. I'm about to distribute some report programs to three people who work together and should be using the same version of Python and libraries that I use, and a common application to us. However, to create two new report programs they do not have, I need to make sure we do have the same versions, and in particular need to use the numpy 1.2.0 I used for the report programs. To replace the old numpy and make sure we are all on on the same wave length, I wrote the description below. I want to know if what I say about removing 1.1.0 and installing 1.2.0 is correct. I think deleting the folder as below is the right way to go. numpy is not in Control Panel Add/Remove. I have some numpy versions in there I would recommend to first check Add/Remove, and delete from there and then delete the numpy folder in site-packages if it is still there. Why don't you upgrade to the latest release. If you are already upgrading, jumping to the latest might be better. Josef Comments? = Required Libraries for Sentinel Report Programs for Win XP ... You will need to replace your numpy-1.1.0 library. This is accomplished by first removing numpy-1.1.0 from the Python library located at C:\Python2.5\Lib\site-packages. Delete the numpy folder. Next execute the numpy-1.2.0-win32-superpack-python2.5.exe to install numpy 1.2.0. Try sentuser.py to make sure it runs properly. If not, contact me. == -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7 N, 121° 2' 32 W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:www.speckledwithstars.net/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
