Re: [Numpy-discussion] Convert array type

[Robert Kern]

Adam Mercer wrote:
 On 08/10/2007, Ryan May [EMAIL PROTECTED] wrote:
 
 Why not use numpy.fromstring?
 
 because that results in the array being filled with gibberish
 
 values = numpy.fromstring(wavearray, dtype=float, count=-1, sep='')

Use sep=' '. As the docstring says, if sep is empty, then the string is
interpreted as binary data. If it is not empty, then the string is interpreted
as ASCII.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fortran 90 compiler problem

[Robert Kern]

[EMAIL PROTECTED] wrote:
   Hello,
 
 I'm using the last svn version of numpy (.
 
 I tried the solution found on the mail list:
 
 python setup.py config_fc --fcompiler=gnu95 build
 
 the two interesting part (at least for me) in the building process are:
 
 customize GnuFCompiler Found executable /usr/bin/g77 
 gnu: no Fortran 90 compiler found
 gnu: no Fortran 90 compiler found 
 customize GnuFCompiler
 gnu: no Fortran 90 compiler found 
 gnu: no Fortran 90 compiler found 
 customize GnuFCompiler using config
 
 and later: 
 customize G95FCompiler Found executable /usr/local/bin/g95
 
 but I still have the same answer when I'm trying to do: f2py -c -m hello 
 hello.f90
 
 customize GnuFCompiler Found executable /usr/bin/g77 
 gnu: no Fortran 90 compiler found gnu: no Fortran 90 
 compiler found customize GnuFCompiler 
 gnu: no Fortran 90 compiler found 
 gnu: no Fortran 90 compiler found

This is a correct message and does not imply that no Fortran 90 compiler exists.
The GnuFCompiler (gnu) only looks at g77, which is not a Fortran 90 compiler.
That message only applies to the gnu compiler, not any other.

 and so:
 
 error: f90 not supported by GnuFCompiler needed for hello.f90
 
 I can't compile my fortran module because of this. I'll attach the file 
 obtain 
 with
 
 python setup.py -v config_fc --fcompiler=gnu95 build
 
 And I have the same result for g95.
 
 thank you for numpy and f2py. 

Please give us the build log that actually failed and the command that you used
to obtain it. The one you show, for numpy, succeeded.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fortran 90 compiler problem

[Robert Kern]

[EMAIL PROTECTED] wrote:
 This is the log I have when I'm trying to use f2py with a fortran 90 file. 
 The last line seems to tell me that f2py cannot see gfortran and g95 even if 
 it detected them when I installed f2py.
 
 thanks,
 
 N.
 
 gfortran and g95 are installed and I can use them:
 
 gfortran
 gfortran: no input files
 
 gfortran -c hello.f90
 ls hello*
 hello.f90 hello.o
 
 g95 
 g95: no input files
 
 g95 -c hello.f90
 ls hello*
 hello.f90 hello.o
 
 and now
 
 f2py -c -m hello hello.f90

Tell f2py that you want it to use gfortran instead of the default.

  f2py --fcompiler=gnu95 -c -m hello hello.f90

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fortran 90 compiler problem

[Robert Kern]

Charles R Harris wrote:
 On 10/10/07, *Robert Kern* [EMAIL PROTECTED]
 mailto:[EMAIL PROTECTED] wrote:
 
 [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote:
Hello,
 
  I'm using the last svn version of numpy (.
 
  I tried the solution found on the mail list:
 
  python setup.py config_fc --fcompiler=gnu95 build
 
  the two interesting part (at least for me) in the building process
 are:
 
  customize GnuFCompiler Found executable /usr/bin/g77
  gnu: no Fortran 90 compiler found
  gnu: no Fortran 90 compiler found
  customize GnuFCompiler
  gnu: no Fortran 90 compiler found
  gnu: no Fortran 90 compiler found
  customize GnuFCompiler using config
 
  and later:
  customize G95FCompiler Found executable /usr/local/bin/g95
 
  but I still have the same answer when I'm trying to do: f2py -c -m
 hello
  hello.f90
 
  customize GnuFCompiler Found executable /usr/bin/g77
  gnu: no Fortran 90 compiler found gnu: no Fortran 90
  compiler found customize GnuFCompiler
  gnu: no Fortran 90 compiler found
  gnu: no Fortran 90 compiler found
 
 This is a correct message and does not imply that no Fortran 90
 compiler exists.
 The GnuFCompiler (gnu) only looks at g77, which is not a Fortran
 90 compiler.
 That message only applies to the gnu compiler, not any other.
 
 
 I have always found these messages confusing. Perhaps they could be
 better worded to reflect the process, or, even better, silenced. All
 that really matters is success or failure and the reason, not every
 little dead end explored on the way. If they are needed for debugging,
 write the whole mess to a file in the build directory instead of the
 console.

Go for it.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] appending extra items to arrays

[Robert Kern]

Adam Mercer wrote:
 Hi
 
 In some code I have, I need to append some extra data to a given
 array.  At the moment I construct the data in a list, append the extra
 information I need and then convert the final list to an array.  Is
 there a way that I can append extra information to an existing array
 thereby negating the need for the list, as the array object doesn't
 seem to have an append() method?

Appending to a list then converting the list to an array is the most
straightforward way to do it. If the performance of this isn't a problem, I
recommend leaving it alone.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] appending extra items to arrays

[Robert Kern]

Anne Archibald wrote:
 On 11/10/2007, Robert Kern [EMAIL PROTECTED] wrote:
 
 Appending to a list then converting the list to an array is the most
 straightforward way to do it. If the performance of this isn't a problem, I
 recommend leaving it alone.
 
 Just a speculation:
 
 Python strings have a similar problem - they're immutable, and so are
 even more resistant to growth than numpy arrays. For those situations
 where you really really want to grow a srting, python provides
 StringIO, where you keep efficiently adding to the string, then
 finalize it to get the real string out. Would something analogous be
 interesting for arrays?

The Python version of StringIO, at least, just keeps a list, too. The primary
benefit of StringIO is that it exposes the file interface. For building up a
string/array, using the list is just as convenient (and for arrays, probably
more convenient given the variety of shapes one might want to build) and at
least as readable.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] For review: first milestone of scons support in numpy

[Robert Kern]

Pearu Peterson wrote:
 David Cournapeau wrote:
 Pearu Peterson wrote:
 I think this is good.
 Does scons require python-dev? If not then this will solve one of the
 frequent issues that new users may experience: not installed distutils.
 Isn't distutils included in python library ?
 
 Not always. For example, in debian one has to install python-dev 
 separately from python.

You'd still need python-dev(el) for the headers if not distutils.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] .bytes

[Robert Kern]

Yves Revaz wrote:
 Nadav Horesh wrote:
 
 array(1, dtype=float32).itemsize

 ok, it will work fine for my purpose.
 In numpy, is there any reason to supress the attribute .bytes from the 
 type object itself ?
 Is it simply because the native python types (int, float, complex, etc.) 
 do not have this attribute ?

The problem is that the instances of the scalar types do have the itemsize
attribute. The implementation of type objects is such that the type object will
also have that attribute, but it will be a stub:

In [15]: float64.itemsize
Out[15]: attribute 'itemsize' of 'numpy.generic' objects

A more straightforward way to get the itemsize is this:

In [17]: dtype(float64).itemsize
Out[17]: 8

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] A basic question on the dot function

[Robert Kern]

Julien Hillairet wrote:
 Hello,
 
 
 First of all, I'm sorry if this question had already been asked. I've
 searched on the gmane archive and elsewhere on internet, but I didn't
 found the answer to my question.
 
 As expected, the dot product of 2 'classical' vectors works fine :
 
 In [50]: a0 = numpy.array([1,2,3])
 In [51]: numpy.dot(a0,a0)
 Out[51]: 14
 
 What I don't understand, is why the dot product of a (3,N) vector gives
 an error :
 
 In [52]: a1 = numpy.array([[1,2,3],[1,2,3],[1,2,3],[1,2,3]])
 In [54]: numpy.dot(a1,a1)
 type 'exceptions.ValueError'Traceback (most recent call last)
 type 'exceptions.ValueError': objects are not aligned
 
 instead of what I've expected ; an array([14 14 14 14]).

When given two 2-D arrays, dot() essentially does matrix multiplication. The
last dimension of the first argument is matched with the next-to-last dimension
of the second argument.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Sum of the product of three or more arrays

[Robert Kern]

Geoffrey Zhu wrote:
 Hi All,
 
 Given three vectors of the same lengths, X, Y, and Z, I am looking for
 an efficient way to calculate the following:
 
 sum(x[i]*y[i]*z[i], for i=1..n )

(x*y*z).sum()

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] vectorizing loops

[Robert Kern]

Gael Varoquaux wrote:
 On Thu, Oct 25, 2007 at 04:16:06PM -0700, Mathew Yeates wrote:
 Anybody know of any tricks for handling something like
 
 z[0]=1.0
 for i in range(100):
 out[i]=func1(z[i])
 z[i+1]=func2(out[i])
 
 Something like:
 
 z[0] = 1.
 out = func1(z)
 z[1:] = func2(out[:-1])

No, that doesn't work. The way you have it, for each i0,

  z[i] = func2(func1(0))

What Matthew wants is this

  z[0] = 1.0
  z[1] = func2(func1(1.0))
  z[2] = func2(func1(func2(func1(1.0
  ...

I'm afraid that there is no fast, elegant way to do this. If you could turn
func2(func1(x)) into a binary ufunc f(x, y) with an ignored y, then you could
call f.accumulate(z). However, if that ufunc is not implemented in C, there's
not much point.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matrix problem: float to matrix power

[Robert Kern]

Charles R Harris wrote:
 On 10/31/07, Robert Kern [EMAIL PROTECTED] wrote:
 Charles R Harris wrote:
 On 10/31/07, Alan G Isaac [EMAIL PROTECTED] wrote:
 1.0**numpy.array([1,2,3])
 array([ 1.,  1.,  1.])
 1.0**numpy.mat([1,2,3])
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: unsupported operand type(s) for ** or pow(): 'float' and 
 'matrix'

 Why the restriction for matrices?
 Looks like it just got overlooked. It works fine for 2D arrays.

 In [11]: 1.0**eye(3)
 Out[11]:
 array([[ 1.,  1.,  1.],
[ 1.,  1.,  1.],
[ 1.,  1.,  1.]])
 For ndarrays, it does elementwise exponentiation, so it is neither here nor
 there with respect to matrices. What is the definition of a scalar raised to 
 a
 matrix power? I don't ever recall seeing one.
 
 I suppose the usual:
 
 a**m = exp(ln(a)*m)
 
 but that isn't the broadcast version.

Ah, yes. Of course. How silly of me.

That raises the question of how to calculate the bloody thing, though. We have
three implementations of expm() in scipy.linalg.matfuncs.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Problem with numpy on Leopard

[Robert Kern]

Matthieu Brucher wrote:
 Hi,
 
 The problem will arise for every package, not only numpy, so Apple
 fixing this is the best solution IMHO.

It's unlikely they are going to. If they put that stuff there, it's because they
are using it for something, not as an (in)convenience to you. I don't recommend
using the Python.framework in /System for anything except for distributing
lightweight .apps. In that case, you can control your sys.path very well. For
regular work, use the binary from www.python.org.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Assessing the use of packages

[Robert Kern]

Pierre GM wrote:
 All,
 
 It's evaluation time in my department (Bio.  Ag. Engng, UGA), and I'd need 
 to 
 document the impact of my Python contributions on the scientific community at 
 large, or more realistically on the numpy/scipy user community...
 
 * Is there a way to estimate how many people installed one particular package 
 from the SVN ?

No. We could only record who has checked them out from SVN. However, everyone
who has checked out the scipy trunk will have gotten the packages you are
concerned with. Whether or not they've built them or used them is another matter
which we cannot determine.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy 1.04 numpy.test() hang

[Robert Kern]

Geoffrey Zhu wrote:
 Good morning.
 
 I just installed the Windows binary of numpy 1.04. When I ran
 numpy.test() in IDLE (the Python shell that comes with Python), the
 program hang (or at least is running for half an hour). I am using
 Windows XP, duel core intel CPU.
 
 Does anyone know what is going on?

No. Run numpy.test(verbosity=2) so it will print out each test name before
running the test. Then we might have some idea about where the hang is coming 
from.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] weibull distribution has only one parameter?

[Robert Kern]

D.Hendriks (Dennis) wrote:
 According to (for instance) 
 http://en.wikipedia.org/wiki/Weibull_distribution the Weibull 
 distribution has two parameters: lambda  0 is the scale parameter 
 (real) and k  0 is the shape parameter (real). However, the 
 numpy.random.weibull function has only a single 'a' parameter (except 
 for the size parameter which indicates the size of the array to fill 
 with values - this is NOT a parameter of the distribution itself). My 
 question is how this 'a' parameter translates to the Weibull 
 distribution as it 'normally' is and how to sample the distribution when 
 I have the lambda and k parameters?

lambda * numpy.random.weibull(k)

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] weibull distribution has only one parameter?

[Robert Kern]

D.Hendriks (Dennis) wrote:
 Alan G Isaac wrote:
 On Mon, 12 Nov 2007, D.Hendriks (Dennis) apparently wrote: 
   
 All of this makes me doubt the correctness of the formula 
 you proposed. 
 
 It is always a good idea to hesitate before doubting Robert.
 URL:http://en.wikipedia.org/wiki/Weibull_distribution#Generating_Weibull-distributed_random_variates

 hth,
 Alan Isaac
   
 So, you are saying that it was indeed correct? That still leaves the
 question why I can't seem to confirm that in the figure I mentioned (red
 and green lines). Also, if you refer to X = lambda*(-ln(U))^(1/k) as
 'proof' for the validity of the formula, I have to ask if
 Weibull(a,Size) does actually correspond to (-ln(U))^(1/a)?

double rk_standard_exponential(rk_state *state)
{
/* We use -log(1-U) since U is [0, 1) */
return -log(1.0 - rk_double(state));
}

double rk_weibull(rk_state *state, double a)
{
return pow(rk_standard_exponential(state), 1./a);
}

Like Ryan says, multiplying a random deviate by a number is different from
multiplying the PDF by a number. Multiplying the random deviate by lambda is
equivalent to transforming pdf(x) to pdf(x/lambda) not lambda*pdf(x).

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposing Ubiquity

[Robert Kern]

dieter h. wrote:
 Hi all,
 
 Would it make sense for all functionality in Numpy/Scipy to have
 ubiquitous returns? In that I'm proposing that every func/method
 (where appropriate) have a flag in its arg list establishing a return
 for either [INDEX, BOOLEAN, VALUE=default]. this could be by a
 singular Enum flag, I believe. Obvioulsy, this proposal gives the
 option for a return of Index/Bool to be used for Indexing arrays, or
 the actual values. Since I mosty index existing arrays, this would be
 a huge plus for me.

I'm afraid that I don't understand this proposal. Can you show some examples of
what you would like to see? Preferably, can you write some working code that
demonstrates this feature?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Should array iterate over a set?

[Robert Kern]

Michael McNeil Forbes wrote:
 My expectation was that array would iterate over a set.  This is  
 incorrect:
 
   array(set([1,2,3]))
 array(set([1, 2, 3]), dtype=object)
 
 Is this the intended behaviour?  A trivial work-around that does what  
 I need is
 
   array(list(set([1,2,3])))
 array([1, 2, 3])
 
 but I was wondering if this was by design or just a forgotten  
 corner.  (Maybe a vestige of the tuple special case for record arrays?)

We can recognize most sequences (i.e. for all i in range(len(x)), x[i] responds
correctly), but we cannot easily deal with arbitrary iterables which are not
sequences in the array() function. There are a lot of special cases and
heuristics going on in array() as it is. Instead, we have a fromiter() function
which will take an iterable and construct an array from it. It is limited to 1D
arrays, but this is by far the most common use for constructing an array from an
iterable.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Building NumPy on Mac OS X without Apple GCC

[Robert Kern]

Joshua Lippai wrote:
 I updated my GCC to a more recent version a day ago, since Apple's
 Xcode Tools only provide GCC 4.0 and the current release of GNU's GCC
 is 4.2. I successfully achieved this, but now I run into a problem
 when trying to build NumPy:
 
 gcc: unrecognized option '-no-cpp-precomp'
 cc1: error: unrecognized command line option -arch
 cc1: error: unrecognized command line option -arch
 cc1: error: unrecognized command line option -Wno-long-double
 gcc: unrecognized option '-no-cpp-precomp'
 cc1: error: unrecognized command line option -arch
 cc1: error: unrecognized command line option -arch
 cc1: error: unrecognized command line option -Wno-long-double
 
 
 Upon investigation into the matter, I found out that these options
 (no-cpp-precomp and Wno-long-double) are only valid in Apple's GCC and
 not the regular GNU release. Yet it seems NumPy automatically assumes
 Apple's GCC is being used when it realizes the target is OS X. Is
 there a way around this, or at least some way to specify Apple's GCC?
 NumPy is the only package I've tried building so far that has a
 problem with this.

I'm surprised that you've built other Python extension modules because numpy
does not add these flags; Python does. Python extensions should be built with
the same compiler that Python itself was built with. If you are using the binary
distribution from www.python.org, you should use Apple's gcc, not a different 
one.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Building NumPy on Mac OS X without Apple GCC

[Robert Kern]

Joshua Lippai wrote:

 Thanks for the reply. Well, I built my Python stuff, including NumPy
 previously, before I changed to the higher GCC version. Do you know if
 there's an option I can toggle that will specify Apple's GCC to be
 used?

$ CC=/usr/bin/gcc python setup.py build

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] display numpy array as image

[Robert Kern]

Zachary Pincus wrote:
 Hello all,
 
 I'm curious if people have experience with / preferences for how to  
 display a numpy array onscreen as an image.
 
 Pyglet looks relatively easy -- you can feed an image buffer object  
 with a string or a ctypes pointer. I presume getting a string from an  
 array is plenty fast, but the ctypes pointer option is intriguing as  
 it allows for dealing with simple strided arrays (the image objects  
 allow for an arbitrary number of bytes between rows). Is it possible  
 to get a ctypes pointer to the beginning of the array buffer from  
 numpy without too much ugliness?

In [16]: from numpy import *

In [17]: a = arange(10)

In [18]: dir(a.ctypes)
Out[18]:
['__class__',
 '__delattr__',
 '__dict__',
 '__doc__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__module__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__str__',
 '__weakref__',
 '_arr',
 '_as_parameter_',
 '_ctypes',
 '_data',
 '_zerod',
 'data',
 'data_as',
 'get_as_parameter',
 'get_data',
 'get_shape',
 'get_strides',
 'shape',
 'shape_as',
 'strides',
 'strides_as']

In [22]: import ctypes

In [24]: a.ctypes.data_as(ctypes.POINTER(ctypes.c_long))
Out[24]: ctypes.LP_c_long object at 0x1b353a0

In [25]: a.ctypes.get_shape()
Out[25]: numpy.core._internal.c_long_Array_1 object at 0x1c096c0

In [26]: a.ctypes.get_strides()
Out[26]: numpy.core._internal.c_long_Array_1 object at 0x1c09710

In [27]: a.ctypes.get_as_parameter()
Out[27]: c_void_p(27442576)


You might want to use the new ctypes-based OpenGL 3.0+ package. It has numpy
support a bit more directly. You can use Pyglet for its windowing and all of the
other surrounding infrastructure and use OpenGL directly for the drawing.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

[Robert Kern]

Barry Wark wrote:

 Some remaining issues:
 - which SDK to build against. Leopard ships with a Python build
 against the 10.5 SDK. It would be much easier, at least initially, for
 us to produce binaries against the Leopard Python 2.5.

I would prefer that we use the Python binary from www.python.org. That should
work on 10.3.9+.

 - how to deal with library dependencies such as fftw2. We currently
 use MacPorts but I suppose it needs to be compiled statically or do we
 just require that users install MacPort's fftw2?

Official binaries intended for distribution from scipy.org or scipy.sf.net
should not be linked against FFTW or UMFPACK since they are GPLed.

The real problem is the Fortran runtime. It is possible to link against the
Fortran runtime libraries statically so as to avoid needing the user to also
install a Fortran compiler just to run scipy. Use the gfortran compiler built
for R's use (and do not use the one from hpc.sf.net):

  http://r.research.att.com/tools/

Basically, copy the libgfortran.a library to a separate location, say
~/staticlibs/. Then do the following:

  $ export LDFLAGS=-undefined dynamic_lookup -bundle -arch i386 -arch ppc
-Wl,-search_paths_first
  $ python setup.py config_fc --fcompiler=gnu95 --arch=-arch i386 -arch ppc
build_ext -L ~/staticlibs/ build

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

[Robert Kern]

Christopher Barker wrote:
 Robert Kern wrote:

 Official binaries intended for distribution from scipy.org or scipy.sf.net
 should not be linked against FFTW or UMFPACK since they are GPLed.
 
 Does that apply to binaries put up on pythonmac.org? It would be nice to 
 have a complete version somewhere standard.

I would argue yes, it does apply to pythonmac.org, too. scipy is a
BSD-licensed package. Putting GPLed components into the binaries which are
distributed at standard locations like our website, PyPI, or pythonmac.org
confuses that picture. I would be mollified if the download link had an
explanation attached, but unfortunately, pythonmac.org/packages does not have
any space for that.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] random.hypergeometric bug

[Robert Kern]

Chris wrote:
 There appears to be a bug in numpy's hypergeometric
 random number generator. Here is an example -- if I 
 generate 1000 hg samples with 4 draws from a space
 with 30 successes and 10 failures:
 
 In [39]: x = hg(30, 10, 4, 1000)
 
 I should get a mean value of:
 
 In [40]: 4*30./40
 Out[40]: 3.0
 
 But the sample mean is way to small:
 
 In [41]: mean(x)
 Out[41]: 0.996

Fixed in r4527. My original source for the algorithm was incorrect, it seems.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:
 Some of the most interesting things I can think of which work with scons:
 - you can control fortran and C flags from the command line: CFLAGS 
 and FFLAGS won't override necessary flags, only optimization flags, so 
 you can easily play with warning, optimization flags. For example:
 
 CFLAGS='-W -Wall -Wextra -DDEBUG' FFLAGS='-DDEBUG -W -Wall -Wextra' 
 python setupscons build
 
 for debugging will work. No need to care about -fPIC and co, all this is 
 handled automatically.

Can I override the flags which are handled automatically without modifying
numpy? I just spent much of last night trying to get Intel Fortran on OS X
working, and I had to dive into numpy.distutils.fcompiler.intel a lot. This is
mildly acceptable, if irritating, for a numpy developer, but not acceptable for
even a sophisticated user. Even if we were to keep our knowledge-base of Fortran
compiler flags immediately up-to-date with every release of every Fortran
compiler we follow, people will still be stuck with older versions of numpy.

numpy.distutils' behavior of using LDFLAGS, for example, to completely replace
the flags instead of extending them mitigated this, somewhat. It allowed someone
to work around the stale flags in numpy.distutils in order to get something
built. This is a hack, and it causes confusion when this isn't the desired
behavior, but it worked.

But can we do something better with scons? One option which would work both with
scons and the current numpy.distutils is to provide something like
LDFLAGS_NO_REALLY which replaces the flags and let LDFLAGS just extend the
flags. That doesn't help the ultimate problem, but it makes the workaround more
user-friendly. Another option is to have our Fortran compiler knowledge-base
separable from the rest of the package. scons could try to import them from,
say, numpy_fcompilers first and then look inside numpy.distutils if
numpy_fcompilers is not found. That way, a user could download a fresh
knowledge-base into their source tree (and possibly tweak it) without the
burden of installing a new numpy.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

Fernando Perez wrote:
 On Dec 4, 2007 12:27 PM, Robert Kern [EMAIL PROTECTED] wrote:
 
 user-friendly. Another option is to have our Fortran compiler 
 knowledge-base
 separable from the rest of the package. scons could try to import them from,
 say, numpy_fcompilers first and then look inside numpy.distutils if
 numpy_fcompilers is not found. That way, a user could download a fresh
 knowledge-base into their source tree (and possibly tweak it) without the
 burden of installing a new numpy.
 
 Is this something that really needs to be a code package?  Why can't
 this knowledge (or at least the easily overridable part of it) be
 packaged in one or more .conf/.ini plaintext files?  In that way,
 users could easily grab new data files or tweak the builtin ones, and
 at build time say
 
 setup.py install  --compiler_conf=~/my_tweaked.conf
 
 Is that impossible/unreasonable for some reason?

It's not impossible, but there are at least a couple of places where it might be
unreasonable. For example, look at the get_flags_arch() for Intel compilers:

def get_flags_arch(self):
v = self.get_version()
opt = []
if cpu.has_fdiv_bug():
opt.append('-fdiv_check')
if cpu.has_f00f_bug():
opt.append('-0f_check')
if cpu.is_PentiumPro() or cpu.is_PentiumII() or cpu.is_PentiumIII():
opt.extend(['-tpp6'])
elif cpu.is_PentiumM():
opt.extend(['-tpp7','-xB'])
elif cpu.is_Pentium():
opt.append('-tpp5')
elif cpu.is_PentiumIV() or cpu.is_Xeon():
opt.extend(['-tpp7','-xW'])
if v and v = '7.1':
if cpu.has_mmx() and (cpu.is_PentiumII() or cpu.is_PentiumIII()):
opt.append('-xM')
elif v and v = '8.0':
if cpu.is_PentiumIII():
opt.append('-xK')
if cpu.has_sse3():
opt.extend(['-xP'])
elif cpu.is_PentiumIV():
opt.append('-xW')
if cpu.has_sse2():
opt.append('-xN')
elif cpu.is_PentiumM():
opt.extend(['-xB'])
if (cpu.is_Xeon() or cpu.is_Core2() or cpu.is_Core2Extreme()) and
cpu.getNCPUs()==2:
opt.extend(['-xT'])
if cpu.has_sse3() and (cpu.is_PentiumIV() or cpu.is_CoreDuo() or
cpu.is_CoreSolo()):
opt.extend(['-xP'])

if cpu.has_sse2():
opt.append('-arch SSE2')
elif cpu.has_sse():
opt.append('-arch SSE')
return opt


Expressing that without code could be hairy.

That said, using configuration files as override mechanisms for each of the
get_flags_*() methods would be feasible especially if there were a script to
dump the current flag set to the configuration file.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

Fernando Perez wrote:
 On Dec 4, 2007 1:24 PM, Robert Kern [EMAIL PROTECTED] wrote:
 Fernando Perez wrote:
 
 Is this something that really needs to be a code package?  Why can't
 this knowledge (or at least the easily overridable part of it) be
 packaged in one or more .conf/.ini plaintext files?  In that way,
 users could easily grab new data files or tweak the builtin ones, and
 at build time say

 setup.py install  --compiler_conf=~/my_tweaked.conf

 Is that impossible/unreasonable for some reason?
 It's not impossible, but there are at least a couple of places where it 
 might be
 unreasonable. For example, look at the get_flags_arch() for Intel compilers:
 
 [...]
 
 I see.  How about an alternate approach: exposing a simple api and
 allowing users to declare a *python* file to execfile() at load time
 looking for the config?
 
 Something like:
 
 setup.py install  --compiler_conf=~/my_tweaked_config.py
 
 where the config file would be (sketch, not real code here):
 
 def make_flags(compiler, etc...):
   flags = []
   
  return flags
 
 There could be a simple API for what functions the config file (down
 to their names and signatures) can declare, and if any of them are
 declared, they get called and their output is used.  They get fed the
 default state of the same variables, so that they can choose to modify
 or outright replace them based on the user's need.
 
 The config code would then
 
 user_ns = {}
 execfile(user_config_filename,user_ns)
 for name,val in user_ns.items():
   if name in approved_functions and callable(val):
 flags[name] = val(*approved_functions[name].default_args)
 
 
 
 What say you?

Well, like I said, for tweaking, a simple data file works better than code.
There's no need to do all of those if tests since I know what platform I'm on.
We should definitely have a simple data file that we can read flags from. It's
just the general case that requires code.

One thing in favor of numpy_fcompilers is that we can ship the updates to the
general case more frequently. This means that other packages using Fortran (but
not tied to a particular platform) can ship the updated code instead of telling
all of their users to read their Fortran compiler manuals or ask on the mailing
list for the correct settings to work around our old defects.

Tweaking should be the province of the developer and the desperate. We have to
be able to tweak, but we should spend more time on preventing the need for 
tweaking.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:

 - I have not yet tweaked fortran compiler configurations for 
 optimizations except for gnu compilers

Can you give us a brief overview about how to do this? For example, the Intel
Fortran compiler's SHLINKFLAGS in scons-local/.../SCons/Tool/ifort.py are
incorrect for version 10 on OS X. Would I copy that file to scons/tool/ and make
my edits there? Do I then add 'ifort' to the list in scons/core/default.py?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:

 So to go back to your problem: if I understand correctly, what is needed 
 is to update the scons tools. Since those are kept at one place, I think 
 it would be safe to update them independently. But I don't understand 
 exactly how this could be installed in the source tree without 
 reinstalling numpy ? I think this is better than completely overriding 
 the compilation flags, personally. But if you really want this 
 possibility, I can add it, too, without too much trouble.

I don't think you could install it into an already installed numpy package. My
suggestion is to keep the implementations of the tools inside the numpy package
as they are now *except* that we look for another package first before going
inside numpy.distutils.scons.tools . I called it numpy_fcompilers though I now
suspect numpy_scons_tools might be more appropriate. If the package
numpy_scons_tools doesn't exist, the implementations inside
numpy.distutils.scons.tools are used. A variation on this would be to provide an
--option to the scons command to provide a package path to look for tools. 
E.g.

  python setup.py scons
--tool-path=my_local_fcompilers,site_fcompilers,numpy.distutils.scons.tools

This, too, is a workaround for the less-than-desirable situation of having
numpy's sizable build infrastructure bundled with the numpy package itself. If
this stuff were an entirely separate package focused on providing this
scons-based build infrastructure, then we wouldn't have a problem. We could just
update it on its own release schedule. People would probably be more willing to
use the development versions of it, too, instead of having to also buy into the
development version of their array package as well.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:
 Robert Kern wrote:
 David Cournapeau wrote:

 - I have not yet tweaked fortran compiler configurations for 
 optimizations except for gnu compilers
 Can you give us a brief overview about how to do this? For example, the Intel
 Fortran compiler's SHLINKFLAGS in scons-local/.../SCons/Tool/ifort.py are
 incorrect for version 10 on OS X. Would I copy that file to scons/tool/ and 
 make
 my edits there? Do I then add 'ifort' to the list in scons/core/default.py?

 The basic rule is: if the code cannot run without a flag, the flag 
 should be put in a tool, or at worse (but really if you have no choice) 
 in numpyenv.py. If the flag is optimization, warning, etc... then it 
 should be put into default.py. Basically, tools are not always 
 up-to-date in scons, perticularly for fortran. So I provided a way to 
 override the tools: as you noticed, you can put tools in 
 .../scons/tools/, those will be picked up first. This is independent 
 from adding ifort in scons/core/default.py.

Right. In this case, -shared needs to be -dynamiclib on OS X, so this should
go into the tool.

 For Mac OS X, you may be bitten by  -undefined dynamic_lookup. This is 
 my fault: this flag is added at the wrong place, I put it temporarily in 
 the python extension builder, but this is not where it should be put. 
 Depending on its meaning, I can put it at the right place: does it give 
 the traditional unix semantic of enabling unresolved symbols instead of 
 the default one, which is similar to windows (even for shared code, 
 every symbol must be resolved) ?

That's the basic idea. Rigorously, it's probably a bit more involved when you
start considering two-level namespaces and framework.

One thing to note is that this option is only valid for GNU compilers. Linking
with ifort, I need to use -Wl,-undefined,dynamic_lookup .

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:
 On Dec 5, 2007 1:14 PM, Robert Kern [EMAIL PROTECTED] wrote:

 This, too, is a workaround for the less-than-desirable situation of having
 numpy's sizable build infrastructure bundled with the numpy package itself. 
 If
 this stuff were an entirely separate package focused on providing this
 scons-based build infrastructure, then we wouldn't have a problem. We could 
 just
 update it on its own release schedule. People would probably be more willing 
 to
 use the development versions of it, too, instead of having to also buy into 
 the
 development version of their array package as well.
 
 The only problem I see is that this increases the chance of losing
 synchronization. I don't know if this is significant.

The problem I see is that numpy-the-array-library and
numpy.distutils-the-build-infrastructure are two related packages with
*over*-synchronized cycles. We aren't going to push out a micro-release of
numpy-the-array-library just because a new version of Intel Fortran comes out
and changes its flags.

 IMHO, the only
 real solution would be to fix distutils (how many people want a shared
 library builder in python community, for example ?), but well, it is
 not gonna happen in a foreseeable future, unfortunately

I don't see how that's relevant to the problem I raised. Supporting Fortran in
the standard library would make the problem even worse. distutils is certainly
not broken because it doesn't support Fortran.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:
 On Dec 5, 2007 1:19 PM, Robert Kern [EMAIL PROTECTED] wrote:
 David Cournapeau wrote:
 Robert Kern wrote:
 David Cournapeau wrote:

 - I have not yet tweaked fortran compiler configurations for
 optimizations except for gnu compilers
 Can you give us a brief overview about how to do this? For example, the 
 Intel
 Fortran compiler's SHLINKFLAGS in scons-local/.../SCons/Tool/ifort.py are
 incorrect for version 10 on OS X. Would I copy that file to scons/tool/ 
 and make
 my edits there? Do I then add 'ifort' to the list in scons/core/default.py?

 The basic rule is: if the code cannot run without a flag, the flag
 should be put in a tool, or at worse (but really if you have no choice)
 in numpyenv.py. If the flag is optimization, warning, etc... then it
 should be put into default.py. Basically, tools are not always
 up-to-date in scons, perticularly for fortran. So I provided a way to
 override the tools: as you noticed, you can put tools in
 .../scons/tools/, those will be picked up first. This is independent
 from adding ifort in scons/core/default.py.
 Right. In this case, -shared needs to be -dynamiclib on OS X, so this 
 should
 go into the tool.
 That's strange: -shared should not be used at all on mac os X. Either
 -bundle or -dynamiclib should be used (this is in applelink tool, so
 this is independant from the compiler used, normally).

I was only reading code; I haven't tested building Fortran extensions, yet.
However, using a generic link tool would be the wrong thing to do for most
Fortran extensions, I think. Where does it get the correct Fortran runtime
libraries from? Some Fortran compilers really like to be the linker when mixing
languages.

 But I may have done something wrong, because I don't know much about
 mac os X idiosyncraties on this: basically, what's the difference
 between -dynamiclib and -bundle ?
 
 When I build python extension, I used the module scons builder, which
 is the same than shared library except on mac os X (on mac os X,
 shared libraries use -dynamiclib, modules use -bundle). I must confess
 that I used the thing which worked in thise case, without knowing
 exactly what i was doing.

ifort only supports -dynamiclib. For the regular linker, -bundle is correct for
building Python extensions; I may have to rethink about using ifort to link,
then. Basically, a bundle can be dynamically loaded while dylibs can't, so
Python uses bundles for extension modules.

  http://www.finkproject.org/doc/porting/shared.php

What confuses me is that I successfully built some Fortran modules last night
using numpy.distutils and ifort -dynamiclib. Hmm.

 One thing to note is that this option is only valid for GNU compilers. 
 Linking
 with ifort, I need to use -Wl,-undefined,dynamic_lookup .
 Can't we just add a linker flag instead of using it from the compiler
 ? We still use the apple linker with ifort/icc, no ?

I don't know. We'd have to locate all of the Fortran runtime libraries and add
them. How do we do that? Or is that already done?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] scipy.scons branch: building numpy and scipy with scons

[Robert Kern]

David Cournapeau wrote:
 On Dec 5, 2007 3:07 PM, Robert Kern [EMAIL PROTECTED] wrote:

 I don't see how that's relevant to the problem I raised. Supporting Fortran 
 in
 the standard library would make the problem even worse. distutils is 
 certainly
 not broken because it doesn't support Fortran.

 Fortran support is indeed certainly not in the scope of distutils. I
 was just answering to the general problem that we have a huge build
 infrastructure, not to the particular fortran problem. Having an
 infrastructure for adding easily new tools, that can be distributed
 separately, is something that distutils severely lacks IMHO.

Ah, yes. I agree.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] site down...

[Robert Kern]

Fernando Perez wrote:
 The whole scipy.org site seems down.  Is it just on my end?

No. Our new IT guy, Ryan Earl jre at enthought.com, is on the case.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] site down...

[Robert Kern]

Steven H. Rogers wrote:
 Fernando Perez wrote:
 The whole scipy.org site seems down.  Is it just on my end?
   
 Works for me, though it seems pretty slow.

The system has been restarted.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] multinomial question

[Robert Kern]

Alan G Isaac wrote:
 I would think that
 multinomial(1,prob,size=ntrials).sum(axis=0)
 would be equivalent to
 multinomial(ntrials,prob)
 but the first gives a surprising result.  (See below.)
 Explanation?

A bug in rk_binomial_inversion(). Unfortunately, this looks like a logical bug
in the sources I was deriving this code from. The safety bound on the search
inversion search cuts out too early. Now that I re-examine it, it looks the
bound (whichever of the multiple choice of bounds one could use) could always be
legitimately exceeded, so there shouldn't be a bound at all. I'll have to dive
deeper to figure out what is going on. This makes me grumpy.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] multinomial question

[Robert Kern]

Alan G Isaac wrote:
 I would think that
 multinomial(1,prob,size=ntrials).sum(axis=0)
 would be equivalent to
 multinomial(ntrials,prob)
 but the first gives a surprising result.  (See below.)
 Explanation?

Pretty much anyone who derives their binomial distribution algorithm from
http://www.unc.edu/~gfish/fcmc.html is also wrong.

SVN now has a bound such that CDF(bound) is within 1e-16 (or so) of 1.0.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Max on Complex Array

[Robert Kern]

Tom Johnson wrote:
 What is the meaning of numpy's maximum on a complex-valued array?

We impose a lexicographical ordering on the complex space. Complex numbers are
compared first by their real component and then, if the real components are
equal, by their imaginary component.


In [1]: from numpy import *

In [19]: c = random.randint(0, 5, 10) + random.random(10)*1j

In [20]: c
Out[20]:
array([ 0.+0.68448275j,  1.+0.97849291j,  3.+0.22114928j,  4.+0.65409519j,
3.+0.91550523j,  4.+0.50667105j,  1.+0.34576644j,  4.+0.97286048j,
1.+0.07268317j,  0.+0.52885086j])

In [21]: c.sort()

In [22]: c
Out[22]:
array([ 0.+0.52885086j,  0.+0.68448275j,  1.+0.07268317j,  1.+0.34576644j,
1.+0.97849291j,  3.+0.22114928j,  3.+0.91550523j,  4.+0.50667105j,
4.+0.65409519j,  4.+0.97286048j])

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Changing the distributed binary for numpy 1.0.4 for windows ?

[Robert Kern]

David M. Cooke wrote:
 On Dec 10, 2007, at 10:30 , Matthieu Brucher wrote:
 2007/12/10, Alexander Michael [EMAIL PROTECTED]: On Dec 10, 2007  
 6:48 AM, David Cournapeau [EMAIL PROTECTED] wrote:
 Hi,

 Several people reported problems with numpy 1.0.4 (See #627 and
 #628, but also other problems mentionned on the ML, which I cannot
 find). They were all solved, as far as I know, by a binary I  
 produced
 (simply using mingw + netlib BLAS/LAPACK,  no ATLAS). Maybe it  
 would be
 good to use those instead ? (I can recompile them if there is a  
 special
 thing to do to build them)
 Do I understand correctly that you are suggesting removing ATLAS from
 the Windows distribution? Wouldn't this make numpy very slow? I know
 on RHEL5 I see a very large improvement between the basic BLAS/LAPACK
 and ATLAS. Perhaps we should make an alternative Windows binary
 available without ATLAS just for those having problems with ATLAS?
 That's why David proposed the netlib version of BLAS/LAPACK and not  
 the default implementation in numpy.

 I would agree with David ;)
 
 Our versions of BLAS/LAPACK are f2c'd versions of the netlib 3.0 BLAS/ 
 LAPACK (actually, of Debian's version of these -- they include several  
 fixes that weren't upstream).
 
 So netlib's versions aren't going to be any faster, really. And  
 netlib's BLAS is slow. Now, if there is a BLAS that's easier to  
 compile than ATLAS on windows, that'd be improvement.

The current situation is untenable. I will gladly accept a slow BLAS for an
official binary that won't segfault anywhere. We can look for a faster BLAS 
later.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Traceback on divide by zero error

[Robert Kern]

Tony S Yu wrote:
 Hello,
 
 This is something that's been bothering for awhile. When numpy raises
 the following divide by zero error:
 Warning: divide by zero encountered in double_scalars
 is there a way to get a Traceback on where that warning occurred.

In [1]: from numpy import *

In [2]: seterr?
Type: function
Base Class:   type 'function'
Namespace:Interactive
File: /Users/rkern/svn/numpy/numpy/core/numeric.py
Definition:   seterr(all=None, divide=None, over=None, under=None, 
invalid=None)
Docstring:
Set how floating-point errors are handled.

Valid values for each type of error are the strings
ignore, warn, raise, and call. Returns the old settings.
If 'all' is specified, values that are not otherwise specified
will be set to 'all', otherwise they will retain their old
values.

Note that operations on integer scalar types (such as int16) are
handled like floating point, and are affected by these settings.

Example:

 seterr(over='raise') # doctest: +SKIP
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 
'ignore'}

 seterr(all='warn', over='raise') # doctest: +SKIP
{'over': 'raise', 'divide': 'ignore', 'invalid': 'ignore', 'under': 
'ignore'}

 int16(32000) * int16(3) # doctest: +SKIP
Traceback (most recent call last):
  File stdin, line 1, in ?
FloatingPointError: overflow encountered in short_scalars
 seterr(all='ignore') # doctest: +SKIP
{'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 
'ignore'}

In [3]: seterr(divide='raise')
Out[3]: {'divide': 'print', 'invalid': 'print', 'over': 'print', 'under': 
'ignore'}

In [4]: ones(10) / zeros(10)
---
FloatingPointErrorTraceback (most recent call last)

/Users/rkern/svn/mpl-toolkits/ipython console in module()

FloatingPointError: divide by zero encountered in divide

In [5]: seterr(all='ignore')
Out[5]: {'divide': 'raise', 'invalid': 'print', 'over': 'print', 'under': 
'ignore'}

In [6]: ones(10) / zeros(10)
Out[6]: array([ Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf])


-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Overloading sqrt(5.5)*myvector

[Robert Kern]

Gary Ruben wrote:
 Sorry this isn't an answer, just noise, but for those here who don't 
 know, Bruce is the chief maintainer of the vpython project. I have found 
 vpython aka the visual module to be a highly attractive and useful 
 module for teaching physics. It would be great if someone with Boost 
 experience would try to help him out. I wouldn't want him to get falsely 
 disillusioned with this list as I for one have been looking forward to a 
 fully numpy-compatible version of vpython.

Please keep in mind that for many of us, this is the holiday season and we are
on vacation. While I'm happy to check the list and give answers that are at the
front of my head, deeper answers that require exploration or experimentation are
beyond my available time. I'm sure others are in a similar situation.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-dev] Doc-day

[Robert Kern]

Matthieu Brucher wrote:
 Matthew B. will be working on converting SciPy tests to use nose per
 Fernando's email.  If you are familiar with nose and want to help,
 please make sure to check with Matthew or Fernando first.
 
 
 I must have missed Fernando's email because I can't find the references
 for nose :(

Look in SciPy Sprint Results. It's only a brief mention, though.

 What are its advantages against the current numpy.testing framework ?

Primarily:

* It is supported by someone else and gaining wide adoption by the rest of the
Python community.

Secondarily:

* More flexible organization of tests. For nose, if it looks like a test, it's a
test. numpy.testing collects test modules which are named like the module it is
testing. E.g. for module.py = tests/test_module.py.

* Test generators:

  def test_evens():
for i in range(0, 5):
  yield check_even, i, i*3

  def check_even(n, nn):
assert n % 2 == 0 or nn % 2 == 0

* Package- and module-level setup() and teardown() functions.

* Test functions can be simple functions. They do not need to be organized into
classes if you don't need classes.

* Integrated doctest collection.

* Detailed error/failure reporting. nose can print out the values of variables
at the location of the error.

* Integrated code coverage and profiling.

* Dropping into pdb on errors and failures.

* More flexible running of specific tests. E.g. when I'm working on getting a
particular test function running, I can specify that exact test and not run the
rest of the test suite.

* Output capture. Tests can print out anything they like to be more informative,
but they won't appear unless if the test fails.

More thoroughly:

  http://somethingaboutorange.com/mrl/projects/nose/

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] unexpected behavior with allclose( scalar, empty array)

[Robert Kern]

Andrew Straw wrote:
 Apologies if I've missed the discussion of this, but I was recently
 surprised by the following behavior (in svn trunk 4673). The following
 code runs without triggering the assertion.
 
 import numpy as np
 print np.__version__
 a=np.int32(42)
 b=np.array([],dtype=np.int32)
 assert np.allclose(a,b)
 
 Is this expected behavior of numpy or is this a bug I should report?

Bug, I think.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] unexpected behavior with allclose( scalar, empty array)

[Robert Kern]

Matthew Brett wrote:
 So, currently we have all and allclose giving the same answer:
 
 In [19]: a = array([])
 
 In [20]: b = array([1])
 
 In [21]: all(a == b)
 Out[21]: True
 
 In [22]: allclose(a, b)
 Out[22]: True
 
 Would we want the answers to be different?

No. I wasn't thinking correctly, previously.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy code coverage

[Robert Kern]

Stefan van der Walt wrote:
 Hi all,
 
 I read about Titus Brown's Figleaf code coverage tool [1] on the
 Planet SciPy aggregator [2].  The results of running figleaf on the
 numpy test-suite [3] covers Python code only.
 
 What the best way of discovering the C and C++ code coverage as well?

I've never encountered any documents about checking C code coverage of Python
extension modules. It is possible that the standard tools for C code coverage
(e.g. gcov) would work fine with some finagling.

If you try this, I would love to hear what you did.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] weird indexing

[Robert Kern]

Mathew Yeates wrote:
 Hi
 Okay, here's a weird one. In Fortran you can specify the upper/lower 
 bounds of an array
 e.g. REAL A(3:7)
 
 What would be the best way to translate this to a Numpy array? I would 
 like to do something like
 A=numpy.zeros(shape=(5,))
 and have the expression A[3] actually return A[0].
 
 Or something. Any thoughts?

The best way would depend on what you want to do with it. Why do you need this?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_FromAny does not accept a generator?

[Robert Kern]

Neal Becker wrote:
 It seems that PyArray_FromAny does not accept a generator?
 
 Seems like this would be useful.

It's difficult to do all the magical interpretation that PyArray_FromAny() does
with a iterator of unknown length. In Python, we have fromiter() which will
consume an iterator to make (only) a rank-1 array and thus sidestep a big chunk
of the magical interpretation.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Nasty bug using pre-initialized arrays

[Robert Kern]

Stuart Brorson wrote:
 On Fri, 4 Jan 2008, Stuart Brorson wrote:
 
 I just discovered this today.  It looks like a bug to me.  Please
 flame me mercilessly if I am wrong!  :-)
 
 FWIW, here's what Matlab does:
 
 A = rand(1, 4) + rand(1, 4)*i
 
 A =
 
Columns 1 through 3
 
 0.7833 + 0.7942i   0.6808 + 0.0592i   0.4611 + 0.6029i
 
Column 4
 
 0.5678 + 0.0503i
 
 B = zeros(1, 4)
 
 B =
 
   0 0 0 0
 
 for idx=1:4; B(idx) = A(idx); end
 B
 
 B =
 
Columns 1 through 3
 
 0.7833 + 0.7942i   0.6808 + 0.0592i   0.4611 + 0.6029i
 
Column 4
 
 0.5678 + 0.0503i
 
 
 I realize NumPy != Matlab, but I'd wager that most users would think
 that this is the natural behavior..

Well, that behavior won't happen. We won't mutate the dtype of the array because
of assignment. Matlab has copy(-on-write) semantics for things like slices while
we have view semantics. We can't safely do the reallocation of memory [1].

[1] Well, we do have a .resize() method which will do the reallocation and raise
an exception if there are views lingering about. However, this is only done when
explicitly asked for because this is a feature that is useful in a limited
number of situations. We will not allow it to be done implicitly.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

Charles R Harris wrote:
 On Jan 4, 2008 2:05 PM, David Cournapeau [EMAIL PROTECTED]
 mailto:[EMAIL PROTECTED] wrote:

 bzr is not tied to linux.
 
 It is, in that development is funded by Canonical, but I haven't used
 either on windows, so don't have any idea how they compare in that regard.

In that sense, it's all pretty much a wash between the three. Selenic initially
developed Mercurial in the aftermath of the Linux kernel Bitkeeper foofoorah,
and they continue to use it to manage their kernel modules.

If we want to talk about Windows support, we should stick to more concrete facts
 (like the availability of Windows shell integration, etc.) instead of vague
inferences.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Nasty bug using pre-initialized arrays

[Robert Kern]

Zachary Pincus wrote:
 Hello all,
 
 That's well and good.  But NumPy should *never* automatically -- and
 silently -- chop the imaginary part off your complex array elements,
 particularly if you are just doing an innocent assignment!
 Doing something drastic like silently throwing half your data away can
 lead to all kinds of bugs in code written by somebody who is unaware
 of this behavior (i.e. most people)!

 It sounds to me like the right thing is to throw an exception instead
 of downcasting a data object.
 
 I'm not sure that I agree! I'd rather not have to litter my code with  
 casting operations every time I wanted to down-convert data types  
 (creating yet more temporary arrays...) via assignment. e.g.:
 
 A[i] = calculate(B).astype(A.dtype)
 vs.
 A[i] = calculate(B)
 
 Further, writing functions to operate on generic user-provided output  
 arrays (or arrays of user-provided dtype; both of which are common  
 e.g. in scipy.ndimage) becomes more bug-prone, as every assignment  
 would need to be modified as above.
 
 This change would break a lot of the image-processing code I've  
 written (where one often does computations in float or even double,  
 and then re-scales and down-converts the result to integer for  
 display), for example.
 
 I guess that this could be rolled in via the geterr/seterr mechanism,  
 and could by default just print warnings. I agree that silent  
 truncation can be troublesome, but not having to spell every  
 conversion out in explicit ( and error-prone) detail is also pretty  
 useful. (And arguably more pythonic.)

There's some confusion in the conversation here. Tim already identified it, but
since it's continuing, I'm going to reiterate.

There are two related hierarchies of datatypes: different kinds (integer,
floating point, complex floating point) and different precisions within a given
kind (int8, int16, int32, int64). The term downcasting should probably be
reserved for the latter only.

It seems to me that Zach and Scott are defending downcasting of precisions
within a given kind. It does not necessarily follow that the behavior we choose
for dtypes within a given kind should be the behavior when we are dealing with
dtypes across different kinds. We can keep the precision downcasting behavior
that you want while raising an error when one attempts to assign a complex
number into a floating point array.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

Chris Barker wrote:
 hmmm. Everyone posting so far seems to be positive on this idea, but I'm 
 not so sure. A few thoughts:
 
 1) change is bad. It may be worth it, but this decision needs to be made 
 very differently than if we were starting from scratch.
 
 2) apparently svn merge sucks compared  to other merge technology. svn 
 (and cvs before it) is the only system I'm used, and yes, merging is 
 painful, but I have to say that it appeared to be painful because it's a 
 very hard problem. Can anyone comment on why these other systems seem so 
 much better? Does it have anything to do with Centralized vs. 
 Distributed at all?

Tangentially, yes. DVCSes need to keep track of more information in order to be
distributed. That information is extremely useful for managing merges properly.
Centralized systems could track this information, but they don't *need* to in
order to be functional, so they mostly haven't, yet.

For each revision, the DVCS knows what revisions it derives from. SVN does not
keep this information. SVN primarily just knows the text diffs from revision to
revision. In particular, if I have a long-lived branch, I am going to merge in
changes from the trunk while I'm working on it. When I go to merge the branch
back into the trunk, I need to know which trunk-revisions I've already merged
into the branch. SVN does not track this information. Tools like svnmerge.py
track some of this information at the expense of some added clumsiness.

It's worth noting that SVN 1.5 will be tracking such information. But that
release is a long ways off.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] how to create an array of objects that are sequences?

[Robert Kern]

Chris Barker wrote:
 Travis E. Oliphant wrote:
 
 I like the depth option, and that could be used with the current 
 interface without too much difficulty, I think.

 This question does come up quite often and should probably be addressed.
 
 Yes, it has, but the depth option was discussed too, and while it might 
 solve some problems, it would still be ambiguous:
 
 What if your objects were nested sequences, and you wanted to partly 
 flatten them -- which would you flatten?

I'm pretty sure that that is exactly the ambiguity that the depth option
resolves. Can you give me an example where it's still ambiguous with the depth
information provided?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

Travis E. Oliphant wrote:

 I don't think it is time to move wholesale to something like Mercurial 
 or bzr.   I would prefer it if all of the Enthought-hosted projects 
 moved to the (new) system at once,  which is not going to happen in the 
 short term (but long term of course it's an open question).

I think that's irrelevant. There is absolutely no reason that we should force 
all of the Enthought-hosted projects to move in sync. We would have reason if 
we 
were being asked to host a different centralized VCS with a complicated server, 
but hosting Mercurial or bzr is nearly trivial. We already do it for me:

   http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi

The remaining thing we would have to support is the Trac integration. While not 
as trivial as simply hosting the repositories, it's not a very large commitment.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] how to create an array of objects that are sequences?

[Robert Kern]

Christopher Barker wrote:
 Robert Kern wrote:
 Chris Barker wrote:
 What if your objects were nested sequences, and you wanted to partly 
 flatten them -- which would you flatten?
 I'm pretty sure that that is exactly the ambiguity that the depth option
 resolves. Can you give me an example where it's still ambiguous with the 
 depth
 information provided?
 
 I was imagining:
 
 Say you have a bunch of nested sequences that, fully expanded, would 
 yield a (i,j,k) rank-3 array. If you wanted a rank-2 array, you could 
 have either:
 a (i,j) array with each element being a length-k sequence
 or:
 a (i,k) array, with each element being a length-j sequence.
 
 This is quite trivial with numpy n-d arrays.
 
 However, while you could build either of those from nested sequences, 
 there isn't, in fact, and object in there that is that length-j sequence.

And I think that's the key difference. I don't think array() should be 
responsible for *arbitrarily* flattening nested sequences in order to 
reinterpret structure. Instead, I think we really only need array() to be able 
to answer the question, is this object an atom or a sequence I need to descend 
into? Essentially, for all valid index-tuples (say, [i,j,k]) in the array:

   arr[i,j,k] == original[i,j,k]

 I guess this really points to the power of n-d arrays!

Yes. Since one can get the (i,k) array after getting the (i,j,k) array and 
reshaping, I don't think we need to support it in array().

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

Bill Baxter wrote:
 http://www.selenic.com/mercurial/wiki/index.cgi/MergeProgram
 
 This is a bit puzzling.  I understand better merging isn't the only
 reason to choose DVCS, but the above page basically says that
 Mercurial just uses whatever external merge program it can find.  So
 the file-level merging sounds like it must really be no different from
 other VCSs.
 
 So it is really just proper merging of directory renames and the like
 that make it superior?

No. If you'll pardon my repeating myself:


DVCSes need to keep track of more information in order to be
distributed. That information is extremely useful for managing merges properly.
Centralized systems could track this information, but they don't *need* to in
order to be functional, so they mostly haven't, yet.

For each revision, the DVCS knows what revisions it derives from. SVN does not
keep this information. SVN primarily just knows the text diffs from revision to
revision. In particular, if I have a long-lived branch, I am going to merge in
changes from the trunk while I'm working on it. When I go to merge the branch
back into the trunk, I need to know which trunk-revisions I've already merged
into the branch. SVN does not track this information. Tools like svnmerge.py
track some of this information at the expense of some added clumsiness.


-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

David Cournapeau wrote:

 Does good merging only depends on the above ? Martin Pool, one of the 
 bzr programmer, wrote this article two years ago:
 
 http://sourcefrog.net/weblog/software/vc/derivatives.html
 
 which I found both enlightening and easy to follow.

My terminology was fuzzy/incorrect. By revision, I meant changeset rather 
than snapshot.

The main thrust of that article is about the value of viewing VCS history as a 
sequence of changesets rather than a sequence of snapshots. FWIW, svnmerge and 
SVN 1.5 Merge Tracking are changeset-oriented rather than snapshot-oriented.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Moving away from svn ?

[Robert Kern]

Bill Baxter wrote:
 On Jan 6, 2008 6:38 PM, Robert Kern [EMAIL PROTECTED] wrote:
 Bill Baxter wrote:
 http://www.selenic.com/mercurial/wiki/index.cgi/MergeProgram

 This is a bit puzzling.  I understand better merging isn't the only
 reason to choose DVCS, but the above page basically says that
 Mercurial just uses whatever external merge program it can find.  So
 the file-level merging sounds like it must really be no different from
 other VCSs.

 So it is really just proper merging of directory renames and the like
 that make it superior?
 No. If you'll pardon my repeating myself:

 
 DVCSes need to keep track of more information in order to be
 distributed. That information is extremely useful for managing merges 
 properly.
 Centralized systems could track this information, but they don't *need* to in
 order to be functional, so they mostly haven't, yet.

 For each revision, the DVCS knows what revisions it derives from. SVN does 
 not
 keep this information. SVN primarily just knows the text diffs from revision 
 to
 revision. In particular, if I have a long-lived branch, I am going to merge 
 in
 changes from the trunk while I'm working on it. When I go to merge the branch
 back into the trunk, I need to know which trunk-revisions I've already merged
 into the branch. SVN does not track this information. Tools like svnmerge.py
 track some of this information at the expense of some added clumsiness.

 
 
 Ok.  Sorry for not reading that closer.  So what you're saying is that
 the magic is in the deciding of exactly which revisions of which files
 to run the merge program on?

That's the main idea.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] parallel numpy (by Brian Granger) - any info?

[Robert Kern]

dmitrey wrote:
 The only one thing I'm very interested in for now - why the most 
 simplest matrix operations are not implemented to be parallel in numpy 
 yet (for several-CPU computers, like my AMD Athlon X2). First of all 
 it's related to matrix multiplication and devision, either point or 
 matrix (i.e. like A\B, A*B, dot(A,B)).

Eric Jones has made an attempt.

   http://svn.scipy.org/svn/numpy/branches/multicore/

Unfortunately, the overhead of starting the threads and acquiring/releasing 
thread locks wipes out most of the performance gains until you get fairly large 
arrays. It is possible that this comes from the particular implementation, 
rather than being intrinsic to the problem.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Using svnmerge on numpy: am I missing something ?

[Robert Kern]

David Cournapeau wrote:
 Matthieu Brucher wrote:
  Oups, safe for the /trunk:1-2871 part. This should be deleted
 before
  a commit to the trunk, I think.
 Yes, that's what I (quite unclearly) meant: since revision numbers are
 per- repository in svn, I don't understand the point of tracking trunk
 revisions: I would think that tracking the last merged version for
 each
 branch to be enough (for the kind of merge svn does, at least). If
 trunk
 version are tracked, then I would expect two branches using
 svnmerge to
 clash each other,


 In fact, the trunk should be tracked from all the branches, although 
 there will be the problem with merging the different branches (I did 
 not have many troubles with that, but I only tried with a few 
 differences) into the trunk. I don't think only one branch wants to be 
 up to date with the trunk ;).
 Sure, that's what I was trying to do (tracking the trunk). But if 
 svnmerge needs the information from the trunk, this means each branch 
 will need a different value, whereas only one value is possible...

IIRC, its presence in the trunk/'s svnmerge-integrated property is a mistake. 
It 
belongs in each of the branches independently. For some reason, it got merged 
into the trunk from one of the branches by mistake, possibly because someone 
started merge tracking with svnmerge then another used svn merge. It should 
be 
removed.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Neal Becker wrote:
 I noticed that if I generate complex rv i.i.d. with var=1, that numpy says:
 
 var (real part) - (close to 1.0)
 var (imag part) - (close to 1.0)
 
 but
 
 var (complex array) - (close to complex 0)
 
 Is that not a strange definition?

There is some discussion on this in the tracker.

   http://projects.scipy.org/scipy/numpy/ticket/638

The current state of affairs is that the implementation of var() just naively 
applies the standard formula for real numbers.

   mean((x - mean(x)) ** 2)

I think this is pretty obviously wrong prima facie. AFAIK, no one considers 
this 
a valid definition of variance for complex RVs or in fact a useful value. I 
think we should change this. Unfortunately, there is no single alternative but 
several.

1. Punt. Complex numbers are inherently multidimensional, and a single scale 
parameter doesn't really describe most distributions of complex numbers. 
Instead, you need a real covariance matrix which you can get with cov([z.real, 
z.imag]). This estimates the covariance matrix of a 2-D Gaussian distribution 
over RR^2 (interpreted as CC).

2. Take a slightly less naive formula for the variance which seems to show up 
in 
some texts:

   mean(absolute(z - mean(z)) ** 2)

This estimates the single parameter of a circular Gaussian over RR^2 
(interpreted as CC). It is also the trace of the covariance matrix above.

3. Take the variances of the real and imaginary components independently. This 
is equivalent to taking the diagonal of the covariance matrix above. This 
wouldn't be the definition of *the* complex variance that anyone else uses, 
but rather another form of punting. There isn't a single complex variance to 
give you, but in the spirit of broadcasting, we'll compute the marginal 
variances of each dimension independently.

Personally, I like 1 a lot. I'm hesitant to support 2 until I've seen an actual 
application of that definition. The references I have been given in the ticket 
comments are all early parts of books where the authors are laying out 
definitions without applications. Personally, it feels to me like the authors 
are just sticking in the absolute()'s ex post facto just so they can extend the 
definition they already have to complex numbers. I'm also not a fan of the 
expectation-centric treatments of random variables. IMO, the variance of an 
arbitrary RV isn't an especially important quantity. It's a parameter of a 
Gaussian distribution, and in this case, I see no reason to favor circular 
Gaussians in CC over general ones.

But if someone shows me an actual application of the definition, I can amend my 
view.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Charles R Harris wrote:

 Suppose you have a set of z_i and want to choose z to minimize the 
 average square error $ \sum_i |z_i - z|^2 $. The solution is that 
 $z=\mean{z_i}$ and the resulting average error is given by 2). Note that 
 I didn't mention Gaussians anywhere. No distribution is needed to 
 justify the argument, just the idea of minimizing the squared distance. 
 Leaving out the ^2 would yield another metric, or one could ask for a 
 minmax solution. It is a question of the distance function, not 
 probability. Anyway, that is one justification for the approach in 2) 
 and it is one that makes a lot of applied math simple. Whether of not a 
 least squares fit is useful is different question.

If you're not doing probability, then what are you using var() for? I can 
accept 
that the quantity is meaningful for your problem, but I'm not convinced it's a 
variance.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Neal Becker wrote:

 2 is what I expected.  Suppose I have a complex signal x, with additive
 Gaussian noise (i.i.d, real and imag are independent). 
 y = x + n

Not only do the real and imag marginal distributions have to be independent, 
but 
also of the same scale, i.e. Re(n) ~ Gaussian(0, sigma) and Im(n) ~ Gaussian(0, 
sigma) for the same sigma.

 Consider an estimate \hat{x} = y.
 
 What is the mean-squared-error E[(y - x)^2] ?
 
 Definition 2 is consistent with that, and gets my vote.

Ah, you have to be careful. What you wrote is what is implemented. Definition 2 
is consistent with this, instead:

   E[|y - x|^2]

But like I said, I see no particular reason to favor circular Gaussians over 
the 
general form for the implementation of numpy.var().

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Does float16 exist?

[Robert Kern]

Charles R Harris wrote:

 I see that there are already a number of parsers available for Python, 
 SPARK, for instance is included in the 2.5.1 distribution.

No, it isn't.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Charles R Harris wrote:
 
 
 On Jan 8, 2008 7:48 PM, Robert Kern [EMAIL PROTECTED] 
 mailto:[EMAIL PROTECTED] wrote:
 
 Charles R Harris wrote:
 
   Suppose you have a set of z_i and want to choose z to minimize the
   average square error $ \sum_i |z_i - z|^2 $. The solution is that
   $z=\mean{z_i}$ and the resulting average error is given by 2).
 Note that
   I didn't mention Gaussians anywhere. No distribution is needed to
   justify the argument, just the idea of minimizing the squared
 distance.
   Leaving out the ^2 would yield another metric, or one could ask
 for a
   minmax solution. It is a question of the distance function, not
   probability. Anyway, that is one justification for the approach in 2)
   and it is one that makes a lot of applied math simple. Whether of
 not a
   least squares fit is useful is different question.
 
 If you're not doing probability, then what are you using var() for?
 I can accept
 that the quantity is meaningful for your problem, but I'm not
 convinced it's a
 variance.
 
 
 Lots of fits don't involve probability distributions. For instance, one 
 might want to fit a polynomial to a mathematical curve. This sort of 
 distinction between probability and distance goes back to Gauss himself, 
 although not in his original work on least squares.  Whether or not 
 variance implies probability is a semantic question.

Well, the problem in front of us is entirely semantics: What does the string 
var(z) mean? Are we going to choose an mechanistic definition: var(z) is 
implemented in such and such a way and interpretations are left open? In that 
case, why are we using the string var(z) rather than something else? We're 
also still left with the question as to which such and such implementation to 
use.

Alternatively, we can look at what people call variances and try to implement 
the calculation of such. In that case, the term variance tends to crop up 
(and 
in my experience *only* crop up) in statistics and probability. Certain 
implementations of the calculations of such quantities have cognates elsewhere, 
but those cognates are not themselves called variances.

My question to you is, is the resulting average error a variance? I.e., do 
people call it a variance outside of SP? There are any number of computations 
that are useful but are not variances, and I don't think we should make 
var(z) 
implement them.

In SP, the single quantity variance is well defined for real RVs, even if 
you 
step away from Gaussians. It's the second central moment of the PDF of the RV. 
When you move up to CC (or RR^2), the definition of moment changes. It's no 
longer a real number or even a scalar; the second central moment is a 
covariance 
matrix. If we're going to call something the variance, that's it. The 
circularly symmetric forms are special cases. Although option #2 is a useful 
quantity to calculate in some circumstances, I think it's bogus to give it a 
special status.

 I think if we are 
 going to compute a single number,  2) is as good as anything even if it 
 doesn't capture the shape of the scatter plot. A 2D covariance wouldn't 
 necessarily capture the shape either.

True, but it is clear exactly what it is. The function is named cov(), and it 
computes covariances. It's not called shape_of_2D_pdf(). Whether or not one 
ought to compute a covariance is not cov()'s problem.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Travis E. Oliphant wrote:
 Robert Kern wrote:
 Neal Becker wrote:
   
 I noticed that if I generate complex rv i.i.d. with var=1, that numpy says:

 var (real part) - (close to 1.0)
 var (imag part) - (close to 1.0)

 but

 var (complex array) - (close to complex 0)

 Is that not a strange definition?
 


 2. Take a slightly less naive formula for the variance which seems to show 
 up in 
 some texts:

mean(absolute(z - mean(z)) ** 2)

 This estimates the single parameter of a circular Gaussian over RR^2 
 (interpreted as CC). It is also the trace of the covariance matrix above.
   
 
 I tend to favor this interpretation because it is used quite heavily in 
 signal processing applications where circular Gaussian random 
 variables show up quite a bit --- so much so, in fact, that most EE 
 folks would expect this as the output and you would have to explain to 
 them why there may be other choices that make sense.  
 
 So, #2 is kind of a nod to the signal-processing community (especially 
 the communication section).

sigh Fair enough. I relent. You implement it; I'll document the 
correct^Wcov() 
alternative.  :-)

 But, there is also merit to me in #3 (although it may be harder to 
 explain why the variance returns a complex number --- if that is what 
 you meant).

Yes.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Template system

[Robert Kern]

Charles R Harris wrote:
 Hi All,
 
 I'm thinking of adding the template system I pulled out of Django to 
 Numpy to use in place of the current code generator. Its advantages are 
 documentation, flexibility, and maintainance. The code totals about 470 
 KB, comes with a BSD license, and is compatible with Python = 2.3. I 
 want to bring this up on the list because I'm sure there will be 
 objections along the lines of why do we need that?  So I want to see 
 how strenuous the objections are.

It's too big. If you want a better template system than the current one, there 
are a few which are contained in a single file. That would be fine to include, 
but I object strongly to incorporating Jinja.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Template system

[Robert Kern]

Charles R Harris wrote:
 
 
 On Jan 11, 2008 2:58 PM, Robert Kern [EMAIL PROTECTED] 
 mailto:[EMAIL PROTECTED] wrote:
 
 Charles R Harris wrote:
   Hi All,
  
   I'm thinking of adding the template system I pulled out of Django to
   Numpy to use in place of the current code generator. Its
 advantages are
   documentation, flexibility, and maintainance. The code totals
 about 470
   KB, comes with a BSD license, and is compatible with Python = 2.3. I
   want to bring this up on the list because I'm sure there will be
   objections along the lines of why do we need that?  So I want
 to see
   how strenuous the objections are.
 
 It's too big. If you want a better template system than the current
 one, there
 are a few which are contained in a single file. That would be fine
 to include,
 but I object strongly to incorporating Jinja.
 
 
 Ummm, harsh. I'm certainly open to suggestions. I took a closer look at 
 the django system and reduced it to 308K, 70K zipped. There was a lot of 
 space taken up by the .svn directories inherited from django and *.pyc 
 files. I could probably knock another 50K off by removing some 
 functionality we don't really need: html, http, dates, time, and maybe 
 template inheritance. Just for comparison, here are some things that are 
 in a fresh checkout of numpy, including the .svn files
 
 Numpy 23 MB
 f2py   5.4 MB
 doc/swig 1.5 MB
 code_generators 192K
 
 So adding the template subsystem would increase the repository size by 
 about 1.4%.  Do you have any objections in addition to the size?

Yes. I'm not convinced that we need anything better than we have now. Show me 
how using Jinja templates makes the current code better. Check the one-file 
templating modules (like YAPTU[1] or Cog[2]) out there to see if those are 
sufficient. Jinja and its kin (Mako, Cheetah, etc.) are intended for a very 
different use case, and their design, implementation, and footprint are 
optimized for that.

[1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52305
[2] http://nedbatchelder.com/code/cog/

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Template system

[Robert Kern]

Charles R Harris wrote:
 OK, so far I've knocked it down to 35KB by removing stuff I'm not 
 interested in. It is now smaller than Cog, and 7x larger than the file 
 we now use to do the same job. I'm pretty sure I can make it leaner than 
 that. It remains extensible.

Can you put a tarball up somewhere? Django generally requires environment 
variables to configure itself. There is a way to configure these from code, but 
it looks like that pulls in django.conf. This is my primary source of 
discomfort 
with using Django templates. My experience with using Django components outside 
of a Django web app has been frustrating. The more modifications we make, the 
more we have to maintain. And if any of it causes problems for someone trying 
to 
build numpy, that's more questions we need to answer. I want to answer numpy 
questions, not Django questions.

I would be much, much more comfortable with an implementation that can simply 
be 
dropped in without modification.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] casting

[Robert Kern]

Neal Becker wrote:
 numpy frequently refers to 'casting'.  I'm not sure if that term is ever
 defined.  I believe it has the same meaning as in C.  In that case, it is
 unfortunately used to mean 2 different things.  There are casts that do not
 change the underlying bits (such as a pointer cast), and there are casts
 that actually convert to different bits (such as float - double).
 
 I think numpy means the latter.  When an array where the underlying data is
 one type, a cast to another type means actually reallocating and converting
 the data.

Yes, that is usually what people mean when they use _casting_ in the context of 
numpy. It is the more frequently performed operation of the two. The former can 
be accomplished with the .view(dtype) method of ndarrays.

 It often occurs that I have an algorithm that can take any integral type,
 because it is written with c++ templates.  In that case, I don't want to
 use PyArray_FROMANY, because I don't want to unecessarily convert the array
 data.  Instead, I'd like to inquire what is the preferred type of the data.
 
 The solution I'm exploring is to use a function I
 call 'preferred_array_type'.  This uses the __array_struct__ interface to
 find the native data type.  I chose to use this interface, because then it
 will work with both numpy arrays and other array-like types.
 
 Any thoughts on all of this? 

I'm not sure what you mean by preferred type of the data. Do you mean the 
dtype of the array as it comes in? There are several functions and function 
macros in the numpy C API which take differing amounts of information. For 
example,

  * PyArray_FROM_O(PyObject*onj) just takes an object.
  * PyArray_FROM_OF(PyObject* obj, int req) takes an object and flags like
NPY_CONTIGUOUS.
  * PyArray_FROM_OT(PyObject* obj, int typenum) takes an object and a type
number.
  * PyArray_FROM_OTF(PyObject* obj, int typenum, int req) takes an object, type,
and flags.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] casting

[Robert Kern]

Neal Becker wrote:
 Robert Kern wrote:
 
 Neal Becker wrote:
 numpy frequently refers to 'casting'.  I'm not sure if that term is ever
 defined.  I believe it has the same meaning as in C.  In that case, it is
 unfortunately used to mean 2 different things.  There are casts that do
 not change the underlying bits (such as a pointer cast), and there are
 casts that actually convert to different bits (such as float - double).

 I think numpy means the latter.  When an array where the underlying data
 is one type, a cast to another type means actually reallocating and
 converting the data.
 Yes, that is usually what people mean when they use _casting_ in the
 context of numpy. It is the more frequently performed operation of the
 two. The former can be accomplished with the .view(dtype) method of
 ndarrays.

 It often occurs that I have an algorithm that can take any integral type,
 because it is written with c++ templates.  In that case, I don't want to
 use PyArray_FROMANY, because I don't want to unecessarily convert the
 array
 data.  Instead, I'd like to inquire what is the preferred type of the
 data.

 The solution I'm exploring is to use a function I
 call 'preferred_array_type'.  This uses the __array_struct__ interface to
 find the native data type.  I chose to use this interface, because then
 it will work with both numpy arrays and other array-like types.

 Any thoughts on all of this?
 I'm not sure what you mean by preferred type of the data. Do you mean
 the dtype of the array as it comes in? There are several functions and
 function macros in the numpy C API which take differing amounts of
 information. For example,

   * PyArray_FROM_O(PyObject*onj) just takes an object.
   * PyArray_FROM_OF(PyObject* obj, int req) takes an object and flags like
 NPY_CONTIGUOUS.
   * PyArray_FROM_OT(PyObject* obj, int typenum) takes an object and a type
 number.
   * PyArray_FROM_OTF(PyObject* obj, int typenum, int req) takes an object,
   type,
 and flags.

 
 Let me try again to explain.  I don't want to convert to some type first -
 that would be a waste.  I need to find out what is the native data type of
 the input array first.  Also, I'd like to allow that the input is not a
 PyArray, but could be something conforming to __array_struct__ interface. 
 So, I need to find the native data type _first_, _then_ call the
 appropriate PyArray_FROM...

I'm sorry, I still think we're talking past each other. What do you mean by 
native data type? If you just want to get an ndarray without specifying a 
type, use PyArray_FROM_O(). That's what it's for. You don't need to know the 
data type beforehand.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] RFC: out of range slice indexes

[Robert Kern]

Neal Becker wrote:
 I've never liked that python silently ignores slices with out of range
 indexes.  I believe this is a source of bugs (it has been for me).  It goes
 completely counter to the python philosophy.
 
 I vote to ban them from numpy.
 from numpy import array
 x = array (xrange (10))
 x[11]
 Traceback (most recent call last):
   File stdin, line 1, in module
 IndexError: index out of bounds
 x[:12] = 2
 x
 array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
 len (x)
 10
 
 Silently ignoring the error x[:12] is a bad idea, IMO.  If it meant to
 _extend_ x to have lenght 12, at least _that_ would be reasonable (but I'm
 not advocating that position).
 
 I believe that out of bounds indexes should always throw IndexError.  We
 can't change that in Python now, but maybe we can in numpy.

-1. Regardless of the merits if we had a blank slate, there is code that 
depends 
on this, specifically my code. It simplifies certain operations that would 
otherwise need tedious special case-handling.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] RFC: out of range slice indexes

[Robert Kern]

Neal Becker wrote:
 Robert Kern wrote:
 
 Neal Becker wrote:
 I've never liked that python silently ignores slices with out of range
 indexes.  I believe this is a source of bugs (it has been for me).  It
 goes completely counter to the python philosophy.

 I vote to ban them from numpy.
 from numpy import array
 x = array (xrange (10))
 x[11]
 Traceback (most recent call last):
   File stdin, line 1, in module
 IndexError: index out of bounds
 x[:12] = 2
 x
 array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
 len (x)
 10

 Silently ignoring the error x[:12] is a bad idea, IMO.  If it meant to
 _extend_ x to have lenght 12, at least _that_ would be reasonable (but
 I'm not advocating that position).

 I believe that out of bounds indexes should always throw IndexError.  We
 can't change that in Python now, but maybe we can in numpy.
 -1. Regardless of the merits if we had a blank slate, there is code that
 depends on this, specifically my code. It simplifies certain operations
 that would otherwise need tedious special case-handling.
 
 For example?

def ichunk(arr, chunk_size=10):
 for i in range(0, len(arr), chunk_size):
 yield arr[i:i+chunk_size]

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] casting

[Robert Kern]

Neal Becker wrote:
 Jon Wright wrote:
 
 I'm sorry, I still think we're talking past each other. What do you mean
 by native data type? If you just want to get an ndarray without
 specifying a type, use PyArray_FROM_O(). That's what it's for. You don't
 need to know the data type beforehand.
 What I have wanted in the past (and what I thought Neal was after) is a
 way to choose which function to call according to the typecode of the
 data as it is currently in memory. I don't want to convert (or cast or
 even touch the data) but just call a type specific function instead. C++
 templates can take some of the tedium out of that, but in some cases
 algorithms may be different too. Guessing which sort algorithm to use
 springs to mind.

 Rather than saying give me the right kind of array, I think there is
 an interest in saying choose which function is the best for this data.
 Something like:

PyArrayObject* array = PyArray_FROM_O( (PyObject*) O );
type = array - descr - type_num ;
switch (type){
   caseNPY_BYTE   : signed_func(array);
   caseNPY_UBYTE  : unsigned_func(array);
   // etc

 It sort of implies having a C++ type hierarchy for numpy arrays and
 casting array to be a PyFloatArray or PyDoubleArray etc?

 The extra confusion might be due to the way arrays can be laid out in
 memory - indexing into array slices is not always obvious. Also if you
 want to make sure your inner loop goes over the fast index you might
 want an algorithm which reads the strides when it runs.

 Sorry if I've only added to the confusion.

 Cheers,

 Jon
 
 This is close to what I'm doing.  If I really can handle any type, then
 FROM_O is fine.
 
 Commonly, it's a little more complicated.
 
 Here is an example, in pseudo-code (real code is in c++):  
 
 if (native_type_of (x) is int or long):
   do_something_with (convert_to_long_array(x))
 elif (native_type_of (x) is complex):
   do_something_with (convert_to_complex_array (x))
 
 In the above, native_type_of means:
 if (hasattr (x, __array_struct__)):
array_if = PyCObject_AsVoidPtr (x.__array_struct__)
return (map_array_if_to_typenum (array_if))
 
 So this means for any numpy array or any type supporting __array_struct__
 protocol, find out the native data type.
 
 I don't want to use FROM_O here, because I really can only handle certain
 types.  If I used FROM_O, then after calling FROM_O, if the type was not
 one I could handle, I'd have to call FromAny and convert it.  Or, in the
 case above, if I was given an array of int32 which I'd want to handle as
 long (int64), I'd have to convert it again.

Okay, I think I see now. I'm not sure what numpy could do to make your code 
more 
elegant. I would recommend just using PyArray_FROM_O() or PyArray_EnsureArray() 
to get a real ndarray object. They should not copy when the object is already 
an 
ndarray or if it satisfies the array interface; however, it will also handle 
the 
cases when you have a Python-level __array_interface__ or just nested 
sequences, 
too. You can look at the PyArray_Descr directly, dispatch the types that you 
can 
handle directly, and then convert for the types which you can't. You will not 
get any extraneous conversions or copies of data.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy quit working

[Robert Kern]

Kevin Christman wrote:
 I've started to use numpy (1.0.4) and matplotlib but it suddenly quit 
 working.  Now when I import a simple test script:
 import numpy
 numpy.linspace(0,1)
 
 it gives the following error:
 
 Traceback (most recent call last):
   File D:/anothertest.py, line 1, in module
 import numpy
   File C:\Python25\lib\site-packages\numpy\__init__.py, line 37, in module
 import testing
   File C:\Python25\lib\site-packages\numpy\testing\__init__.py, line 3, in 
 module
 from numpytest import *
   File C:\Python25\lib\site-packages\numpy\testing\numpytest.py, line 19, 
 in module
 from numpy.distutils.exec_command import splitcmdline
   File C:\Python25\lib\site-packages\numpy\distutils\__init__.py, line 6, 
 in module
 import ccompiler
   File C:\Python25\lib\site-packages\numpy\distutils\ccompiler.py, line 46, 
 in module
 replace_method(CCompiler, 'spawn', CCompiler_spawn)
   File C:\Python25\lib\site-packages\numpy\distutils\ccompiler.py, line 24, 
 in replace_method
 m = new.instancemethod(func, None, klass)
 AttributeError: 'module' object has no attribute 'instancemethod'
 
 However, other import statements, like import time work ok.
 The problem first started when a script would only run if on the root of the 
 drive (e.g. D:\), but now even if I run it from there it doesn't work. What's 
 going on?

Do you have a module called new.py somewhere? If so, then it is overriding the 
standard library's new module which we use.

  import new
  new
module 'new' from 'C:\Python25\lib\new.pyc'

If that filename does not point to 'C:\Python25\lib\new.pyc', then you have a 
problem.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] def of var of complex

[Robert Kern]

Neal Becker wrote:
 Neal Becker wrote:
 
 I noticed that if I generate complex rv i.i.d. with var=1, that numpy
 says:

 var (real part) - (close to 1.0)
 var (imag part) - (close to 1.0)

 but

 var (complex array) - (close to complex 0)

 Is that not a strange definition?
 
 I don't think there is any ambiguity about the definition of the variance of
 complex.
 
 Var(x) = E{(x-E[x])^2} = E{x}^2 - E{x}

That's currently what's implemented, but there is simply no evidence that 
anyone 
ever uses this definition for complex random variables. Note that for real 
variables,

   E{(x-E[x])^2} = E{|x-E[x]|^2}

but for complex variables, there is a large difference. Since the || are 
superfluous with real variables, probability texts rarely include them unless 
if 
they are then going on to talk about complex variables. If you want to extend 
the definition for real variables to complex variables, that is an ambiguity 
you 
have to resolve.

There is, apparently, a large body of statistical signal processing literature 
that defines the complex variance as

   E{|z-E[z]|^2}

so, I (now) believe that this is what should be implemented.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Docstring page, out of date?

[Robert Kern]

David Cournapeau wrote:
 Nobody likes to write doc, specially if you have to follow many rules.
 Not being able to see the result does not help. I don't know much
 about the current documation tools situation with python, but I have
 seen good, automatically generated doc for python modules: for
 example, the current developement doc of python
 (http://docs.python.org/dev/) looks pretty good to me, and using the
 same tools as python makes more sense than forking our own, no ? Or am
 I missing something fundamental ?

Yes. Sphinx does not do automatically generated documentation.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Building a static libnumpy

[Robert Kern]

Jussi Enkovaara wrote:
 Hi,
 I need to use numpy in an environment which does not support shared libraries.
 Previously, I have used the old Numeric where I created a Makefile to build a
 static Numeric library which was later on linked to the python interpreter.
 
 As far as I understood, numpy uses sort of extended distutils. I was 
 wondering if
 this extended distutils enables building of static libraries or do I have to 
 go
 the cumbersome Makefile-way again?

Cumbersome Makefile, sorry.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Docstring standard: how to specify variable types

[Robert Kern]

Stefan van der Walt wrote:
 Hi Charles
 
 On Wed, Jan 23, 2008 at 09:34:44AM -0700, Charles R Harris wrote:
 2. How do we specify default values?

 I like to put them first in the list: {-1, integer}
 
 When exactly is this list used?  That should be made clear in the
 standard as well.

No special processing is done. I omit it. If I need to talk about the default 
value, I do it in prose in the argument's description. Note that the default 
value is not always the same thing as the value in the signature, particularly 
when kwd=None. But in those cases, I usually need a sentence to describe what's 
happening, so that's where I always put that information.

 3. Why do we need the optional keyword (the function signature
   already indicates that the parameter is optional).

 It's extra information, true, but that isn't always a bad thing. It's like
 looking up whole numbers in a book index and, instead of the page 
 reference,
 the index directs you to numbers, whole. Flip, flip, flip. Drives me crazy.
 Besides, the function signature might be missing.
 
 When would the function signature be missing?  In C functions we copy
 the signature into the docstring.  I am concerned about duplicating
 information that may change.
 
 4. Do we really need the Other Parameters list?  It would make more
   sense to split positional and keyword arguments, but I'm not even
   sure that is necessary, since that information is already specified in
 the
   function signature.

 I agree, but Travis likes this section and I don't feel strongly
   about it.
 
 If I understood its role properly, I would be happier to use it, but
 at the moment I'm not convinced that it contributes anything useful.
 Parameters are normally sorted from most to least valuable anyway.

Look at the scipy.optimize.fmin*() functions, for instance. If you don't feel 
like using it, don't. None of the numpy function signatures should be long 
enough to need it.

 5. Is the {'hi', 'ho'} syntax used when a parameter can only assume a
   limited number of values?  In Python {} is a dictionary, so why not
   use ('hi','ho') instead?

 Either would be fine. IIRC, the {} was inherited from epydoc consolidated
 fields.
 
 I thought that, since we threw all epydoc guidelines out the window,
 this would be a good time talk about it -- the next docday is just
 around the corner!

Personally, I don't think it's worth standardizing. If it's readable and valid 
reST, just do it.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [ANN] numscons 0.3.0 release

[Robert Kern]

dmitrey wrote:
 Hi all,
 I don't know much about what are these scons are, if it's something 
 essential (as it seems to be from amount of mailing list traffic) why 
 can't it be just merged to numpy, w/o making any additional branches?

It's a very large, still experimental change to the entire build 
infrastructure. 
It requires branches for us to evaluate.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] difference numpy/matlab

[Robert Kern]

Stuart Brorson wrote:
 I have to agree with Lorenzo.  There is no natural ordering of the
 complex numbers.   Any way you order them is arbitrary.
 
 Accepting this, the question then becomes what should NumPy do when
 the user tries to do order comparison operations on complex numbers.
 The problem is that NumPy is schizophrenic.  Watch this:
 
 --  session log  -
 
 In [20]: A = numpy.array([3+1j, 1+3j, -1-3j, -1+3j, -3-1j])
 
 In [21]: B = A[numpy.random.permutation(5)]
 
 In [22]:
 
 In [22]: A
 Out[22]: array([ 3.+1.j,  1.+3.j, -1.-3.j, -1.+3.j, -3.-1.j])
 
 In [23]: B
 Out[23]: array([-1.+3.j,  3.+1.j, -1.-3.j,  1.+3.j, -3.-1.j])
 
 In [24]: numpy.greater(A, B)
 Out[24]: array([ True, False, False, False, False], dtype=bool)
 
 In [25]: numpy.maximum(A, B)
 Out[25]: array([ 3.+1.j,  3.+1.j, -1.-3.j,  1.+3.j, -3.-1.j])
 
 In [26]:
 
 In [26]: 3+1j  -1+3j
 ---
 type 'exceptions.TypeError' Traceback (most recent call
 last)
 
 /tmp/test/web/ipython console in module()
 
 type 'exceptions.TypeError': no ordering relation is defined for
 complex numbers
 
   /session log  --

No, numpy is entirely consistent. In[26] has Python's complex numbers, not 
numpy's.

In [1]: from numpy import complex64

In [2]: complex64(3+1j)  complex64(-1+3j)
Out[2]: True

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [F2PY]: Allocatable Arrays

[Robert Kern]

Pearu Peterson wrote:
 On Fri, February 1, 2008 1:28 pm, Andrea Gavana wrote:
 Hi All,

 I sent a couple of messages to f2py mailing list, but it seems
 like my problem has no simple solution so I thought to ask for some
 suggestions here.
 
 Sorry, I haven't been around there long time.

Are you going to continue not reading the f2py list? If so, you should point 
everyone there to this list and close the list.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Unexpected behavior with numpy array

[Robert Kern]

Damian Eads wrote:
 Here's another question: is there any way to construct a numpy array and 
 specify the buffer address where it should store its values? I ask 
 because I would like to construct numpy arrays that work on buffers that 
 come from mmap.

Can you clarify that a little? By buffer do you mean a Python buffer() 
object? 
By mmap do you mean Python's mmap in the standard library?

numpy has a memmap class which subclasses ndarray to wrap a mmapped file. It 
handles the opening and mmapping of the file itself, but it could be subclassed 
to override this behavior to take an already opened mmap object.

In general, if you have a buffer() object, you can make an array from it using 
numpy.frombuffer(). This will be a standard ndarray and won't have the 
conveniences of syncing to disk that the memmap class provides.

If you don't have a buffer() object, but just have a pointer allocated from 
some 
C code, then you *could* fake an object which exposes the __array_interface__() 
method to describe the memory. The numpy.asarray() constructor will use that to 
make an ndarray object that uses the specified memory. This is advanced stuff 
and difficult to get right because of memory ownership and object lifetime 
issues. If you can modify the C code, it might be easier for you to have numpy 
allocate the memory, then make the C code use that pointer to do its operations.

But look at numpy.memmap first and see if it fits your needs.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Unexpected behavior with numpy array

[Robert Kern]

Damian Eads wrote:
 Robert Kern wrote:
 Damian Eads wrote:
 Here's another question: is there any way to construct a numpy array and 
 specify the buffer address where it should store its values? I ask 
 because I would like to construct numpy arrays that work on buffers that 
 come from mmap.
 Can you clarify that a little? By buffer do you mean a Python buffer() 
 object? 
 
 Yes, I mean the .data field of a numpy array, which is a buffer object, 
 and points to the memory where an array's values are stored.

Actually, the .data field is always constructed by ndarray; it is never 
provided 
*to* ndarray even if you construct the ndarray from a buffer object. The buffer 
object's information is interpreted to construct the ndarray object and then 
the 
original buffer object is ignored. The .data attribute will be constructed 
on-the-fly when it is requested.


In [9]: from numpy import *

In [10]: s = ''

In [11]: b = buffer(s)

In [12]: a = frombuffer(b, dtype=int32)

In [13]: a.data is b
Out[13]: False

In [14]: d1 = a.data

In [15]: d2 = a.data

In [16]: d1 is d2
Out[16]: False


 By mmap do you mean Python's mmap in the standard library?
 
 I actually was referring to the C Standard Library's mmap. My intention 
 was to use a pointer returned by C-mmap as the .data buffer to store 
 array values.
 
 numpy has a memmap class which subclasses ndarray to wrap a mmapped file. It 
 handles the opening and mmapping of the file itself, but it could be 
 subclassed 
 to override this behavior to take an already opened mmap object.
 
 This may satisfy my needs. I'm going to look into it and get back to you.
 
 In general, if you have a buffer() object, you can make an array from it 
 using 
 numpy.frombuffer(). This will be a standard ndarray and won't have the 
 conveniences of syncing to disk that the memmap class provides.
 
 This is good to know because there have been a few situations when this 
 would have been very useful.
 
 Suppose I do something like (in Python):
 
import ctypes
mylib = ctypes.CDLL('libmylib.so')
y = mylib.get_float_array_from_c_function()
 
 which returns a float* as a Python int, and then I do
 
nelems = mylib.get_float_array_num_elems()
x = numpy.frombuffer(ctypes.c_buffer(y), 'float', nelems)
 
 This gives me an ndarray x with its (.data) buffer pointing to the 
 memory address give by y. When the ndarray x is no longer referenced 
 (even as another array's base), does numpy attempt to free the memory 
 pointed to by y? In other words, does numpy always deallocate the 
 (.data) buffer in the __del__ method? Or, does fromarray set a flag 
 telling it not to?

By default, frombuffer() creates an array that is flagged as not owning the 
data. That means it will not delete the data memory when the ndarray object is 
destroyed.


In [69]: import ctypes

In [70]: ca = (ctypes.c_int*8)()

In [71]: a = frombuffer(ci, int)

In [72]: a
Out[72]: array([0, 0, 0, 0, 0, 0, 0, 0])

In [73]: a.flags
Out[73]:
   C_CONTIGUOUS : True
   F_CONTIGUOUS : True
   OWNDATA : False
   WRITEABLE : True
   ALIGNED : True
   UPDATEIFCOPY : False

 If you don't have a buffer() object, but just have a pointer allocated from 
 some 
 C code, then you *could* fake an object which exposes the 
 __array_interface__() 
 method to describe the memory. The numpy.asarray() constructor will use that 
 to 
 make an ndarray object that uses the specified memory. This is advanced 
 stuff 
 and difficult to get right because of memory ownership and object lifetime 
 issues.
 
 Allocating memory in C code would be very useful for me. If I were to 
 use such a numpy.asarray() function (seems the frombuffer you mentioned 
 would also work as described above),

Yes, if you can create the buffer object or something that obeys the buffer 
protocol. ctypes arrays work fine; ctypes pointers don't.

 it makes sense for the C code to be 
 responsible for deallocating the memory, not numpy. I understand that I 
 would need to ensure that the deallocation happens only when the 
 containing ndarray is no longer referenced anywhere in Python 
 (hopefully, ndarray's finalization code does not need access to the 
 .data buffer).

My experience has been that this is fairly difficult to do. If you have 
*complete* control of the ndarray object over its entire lifetime, then this is 
reasonable. If you don't, then you are going to run into (nondeterministic!) 
segfaulting bugs eventually. For example, if you are only using it as a 
temporary inside a function and never return it, this is fine. You will also 
need to be very careful about constructing views from the ndarray; these will 
need to be controlled, too. You will have a bug if you delete myarray but 
return 
reversed_array=myarray[::-1], for example.

I see that you are using ctypes. Be sure to take a look at the .ctypes 
attribute 
on ndarrays. This allows you to get a ctypes pointer object from an array. This 
might help you use numpy to allocate

Re: [Numpy-discussion] Bug in numpy all() function

[Robert Kern]

Dan Goodman wrote:
 Hi all,
 
 I think this is a bug (I'm running Numpy 1.0.3.1):
 
 from numpy import *
 def f(x): return False
 
 all(f(x) for x in range(10))
 True
 
 I guess the all function doesn't know about generators?

Yup. It works on arrays and things it can turn into arrays by calling the C API 
equivalent of numpy.asarray(). There's a ton of magic and special cases in 
asarray() in order to interpret nested Python sequences as arrays. That magic 
works fairly well when we have sequences with known lengths; it fails utterly 
when given an arbitrary iterator of unknown length. So we punt. Unfortunately, 
what happens then is that asarray() sees an object that it can't interpret as a 
sequence to turn into a real array, so it makes a rank-0 array with the 
iterator 
object as the value. This evaluates to True.

It's possible that asarray() should raise an exception for generators, but it 
would be a special case. We wouldn't be able to test for arbitrary iterables.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] random enhancement

[Robert Kern]

Neal Becker wrote:
 One thing missing from random is a mechanism to share a single underlying
 rng with other code that is not part of numpy.random.
 
 For example, I have code that generates distributions that expect a mersenne
 twister (the shared, underlying rng) to be passed in as a constructor
 argument.
 
 numpy.random shares a single rng amongst it's own distributions, but I don't
 see any way to share with others.

Are you talking about C or Python?

In Python, just instantiate numpy.random.RandomState() and pass it around. All 
of the functions in numpy.random are just aliases to the methods on a global 
RandomState() instance.

C is a problem because the module is implemented in Pyrex, and RandomState is 
an 
extension type. I've tried working on exposing the C API as a PyCObject like 
numpy does, but it is incredibly tedious and, furthermore, is unlikely to 
capture the higher-level methods like multivariate_normal(). I believe that 
Cython has a way to automatically expose the C API of a Pyrex/Cython module, 
but 
I haven't had the time to investigate it.

For everything but the higher level methods like multivariate_normal(), we 
might 
be able to expose the pointer to the rk_state struct on the RandomState object 
as a PyCObject and punt on exposing the API. The C user can copy the 
randomkit.[ch] and distributions.[ch] files into their own code and operate on 
the rk_state pointer with those functions. We may be thwarted by symbol 
conflicts on some platforms, but I'm not sure.

Contributions are, of course, welcome.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Bug in numpy all() function

[Robert Kern]

Anne Archibald wrote:
 On 06/02/2008, Robert Kern [EMAIL PROTECTED] wrote:
 
 I guess the all function doesn't know about generators?
 Yup. It works on arrays and things it can turn into arrays by calling the C 
 API
 equivalent of numpy.asarray(). There's a ton of magic and special cases in
 asarray() in order to interpret nested Python sequences as arrays. That magic
 works fairly well when we have sequences with known lengths; it fails utterly
 when given an arbitrary iterator of unknown length. So we punt. 
 Unfortunately,
 what happens then is that asarray() sees an object that it can't interpret 
 as a
 sequence to turn into a real array, so it makes a rank-0 array with the 
 iterator
 object as the value. This evaluates to True.

 It's possible that asarray() should raise an exception for generators, but it
 would be a special case. We wouldn't be able to test for arbitrary iterables.
 
 Would it be possible for asarray() to pull out the first element from
 the iterable, make an array out of it, then assume that all other
 values out of the iterable will have the same shape (raising an error,
 of course, when they aren't)? I guess this has high foot-shooting
 potential, but is it that much worse than numpy's shpe-guessing
 generally?

I'm skeptical. Personally, it comes down to this: if you provide code that 
implements this safely and efficiently without making a confusing API, I'm more 
than happy to consider it for inclusion. But I'm not going to spend time trying 
to write the code.

 It would be handy to be able to use an iterable to fill an array, so
 that you'd never need to store the values in anything else first:
 
 a = N.array((sin(N.pi*x/n) for x in xrange(n)))

If n is large enough that storage matters,

   a = N.sin(N.linspace(0, np.pi, n))

is always faster, more memory efficient, and more readable. Remember that the 
array will have to be dynamically resized as we go through the iterator. The 
memory movement is going to wipe out much of the benefit of having an iterator 
in the first place.

For 1D arrays, remember that we have numpy.fromiter() already, so we can test 
this.


In [39]: import numpy as np

In [40]: from math import sin

In [41]: n = 10

In [42]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float)
10 loops, best of 3: 11.5 µs per loop

In [43]: %timeit np.sin(np.linspace(0, np.pi, n))
1 loops, best of 3: 26.1 µs per loop

In [44]: n = 100

In [45]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float)
1 loops, best of 3: 84 µs per loop

In [46]: %timeit np.sin(np.linspace(0, np.pi, n))
1 loops, best of 3: 32.3 µs per loop

In [47]: n = 1000

In [48]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float)
1000 loops, best of 3: 794 µs per loop

In [49]: %timeit np.sin(np.linspace(0, np.pi, n))
1 loops, best of 3: 91.8 µs per loop


So, for n=10, the generator wins, but is n=10 really the case that you want to 
use a generator for?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy-discussion Digest, Vol 17, Issue 15

[Robert Kern]

Matthew, please do not reply to the digests. Think of them as read-only. If you 
want to start a new thread, send your mail, with a descriptive Subject line, to 
numpy-discussion@scipy.org . If you want to reply to individual messages, 
please 
turn digest delivery *off* and receive and respond to messages normally.

Thank you.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Setting contents of buffer for array object

[Robert Kern]

On Feb 10, 2008 5:15 PM, Matthew Brett [EMAIL PROTECTED] wrote:
 Hi,

 I am sorry if I have missed something obvious, but is there any way in
 python of doing this:

 import numpy as np
 a = np.arange(10)
 b = np.arange(10)+1
 a.data = b.data # raises error, but I hope you see what I mean

 ?

Not really, no. Can you describe your use case in more detail?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Setting contents of buffer for array object

[Robert Kern]

On Feb 10, 2008 6:48 PM, Matthew Brett [EMAIL PROTECTED] wrote:
   import numpy as np
   a = np.arange(10)
   b = np.arange(10)+1
   a.data = b.data # raises error, but I hope you see what I mean
  
   ?
 
  Not really, no. Can you describe your use case in more detail?

 Yes - I am just writing the new median implementation.   To allow
 future optimization, I would like to have the same signature as
 mean():

 def median(a, axis=0, dtype=None, out=None)

 (axis=0 to change to axis=None default at some point).

 To do this, I need to copy the results of the median calculation in
 the routine into the array object given by 'out' - when passed.

Ah, I see. You definitely do not want to reassign the .data buffer in
this case. An out= parameter does not reassign the memory location
that the array object points to. It should use the allocated memory
that was already there. It shouldn't copy anything at all;
otherwise, median(x, out=out) is no better than out[:] =
median(x). Personally, I don't think that a function should expose an
out= parameter unless if it can make good on that promise of memory
efficency. Can you show us the current implementation that you have?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] non-contiguous array error

[Robert Kern]

On Feb 10, 2008 7:43 PM, Brad Malone [EMAIL PROTECTED] wrote:
 Hi, I am receiving a AttributeError: incompatible shape for a
 non-contiguous array error.  A quick illustration of the type of code
 that gives me the error is shown below:
 
 from numpy import *
 list=[i for i in range(0,27)]
 c=array(list)
 c.shape=(3,3,3)
 d=fft.fftn(c)
 d.shape=(27)
 error here
 

 I suppose this has something to do with the fact that the fourier
 transform of c has imaginary parts, which affects the way the
 information is stored in memory, and this messed up the call to
 .shape?

No. The problem is that (27) is not a tuple. Parentheses are also
used for grouping expressions in Python, so a single-element tuple
needs a comma to disambiguate. You want d.shape = (27,).

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Setting contents of buffer for array object

[Robert Kern]

On Feb 10, 2008 7:17 PM, Matthew Brett [EMAIL PROTECTED] wrote:
  Ah, I see. You definitely do not want to reassign the .data buffer in
  this case. An out= parameter does not reassign the memory location
  that the array object points to. It should use the allocated memory
  that was already there. It shouldn't copy anything at all;
  otherwise, median(x, out=out) is no better than out[:] =
  median(x). Personally, I don't think that a function should expose an
  out= parameter unless if it can make good on that promise of memory
  efficency.

 I agree - but there are more efficient median algorithms out there
 which can make use of the memory efficiently.  I wanted to establish
 the call signature to allow that.  I don't feel strongly about it
 though.

I say add the out= parameter when you use such an algorithm. But if
you like, just use slice assignment for now.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New bug with setup,py develop

[Robert Kern]

On Feb 11, 2008 3:49 AM, David Cournapeau [EMAIL PROTECTED] wrote:
 On Feb 11, 2008 5:38 PM, Robert Kern [EMAIL PROTECTED] wrote:
  On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote:
   I've just updated the SVN trunk to get the latest numscons merge.
   Something broke the support I put in for the setuptools develop
   command. In order to make sure that setuptools' develop works with
   numpy.distutils' build_src, we override the develop command to
   reinitialize the build_src command to add the --inplace option. This
   used to work as of r4772, but now any Fortran Extensions have the
   generated sources added twice.
 
  Spoke too soon. It fails with r4772, too.

 Does it mean that it was already broken before numscons merge or not ?

Yes. I think I figured out the problem. python setup.py develop
works fine. However, I ran into the problem when I added develop to
the end of a command line that already had build. So essentially,
build_src gets run twice. I'm not sure there is anything that can be
done about that given distutils' handling of options.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?

[Robert Kern]

On Feb 12, 2008 12:41 AM, Damian Eads [EMAIL PROTECTED] wrote:
 Robert Kern wrote:
  On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote:
  David Cournapeau wrote:
  On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote:
  Dear Lou,
 
  You may want to try using distutils or setuputils, which makes compiling
  extensions much easier. It does the hard work of finding out which flags
  are needed to compile extensions on the host platform. There are many
  examples on the web on how to use distutils to build C extensions
  (http://docs.python.org/ext/building.html).
  Unfortunately, this does not work. Distutils only knows how to build
  python extensions, not shared libraries. Depending on the platform, this
  is not the same thing, and mac os X is such a platform where both are
  not the same.
 
  cheers,
 
  David
  Really? distutils generates .so files for me, which I assume are shared
  libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not
  generate a shared library on a mac?
 
  Python extension modules are shared libraries, yes. But they must
  follow a particular format, namely exposing a correct
  initmodulename function. distutils/setuptools only maked Python
  extension modules, not arbitrary shared libraries.

 Perhaps I was a bit too liberal in my use of the term extension
 module. Several small libraries for a project at work do not define the
 standard initmodulename function, and yet they build with distutils. I
 can load them into ctypes without any hitches. Perhaps distutils does
 not check for the presence of the initmodulename function and required
 data structures? I'll admit I may be abusing distutils by using it for
 something for which it wasn't designed.

Yup. It usually works on Linux because the initmodulename bit isn't
checked. On Windows, it is, and the build will fail.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?

[Robert Kern]

On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote:
 David Cournapeau wrote:
  On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote:
  Dear Lou,
 
  You may want to try using distutils or setuputils, which makes compiling
  extensions much easier. It does the hard work of finding out which flags
  are needed to compile extensions on the host platform. There are many
  examples on the web on how to use distutils to build C extensions
  (http://docs.python.org/ext/building.html).
 
  Unfortunately, this does not work. Distutils only knows how to build
  python extensions, not shared libraries. Depending on the platform, this
  is not the same thing, and mac os X is such a platform where both are
  not the same.
 
  cheers,
 
  David

 Really? distutils generates .so files for me, which I assume are shared
 libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not
 generate a shared library on a mac?

Python extension modules are shared libraries, yes. But they must
follow a particular format, namely exposing a correct
initmodulename function. distutils/setuptools only maked Python
extension modules, not arbitrary shared libraries.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?

[Robert Kern]

On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote:
 David Cournapeau wrote:
  On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote:
  Dear Lou,
 
  You may want to try using distutils or setuputils, which makes compiling
  extensions much easier. It does the hard work of finding out which flags
  are needed to compile extensions on the host platform. There are many
  examples on the web on how to use distutils to build C extensions
  (http://docs.python.org/ext/building.html).
 
  Unfortunately, this does not work. Distutils only knows how to build
  python extensions, not shared libraries. Depending on the platform, this
  is not the same thing, and mac os X is such a platform where both are
  not the same.
 
  cheers,
 
  David

 Really? distutils generates .so files for me, which I assume are shared
 libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not
 generate a shared library on a mac?

As to David's point, yes, distutils makes a .so shared library on
Macs. This is not the same thing as a dynamic library (on Macs) which
is what ctypes needs (on Macs), IIRC. There is a subtle, but important
difference between the two.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] New bug with setup,py develop

[Robert Kern]

I've just updated the SVN trunk to get the latest numscons merge.
Something broke the support I put in for the setuptools develop
command. In order to make sure that setuptools' develop works with
numpy.distutils' build_src, we override the develop command to
reinitialize the build_src command to add the --inplace option. This
used to work as of r4772, but now any Fortran Extensions have the
generated sources added twice. This causes links to fail since the
same symbol shows up twice.

David, any ideas?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New bug with setup,py develop

[Robert Kern]

On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote:
 I've just updated the SVN trunk to get the latest numscons merge.
 Something broke the support I put in for the setuptools develop
 command. In order to make sure that setuptools' develop works with
 numpy.distutils' build_src, we override the develop command to
 reinitialize the build_src command to add the --inplace option. This
 used to work as of r4772, but now any Fortran Extensions have the
 generated sources added twice.

Spoke too soon. It fails with r4772, too.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Median advice

[Robert Kern]

On Feb 12, 2008 6:33 AM, Joris De Ridder
[EMAIL PROTECTED] wrote:

 On 12 Feb 2008, at 12:31, Matthew Brett wrote:

  def median(a, axis=0, out=None)
  (same signature as max, min etc)

 I would be slightly in favour of this option.
 Using the same signature would be convenient in code like

 def myfunc(myarray, somefunc):

 # do stuff
  ...
 x = somefunc(myarray, axis = 0, out = None)

 # do more stuff
  ...


 where somefunc could be median(), mean(), max(), min(), std() etc. I
 once wrote this kind of function to provide (small) image filtering.
 If the same signature is used, there is no need to special-case
 median(). I realise it's kind of a niche example, though.

I'm happy with that use case. The docstring should mention that out=
is not memory-optimized like it is for the others, though.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] unexpected downcast

[Robert Kern]

Alan G Isaac wrote:
 On Tue, 12 Feb 2008, dmitrey apparently wrote:
 from numpy import * 
 a = array((1.0, 2.0), float128)
 b=asfarray(a)
 type(a[0])
 #type 'numpy.float128' 
 type(b[0])
 #type 'numpy.float64' 
  __version__
 '1.0.5.dev4767'
 
 
 Dmitrey noted an unexpected down cast (above).
 Is there a reason for it?

That's just what asfarray is designed to do. If you don't give it a dtype, it 
uses float64. Changing it would be a redesign of the function that may break 
code. The amount of code is probably minimal, so I'm only -0 on changing it.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth.
   -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion