[Numpy-discussion] ANN: numexpr 2.3.1 released

2014-02-18 Thread Francesc Alted
== Announcing Numexpr 2.3.1 == Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python. It wears

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Julian Taylor
On Tue, Feb 18, 2014 at 1:47 AM, David Cournapeau courn...@gmail.com wrote: On Mon, Feb 17, 2014 at 7:31 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: hi, I noticed that during some simplistic benchmarks (e.g. https://github.com/numpy/numpy/issues/4310) a lot of time is spent in

Re: [Numpy-discussion] Proposal: Chaining np.dot with mdot helper function

2014-02-18 Thread Stefan Otte
Just to give an idea about the performance implications I timed the operations on my machine %timeit reduce(dotp, [x, v, x.T, y]).shape 1 loops, best of 3: 1.32 s per loop %timeit reduce(dotTp, [x, v, x.T, y][::-1]).shape 1000 loops, best of 3: 394 µs per loop I was just interested in a nicer

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Sturla Molden
Julian Taylor jtaylor.deb...@googlemail.com wrote: I was thinking of something much simpler, just a layer of pointer stacks for different allocations sizes, the larger the size the smaller the cache with pessimistic defaults. e.g. the largest default cache layer is 128MB and with one or two

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Sturla Molden
Charles R Harris charlesr.har...@gmail.com wrote: This is apropos issue #899 a href=https://github.com/numpy/numpy/issues/899;https://github.com/numpy/numpy/issues/899/a, where it is suggested that power promote integers to float. That sounds reasonable to me, but such a change in behavior

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 9:51 AM, Sturla Molden sturla.mol...@gmail.com wrote: Charles R Harris charlesr.har...@gmail.com wrote: This is apropos issue #899 a href=https://github.com/numpy/numpy/issues/899;https://github.com/numpy/numpy/issues/899/a, where it is suggested that power promote

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Todd
On Feb 18, 2014 11:55 AM, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 9:51 AM, Sturla Molden sturla.mol...@gmail.com wrote: Charles R Harris charlesr.har...@gmail.com wrote: This is apropos issue #899 a href=https://github.com/numpy/numpy/issues/899;

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 10:59 AM, Todd toddr...@gmail.com wrote: On Feb 18, 2014 11:55 AM, Robert Kern robert.k...@gmail.com wrote: I am -1 on the proposal to make power(x:int, y:int) always return a float. It is usually trivial to just make the exponent a float if one wants a float

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Sturla Molden
Robert Kern robert.k...@gmail.com wrote: That's fine if you only have one value for each operand. When you have multiple values for each operand, say an exponent array containing both positive and negative integers, that becomes a problem. I don't really see why. If you have a negative

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 11:19 AM, Sturla Molden sturla.mol...@gmail.com wrote: Robert Kern robert.k...@gmail.com wrote: That's fine if you only have one value for each operand. When you have multiple values for each operand, say an exponent array containing both positive and negative

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Sturla Molden
Robert Kern robert.k...@gmail.com wrote: We're talking about numpy.power(), not just ndarray.__pow__(). The equivalence of the two is indeed an implementation detail, but I do think that it is useful to maintain the equivalence. If we didn't, it would be the only exception, to my knowledge.

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 11:44 AM, Sturla Molden sturla.mol...@gmail.com wrote: Robert Kern robert.k...@gmail.com wrote: We're talking about numpy.power(), not just ndarray.__pow__(). The equivalence of the two is indeed an implementation detail, but I do think that it is useful to maintain

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Nathaniel Smith
Perhaps integer power should raise an error on negative powers? That way people will at least be directed to use arr ** -1.0 instead of silently getting nonsense from arr ** -1. On 18 Feb 2014 06:57, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 11:44 AM, Sturla Molden

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 12:00 PM, Nathaniel Smith n...@pobox.com wrote: Perhaps integer power should raise an error on negative powers? That way people will at least be directed to use arr ** -1.0 instead of silently getting nonsense from arr ** -1. Controllable by np.seterr(invalid=...)? I

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Sebastian Berg
On Di, 2014-02-18 at 11:44 +, Sturla Molden wrote: Robert Kern robert.k...@gmail.com wrote: We're talking about numpy.power(), not just ndarray.__pow__(). The equivalence of the two is indeed an implementation detail, but I do think that it is useful to maintain the equivalence. If we

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Sturla Molden
Nathaniel Smith n...@pobox.com wrote: Perhaps integer power should raise an error on negative powers? That way people will at least be directed to use arr ** -1.0 instead of silently getting nonsense from arr ** -1. That sounds far better than silently returning errorneous results. Sturla

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Nathaniel Smith
On 18 Feb 2014 07:07, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 12:00 PM, Nathaniel Smith n...@pobox.com wrote: Perhaps integer power should raise an error on negative powers? That way people will at least be directed to use arr ** -1.0 instead of silently getting

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 1:46 PM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 07:07, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 12:00 PM, Nathaniel Smith n...@pobox.com wrote: Perhaps integer power should raise an error on negative powers? That way people will

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread josef . pktd
On Tue, Feb 18, 2014 at 8:53 AM, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 1:46 PM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 07:07, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 12:00 PM, Nathaniel Smith n...@pobox.com wrote: Perhaps

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread Robert Kern
On Tue, Feb 18, 2014 at 2:37 PM, josef.p...@gmail.com wrote: On Tue, Feb 18, 2014 at 8:53 AM, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 1:46 PM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 07:07, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at

[Numpy-discussion] Numpy type inheritance

2014-02-18 Thread Charles R Harris
Hi All, I'm looking for some expert explication of the problems in issues #1398https://github.com/numpy/numpy/issues/1398and #1397 https://github.com/numpy/numpy/issues/1397. The segfault in the second looks nasty. Chuck ___ NumPy-Discussion mailing

Re: [Numpy-discussion] Proposal to make power return float, and other such things.

2014-02-18 Thread josef . pktd
On Tue, Feb 18, 2014 at 9:42 AM, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 2:37 PM, josef.p...@gmail.com wrote: On Tue, Feb 18, 2014 at 8:53 AM, Robert Kern robert.k...@gmail.com wrote: On Tue, Feb 18, 2014 at 1:46 PM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb

[Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Frédéric Bastien
Hi, In a ticket I did a coment and Charles suggested that I post it here: In Theano we have an C implementation of a faster RNG: MRG31k3p. It is faster on CPU, and we have a GPU implementation. It would be relatively easy to parallize on the CPU with OpenMP. If someone is interested to port

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Julian Taylor
On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith n...@pobox.com wrote: On 17 Feb 2014 15:17, Sturla Molden sturla.mol...@gmail.com wrote: Julian Taylor jtaylor.deb...@googlemail.com wrote: When an array is created it tries to get its memory from the cache and when its deallocated it

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Nathaniel Smith
On 18 Feb 2014 10:21, Julian Taylor jtaylor.deb...@googlemail.com wrote: On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith n...@pobox.com wrote: On 17 Feb 2014 15:17, Sturla Molden sturla.mol...@gmail.com wrote: Julian Taylor jtaylor.deb...@googlemail.com wrote: When an array is

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Sturla Molden
AFAIK, CMRG (MRG31k3p) is more equidistributed than Mersenne Twister, but the period is much shorter. However, MT is getting acceptance as the PRNG of choice for numerical work. And when we are doing stochastic simulations in Python, the speed of the PRNG is unlikely to be the bottleneck. Sturla

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Sturla Molden
I am cross-posting this to Cython user group to make sure they see this. Sturla Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 10:21, Julian Taylor jtaylor.deb...@googlemail.com wrote: On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith n...@pobox.com wrote: On 17 Feb 2014 15:17, Sturla

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Frédéric Bastien
I won't go in the discussion of which RNG is better for some problems. I'll just tell why we pick this one. We needed a parallel RNG and we wanted to use the same RNG on CPU and on GPU. We discussed with a professor in our department that is well know in that field(Pierre L'Ecuyer) and he

[Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Charles R Harris
Hi All, There is an old ticket, #1499 https://github.com/numpy/numpy/issues/1499, that suggest adding a segment_axis function. def segment_axis(a, length, overlap=0, axis=None, end='cut', endvalue=0): Generate a new array that chops the given array along the given axis into overlapping

[Numpy-discussion] Rethinking multiple dimensional indexing with sequences?

2014-02-18 Thread Sebastian Berg
Hey all, currently in numpy this is possible: a = np.zeros((5, 5)) a[[0, slice(None, None)]] #this behaviour has its quirks, since the correct way is: a[(0, slice(None, None))] # or identically a[0, :] The problem with using an arbitrary sequence is, that an arbitrary sequence is also typically

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Benjamin Root
yes, but I don't like the name too much. Unfortunately, I can't think of a better one. Ben Root On Tue, Feb 18, 2014 at 11:05 AM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, There is an old ticket, #1499 https://github.com/numpy/numpy/issues/1499, that suggest adding a

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Matthieu Brucher
I won't dive into the discussion as well, except to say that parallel RNGs have to have specific characteristics, mainly to initialize many RNGs at the same time. I don't know how MRG31k3p handles this, as the publications was not very clear on this aspect. I guess it falls down as the other from

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Sebastian Berg
On Di, 2014-02-18 at 09:05 -0700, Charles R Harris wrote: Hi All, There is an old ticket, #1499, that suggest adding a segment_axis function. def segment_axis(a, length, overlap=0, axis=None, end='cut', endvalue=0): Generate a new array that chops the given array along the given

Re: [Numpy-discussion] Rethinking multiple dimensional indexing with sequences?

2014-02-18 Thread Sebastian Berg
On Di, 2014-02-18 at 17:09 +0100, Sebastian Berg wrote: Hey all, snip Now also NumPy commonly uses lists here to build up indexing tuples (since they are mutable), however would it really be so bad if we had to do `arr[tuple(slice_list)]` in the end to resolve this issue? So the proposal

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Nathaniel Smith
On 18 Feb 2014 11:05, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, There is an old ticket, #1499, that suggest adding a segment_axis function. def segment_axis(a, length, overlap=0, axis=None, end='cut', endvalue=0): Generate a new array that chops the given array along the

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Sturla Molden
Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, The main issue with PRNG and MT is that you don't know how to initialize all MT generators properly. A hash-based PRNG is much more efficient in that regard (see Random123 for a more detailed explanation). From what I heard, if MT is

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Andreas Hilboll
On 18.02.2014 17:47, Sturla Molden wrote: Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, The main issue with PRNG and MT is that you don't know how to initialize all MT generators properly. A hash-based PRNG is much more efficient in that regard (see Random123 for a more detailed

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Charles R Harris
On Tue, Feb 18, 2014 at 9:40 AM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 11:05, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, There is an old ticket, #1499, that suggest adding a segment_axis function. def segment_axis(a, length, overlap=0, axis=None,

Re: [Numpy-discussion] Suggestion: Port Theano RNG implementation to NumPy

2014-02-18 Thread Sturla Molden
Andreas Hilboll li...@hilboll.de wrote: On 18.02.2014 17:47, Sturla Molden wrote: Undortunately the DCMT code was LGPL, not BSD, I don't know if this has changed. I just checked. The file dcmt0.6.1b.tgz, available from http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/DC/dc.html, contains

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Jaime Fernández del Río
On Tue, Feb 18, 2014 at 9:03 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, Feb 18, 2014 at 9:40 AM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 11:05, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, There is an old ticket, #1499, that suggest

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Nathaniel Smith
On 18 Feb 2014 12:04, Charles R Harris charlesr.har...@gmail.com wrote: Where does 'shingle' come from. I can see the analogy but haven't seen that as a technical term. It just seems like a good name :-). -n ___ NumPy-Discussion mailing list

Re: [Numpy-discussion] allocated memory cache for numpy

2014-02-18 Thread Julian Taylor
On 18.02.2014 16:21, Julian Taylor wrote: On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith n...@pobox.com wrote: On 17 Feb 2014 15:17, Sturla Molden sturla.mol...@gmail.com wrote: Julian Taylor jtaylor.deb...@googlemail.com wrote: When an array is created it tries to get its memory from the

Re: [Numpy-discussion] New (old) function proposal.

2014-02-18 Thread Tony Yu
On Tue, Feb 18, 2014 at 11:11 AM, Jaime Fernández del Río jaime.f...@gmail.com wrote: On Tue, Feb 18, 2014 at 9:03 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, Feb 18, 2014 at 9:40 AM, Nathaniel Smith n...@pobox.com wrote: On 18 Feb 2014 11:05, Charles R Harris

Re: [Numpy-discussion] Rethinking multiple dimensional indexing with sequences?

2014-02-18 Thread Nathaniel Smith
So to be clear - what's being suggested is that code like this will be deprecated in 1.9, and then in some future release break: slices = [] for i in ...: slices.append(make_slice(...)) subarray = arr[slices] Instead, you will have to do: subarray = arr[tuple(slices)] And the reason is

[Numpy-discussion] Default dtype of genfromtxt

2014-02-18 Thread Charles R Harris
This is apropos issue #1860 https://github.com/numpy/numpy/issues/1860, where it is proposed that the default dtype of genfromtxt should be None rather than float. A decision is needed. Thoughts? Chuck ___ NumPy-Discussion mailing list

Re: [Numpy-discussion] Default dtype of genfromtxt

2014-02-18 Thread Benjamin Root
I can certainly see the advantage of switching over to None. It makes a lot of sense. One thing that concerns me, though is a file full of whole number values. Would that get interpreted as integers? If so, then that affect needs to be aligned with some other proposed changes with respect to

[Numpy-discussion] except expression discussion on python-ideas

2014-02-18 Thread Alexander Belopolsky
I would like to invite numpy community to weigh in on the idea that is getting momentum at https://mail.python.org/pipermail/python-ideas/2014-February/025437.html The main motivation is to provide syntactic alternative to proliferation of default value options, so that x = getattr(u, 'answer',

[Numpy-discussion] Document server error.

2014-02-18 Thread Charles R Harris
From issue #1951 https://github.com/numpy/numpy/issues/1951: The following URL shows an 500 internal server error: http://docs.scipy.org/doc/numpy/reference/generated/numpy.var.html Can someone with access to the server take a look? TIA, Chuck ___