Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Samuel John
On 17.02.2012, at 21:46, Ralf Gommers wrote: [...] So far no one has managed to build the numpy/scipy combo with the LLVM-based compilers, so if you were willing to have a go at fixing that it would be hugely appreciated. See http://projects.scipy.org/scipy/ticket/1500 for details.

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Pauli Virtanen
20.02.2012 08:35, Paul Anton Letnes kirjoitti: In the language wars, I have one question. Why is Fortran not being considered? Fortran is OK for simple numerical algorithms, but starts to suck heavily if you need to do any string handling, I/O, complicated logic, or data structures. Most of

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Stéfan van der Walt
On Mon, Feb 20, 2012 at 1:54 AM, Pauli Virtanen p...@iki.fi wrote: 20.02.2012 08:35, Paul Anton Letnes kirjoitti: In the language wars, I have one question. Why is Fortran not being considered? Fortran is OK for simple numerical algorithms, but starts to suck heavily if you need to do any

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Charles R Harris
On Mon, Feb 20, 2012 at 2:54 AM, Pauli Virtanen p...@iki.fi wrote: 20.02.2012 08:35, Paul Anton Letnes kirjoitti: In the language wars, I have one question. Why is Fortran not being considered? Fortran is OK for simple numerical algorithms, but starts to suck heavily if you need to do any

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 12:43, skrev Charles R Harris: There also used to be a problem with unsigned types not being available. I don't know if that is still the case. Fortran -- like Python and Java -- does not have built-in unsigned integer types. It is never really a problem though. One can e.g.

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 10:54, skrev Pauli Virtanen: Fortran is OK for simple numerical algorithms, but starts to suck heavily if you need to do any string handling, I/O, complicated logic, or data structures For string handling, C is actually worse than Fortran. In Fortran a string can be sliced

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 08:35, skrev Paul Anton Letnes: In the language wars, I have one question. Why is Fortran not being considered? Fortran already implements many of the features that we want in NumPy: Yes ... but it does not make Fortran a systems programming language. Making NumPy is

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 08:35, skrev Paul Anton Letnes: As far as I can understand, implementing element-wise operations, slicing, and a host of other NumPy features is in some sense pointless - the Fortran compiler authors have already done it for us. Only if you know the array dimensions in

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 19.02.2012 00:09, skrev David Cournapeau: There are better languages than C++ that has most of the technical benefits stated in this discussion (rust and D being the most obvious ones), but whose usage is unrealistic today for various reasons: knowledge, availability on esoteric

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth considering. One would be to write NumPy in Python. E.g. we could use LLVM as a JIT-compiler and produce the performance critical code we need on the fly. LLVM and its C/C++ frontend Clang

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Charles R Harris
On Mon, Feb 20, 2012 at 9:55 AM, Sturla Molden stu...@molden.no wrote: Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth considering. One would be to write NumPy in Python. E.g. we could use LLVM as a JIT-compiler and produce the

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 08:55 AM, Sturla Molden wrote: Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth considering. One would be to write NumPy in Python. E.g. we could use LLVM as a JIT-compiler and produce the performance critical code we need on

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
C++11 has this option: for (auto item : container) { // iterate over the container object, // get a reference to each item // // container can be an STL class or // A C-style array with known size. } Which does this: for item in container: pass It is even

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
Would it be fair to say then, that you are expecting the discussion about C++ will mainly arise after the Mark has written the code? I can see that it will be easier to specific at that point, but there must be a serious risk that it will be too late to seriously consider an alternative

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Matthew Brett matthew.br...@gmail.com Hi, On Sat, Feb 18, 2012 at 8:38 PM, Travis Oliphant tra...@continuum.io wrote: We will need to see examples of what Mark is talking about and clarify some of the compiler issues. Certainly there is some risk that once code is written

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:14, skrev Charles R Harris: Would that work for Ruby also? One of the advantages of C++ is that the code doesn't need to be refactored to start with, just modified step by step going into the future. I think PyPy is close to what you are talking about. If we plant to

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Nathaniel Smith n...@pobox.com On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe mwwi...@gmail.com wrote: Is there a specific target platform/compiler combination you're thinking of where we can do tests on

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: You need at least a slightly different Python API to get anywhere, so numexpr/Theano is the right place to work on an implementation of this idea. Of course it would be nice if numexpr/Theano offered something as convenient as with

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/19 Sturla Molden stu...@molden.no Den 19.02.2012 10:28, skrev Mark Wiebe: Particular styles of using templates can cause this, yes. To properly do this kind of advanced C++ library work, it's important to think about the big-O notation behavior of your template instantiations, not

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Christopher Jordan-Squire
On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/20/2012 08:55 AM, Sturla Molden wrote: Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth considering. One would be to write NumPy in Python. E.g. we

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:18, skrev Dag Sverre Seljebotn: I think it is moot to focus on improving NumPy performance as long as in practice all NumPy operations are memory bound due to the need to take a trip through system memory for almost any operation. C/C++ is simply good enough. JIT is when

[Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 09:24 AM, Olivier Delalleau wrote: Hi Dag, Would you mind elaborating a bit on that example you mentioned at the end of your email? I don't quite understand what behavior you would like to achieve Sure, see below. I think we should continue discussion on numpy-discuss. I

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 18:34, skrev Christopher Jordan-Squire: I don't follow this. Could you expand a bit more? (Specifically, I wasn't aware that numpy could be 10-20x slower than a cython loop, if we're talking about the base numpy library--so core operations. I'm also not totally sure why a JIT

Re: [Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 6:46 PM, Dag Sverre Seljebotn wrote: On 02/20/2012 09:24 AM, Olivier Delalleau wrote: Hi Dag, Would you mind elaborating a bit on that example you mentioned at the end of your email? I don't quite understand what behavior you would like to achieve Sure, see below. I

Re: [Numpy-discussion] ndarray and lazy evaluation (was: Proposed Rodmap Overview)

2012-02-20 Thread Olivier Delalleau
Never mind. The link Francesc posted answered my question :) -=- Olivier Le 20 février 2012 12:54, Olivier Delalleau delal...@iro.umontreal.ca a écrit : Le 20 février 2012 12:46, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no a écrit : On 02/20/2012 09:24 AM, Olivier Delalleau wrote: Hi

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 09:34 AM, Christopher Jordan-Squire wrote: On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/20/2012 08:55 AM, Sturla Molden wrote: Den 20.02.2012 17:42, skrev Sturla Molden: There are still other options than C or C++ that are worth

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread Dag Sverre Seljebotn
On 02/20/2012 10:04 AM, Francesc Alted wrote: On Feb 20, 2012, at 6:46 PM, Dag Sverre Seljebotn wrote: On 02/20/2012 09:24 AM, Olivier Delalleau wrote: Hi Dag, Would you mind elaborating a bit on that example you mentioned at the end of your email? I don't quite understand what behavior you

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Francesc Alted
On Feb 20, 2012, at 7:08 PM, Dag Sverre Seljebotn wrote: On 02/20/2012 09:34 AM, Christopher Jordan-Squire wrote: On Mon, Feb 20, 2012 at 9:18 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/20/2012 08:55 AM, Sturla Molden wrote: Den 20.02.2012 17:42, skrev Sturla Molden:

Re: [Numpy-discussion] Forbidden charcter in the names argument of genfromtxt?

2012-02-20 Thread Brett Olsen
On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes hugad...@gwmail.gwu.edu wrote: Hey everyone, I have timeseries data in which the column label is simply a filename from which the original data was taken.  Here's some sample data: name1.txt  name2.txt  name3.txt 32  34    953

Re: [Numpy-discussion] Forbidden charcter in the names argument of genfromtxt?

2012-02-20 Thread Adam Hughes
Thanks Brett. I appreciate you taking the time to help me out. In particular, I did not know the correct syntax for this: data.dtype.names = names Which is very helpful. If I would have known how to access data.dtype.names, I think it would have saved me a great deal of trouble. I guess it's

Re: [Numpy-discussion] Forbidden charcter in the names argument of genfromtxt?

2012-02-20 Thread Skipper Seabold
On Mon, Feb 20, 2012 at 1:35 PM, Brett Olsen brett.ol...@gmail.com wrote: On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes hugad...@gwmail.gwu.edu wrote: Hey everyone, I have timeseries data in which the column label is simply a filename from which the original data was taken.  Here's some sample

Re: [Numpy-discussion] Forbidden charcter in the names argument of genfromtxt?

2012-02-20 Thread Adam Hughes
Thanks for clearing that up. On Mon, Feb 20, 2012 at 1:58 PM, Skipper Seabold jsseab...@gmail.comwrote: On Mon, Feb 20, 2012 at 1:35 PM, Brett Olsen brett.ol...@gmail.com wrote: On Sat, Feb 18, 2012 at 8:12 PM, Adam Hughes hugad...@gwmail.gwu.edu wrote: Hey everyone, I have timeseries

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Daniele Nicolodi
On 18/02/12 04:54, Sturla Molden wrote: This is not true. C++ can be much easier, particularly for those who already know Python. The problem: C++ textbooks teach C++ as a subset of C. Writing C in C++ just adds the complexity of C++ on top of C, for no good reason. I can write FORTRAN in any

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Matthieu Brucher
2012/2/20 Daniele Nicolodi dani...@grinta.net On 18/02/12 04:54, Sturla Molden wrote: This is not true. C++ can be much easier, particularly for those who already know Python. The problem: C++ textbooks teach C++ as a subset of C. Writing C in C++ just adds the complexity of C++ on top of

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 12:28 PM, Francesc Alted franc...@continuum.iowrote: On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: You need at least a slightly different Python API to get anywhere, so numexpr/Theano is the right place to work on an implementation of this idea. Of course

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 1:01 PM, James Bergstra james.bergs...@gmail.comwrote: On Mon, Feb 20, 2012 at 12:28 PM, Francesc Alted franc...@continuum.iowrote: On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: You need at least a slightly different Python API to get anywhere, so

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Lluís
Francesc Alted writes: On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: You need at least a slightly different Python API to get anywhere, so numexpr/Theano is the right place to work on an implementation of this idea. Of course it would be nice if numexpr/Theano offered something

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Neal Becker
Charles R Harris wrote: On Fri, Feb 17, 2012 at 12:09 PM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Feb 17, 2012 at 1:00 PM, Christopher Jordan-Squire cjord...@uw.edu wrote: On Fri, Feb 17, 2012 at 10:21 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Feb 17, 2012 at 11:52 AM, Eric

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Lluís
Lluís writes: Francesc Alted writes: On Feb 20, 2012, at 6:18 PM, Dag Sverre Seljebotn wrote: You need at least a slightly different Python API to get anywhere, so numexpr/Theano is the right place to work on an implementation of this idea. Of course it would be nice if numexpr/Theano

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Paul Anton Letnes
On 20. feb. 2012, at 16:29, Sturla Molden wrote: Den 20.02.2012 08:35, skrev Paul Anton Letnes: In the language wars, I have one question. Why is Fortran not being considered? Fortran already implements many of the features that we want in NumPy: Yes ... but it does not make Fortran a

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread Lluís
James Bergstra writes: [...] I should add that the biggest benefit of expressing things as compound expressions in this way is not in saving temporaries (though that is nice) it's being able to express enough computation work at a time that it offsets the time required to ship the

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread James Bergstra
Looks like Dag forked the discussion of lazy evaluation to a new thread ([Numpy-discussion] ndarray and lazy evaluation). There are actually several projects inspired by this sort of design: off the top of my head I can think of Theano, copperhead, numexpr, arguably sympy, and some non-public

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Sturla Molden
Den 20.02.2012 20:14, skrev Daniele Nicolodi: Hello Sturla, unrelated to the numpy tewrite debate, can you please suggest some resources you think can be used to learn how to program C++ the proper way? Thank you. Cheers, This is totally OT on this list, however ... Scott Meyer's books

Re: [Numpy-discussion] ndarray and lazy evaluation

2012-02-20 Thread James Bergstra
On Mon, Feb 20, 2012 at 2:57 PM, Lluís xscr...@gmx.net wrote: James Bergstra writes: [...] I should add that the biggest benefit of expressing things as compound expressions in this way is not in saving temporaries (though that is nice) it's being able to express enough computation work

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Robert Kern
On Mon, Feb 20, 2012 at 19:55, Paul Anton Letnes paul.anton.let...@gmail.com wrote: On 20. feb. 2012, at 16:29, Sturla Molden wrote: - in newer standards it has some nontrivial mathematical functions: gamma, bessel, etc. that numpy lacks right now That belongs to SciPy. I don't see

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-20 Thread Travis Oliphant
Interesting you bring this up. I actually have a working prototype of using Python to emit LLVM. I will be showing it at the HPC tutorial that I am giving at PyCon.I will be making this available after PyCon to a wider audience as open source. It uses llvm-py (modified to work with

[Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Yaroslav Halchenko
Hi to all Numeric Python experts, could not think of a mailing list with better fit to my question which might have an obvious answer: straightforward (naive) Python code to answer my question would be something like import random, itertools n,p,k=100,50,10 # don't try to run with this

Re: [Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Christopher Jordan-Squire
If you're using numpy 2.0 (the development branch), the function numpy.random.choice might do what you're looking for. -Chris On Mon, Feb 20, 2012 at 8:35 PM, Yaroslav Halchenko li...@onerussian.com wrote: Hi to all Numeric  Python experts, could not think of a mailing list with better fit to

Re: [Numpy-discussion] is there an efficient way to get a random set of subsets/combinations?

2012-02-20 Thread Val Kalatsky
Hi Slava, Since your k is only 10, here is a quickie: import numpy as np arr = np.arange(n) for i in range(k): np.random.shuffle(arr) print np.sort(arr[:p]) If your ever get non-unique entries in a set of k=10 for your n and p, consider yourself lucky:) Val On Mon, Feb 20, 2012 at