Re: [Numpy-discussion] Choosing between NumPy and SciPy functions

2014-10-28 Thread David Cournapeau
On Tue, Oct 28, 2014 at 5:24 AM, Sturla Molden sturla.mol...@gmail.com wrote: Matthew Brett matthew.br...@gmail.com wrote: Is this an option for us? Aren't we a little behind the performance curve on FFT after we lost FFTW? It does not run on Windows because it uses POSIX to allocate

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Jerome Kieffer
On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com wrote: It's definitely attractive. Some potential issues that might need dealing with, based on a quick skim: In my tests, numpy's FFTPACK isn't that bad considering * (virtually) no extra overhead for installation *

Re: [Numpy-discussion] numpy.i and std::complex

2014-10-28 Thread Robert Kern
On Mon, Oct 27, 2014 at 11:36 PM, Sturla Molden sturla.mol...@gmail.com wrote: Robert Kern robert.k...@gmail.com wrote: Please stop haranguing the new guy for not knowing things that you know. I am not doing any of that. You are the only one haranguing here. I understand that it's not your

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Charles R Harris
On Tue, Oct 28, 2014 at 1:32 AM, Jerome Kieffer jerome.kief...@esrf.fr wrote: On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com wrote: It's definitely attractive. Some potential issues that might need dealing with, based on a quick skim: In my tests, numpy's FFTPACK isn't

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread David Cournapeau
On Tue, Oct 28, 2014 at 9:19 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, Oct 28, 2014 at 1:32 AM, Jerome Kieffer jerome.kief...@esrf.fr wrote: On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com wrote: It's definitely attractive. Some potential issues

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Henry Gomersall
On 28/10/14 09:41, David Cournapeau wrote: The real issue with fftw (besides the license) is the need for plan computation, which are expensive (but are not needed for each transform). Handling this in a way that is user friendly while tweakable for advanced users is not easy, and IMO more

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Henry Gomersall
On 28/10/14 04:28, Nathaniel Smith wrote: - not sure if it can handle non-power-of-two problems at all, or at all efficiently. (FFTPACK isn't great here either but major regressions would be bad.) From my reading, this seems to be the biggest issue with FFTS (from my reading as well) and

Re: [Numpy-discussion] Choosing between NumPy and SciPy functions

2014-10-28 Thread Stefan van der Walt
Hi Michael On 2014-10-27 15:26:58, D. Michael McFarland dm...@dmmcf.net wrote: What I would like to ask about is the situation this illustrates, where both NumPy and SciPy provide similar functionality (sometimes identical, to judge by the documentation). Is there some guidance on which is to

Re: [Numpy-discussion] numpy.i and std::complex

2014-10-28 Thread Sturla Molden
Robert Kern robert.k...@gmail.com wrote: The polite, welcoming response to someone coming along with a straightforward, obviously-correct contribution to our SWIG capabilities is Thank you!, not perhaps you overestimate the number of NumPy users who use Swig. That was a response to

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Sturla Molden
Jerome Kieffer jerome.kief...@esrf.fr wrote: Because the plan creation was taking ages with FFTw, numpy's FFTPACK was often faster (overall) Matlab switched from FFTPACK to FFTW because the latter was faster in general. If FFTW guesses a plan it does not take very long. Actual measurements can

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Sturla Molden
David Cournapeau courn...@gmail.com wrote: The real issue with fftw (besides the license) is the need for plan computation, which are expensive (but are not needed for each transform). This is not a problem if you thell FFTW to guess a plan instead of making measurements. FFTPACK needs to set

Re: [Numpy-discussion] Choosing between NumPy and SciPy functions

2014-10-28 Thread Pierre Barbier de Reuille
I would add one element to the discussion: for some (odd) reasons, SciPy is lacking the functions `rfftn` and `irfftn`, functions using half the memory space compared to their non-real equivalent `fftn` and `ifftn`. However, I haven't (yet) seriously tested `scipy.fftpack.fftn` vs. `np.fft.rfftn`

Re: [Numpy-discussion] Choosing between NumPy and SciPy functions

2014-10-28 Thread Sturla Molden
Pierre Barbier de Reuille pie...@barbierdereuille.net wrote: I would add one element to the discussion: for some (odd) reasons, SciPy is lacking the functions `rfftn` and `irfftn`, functions using half the memory space compared to their non-real equivalent `fftn` and `ifftn`. In both NumPy

Re: [Numpy-discussion] multi-dimensional c++ proposal

2014-10-28 Thread Sturla Molden
Neal Becker ndbeck...@gmail.com wrote: That's harsh! Do you have any specific features you dislike? Are you objecting to the syntax? I have programmed C++ for almost 15 years. But I cannot look at the proposed code an get a mental image of what it does. It is not a specific feature, but

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Nathaniel Smith
On 28 Oct 2014 07:32, Jerome Kieffer jerome.kief...@esrf.fr wrote: On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com wrote: It's definitely attractive. Some potential issues that might need dealing with, based on a quick skim: In my tests, numpy's FFTPACK isn't that bad

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Eelco Hoogendoorn
If I may 'hyjack' the discussion back to the meta-point: should we be having this discussion on the numpy mailing list at all? Perhaps the 'batteries included' philosophy made sense in the early days of numpy; but given that there are several fft libraries with their own pros and cons, and that

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread David Cournapeau
I On Tue, Oct 28, 2014 at 2:31 PM, Nathaniel Smith n...@pobox.com wrote: On 28 Oct 2014 07:32, Jerome Kieffer jerome.kief...@esrf.fr wrote: On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com wrote: It's definitely attractive. Some potential issues that might need

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Nathaniel Smith
On 28 Oct 2014 14:48, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: If I may 'hyjack' the discussion back to the meta-point: should we be having this discussion on the numpy mailing list at all? Of course we should. Perhaps the 'batteries included' philosophy made sense in the early

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread David Cournapeau
On Tue, Oct 28, 2014 at 3:06 PM, David Cournapeau courn...@gmail.com wrote: I On Tue, Oct 28, 2014 at 2:31 PM, Nathaniel Smith n...@pobox.com wrote: On 28 Oct 2014 07:32, Jerome Kieffer jerome.kief...@esrf.fr wrote: On Tue, 28 Oct 2014 04:28:37 + Nathaniel Smith n...@pobox.com

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Alexander Belopolsky
On Mon, Oct 27, 2014 at 9:41 PM, Yuxiang Wang yw...@virginia.edu wrote: In my opinion - because they don't do the same thing, especially when you think in terms in lower-level. ndarray.flat returns an iterator; ndarray.flatten() returns a copy; ndarray.ravel() only makes copies when

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Sturla Molden
Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: Perhaps the 'batteries included' philosophy made sense in the early days of numpy; but given that there are several fft libraries with their own pros and cons, and that most numpy projects will use none of them at all, why should numpy

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Nathaniel Smith
On 28 Oct 2014 16:58, Alexander Belopolsky ndar...@mac.com wrote: On Mon, Oct 27, 2014 at 9:41 PM, Yuxiang Wang yw...@virginia.edu wrote: In my opinion - because they don't do the same thing, especially when you think in terms in lower-level. ndarray.flat returns an iterator;

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Daniele Nicolodi
On 28/10/14 16:50, David Cournapeau wrote: Nothing impossible (looks like Sony at least uses this code on windows: https://github.com/anthonix/ffts/issues/27#issuecomment-40204403), but not a 2 hours thing either. One of the downsides of the BSD license :) Cheers, Daniele

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Alan G Isaac
On 10/28/2014 1:25 PM, Nathaniel Smith wrote: I too would be curious to know why .flat exists (beyond it seemed like a good idea at the time How else would you iterate over all items of a multidimensional array? As an example application, use it to assign to an arbitrary diagonal. (It can be

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Stephan Hoyer
On Tue, Oct 28, 2014 at 10:25 AM, Nathaniel Smith n...@pobox.com wrote: I too would be curious to know why .flat exists (beyond it seemed like a good idea at the time ;-)). I've always treated it as some weird legacy thing and ignored it, and this has worked out well for me. Is there any

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Stefan van der Walt
On 2014-10-28 19:37:17, Daniele Nicolodi dani...@grinta.net wrote: On 28/10/14 16:50, David Cournapeau wrote: Nothing impossible (looks like Sony at least uses this code on windows: https://github.com/anthonix/ffts/issues/27#issuecomment-40204403), but not a 2 hours thing either. One of the

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Daniele Nicolodi
On 28/10/14 18:44, Stefan van der Walt wrote: On 2014-10-28 19:37:17, Daniele Nicolodi dani...@grinta.net wrote: On 28/10/14 16:50, David Cournapeau wrote: Nothing impossible (looks like Sony at least uses this code on windows: https://github.com/anthonix/ffts/issues/27#issuecomment-40204403),

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Alan G Isaac
On 10/28/2014 1:42 PM, Stephan Hoyer wrote: np.nditer is a reasonable alternative to .flat (and it's documented as such), but it's a rather inelegant, kitchen-sink type function. I'm not sure what reasonable means here, other than in principle, possible to use. In particular, `flat` is much

[Numpy-discussion] [ANN] Python for Scientific Computing conference in Boulder, CO; April'15

2014-10-28 Thread Fernando Perez
Hi folks, a colleague from NCAR in Boulder just sent me this link about a conference they are organizing in the spring: https://sea.ucar.edu/conference/2015 I figured this might be of interest to many on these lists. The actual call isn't up yet, so if you're interested, watch that site for an

Re: [Numpy-discussion] [ANN] Python for Scientific Computing conference in Boulder, CO; April'15

2014-10-28 Thread Jean-Baptiste Marquette
Le 28 oct. 2014 à 19:09, Fernando Perez fperez@gmail.com a écrit : a colleague from NCAR in Boulder just sent me this link about a conference they are organizing in the spring: Wrong year on the web page: April 13 - 17, 2014 Cheers, JB

Re: [Numpy-discussion] [ANN] Python for Scientific Computing conference in Boulder, CO; April'15

2014-10-28 Thread Fernando Perez
thanks, reported. On Tue, Oct 28, 2014 at 11:23 AM, Jean-Baptiste Marquette marqu...@iap.fr wrote: Le 28 oct. 2014 à 19:09, Fernando Perez fperez@gmail.com a écrit : a colleague from NCAR in Boulder just sent me this link about a conference they are organizing in the spring: Wrong

Re: [Numpy-discussion] Generalize hstack/vstack -- stack; Block matrices like in matlab

2014-10-28 Thread Nathaniel Smith
On 28 Oct 2014 18:34, Stefan Otte stefan.o...@gmail.com wrote: Hey, In the last weeks I tested `np.asarray(np.bmat())` as `stack` function and it works quite well. So the question persits: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Din Vadhia
It would be nice if there were a single meta numpy.flatten() function with kind: {'ravel', 'flatten', 'flat', 'reshape'} options, similar to the numpy.sort() function kind : {‘quicksort’, ‘mergesort’, ‘heapsort’} options. It would also make it easier to select the best option for each problem

Re: [Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt

2014-10-28 Thread Chris Barker
A few thoughts: 1) yes, a faster, more memory efficient text file parser would be great. Yeah, if your workflow relies on parsing lots of huge text files, you probably need another workflow. But it's a really really common thing to nee to do -- why not do it fast? 2) you are describing a special

Re: [Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt

2014-10-28 Thread Nathaniel Smith
On 28 Oct 2014 20:10, Chris Barker chris.bar...@noaa.gov wrote: Memory efficiency -- somethign like my growable array is not all that hard to implement and pretty darn quick -- you just do the usual trick_ over allocate a bit of memory, and when it gets full re-allocate a larger chunk. Can't

Re: [Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt

2014-10-28 Thread Benjamin Root
As a bit of an aside, I have just discovered that for fixed-width text data, numpy's text readers seems to edge out pandas' read_fwf(), and numpy has the advantage of being able to specify the dtypes ahead of time (seems that the pandas version just won't allow it, which means I end up with

Re: [Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt

2014-10-28 Thread Julian Taylor
On 28.10.2014 21:24, Nathaniel Smith wrote: On 28 Oct 2014 20:10, Chris Barker chris.bar...@noaa.gov mailto:chris.bar...@noaa.gov wrote: Memory efficiency -- somethign like my growable array is not all that hard to implement and pretty darn quick -- you just do the usual trick_ over allocate

Re: [Numpy-discussion] Memory efficient alternative for np.loadtxt and np.genfromtxt

2014-10-28 Thread Chris Barker
On Tue, Oct 28, 2014 at 1:24 PM, Nathaniel Smith n...@pobox.com wrote: Memory efficiency -- somethign like my growable array is not all that hard to implement and pretty darn quick -- you just do the usual trick_ over allocate a bit of memory, and when it gets full re-allocate a larger

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-28 Thread Stefan van der Walt
On 2014-10-28 19:55:57, Daniele Nicolodi dani...@grinta.net wrote: On 28/10/14 18:44, Stefan van der Walt wrote: On 2014-10-28 19:37:17, Daniele Nicolodi dani...@grinta.net wrote: On 28/10/14 16:50, David Cournapeau wrote: Nothing impossible (looks like Sony at least uses this code on windows:

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Alexander Belopolsky
On Tue, Oct 28, 2014 at 1:42 PM, Stephan Hoyer sho...@gmail.com wrote: .flat lets you iterate over all elements of a N-dimensional array as if it was 1D, without ever needing to make a copy of the array. In contrast, ravel() and reshape(-1) cannot always avoid a copy, because they need to

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Nathaniel Smith
On Wed, Oct 29, 2014 at 12:37 AM, Alexander Belopolsky ndar...@mac.com wrote: On Tue, Oct 28, 2014 at 1:42 PM, Stephan Hoyer sho...@gmail.com wrote: .flat lets you iterate over all elements of a N-dimensional array as if it was 1D, without ever needing to make a copy of the array. In

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Alexander Belopolsky
On Tue, Oct 28, 2014 at 9:23 PM, Nathaniel Smith n...@pobox.com wrote: OTOH trying to make .flat into a full duck-compatible ndarray-like type is a non-starter; it would take a tremendous amount of work for no clear gain. I don't think so - I think all the heavy lifting is already done in

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-28 Thread Nathaniel Smith
On 29 Oct 2014 01:47, Alexander Belopolsky ndar...@mac.com wrote: On Tue, Oct 28, 2014 at 9:23 PM, Nathaniel Smith n...@pobox.com wrote: OTOH trying to make .flat into a full duck-compatible ndarray-like type is a non-starter; it would take a tremendous amount of work for no clear gain.

[Numpy-discussion] Deprecate pkgload, PackageLoader

2014-10-28 Thread Charles R Harris
Hi All, It is proposed to deprecate, then remove, pkgload and PackageLoader. Complaints? Cries of Anguish? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion