Re: [Numpy-discussion] numpy 1.7.0 release?

2011-12-06 Thread Wes McKinney
On Tue, Dec 6, 2011 at 4:11 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, Dec 5, 2011 at 8:43 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: Hi all, It's been a little over 6 months since the release of 1.6.0 and the NA debate has quieted down, so I'd like to ask your

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Wes McKinney
On Mon, Dec 12, 2011 at 12:34 PM, Warren Weckesser warren.weckes...@enthought.com wrote: On Mon, Dec 12, 2011 at 10:22 AM, Chris.Barker chris.bar...@noaa.gov wrote: On 12/11/11 8:40 AM, Ralf Gommers wrote: On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker chris.bar...@noaa.gov     * If we

Re: [Numpy-discussion] PyInt and Numpy's int64 conversion

2011-12-23 Thread Wes McKinney
On Fri, Dec 23, 2011 at 4:37 AM, xantares 09 xantare...@hotmail.com wrote: Hi, I'm using Numpy from the C python api side while tweaking my SWIG interface to work with numpy array types. I want to convert a numpy array of integers (whose elements are numpy's 'int64') The problem is that it

Re: [Numpy-discussion] PyInt and Numpy's int64 conversion

2011-12-24 Thread Wes McKinney
On Sat, Dec 24, 2011 at 3:11 AM, xantares 09 xantare...@hotmail.com wrote: From: wesmck...@gmail.com Date: Fri, 23 Dec 2011 12:31:45 -0500 To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] PyInt and Numpy's int64 conversion On Fri, Dec 23, 2011 at 4:37 AM, xantares 09

Re: [Numpy-discussion] GSOC

2011-12-29 Thread Wes McKinney
On Thu, Dec 29, 2011 at 4:36 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Dec 29, 2011 at 9:50 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, I thought I'd raise this topic just to get some ideas out there. At the moment I see two areas that I'd like to see

Re: [Numpy-discussion] Enum type

2012-01-03 Thread Wes McKinney
On Tue, Jan 3, 2012 at 12:46 PM, Ognen Duzlevski og...@enthought.com wrote: Hello, I am playing with adding an enum dtype to numpy (to get my feet wet in numpy really). I have looked at the https://github.com/martinling/numpy_quaternion and I feel comfortable with my understanding of adding

Re: [Numpy-discussion] Enum type

2012-01-03 Thread Wes McKinney
On Tue, Jan 3, 2012 at 1:06 PM, Jim Vickroy jim.vick...@noaa.gov wrote: On 1/3/2012 10:46 AM, Ognen Duzlevski wrote: Hello, I am playing with adding an enum dtype to numpy (to get my feet wet in numpy really). I have looked at the https://github.com/martinling/numpy_quaternion and I feel

Re: [Numpy-discussion] PyInt and Numpy's int64 conversion

2012-01-06 Thread Wes McKinney
On Wed, Jan 4, 2012 at 5:22 AM, xantares 09 xantare...@hotmail.com wrote: From: wesmck...@gmail.com Date: Sat, 24 Dec 2011 19:51:06 -0500 To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] PyInt and Numpy's int64 conversion On Sat, Dec 24, 2011 at 3:11 AM, xantares 09

Re: [Numpy-discussion] Numpy 'groupby'

2012-01-11 Thread Wes McKinney
On Wed, Jan 11, 2012 at 7:05 AM, Neal Becker ndbeck...@gmail.com wrote: Michael Hull wrote: Hi Everyone, First off, thanks for all your hard work on numpy, its a really great help! I was wondering if there was a standard 'groupby' in numpy, that similar to that in itertools. I know its not

Re: [Numpy-discussion] fast method to to count a particular value in a large matrix

2012-02-05 Thread Wes McKinney
On Sun, Feb 5, 2012 at 7:02 PM, David Verelst david.vere...@gmail.com wrote: Just out of curiosity, what speed-up factor did you achieve? Regards, David On 04/02/12 22:20, Naresh wrote: Warren Weckesserwarren.weckesserat  enthought.com  writes: On Sat, Feb 4, 2012 at 2:35 PM, Benjamin

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 6:23 PM, Marcel Oliver m.oli...@jacobs-university.de wrote: Hi, I have a short piece of code where the use of an index array feels right, but incurs a severe performance penalty: It's about an order of magnitude slower than all other operations with arrays of that

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
loop In [37]: timeit arr[:] += 1 1000 loops, best of 3: 1.13 ms per loop - Wes On Feb 14, 2012 12:18 AM, Wes McKinney wesmck...@gmail.com wrote: On Mon, Feb 13, 2012 at 6:23 PM, Marcel Oliver m.oli...@jacobs-university.de wrote: Hi, I have a short piece of code where the use of an index

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 7:46 PM, Wes McKinney wesmck...@gmail.com wrote: On Mon, Feb 13, 2012 at 7:30 PM, Nathaniel Smith n...@pobox.com wrote: How would you fix it? I shouldn't speculate without profiling, but I'll be naughty. Presumably the problem is that python turns that into something

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Wes McKinney
On Mon, Feb 13, 2012 at 7:48 PM, Wes McKinney wesmck...@gmail.com wrote: On Mon, Feb 13, 2012 at 7:46 PM, Wes McKinney wesmck...@gmail.com wrote: On Mon, Feb 13, 2012 at 7:30 PM, Nathaniel Smith n...@pobox.com wrote: How would you fix it? I shouldn't speculate without profiling, but I'll

Re: [Numpy-discussion] Index Array Performance

2012-02-14 Thread Wes McKinney
On Tue, Feb 14, 2012 at 4:03 AM, Francesc Alted franc...@continuum.io wrote: On Feb 14, 2012, at 1:50 AM, Wes McKinney wrote: [clip] But: In [40]: timeit hist[i, j] 1 loops, best of 3: 32 us per loop So that's roughly 7-8x slower than a simple Cython method, so I sincerely hope

[Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
dear all, I haven't read all 180 e-mails, but I didn't see this on Travis's initial list. All of the existing flat file reading solutions I have seen are not suitable for many applications, and they compare very unfavorably to tools present in other languages, like R. Here are some of the main

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
better performance and memory usage that way. That's soon to change, though, I gather, at which point I'll almost definitely (!) move to pointer arrays instead of dtype=object arrays. - Wes On Feb 23, 2012, at 1:53 PM, Pauli Virtanen wrote: Hi, 23.02.2012 20:32, Wes McKinney kirjoitti

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
PM, Pauli Virtanen wrote: Hi, 23.02.2012 20:32, Wes McKinney kirjoitti: [clip] To be clear: I'm going to do this eventually whether or not it happens in NumPy because it's an existing problem for heavy pandas users. I see no reason why the code can't emit structured arrays, too, so

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
On Thu, Feb 23, 2012 at 3:23 PM, Erin Sheldon erin.shel...@gmail.com wrote: Wes - I designed the recfile package to fill this need.  It might be a start. Some features:    - the ability to efficiently read any subset of the data without      loading the whole file.    - reads directly

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
On Thu, Feb 23, 2012 at 3:31 PM, Éric Depagne e...@depagne.org wrote: Le jeudi 23 février 2012 21:24:28, Wes McKinney a écrit : That would indeed be great. Reading large files is a real pain whatever the python method used. BTW, could you tell us what you mean by large files? cheers, Éric

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
On Thu, Feb 23, 2012 at 3:55 PM, Erin Sheldon erin.shel...@gmail.com wrote: Excerpts from Wes McKinney's message of Thu Feb 23 15:45:18 -0500 2012: Reasonably wide CSV files with hundreds of thousands to millions of rows. I have a separate interest in JSON handling but that is a different kind

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-23 Thread Wes McKinney
On Thu, Feb 23, 2012 at 4:20 PM, Erin Sheldon erin.shel...@gmail.com wrote: Excerpts from Wes McKinney's message of Thu Feb 23 16:07:04 -0500 2012: That's pretty good. That's faster than pandas's csv-module+Cython approach almost certainly (but I haven't run your code to get a read on how much

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-25 Thread Wes McKinney
On Fri, Feb 24, 2012 at 9:07 AM, Erin Sheldon erin.shel...@gmail.com wrote: Excerpts from Travis Oliphant's message of Thu Feb 23 15:08:52 -0500 2012: This is actually on my short-list as well --- it just didn't make it to the list. In fact, we have someone starting work on it this week.  It

Re: [Numpy-discussion] Migrating issues to GitHub

2012-02-26 Thread Wes McKinney
On Thu, Feb 16, 2012 at 4:32 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Feb 16, 2012 at 10:20 PM, Thouis (Ray) Jones tho...@gmail.com wrote: On Thu, Feb 16, 2012 at 19:25, Ralf Gommers ralf.gomm...@googlemail.com wrote: In another thread Jira was proposed as an

Re: [Numpy-discussion] draft enum NEP

2012-03-11 Thread Wes McKinney
On Fri, Mar 9, 2012 at 5:48 PM, David Gowers (kampu) 00a...@gmail.com wrote: Hi, On Sat, Mar 10, 2012 at 3:25 AM, Bryan Van de Ven bry...@continuum.io wrote: Hi all, I have started working on a NEP for adding an enumerated type to NumPy. It is on my GitHub:    

[Numpy-discussion] Improving NumPy's indexing / subsetting / fancy indexing implementation

2012-04-05 Thread Wes McKinney
dear all, I've routinely found that: 1) ndarray.take is up to 1 order of magnitude faster than fancy indexing 2) Hand-coded Cython boolean indexing is many times faster than ndarray indexing 3) putmask is significantly faster than ndarray indexing For example, I stumbled on this tonight:

Re: [Numpy-discussion] datetime dtype possible regression

2012-04-28 Thread Wes McKinney
On Fri, Apr 27, 2012 at 4:57 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Apr 27, 2012 at 21:52, Travis Vaught tra...@vaught.net wrote: With NumPy 1.6.1 (from EPD 7.2-2) I get this behavior: ~ In [1]: import numpy as np In [2]: schema =

Re: [Numpy-discussion] datetime dtype possible regression

2012-04-29 Thread Wes McKinney
On Sat, Apr 28, 2012 at 11:18 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sat, Apr 28, 2012 at 9:13 AM, Wes McKinney wesmck...@gmail.com wrote: On Fri, Apr 27, 2012 at 4:57 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Apr 27, 2012 at 21:52, Travis Vaught tra

Re: [Numpy-discussion] Issue Tracking

2012-05-02 Thread Wes McKinney
On Wed, May 2, 2012 at 9:48 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, May 1, 2012 at 11:47 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Wed, May 2, 2012 at 1:48 AM, Pauli Virtanen p...@iki.fi wrote: 01.05.2012 21:34, Ralf Gommers kirjoitti: [clip] At this

Re: [Numpy-discussion] Status of np.bincount

2012-05-03 Thread Wes McKinney
On Thu, May 3, 2012 at 12:51 PM, Tony Yu tsy...@gmail.com wrote: On Thu, May 3, 2012 at 9:57 AM, Robert Kern robert.k...@gmail.com wrote: On Thu, May 3, 2012 at 2:50 PM, Robert Elsner ml...@re-factory.de wrote: Am 03.05.2012 15:45, schrieb Robert Kern: On Thu, May 3, 2012 at 2:24 PM,

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Wes McKinney
On Wed, Jun 13, 2012 at 2:12 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 5:44 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 8:33 AM, Nathaniel Smith wrote: Hi Bryan, I skimmed over the diff:     https://github.com/bryevdv/numpy/compare/master...enum It was

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Wes McKinney
On Wed, Jun 13, 2012 at 5:19 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 1:54 PM, Wes McKinney wrote: OK, I need to spend some time on this as it will directly impact me. Random thoughts here. It looks like the levels can only be strings. This is too limited for my needs. Why

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-17 Thread Wes McKinney
On Sun, Jun 17, 2012 at 6:10 AM, Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 7:54 PM, Wes McKinney wesmck...@gmail.com wrote: It looks like the levels can only be strings. This is too limited for my needs. Why not support all possible NumPy dtypes? In pandas world, the levels

Re: [Numpy-discussion] Created NumPy 1.7.x branch

2012-06-21 Thread Wes McKinney
On Thu, Jun 21, 2012 at 2:49 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Jun 21, 2012 at 5:25 PM, Travis Oliphant tra...@continuum.io wrote: I thought it was clear we were doing a 1.7 release before SciPy.   It seems pretty urgent that we get something out sooner than later.

Re: [Numpy-discussion] NumPy 1.7 release delays

2012-06-26 Thread Wes McKinney
On Tue, Jun 26, 2012 at 8:15 PM, Skipper Seabold jsseab...@gmail.com wrote: On Tue, Jun 26, 2012 at 7:59 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Jun 26, 2012 at 1:10 PM, Travis Oliphant tra...@continuum.io wrote: One issues is the one that Sage identified about the array

Re: [Numpy-discussion] memory-efficient loadtxt

2012-10-03 Thread Wes McKinney
On Monday, October 1, 2012, Chris Barker wrote: Paul, Nice to see someone working on these issues, but: I'm not sure the problem you are trying to solve -- accumulating in a list is pretty efficient anyway -- not a whole lot overhead. But if you do want to improve that, it may be better

[Numpy-discussion] ANN: pandas 0.10.0 released

2012-12-17 Thread Wes McKinney
/pypi/pandas $ git log v0.9.1..v0.10.0 --pretty=format:%aN | sort | uniq -c | sort -rn 246 Wes McKinney 140 y-p 99 Chang She 45 jreback 18 Abraham Flaxman 17 Jeff Reback 14 locojaydev 11 Keith Hughitt 5 Adam Obeng 2 Dieter Vandenbussche 1 zach

[Numpy-discussion] ANN: pandas 0.10.1 is released

2013-01-22 Thread Wes McKinney
://pypi.python.org/pypi/pandas $ git log v0.10.0..v0.10.1 --pretty=format:%aN | sort | uniq -c | sort -rn 66 jreback 59 Wes McKinney 43 Chang She 12 y-p 5 Vincent Arel-Bundock 4 Damien Garaud 3 Christopher Whelan 3 Andy Hayden 2 Jay Parlar 2 Dan Allan

[Numpy-discussion] ANN: pandas 0.11.0 released!

2013-04-23 Thread Wes McKinney
://pandas.pydata.org/pandas-docs/stable/whatsnew.html Installers: http://pypi.python.org/pypi/pandas $ git log v0.10.1..v0.11.0 --pretty=format:%aN | sort | uniq -c | sort -rn 308 y-p 279 jreback 85 Vytautas Jancauskas 74 Wes McKinney 25 Stephen Lin 22 Andy Hayden 19 Chang

Re: [Numpy-discussion] Anyone with Core i7 and Ubuntu 10.04?

2010-11-08 Thread Wes McKinney
On Mon, Nov 8, 2010 at 11:33 PM, David Warde-Farley warde...@iro.umontreal.ca wrote: On 2010-11-08, at 8:52 PM, David wrote: Please tell us what error you got - saying that something did not working is really not useful to help you. You need to say exactly what fails, and which steps you

Re: [Numpy-discussion] [ANN] Nanny, faster NaN functions

2010-11-20 Thread Wes McKinney
On Sat, Nov 20, 2010 at 6:39 PM, Keith Goodman kwgood...@gmail.com wrote: On Fri, Nov 19, 2010 at 7:42 PM, Keith Goodman kwgood...@gmail.com wrote: I should make a benchmark suite. ny.benchit(verbose=False) Nanny performance benchmark    Nanny 0.0.1dev    Numpy 1.4.1    Speed is numpy time

Re: [Numpy-discussion] [ANN] Nanny, faster NaN functions

2010-11-20 Thread Wes McKinney
On Sat, Nov 20, 2010 at 6:54 PM, Wes McKinney wesmck...@gmail.com wrote: On Sat, Nov 20, 2010 at 6:39 PM, Keith Goodman kwgood...@gmail.com wrote: On Fri, Nov 19, 2010 at 7:42 PM, Keith Goodman kwgood...@gmail.com wrote: I should make a benchmark suite. ny.benchit(verbose=False) Nanny

Re: [Numpy-discussion] [ANN] Nanny, faster NaN functions

2010-11-21 Thread Wes McKinney
On Sat, Nov 20, 2010 at 7:24 PM, Keith Goodman kwgood...@gmail.com wrote: On Sat, Nov 20, 2010 at 3:54 PM, Wes McKinney wesmck...@gmail.com wrote: Keith (and others), What would you think about creating a library of mostly Cython-based domain specific functions? So stuff like rolling

Re: [Numpy-discussion] [ANN] Nanny, faster NaN functions

2010-11-21 Thread Wes McKinney
at 10:25 AM, Wes McKinney wesmck...@gmail.com wrote: On Sat, Nov 20, 2010 at 7:24 PM, Keith Goodman kwgood...@gmail.com wrote: On Sat, Nov 20, 2010 at 3:54 PM, Wes McKinney wesmck...@gmail.com wrote: Keith (and others), What would you think about creating a library of mostly Cython-based

Re: [Numpy-discussion] [ANN] Nanny, faster NaN functions

2010-11-21 Thread Wes McKinney
On Sun, Nov 21, 2010 at 6:37 PM, Keith Goodman kwgood...@gmail.com wrote: On Sun, Nov 21, 2010 at 3:16 PM, Wes McKinney wesmck...@gmail.com wrote: What would you say to a single package that contains: - NaN-aware NumPy and SciPy functions (nanmean, nanmin, etc.) I'd say yes. - moving

Re: [Numpy-discussion] relational join

2011-02-02 Thread Wes McKinney
On Wed, Feb 2, 2011 at 4:46 PM, Robert Kern robert.k...@gmail.com wrote: On Wed, Feb 2, 2011 at 21:42, Ilya Shlyakhter ilya_...@alum.mit.edu wrote: Does numpy have a relational join operation for joining recordarrays? [~] |1 from numpy.lib import recfunctions [~] |2 recfunctions.join_by?

Re: [Numpy-discussion] OT: performance in C extension; OpenMP, or SSE ?

2011-02-15 Thread Wes McKinney
On Tue, Feb 15, 2011 at 11:25 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Use directly restrict in C99 mode (__restrict does not have exactly the same semantics). For a valgrind profil, you can check my blog (http://matt.eifelle.com/2009/04/07/profiling-with-valgrind/) Basically,

Re: [Numpy-discussion] OT: performance in C extension; OpenMP, or SSE ?

2011-02-15 Thread Wes McKinney
. On Tue, Feb 15, 2011 at 5:26 PM, Wes McKinney wesmck...@gmail.com wrote: On Tue, Feb 15, 2011 at 11:25 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Use directly restrict in C99 mode (__restrict does not have exactly the same semantics). For a valgrind profil, you can check my blog

[Numpy-discussion] {zeros, empty, ...}_like functions behavior with subclasses

2011-02-28 Thread Wes McKinney
I'm having some trouble with the zeros_like function via np.fix: def zeros_like(a): if isinstance(a, ndarray): res = ndarray.__new__(type(a), a.shape, a.dtype, order=a.flags.fnc) res.fill(0) return res try: wrap = a.__array_wrap__ except AttributeError:

Re: [Numpy-discussion] {zeros, empty, ...}_like functions behavior with subclasses

2011-02-28 Thread Wes McKinney
On Mon, Feb 28, 2011 at 7:24 PM, Pierre GM pgmdevl...@gmail.com wrote: On Mar 1, 2011, at 1:05 AM, Bruce Southey wrote: On Mon, Feb 28, 2011 at 4:52 PM, Wes McKinney wesmck...@gmail.com wrote: I'm having some trouble with the zeros_like function via np.fix: def zeros_like

Re: [Numpy-discussion] {zeros, empty, ...}_like functions behavior with subclasses

2011-03-01 Thread Wes McKinney
On Mon, Feb 28, 2011 at 10:52 PM, Wes McKinney wesmck...@gmail.com wrote: On Mon, Feb 28, 2011 at 7:24 PM, Pierre GM pgmdevl...@gmail.com wrote: On Mar 1, 2011, at 1:05 AM, Bruce Southey wrote: On Mon, Feb 28, 2011 at 4:52 PM, Wes McKinney wesmck...@gmail.com wrote: I'm having some trouble

Re: [Numpy-discussion] broadcasting behavior for 1.6 (was: Numpy 1.6 schedule)

2011-03-11 Thread Wes McKinney
On Fri, Mar 11, 2011 at 9:57 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Mar 11, 2011 at 7:42 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Mar 11, 2011 at 2:01 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: I'm just going through the very long 1.6

Re: [Numpy-discussion] [Numpy-svn] aada93: ENH: Add 'subok' parameter to PyArray_NewLikeArray...

2011-03-15 Thread Wes McKinney
On Tue, Mar 15, 2011 at 3:25 PM, Sebastian Haase seb.ha...@gmail.com wrote: On Tue, Mar 15, 2011 at 7:22 PM,  nore...@github.com wrote: Branch: refs/heads/master Home:   https://github.com/numpy/numpy Commit: aada93306acfb4e2eb816faf32652edf8825cf45    

Re: [Numpy-discussion] Remove deprecated skiprows

2011-04-05 Thread Wes McKinney
On Tue, Apr 5, 2011 at 5:52 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Tue, Apr 5, 2011 at 11:45 PM, Skipper Seabold jsseab...@gmail.com wrote: On Sun, Apr 3, 2011 at 8:20 PM, Charles R Harris charlesr.har...@gmail.com wrote: Should skiprows be removed?     if skiprows:    

Re: [Numpy-discussion] Remove deprecated skiprows

2011-04-05 Thread Wes McKinney
On Tue, Apr 5, 2011 at 7:32 PM, Robert Kern robert.k...@gmail.com wrote: On Tue, Apr 5, 2011 at 18:20, Wes McKinney wesmck...@gmail.com wrote: FYI: http://docs.python.org/library/warnings.html DeprecationWarning     Base category for warnings about deprecated features (ignored by default

Re: [Numpy-discussion] Data standardizing

2011-04-13 Thread Wes McKinney
On Wed, Apr 13, 2011 at 9:50 AM, Jonathan Rocher jroc...@enthought.com wrote: Hi, I assume you have this data in a txt file, correct? You can load up all of it in a numpy array using import numpy as np data = np.loadtxt(climat_file.txt, skiprows = 1) Then you can compute the mean you want

[Numpy-discussion] Bug? NumPy types circumvent some overloaded methods in ndarray subclasses

2011-05-07 Thread Wes McKinney
This strikes me as a bug-- haven't checked NumPy 1.6 yet but this happens in 1.5.1. Here's a toy example: class MyNdarray(np.ndarray): def __new__(cls, data): subarr = np.array(data, dtype=np.float64).view(cls) return subarr def __radd__(self, other): print

Re: [Numpy-discussion] datarray -- a couple of questions

2011-05-22 Thread Wes McKinney
On Sun, May 22, 2011 at 10:49 AM, Ben Walsh ben_w_...@yahoo.co.uk wrote: Hi I've been looking at various labelled-array packages, including Pandas (https://github.com/wesm/pandas) and datarray (https://github.com/fperez/datarray). I was quite interested in the design discussion for

Re: [Numpy-discussion] labeled axes

2011-05-26 Thread Wes McKinney
On Tue, May 24, 2011 at 6:39 PM, Craig Yoshioka crai...@me.com wrote: Hi all, I've read some discussions about adding labeled axes, and even ticks, to numpy arrays  (such as in Luis' dataarray). I have recently found that the ability to label axes would be very helpful to me, but I'd like to

Re: [Numpy-discussion] fixing up datetime

2011-06-01 Thread Wes McKinney
On Wed, Jun 1, 2011 at 10:29 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Jun 1, 2011 at 3:16 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris charlesr.har...@gmail.com wrote: snip Just a quick comment, as this really needs more

Re: [Numpy-discussion] fixing up datetime

2011-06-06 Thread Wes McKinney
On Mon, Jun 6, 2011 at 8:16 AM, Mark Dickinson mdickin...@enthought.com wrote: On Thu, Jun 2, 2011 at 5:42 PM, Mark Wiebe mwwi...@gmail.com wrote: Leap years are easy compared with leap seconds. Leap seconds involve a hardcoded table of particular leap-seconds that are added or subtracted, and

Re: [Numpy-discussion] fixing up datetime

2011-06-08 Thread Wes McKinney
On Wed, Jun 8, 2011 at 7:36 AM, Chris Barker chris.bar...@noaa.gov wrote: On 6/7/11 4:53 PM, Pierre GM wrote:  Anyhow, each time yo read 'frequency' in scikits.timeseries, think 'unit'. or maybe precision -- when I think if unit, I think of something that can be represented as a floating

Re: [Numpy-discussion] fixing up datetime

2011-06-08 Thread Wes McKinney
On Wed, Jun 8, 2011 at 6:37 AM, Dave Hirschfeld dave.hirschf...@gmail.com wrote: Wes McKinney wesmckinn at gmail.com writes: - Fundamental need to be able to work with multiple time series, especially performing operations involving cross-sectional data - I think it's a bit hard for lay

Re: [Numpy-discussion] fixing up datetime

2011-06-09 Thread Wes McKinney
On Wed, Jun 8, 2011 at 8:53 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Jun 8, 2011 at 4:57 AM, Wes McKinney wesmck...@gmail.com wrote: snip So in summary, w.r.t. time series data and datetime, the only things I care about from a datetime / pandas point of view: - Ability to easily

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-24 Thread Wes McKinney
On Fri, Jun 24, 2011 at 12:33 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 23, 2011 at 8:32 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Jun 23, 2011 at 5:21 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jun 23, 2011 at 7:00 PM, Nathaniel Smith n...@pobox.com wrote: It's

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-24 Thread Wes McKinney
On Fri, Jun 24, 2011 at 7:10 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jun 24, 2011 at 4:21 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jun 24, 2011 at 10:09 PM, Benjamin Root ben.r...@ou.edu wrote: ... Again, there are pros and cons either way and I

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-24 Thread Wes McKinney
On Fri, Jun 24, 2011 at 8:02 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jun 24, 2011 at 5:22 PM, Wes McKinney wesmck...@gmail.com wrote: On Fri, Jun 24, 2011 at 7:10 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jun 24, 2011 at 4:21 PM, Matthew Brett

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-24 Thread Wes McKinney
On Fri, Jun 24, 2011 at 11:59 PM, Nathaniel Smith n...@pobox.com wrote: On Fri, Jun 24, 2011 at 6:57 PM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Jun 24, 2011 at 8:11 PM, Nathaniel Smith n...@pobox.com wrote: This is a situation where I would just... use an array and a mask, rather than a

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-25 Thread Wes McKinney
On Sat, Jun 25, 2011 at 12:42 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jun 24, 2011 at 10:06 PM, Wes McKinney wesmck...@gmail.com wrote: On Fri, Jun 24, 2011 at 11:59 PM, Nathaniel Smith n...@pobox.com wrote: On Fri, Jun 24, 2011 at 6:57 PM, Benjamin Root ben.r

Re: [Numpy-discussion] Concepts for masked/missing data

2011-06-25 Thread Wes McKinney
On Sat, Jun 25, 2011 at 1:05 PM, Nathaniel Smith n...@pobox.com wrote: On Sat, Jun 25, 2011 at 9:26 AM, Matthew Brett matthew.br...@gmail.com wrote: So far I see the difference between 1) and 2) being that you cannot unmask.  So, if you didn't even know you could unmask data, then it would

Re: [Numpy-discussion] Concepts for masked/missing data

2011-06-25 Thread Wes McKinney
On Sat, Jun 25, 2011 at 3:51 PM, Nathaniel Smith n...@pobox.com wrote: On Sat, Jun 25, 2011 at 11:32 AM, Benjamin Root ben.r...@ou.edu wrote: On Sat, Jun 25, 2011 at 12:05 PM, Nathaniel Smith n...@pobox.com wrote: I guess that is a difference, but I'm trying to get at something more

Re: [Numpy-discussion] named ndarray axes

2011-07-13 Thread Wes McKinney
On Wed, Jul 13, 2011 at 7:15 PM, Craig Yoshioka crai...@me.com wrote: Yup exactly.  To enable this sort of tracking I needed to explicitly reverse-engineer the effects of indexing on axes.  I figure overriding indexing catches most cases that modify axes, but other holes need to be plugged

Re: [Numpy-discussion] inconsistent semantics for double-slicing

2011-07-27 Thread Wes McKinney
On Wed, Jul 27, 2011 at 5:36 PM, Alex Flint alex.fl...@gmail.com wrote: When applying two different slicing operations in succession (e.g. select a sub-range, then select using a binary mask) it seems that numpy arrays can be inconsistent with respect to assignment: For example, in this case

Re: [Numpy-discussion] Warning: invalid value encountered in divide

2011-08-08 Thread Wes McKinney
On Mon, Aug 8, 2011 at 7:01 PM, Neal Becker ndbeck...@gmail.com wrote: Warning: invalid value encountered in divide No traceback.  How can I get more info on this?  Can this warning be converted to an exception so I can get a trace? ___

[Numpy-discussion] Questionable reduceat behavior

2011-08-11 Thread Wes McKinney
I'm a little perplexed why reduceat was made to behave like this: In [26]: arr = np.ones((10, 4), dtype=bool) In [27]: arr Out[27]: array([[ True, True, True, True], [ True, True, True, True], [ True, True, True, True], [ True, True, True, True], [ True,

Re: [Numpy-discussion] Questionable reduceat behavior

2011-08-14 Thread Wes McKinney
. -Mark On Thu, Aug 11, 2011 at 11:25 AM, Wes McKinney wesmck...@gmail.com wrote: I'm a little perplexed why reduceat was made to behave like this: In [26]: arr = np.ones((10, 4), dtype=bool) In [27]: arr Out[27]: array([[ True,  True,  True,  True],       [ True,  True,  True,  True

Re: [Numpy-discussion] Questionable reduceat behavior

2011-08-16 Thread Wes McKinney
On Sun, Aug 14, 2011 at 11:58 AM, Wes McKinney wesmck...@gmail.com wrote: On Sat, Aug 13, 2011 at 8:06 PM, Mark Wiebe mwwi...@gmail.com wrote: Looks like this is the second-oldest open bug in the bug tracker. http://projects.scipy.org/numpy/ticket/236 For what it's worth, I'm in favour

Re: [Numpy-discussion] NA masks for NumPy are ready to test

2011-08-24 Thread Wes McKinney
On Wed, Aug 24, 2011 at 8:19 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Aug 19, 2011 at 11:37 AM, Bruce Southey bsout...@gmail.com wrote: Hi, snip 2) Can the 'skipna' flag be added to the methods? a.sum(skipna=True) Traceback (most recent call last):  File stdin, line 1, in module

Re: [Numpy-discussion] Bug in numpy std, etc. with other data structures?

2011-09-17 Thread Wes McKinney
On Sat, Sep 17, 2011 at 4:48 PM, Skipper Seabold jsseab...@gmail.com wrote: Just ran into this. Any objections for having numpy.std and other functions in core/fromnumeric.py call asanyarray before trying to use the array's method? Other data structures like pandas and larry define their own

Re: [Numpy-discussion] Bug in numpy std, etc. with other data structures?

2011-09-17 Thread Wes McKinney
On Sat, Sep 17, 2011 at 8:36 PM, josef.p...@gmail.com wrote: On Sat, Sep 17, 2011 at 5:12 PM, Wes McKinney wesmck...@gmail.com wrote: On Sat, Sep 17, 2011 at 4:48 PM, Skipper Seabold jsseab...@gmail.com wrote: Just ran into this. Any objections for having numpy.std and other functions in core

Re: [Numpy-discussion] Bug in numpy std, etc. with other data structures?

2011-09-17 Thread Wes McKinney
On Sat, Sep 17, 2011 at 10:50 PM, Bruce Southey bsout...@gmail.com wrote: On Sat, Sep 17, 2011 at 4:12 PM, Wes McKinney wesmck...@gmail.com wrote: On Sat, Sep 17, 2011 at 4:48 PM, Skipper Seabold jsseab...@gmail.com wrote: Just ran into this. Any objections for having numpy.std and other

Re: [Numpy-discussion] Preventing an ndarray subclass from returning new subclass instances for std(), etc

2011-09-18 Thread Wes McKinney
On Sun, Sep 18, 2011 at 4:25 PM, Pierre GM pgmdevl...@gmail.com wrote: On Sep 18, 2011, at 21:25 , Stéfan van der Walt wrote: On Sun, Sep 18, 2011 at 9:48 AM, Keith Hughitt keith.hugh...@gmail.com wrote: Interesting. It works as expected when called as a method: In [10]: x =

Re: [Numpy-discussion] NA masks in the next numpy release?

2011-10-23 Thread Wes McKinney
On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing efir...@hawaii.edu wrote: On 10/23/2011 12:34 PM, Nathaniel Smith wrote: like. And in this case I do think we can come up with an API that will make everyone happy, but that Mark's current API probably can't be incrementally evolved to become that

Re: [Numpy-discussion] NA masks in the next numpy release?

2011-10-24 Thread Wes McKinney
On Mon, Oct 24, 2011 at 10:54 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Oct 24, 2011 at 8:40 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Oct 23, 2011 at 11:23 PM, Wes McKinney wesmck...@gmail.com wrote: On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing

Re: [Numpy-discussion] consensus (was: NA masks in the next numpy release?)

2011-10-28 Thread Wes McKinney
On Fri, Oct 28, 2011 at 7:53 PM, Benjamin Root ben.r...@ou.edu wrote: On Friday, October 28, 2011, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Oct 28, 2011 at 4:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, Oct 29, 2011 at 12:37 AM, Matthew Brett

Re: [Numpy-discussion] consensus (was: NA masks in the next numpy release?)

2011-10-29 Thread Wes McKinney
On Fri, Oct 28, 2011 at 9:32 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Oct 28, 2011 at 6:45 PM, Wes McKinney wesmck...@gmail.com wrote: On Fri, Oct 28, 2011 at 7:53 PM, Benjamin Root ben.r...@ou.edu wrote: On Friday, October 28, 2011, Matthew Brett matthew.br

Re: [Numpy-discussion] Question on timeseries, for financial application

2009-12-13 Thread Wes McKinney
On Sun, Dec 13, 2009 at 9:27 AM, Robert Ferrell ferr...@diablotech.com wrote: On Dec 13, 2009, at 7:07 AM, josef.p...@gmail.com wrote: On Sun, Dec 13, 2009 at 3:31 AM, Pierre GM pgmdevl...@gmail.com wrote: On Dec 13, 2009, at 12:11 AM, Robert Ferrell wrote: Have you considered creating a

[Numpy-discussion] [ANN] pandas 0.1, a new NumPy-based data analysis library

2009-12-25 Thread Wes McKinney
are welcome on the scipy-user mailing list. Specialized discussions or design issues should take place on the pystatsmodels mailing list / google group, where scikits.statsmodels and other libraries will also be discussed: http://groups.google.com/group/pystatsmodels Best regards, Wes McKinney

[Numpy-discussion] [ANN] pandas 0.1, a new NumPy-based data analysis library

2009-12-26 Thread Wes McKinney
://groups.google.com/group/pystatsmodels Best regards, Wes McKinney ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [ANN] New open source project for labeled arrays

2010-01-27 Thread Wes McKinney
On Wed, Jan 27, 2010 at 9:10 PM, Keith Goodman kwgood...@gmail.com wrote: I recently opened sourced one of my packages. It is a labeled array that I call larry. A two-dimensional larry, for example, contains a 2d NumPy array with labels on each row and column. A larry can have any dimension.

[Numpy-discussion] [Job] research developer job opening at AQR Capital

2010-05-19 Thread Wes McKinney
We are looking for a scientific Python developer to work on the design and implementation of AQR’s proprietary research and production systems. In this role you will work closely with researchers and portfolio managers to identify problems and create solutions addressing speed, scale, and

Re: [Numpy-discussion] 2D binning

2010-06-01 Thread Wes McKinney
On Tue, Jun 1, 2010 at 4:49 PM, Zachary Pincus zachary.pin...@yale.edu wrote: Hi Can anyone think of a clever (non-lopping) solution to the following? A have a list of latitudes, a list of longitudes, and list of data values. All lists are the same length. I want to compute an average  of

Re: [Numpy-discussion] 2D binning

2010-06-02 Thread Wes McKinney
On Wed, Jun 2, 2010 at 3:41 AM, Vincent Schut sc...@sarvision.nl wrote: On 06/02/2010 04:52 AM, josef.p...@gmail.com wrote: On Tue, Jun 1, 2010 at 9:57 PM, Zachary Pincuszachary.pin...@yale.edu   wrote: I guess it's as fast as I'm going to get. I don't really see any other way. BTW, the

Re: [Numpy-discussion] 2D binning

2010-06-02 Thread Wes McKinney
On Wed, Jun 2, 2010 at 1:23 PM, Mathew Yeates mat.yea...@gmail.com wrote: thanks. I am also getting an error in ndi.mean Were you getting the error RuntimeError: data type not supported? -Mathew On Wed, Jun 2, 2010 at 9:40 AM, Wes McKinney wesmck...@gmail.com wrote: On Wed, Jun 2, 2010

Re: [Numpy-discussion] 2D binning

2010-06-02 Thread Wes McKinney
the type restrictions, but there are/were several tickets for it. Josef On Wed, Jun 2, 2010 at 10:45 AM, Wes McKinney wesmck...@gmail.com wrote: On Wed, Jun 2, 2010 at 1:23 PM, Mathew Yeates mat.yea...@gmail.com wrote: thanks. I am also getting an error in ndi.mean Were you getting

Re: [Numpy-discussion] 2D binning

2010-06-02 Thread Wes McKinney
On Wed, Jun 2, 2010 at 6:18 PM, Stephen Simmons m...@stevesimmons.com wrote: On 1/06/2010 10:51 PM, Wes McKinney wrote:   snip   This is a pretty good example of the group-by problem that will   hopefully work its way into a future edition of NumPy. Wes (or anyone else), please can you

[Numpy-discussion] Tools / data structures for statistical analysis and related applications

2010-06-09 Thread Wes McKinney
Dear all, We've been having discussions on the pystatsmodels mailing list recently regarding data structures and other tools for statistics / other related data analysis applications. I believe we're trying to answer a number of different, but related questions: 1. What are the sets of

Re: [Numpy-discussion] Tools / data structures for statistical analysis and related applications

2010-06-11 Thread Wes McKinney
On Fri, Jun 11, 2010 at 9:46 AM, Bruce Southey bsout...@gmail.com wrote: On 06/09/2010 03:40 PM, Wes McKinney wrote: Dear all, We've been having discussions on the pystatsmodels mailing list recently regarding data structures and other tools for statistics / other related data analysis

Re: [Numpy-discussion] indexing question

2010-06-21 Thread Wes McKinney
On Mon, Jun 21, 2010 at 7:10 PM, Robert Kern robert.k...@gmail.com wrote: On Mon, Jun 21, 2010 at 17:42, Neal Becker ndbeck...@gmail.com wrote: Robert Kern wrote: On Mon, Jun 21, 2010 at 14:01, Neal Becker ndbeck...@gmail.com wrote: Can I find an efficient way to do this? I have a 2d array,

Re: [Numpy-discussion] BOF notes: Fernando's proposal: NumPy ndarray with named axes

2010-07-06 Thread Wes McKinney
On Tue, Jul 6, 2010 at 12:56 PM, Keith Goodman kwgood...@gmail.com wrote: On Tue, Jul 6, 2010 at 9:52 AM, Joshua Holbrook josh.holbr...@gmail.com wrote: On Tue, Jul 6, 2010 at 8:42 AM, Skipper Seabold jsseab...@gmail.com wrote: On Tue, Jul 6, 2010 at 12:36 PM, Joshua Holbrook

  1   2   >