Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-04 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 1:22 PM, T J tjhn...@gmail.com wrote: I agree that it would be ideal if the default were to skip IGNORED values, but that behavior seems inconsistent with its propagation properties (such as when adding arrays with IGNORED values).  To illustrate, when we did x+2, we

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-04 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 11:59 AM, Pauli Virtanen p...@iki.fi wrote: I have a feeling that if you don't start by mathematically defining the scalar operations first, and only after that generalize them to arrays, some conceptual problems may follow. On the other hand, I should note that

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-04 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 3:04 PM, Nathaniel Smith n...@pobox.com wrote: On Fri, Nov 4, 2011 at 11:59 AM, Pauli Virtanen p...@iki.fi wrote: If classified this way, behaviour of items in np.ma arrays is different in different operations, but seems roughly PdX, where X stands for returning a masked

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-04 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 3:08 PM, T J tjhn...@gmail.com wrote: On Fri, Nov 4, 2011 at 2:29 PM, Nathaniel Smith n...@pobox.com wrote: Continuing my theme of looking for consensus first... there are obviously a ton of ugly corners in here. But my impression is that at least for some simple cases

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-04 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 7:43 PM, T J tjhn...@gmail.com wrote: On Fri, Nov 4, 2011 at 6:31 PM, Pauli Virtanen p...@iki.fi wrote: An acid test for proposed rules: given two arrays `a` and `b`,         a = [1, 2, IGNORED(3), IGNORED(4)]        b = [10, IGNORED(20), 30, IGNORED(40)] [...] (A1) 

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-05 Thread Nathaniel Smith
On Fri, Nov 4, 2011 at 8:33 PM, T J tjhn...@gmail.com wrote: On Fri, Nov 4, 2011 at 8:03 PM, Nathaniel Smith n...@pobox.com wrote: Again, I really don't think you're going to be able to sell an API where  [2] + [IGNORED(20)] == [IGNORED(22)] I mean, it's not me you have to convince, it's Gary

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-06 Thread Nathaniel Smith
On Sat, Nov 5, 2011 at 3:22 PM, T J tjhn...@gmail.com wrote: So what do people expect out of ignored values?  It seems that we might need to extend the list you put forward so that it includes these desires. Since my primary use is with MISSING and not so much IGNORED, I'm not in a very good

Re: [Numpy-discussion] Apparently non-deterministic behaviour of complex array multiplication

2011-12-02 Thread Nathaniel Smith
If save/load actually makes a reliable difference, then it would be useful to do something like this, and see what you see: save(X, X) X2 = load(X.npy) diff = (X == X2) # did save/load change anything? any(diff) # if so, then what changed? X[diff] X2[diff] # any subtle differences in floating

Re: [Numpy-discussion] loop through values in a array and find maximum as looping

2011-12-06 Thread Nathaniel Smith
I think you want np.maximum(a, b, out=a) - Nathaniel On Dec 6, 2011 9:04 PM, questions anon questions.a...@gmail.com wrote: thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in

Re: [Numpy-discussion] Enum type

2012-01-03 Thread Nathaniel Smith
On Tue, Jan 3, 2012 at 9:46 AM, Ognen Duzlevski og...@enthought.com wrote: Hello, I am playing with adding an enum dtype to numpy (to get my feet wet in numpy really). I have looked at the https://github.com/martinling/numpy_quaternion and I feel comfortable with my understanding of adding a

Re: [Numpy-discussion] np.zeros(2, 'S') returns empty strings.

2012-01-15 Thread Nathaniel Smith
On Sat, Jan 14, 2012 at 2:12 PM, Charles R Harris charlesr.har...@gmail.com wrote: This sort of makes sense, but is it the 'correct' behavior? In [20]: zeros(2, 'S') Out[20]: array(['', ''],   dtype='|S1') I think of numpy strings as raw fixed-length byte arrays (since, well, that's

Re: [Numpy-discussion] Index Array Performance

2012-02-13 Thread Nathaniel Smith
How would you fix it? I shouldn't speculate without profiling, but I'll be naughty. Presumably the problem is that python turns that into something like hist[i,j] = hist[i,j] + 1 which means there's no way for numpy to avoid creating a temporary array. So maybe this could be fixed by adding a

Re: [Numpy-discussion] Numpy governance update

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 12:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know

Re: [Numpy-discussion] strange behavior of numpy.random.multivariate_normal, ticket:1842

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 2:08 PM, Pauli Virtanen p...@iki.fi wrote: 16.02.2012 14:54, josef.p...@gmail.com kirjoitti: [clip] If I interpret you correctly, this should be a svd ticket, or an svd ticket as duplicate ? I think it should be a multivariate normal ticket. Fixing SVD is in my

Re: [Numpy-discussion] Numpy governance update

2012-02-16 Thread Nathaniel Smith
On Wed, Feb 15, 2012 at 7:46 PM, Benjamin Root ben.r...@ou.edu wrote: Why not the NA discussion?  Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas,

Re: [Numpy-discussion] Numpy governance update

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots

Re: [Numpy-discussion] strange behavior of numpy.random.multivariate_normal, ticket:1842

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 5:20 PM, Pauli Virtanen p...@iki.fi wrote: Hi, 16.02.2012 18:00, Nathaniel Smith kirjoitti: [clip] I agree, but the behavior is still surprising -- people reasonably expect something like svd to be deterministic. So there's probably a doc bug for alerting people

Re: [Numpy-discussion] Numpy governance update

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 8:36 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 1:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's

Re: [Numpy-discussion] Buildbot/continuous integration (was Re: Issue Tracking)

2012-02-16 Thread Nathaniel Smith
On Thu, Feb 16, 2012 at 11:52 PM, Chris Ball ceb...@gmail.com wrote: Buildbot is used by some big projects (e.g. Python, Chromium, and Mozilla), but I'm aware that several projects in the scientific/numeric Python ecosystem use Jenkins (including Cython, IPython, and SymPy), often using a

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-18 Thread Nathaniel Smith
On Sat, Feb 18, 2012 at 10:54 PM, Travis Oliphant tra...@continuum.io wrote: I'm reading very carefully any arguments against using C++ because I've actually pushed back on Mark pretty hard as we've discussed these things over the past months.  I am nervous about corner use-cases that will be

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-18 Thread Nathaniel Smith
On Sat, Feb 18, 2012 at 11:09 PM, David Cournapeau courn...@gmail.com wrote: On Sat, Feb 18, 2012 at 10:50 PM, Sturla Molden stu...@molden.no wrote:   In an ideal world, we would have a better language than C++ that can be spit out as C for portability. What about a statically typed Python?

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau courn...@gmail.com wrote: On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe mwwi...@gmail.com wrote: Is there a specific target platform/compiler combination you're thinking of where we can do tests on this? I don't believe the compile times are as

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 4:13 PM, xavier.gn...@gmail.com xavier.gn...@gmail.com wrote: I'm no sure. If you want to be able to write A=B+C+D; with decent performances, I think you have to use a lib based on expression templates. It would be great if C++ compilers could automatically optimize out

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 7:13 PM, Mark Wiebe mwwi...@gmail.com wrote: On Sun, Feb 19, 2012 at 5:25 AM, Nathaniel Smith n...@pobox.com wrote: Precompiled headers can help some, but require complex and highly non-portable build-system support. (E.g., gcc's precompiled header constraints are here

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 1:42 PM, Neal Becker ndbeck...@gmail.com wrote: On Fedora linux I use ccache, which is completely transparant and makes a huge difference in build times. ccache is fabulous (and it's fabulous for C too), but it only helps when 'make' has screwed up and decided to rebuild

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-21 Thread Nathaniel Smith
On Tue, Feb 21, 2012 at 4:04 AM, Travis Oliphant tra...@continuum.io wrote: It uses llvm-py (modified to work with LLVM 3.0) and code I wrote to do the translation from Python byte-code to LLVM.   This LLVM can then be JITed.   I have several applications that I would like to use this for.   It

Re: [Numpy-discussion] np.longlong casts to int

2012-02-23 Thread Nathaniel Smith
On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted franc...@continuum.io wrote: Exactly.  I'd update this to read: float96    96 bits.  Only available on 32-bit (i386) platforms. float128  128 bits.  Only available on 64-bit (AMD64) platforms. Except float96 is actually 80 bits. (Usually?) Plus

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 5:23 PM, Warren Weckesser warren.weckes...@enthought.com wrote: I haven't pushed it to the extreme, but the big example (in the examples/ directory) is a 1 gig text file with 2 million rows and 50 fields in each row.  This is read in less than 30 seconds (but that's with

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 7:16 PM, Warren Weckesser warren.weckes...@enthought.com wrote: On Sun, Feb 26, 2012 at 1:00 PM, Nathaniel Smith n...@pobox.com wrote: For this kind of benchmarking, you'd really rather be measuring the CPU time, or reading byte streams that are already in memory. If you

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 7:58 PM, Warren Weckesser warren.weckes...@enthought.com wrote: Right, I got that.  Sorry if the placement of the notes about how to clear the cache seemed to imply otherwise. OK, cool, np. Clearing the disk cache is very important for getting meaningful, repeatable

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-27 Thread Nathaniel Smith
On Mon, Feb 27, 2012 at 2:44 PM, Erin Sheldon erin.shel...@gmail.com wrote: What I've got is a solution for writing and reading structured arrays to and from files, both in text files and binary files.  It is written in C and python.  It allows reading arbitrary subsets of the data efficiently

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-28 Thread Nathaniel Smith
[Re-adding the list to the To: field, after it got dropped accidentally] On Tue, Feb 28, 2012 at 12:28 AM, Erin Sheldon erin.shel...@gmail.com wrote: Excerpts from Nathaniel Smith's message of Mon Feb 27 17:33:52 -0500 2012: On Mon, Feb 27, 2012 at 6:02 PM, Erin Sheldon erin.shel...@gmail.com

Re: [Numpy-discussion] Determining the 'viewness' of an array, and flags.owndata confusion

2012-02-28 Thread Nathaniel Smith
On Tue, Feb 28, 2012 at 11:01 PM, Kurt Smith kwmsm...@gmail.com wrote: For an arbitrary numpy array 'a', what does 'a.flags.owndata' indicate? I think what it really indicates is whether a's destructor should call free() on a's data pointer. I originally thought that owndata is False iff 'a'

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-29 Thread Nathaniel Smith
On Wed, Feb 29, 2012 at 3:11 PM, Erin Sheldon erin.shel...@gmail.com wrote: Excerpts from Nathaniel Smith's message of Tue Feb 28 17:22:16 -0500 2012: Even for binary, there are pathological cases, e.g. 1) reading a random subset of nearly all rows.  2) reading a single column when rows are

Re: [Numpy-discussion] [Numpy] quadruple precision

2012-03-02 Thread Nathaniel Smith
On Mar 2, 2012 10:48 AM, Paweł Biernat pw...@wp.pl wrote: The portability is broken for numpy.float128 anyway (as I understand, it behaves in different ways on different architectures), so adding a new type (call it, say, quad128) that properly supports binary128 shouldn't be a drawback. Later

Re: [Numpy-discussion] verbose output when running python script?

2012-03-05 Thread Nathaniel Smith
Try the trace module in the standard library: http://docs.python.org/library/trace.html http://www.doughellmann.com/PyMOTW/trace/ - Nathaniel On Mar 5, 2012 3:27 PM, Chao YUE chaoyue...@gmail.com wrote: Dear all, Sorry this is not the good place to ask but I think there must be someone who

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant tra...@continuum.io wrote: Hi all, Hi Travis, Thanks for bringing this back up. Have you looked at the summary from the last thread? https://github.com/njsmith/numpy/wiki/NA-discussion-status The goal was to try and at least work out what

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 4:38 PM, Mark Wiebe mwwi...@gmail.com wrote: On Tue, Mar 6, 2012 at 5:48 AM, Pierre Haessig pierre.haes...@crans.org wrote: From a potential user perspective, I feel it would be nice to have NA and non-NA cases look as similar as possible. Your code example is

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 9:14 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Tue, Mar 6, 2012 at 9:25 PM, Nathaniel Smith n...@pobox.com wrote: On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant tra...@continuum.io wrote: Hi all, Hi Travis, Thanks for bringing this back up. Have you

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 4:35 PM, Pierre Haessig pierre.haes...@crans.org wrote: Hi, Thanks you very much for your lights ! Le 06/03/2012 21:59, Nathaniel Smith a écrit : Right -- R has a very impoverished type system as compared to numpy. There's basically four types: numeric (meaning double

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig pierre.haes...@crans.org Coming back to Travis proposition bit-pattern approaches to missing data (*at least* for float64 and int32) need to be implemented., I

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:37 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Mar 7, 2012 at 12:26 PM, Nathaniel Smith n...@pobox.com wrote: When it comes to missing data, bitpatterns can do everything that masks can do, are no more complicated to implement, and have better

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:39 PM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Mar 7, 2012 at 1:26 PM, Nathaniel Smith n...@pobox.com wrote: When it comes to missing data, bitpatterns can do everything that masks can do, are no more complicated to implement, and have better performance

Re: [Numpy-discussion] use for missing (ignored) data?

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 8:05 PM, Neal Becker ndbeck...@gmail.com wrote: I'm wondering what is the use for the ignored data feature? I can use: A[valid_A_indexes] = whatever to process only the 'non-ignored' portions of A.  So at least some simple cases of ignored data are already

Re: [Numpy-discussion] draft enum NEP

2012-03-09 Thread Nathaniel Smith
On Fri, Mar 9, 2012 at 4:55 PM, Bryan Van de Ven bry...@continuum.io wrote: Hi all, I have started working on a NEP for adding an enumerated type to NumPy. It is on my GitHub:     https://github.com/bryevdv/numpy/blob/enum/doc/neps/enum.rst It is still very rough, and incomplete in places.

Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM

2012-03-13 Thread Nathaniel Smith
On Tue, Mar 13, 2012 at 5:18 PM, Travis Oliphant tra...@continuum.io wrote: Cython and Numba certainly overlap.  However, Cython requires:        1) learning another language So is the goal for numba to actually handle arbitrary Python code with correct semantics, i.e., it's actually a

Re: [Numpy-discussion] draft enum NEP

2012-03-15 Thread Nathaniel Smith
On Wed, Mar 14, 2012 at 1:44 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Mar 9, 2012 at 8:55 AM, Bryan Van de Ven bry...@continuum.io wrote: Hi all, I have started working on a NEP for adding an enumerated type to NumPy. It is on my GitHub:    

Re: [Numpy-discussion] Missing data again

2012-03-15 Thread Nathaniel Smith
with anything that is stated there? 2) Do you feel like that document accurately summarises your basic idea of what this feature is supposed to do (I assume under the IGNORED heading)? Thanks, -- Nathaniel On Wed, Mar 7, 2012 at 11:10 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Mar 7, 2012 at 7

Re: [Numpy-discussion] draft enum NEP

2012-03-16 Thread Nathaniel Smith
On Mar 16, 2012 1:02 AM, Stéfan van der Walt stefan ste...@sun.ac.za@ste...@sun.ac.za sun.ac.za ste...@sun.ac.za wrote: On Thu, Mar 15, 2012 at 4:02 PM, Nathaniel Smith njs n...@pobox.com@n...@pobox.com pobox.com n...@pobox.com wrote: I'm not sure what it would even mean to treat this kind

Re: [Numpy-discussion] draft enum NEP

2012-03-17 Thread Nathaniel Smith
On Fri, Mar 16, 2012 at 4:26 PM, Bryan Van de Ven bry...@continuum.io wrote: Hi all, I have spent some time thinking about things, and discussing them with folks nearby. I actually got to wondering whether we really need new dtypes for this. It seems like enumerated values or factor levels

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant tra...@continuum.io wrote: While namespaces are a really good idea, I'm not a big fan of both module namespaces and underscore namespaces.   It seems pretty redundant to me to have pad.pad_mean. On the other hand, one could argue that pad.mean

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:38 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant tra...@continuum.io wrote: While namespaces are a really good idea, I'm not a big fan of both module namespaces and underscore namespaces.   It seems pretty redundant to me

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:45 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Mar 29, 2012 at 10:38 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant tra...@continuum.io wrote: While namespaces are a really good idea, I'm not a big fan

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 8:35 AM, Richard Hattersley rhatters...@gmail.com wrote: I like where this is going. Driven by a desire to avoid a million different methods on a single class, we've done something similar in our library. So instead of   thing.mean()   thing.max(...)   etc. we

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 6:53 PM, Tim Cera t...@cerazone.net wrote: If instead you passed in a function:     def padwithzeros(vector, pad_width, iaxis, **kwargs):         bvector = np.zeros(pad_width[0])         avector = np.zeros(pad_width[1])         return bvector, avector     b =

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 1:22 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Mar 30, 2012 at 5:41 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Mar 29, 2012 at 6:53 PM, Tim Cera t...@cerazone.net wrote: If instead you passed in a function:     def padwithzeros(vector

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 2:20 PM, Tim Cera t...@cerazone.net wrote: My suggestion is: Step 1: Change the current PR so that it has only one user-exposed function, something like pad(..., mode=foo), and commit that. Everyone seems to pretty much like that interface, implementing it would take 1

Re: [Numpy-discussion] NumPy EIG much slower than MATLAB EIG

2012-04-02 Thread Nathaniel Smith
To see if this is an effect of numpy using C-order by default instead of Fortran-order, try measuring eig(x.T) instead of eig(x)? -n On Apr 1, 2012 2:28 PM, Kamesh Krishnamurthy kames...@gmail.com wrote: Hello all, I profiled NumPy EIG and MATLAB EIG on the same Macbook pro, and both were

Re: [Numpy-discussion] NumPy EIG much slower than MATLAB EIG

2012-04-02 Thread Nathaniel Smith
On Mon, Apr 2, 2012 at 6:18 PM, Aronne Merrelli aronne.merre...@gmail.com wrote: On Sun, Apr 1, 2012 at 8:28 AM, Kamesh Krishnamurthy kames...@gmail.com wrote: Hello all, I profiled NumPy EIG and MATLAB EIG on the same Macbook pro, and both were linking to the Accelerate framework BLAS.

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-04-03 Thread Nathaniel Smith
On Mon, Apr 2, 2012 at 7:14 PM, Tim Cera t...@cerazone.net wrote: I think the suggestion is pad(a, 5, mode='mean'), which would be consistent with common numpy signatures. The mode keyword should probably have a default, something commonly used. I'd suggest 'mean', Nathaniel suggests 'zero',

Re: [Numpy-discussion] Bitwise operations and unsigned types

2012-04-06 Thread Nathaniel Smith
On Fri, Apr 6, 2012 at 7:19 AM, Travis Oliphant tra...@continuum.io wrote: That is an interesting point of view.     I could see that point of view.  But, was this discussed as a bug prior to this change occurring? I just heard from a very heavy user of NumPy that they are nervous about

Re: [Numpy-discussion] Improving NumPy's indexing / subsetting / fancy indexing implementation

2012-04-06 Thread Nathaniel Smith
Hi Wes, I believe that Mark rewrote a bunch of the fancy-indexing-related code from scratch in the masked-NA branch. I don't know if it affects anything you're talking about here, but just as a heads up, you might want to benchmark master, since it may have a different performance profile. --

Re: [Numpy-discussion] Keyword argument support for vectorize.

2012-04-09 Thread Nathaniel Smith
On Mon, Apr 9, 2012 at 10:53 AM, Michael McNeil Forbes michael.for...@gmail.com wrote: It seems like functools.partial is the appropriate tool to use here which means I will have to deal with the functools was added in Python 2.5, and so far numpy is still trying to maintain 2.4 compatibility.

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-09 Thread Nathaniel Smith
...isn't this an operation that will be performed once per compiled function? Is the overhead of the easy, robust method (calling ctypes.cast) actually measurable as compared to, you know, running an optimizing compiler? I mean, I doubt there'd be any real problem with adding this extra API to

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 1:57 AM, Travis Oliphant tra...@continuum.io wrote: On Apr 9, 2012, at 7:21 PM, Nathaniel Smith wrote: ...isn't this an operation that will be performed once per compiled function? Is the overhead of the easy, robust method (calling ctypes.cast) actually measurable

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 1:39 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 04/10/2012 12:37 PM, Nathaniel Smith wrote: On Tue, Apr 10, 2012 at 1:57 AM, Travis Oliphanttra...@continuum.io  wrote: On Apr 9, 2012, at 7:21 PM, Nathaniel Smith wrote: ...isn't this an operation

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 2:15 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 04/10/2012 03:10 PM, Dag Sverre Seljebotn wrote: On 04/10/2012 03:00 PM, Nathaniel Smith wrote: On Tue, Apr 10, 2012 at 1:39 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no   wrote: On 04/10/2012 12

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 2:38 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 04/10/2012 03:29 PM, Nathaniel Smith wrote: Right, that's what I wasn't getting until you mentioned strcmp :-). That said, the core numpy dtypes are singletons. For this purpose, the signature could

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-12 Thread Nathaniel Smith
On Wed, Apr 11, 2012 at 10:23 PM, Travis Oliphant teoliph...@gmail.com wrote: In the mean-time, I think we could do as Robert essentially suggested and just use Capsule Objects around an agreed-upon simple C-structure:      int   id   /* Some number that can be used as a type-check */      

Re: [Numpy-discussion] 1.7 blockers

2012-04-16 Thread Nathaniel Smith
On Mon, Apr 16, 2012 at 10:09 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, Mar 24, 2012 at 10:13 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, There several problems with numpy master that need to be fixed before a release can be considered. Datetime on

Re: [Numpy-discussion] Different behaviour of python built sum and addition on ndarrays

2012-04-16 Thread Nathaniel Smith
On Mon, Apr 16, 2012 at 11:06 PM, Christopher Mutel cmu...@gmail.com wrote: So, for both 1.5 and 1.6 (at least), it appears that the builtin sum does not add ndarrays the way + (and operator.add) do: a = np.arange(10).reshape((2,5)) b = np.arange(10, 20).reshape((2,5)) sum(a,b) Out[5]:

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 6:44 AM, Travis Oliphant tra...@continuum.io wrote: Basically, there are two sets of changes as far as I understand right now:        1) ufunc infrastructure understands masked arrays        2) ndarray grew attributes to represent masked arrays I am proposing that we

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: Mark and I will have conversations about NumPy while he is in Austin.   There are many other active stake-holders whose opinions and

[Numpy-discussion] the state of NA/masking

2012-04-22 Thread Nathaniel Smith
Hi all, Travis, Mark, and I talked on Skype this week about how to productively move forward with the NA debate, and I got picked to summarize for the list :-). There are three main things we discussed: 1) About process: We seem to agree that this discussion has been ineffective for a variety

[Numpy-discussion] NEP mask code and the 1.7 release

2012-04-22 Thread Nathaniel Smith
We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what it is, we don't actually have consensus yet that it's the best way to give our users what they want/need -- or even an appropriate way. So we need to figure out

[Numpy-discussion] What is consensus anyway

2012-04-22 Thread Nathaniel Smith
If you hang around big FOSS projects, you'll see the word consensus come up a lot. For example, the glibc steering committee recently dissolved itself in favor of governance directly by the consensus of the people active in glibc development[1]. It's the governing rule of the IETF, which defines

Re: [Numpy-discussion] What is consensus anyway

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 1:04 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 4:15 PM, Nathaniel Smith n...@pobox.com wrote: If you hang around big FOSS projects, you'll see the word consensus come up a lot. For example, the glibc steering committee recently

Re: [Numpy-discussion] NEP mask code and the 1.7 release

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 6:18 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith n...@pobox.com wrote: We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what

Re: [Numpy-discussion] NEP mask code and the 1.7 release

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 9:16 PM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Apr 23, 2012 at 12:57 PM, Nathaniel Smith n...@pobox.com wrote: Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were

Re: [Numpy-discussion] Masked Arrays in NumPy 1.x

2012-04-23 Thread Nathaniel Smith
Hi Paul, On Wed, Apr 11, 2012 at 8:57 PM, Paul Hobson pmhob...@gmail.com wrote: Travis et al, This isn't a reply to anything specific in your email and I apologize if there is a better thread or place to share this information. I've been meaning to participate in the discussion for a long

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Tue, Apr 24, 2012 at 2:43 PM, Pierre Haessig pierre.haes...@crans.org wrote: If the idea of having two payloads is to avoid a maximum of skipna friends extra keywords, I would like it much. My feeling with my small experience with R is that I end up calling every function with a different

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 12:49 AM, Charles R Harris charlesr.har...@gmail.com wrote: I think we adhere to these pretty well already, the problem is with the word 'everyone'. I grew up in Massachusetts where town meetings were a tradition. At those meetings the townsfolk voted on the budget,

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Tue, Apr 24, 2012 at 2:14 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Apr 23, 2012 at 11:35 PM, Fernando Perez fperez@gmail.com wrote: On Mon, Apr 23, 2012 at 8:49 PM, Stéfan van der Walt ste...@sun.ac.za wrote: If you are referring to the traditional concept of a

Re: [Numpy-discussion] What is consensus anyway

2012-04-25 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 4:02 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Tue, Apr 24, 2012 at 8:56 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Apr 24, 2012 at 6:12 PM, Charles R Harris charlesr.har...@gmail.com wrote: I admit to a certain curiosity about your own

Re: [Numpy-discussion] What is consensus anyway

2012-04-25 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 1:03 PM, Charles R Harris charlesr.har...@gmail.com wrote: That is one of the reasons that the smaller scikits attract people, they have more freedom to do what they want and fewer people to answer to. Scipy also has some of that advantage because there are a number of

Re: [Numpy-discussion] A crazy masked-array thought

2012-04-27 Thread Nathaniel Smith
On Fri, Apr 27, 2012 at 11:32 AM, Richard Hattersley rhatters...@gmail.com wrote: I know used a somewhat jokey tone in my original posting, but fundamentally it was a serious question concerning a live topic. So I'm curious about the lack of response. Has this all been covered before? Sorry

Re: [Numpy-discussion] A crazy masked-array thought

2012-04-28 Thread Nathaniel Smith
On Sat, Apr 28, 2012 at 7:38 AM, Richard Hattersley rhatters...@gmail.com wrote: So, assuming numpy.ndarray became a strict subclass of some new masked array, it looks plausible that adding just a few checks to numpy.ndarray to exclude the masked superclass would prevent much downstream code

Re: [Numpy-discussion] sparse array data

2012-05-02 Thread Nathaniel Smith
On Wed, May 2, 2012 at 9:53 PM, Francesc Alted franc...@continuum.io wrote: On 5/2/12 11:16 AM, Wolfgang Kerzendorf wrote: Hi all, I'm currently writing a code that needs three dimensional data (for the physicists it's dimensions are atom, ion, level). The problem is that not all

Re: [Numpy-discussion] sparse array data

2012-05-02 Thread Nathaniel Smith
On Wed, May 2, 2012 at 11:26 PM, Francesc Alted franc...@continuum.io wrote: On 5/2/12 4:20 PM, Nathaniel Smith wrote: On Wed, May 2, 2012 at 9:53 PM, Francesc Altedfranc...@continuum.io  wrote: On 5/2/12 11:16 AM, Wolfgang Kerzendorf wrote: Hi all, I'm currently writing a code that needs

Re: [Numpy-discussion] sparse array data

2012-05-03 Thread Nathaniel Smith
On Thu, May 3, 2012 at 4:44 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, May 2, 2012 at 3:20 PM, Nathaniel Smith n...@pobox.com wrote: This coordinate format is also what's used by the MATLAB Tensor Toolbox. They have a paper justifying this choice and describing some tricks

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
On Wed, May 9, 2012 at 5:46 PM, Travis Oliphant tra...@continuum.io wrote: Hey all, Nathaniel and Mark have worked very hard on a joint document to try and explain the current status of the missing-data debate.   I think they've done an amazing job at providing some context, articulating

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
Hi Dag, On Wed, May 9, 2012 at 8:44 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: I'm a heavy user of masks, which are used to make data NA in the statistical sense. The setting is that we have to mask out the radiation coming from the Milky Way in full-sky images of the Cosmic

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Nathaniel Smith
Hi Matthew, On Thu, May 10, 2012 at 12:01 AM, Matthew Brett matthew.br...@gmail.com wrote: The third proposal is certainly the best one from Cython's perspective; and I imagine for those writing C extensions against the C API too. Having PyType_Check fail for ndmasked is a very good way of

[Numpy-discussion] Anyone have a cached copy of NA-discussion-status?

2012-05-11 Thread Nathaniel Smith
Hi all, I'm an idiot and seem to have accidentally deleted the NA-discussion-status web page. I do have a query into support@github, but, does anyone happen to have a local copy of the content, perhaps in their browser cache? Frustratedly yrs, -- Nathaniel

[Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-11 Thread Nathaniel Smith
I've been trying to sort through the changes that landed in master from the missingdata branch to figure out how to separate out changes related to NA support from those that aren't, and noticed that one of them should probably be flagged to the list. Traditionally, arr.diagonal() and

Re: [Numpy-discussion] Masking through generator arrays

2012-05-11 Thread Nathaniel Smith
On Thu, May 10, 2012 at 7:23 PM, Chris Barker chris.bar...@noaa.gov wrote: That is one of my concerns about the bit pattern idea -- we've then created a new binary type that no other standard software understands -- that looks like a a lot of work to me to deal with, or even worse, ripe for

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-11 Thread Nathaniel Smith
On Fri, May 11, 2012 at 9:26 PM, T J tjhn...@gmail.com wrote: On Fri, May 11, 2012 at 1:12 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, May 11, 2012 at 2:18 PM, Pauli Virtanen p...@iki.fi wrote: 11.05.2012 17:54, Frédéric Bastien kirjoitti: In Theano we use a view, but that is not

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-13 Thread Nathaniel Smith
On Sun, May 13, 2012 at 3:28 AM, Travis Oliphant tra...@continuum.io wrote: Another approach would be to introduce a method: a.diag(copy=False) and leave a.diagonal() alone.  Then, a.diagonal() could be deprecated over 2-3 releases. This would be a good idea if we didn't already have both

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-16 Thread Nathaniel Smith
On Tue, May 15, 2012 at 2:49 PM, Frédéric Bastien no...@nouiz.org wrote: Hi, In fact, I would arg to never change the current behavior, but add the flag for people that want to use it. Why? 1) There is probably 10k script that use it that will need to be checked for correctness. There

Re: [Numpy-discussion] Fancy-indexing reorders output in corner cases?

2012-05-16 Thread Nathaniel Smith
On Tue, May 15, 2012 at 5:03 AM, Travis Oliphant tra...@continuum.io wrote: So, the behavior is actually quite predictable, it's just that in some common cases it doesn't do what you would expect --- especially if you think that [0,1] is the same as :2.   When I wrote this code to begin with

  1   2   3   4   5   6   7   8   9   10   >