Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Travis E. Oliphant
Numpy 1.2 is for documentation, bug fixes, and getting the new testing framework in place. Discipline is called for if we are going to have timely releases. We also agreed to a change in the C-API (or at least did not object too loudly). I'm in favor of minimizing that sort of change.

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Jarrod Millman
On Thu, Aug 14, 2008 at 10:54 PM, Charles R Harris [EMAIL PROTECTED] wrote: Numpy 1.2 is for documentation, bug fixes, and getting the new testing framework in place. Discipline is called for if we are going to have timely releases. First, all your points are very valid. And I apologize for

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Travis E. Oliphant
Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. Where did you notice this? I didn't see

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
On Aug 14, 2008, at 11:07 PM, Alan G Isaac wrote: Btw, numpy loads noticeably faster. Any chance of someone reviewing my suggestions for making the import somewhat faster still? http://scipy.org/scipy/numpy/ticket/874 Andrew

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Travis E. Oliphant
Travis E. Oliphant wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t.

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 12:35 AM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Travis E. Oliphant wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Andrew Dalke
On Aug 15, 2008, at 8:36 AM, Charles R Harris wrote: The inline keyword also tends to be gcc/icc specific, although it is part of the C99 standard. For reference, a page on using inline and doing so portably: http://www.greenend.org.uk/rjk/2003/03/inline.html

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 12:45 AM, Andrew Dalke [EMAIL PROTECTED]wrote: On Aug 15, 2008, at 8:36 AM, Charles R Harris wrote: The inline keyword also tends to be gcc/icc specific, although it is part of the C99 standard. For reference, a page on using inline and doing so portably:

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Jarrod Millman
On Thu, Aug 14, 2008 at 9:05 PM, Charles R Harris [EMAIL PROTECTED] wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. It should work now. Let me know if you aren't getting them now. -- Jarrod Millman Computational Infrastructure for Research Labs

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 1:04 AM, Jarrod Millman [EMAIL PROTECTED]wrote: On Thu, Aug 14, 2008 at 9:05 PM, Charles R Harris [EMAIL PROTECTED] wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. It should work now. Let me know if you aren't

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Robert Kern
On Fri, Aug 15, 2008 at 02:59, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/8/15 Robert Kern [EMAIL PROTECTED]: The devil is in the details. What exactly do you propose? When we discussed this last time, the participants more or less agreed that environment variables could cause more

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Stéfan van der Walt
2008/8/15 Robert Kern [EMAIL PROTECTED]: I'm afraid that I still don't understand. Please expand on the Sorry, it's late. My explanation is probably not too lucid. The variable should rather read something like NUMPY_VIA_API, but here goes. 1) NUMPY_FAST_IMPORT=0 (or simply absent)

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Jon Wright
Jarrod Millman wrote: Hey, NumPy 1.2.0b2 is now available. Please test this so that we can uncover any problems ASAP. Windows binary: http://www.enthought.com/~gvaroquaux/numpy-1.2.0b2-win32.zip Hello Again, It seems the new release breaks matplotlib, for those pauvres who are

Re: [Numpy-discussion] min() of array containing NaN

2008-08-15 Thread Joe Harrington
If you're willing to do arithmetic you might even be able to pull it off, since NaNs tend to propagate: if (newmin) min -= (min-new); Whether the speed of this is worth its impenetrability I couldn't say. Code comments cure impenetrability, and have no cost in speed. One could write a

Re: [Numpy-discussion] Generalized ufuncs?

2008-08-15 Thread David Cournapeau
On Fri, Aug 15, 2008 at 1:16 AM, Travis E. Oliphant [EMAIL PROTECTED] wrote: The biggest reason is that the patch requires changing the C-API and we are already doing that for 1.2. I would rather not do it again for another 6 months at least. I don't think we should make the patch wait

Re: [Numpy-discussion] Different results from repeated calculation, part 2

2008-08-15 Thread Bruce Southey
Keith Goodman wrote: On Thu, Aug 14, 2008 at 11:29 AM, Bruce Southey [EMAIL PROTECTED] wrote: Keith Goodman wrote: I get slightly different results when I repeat a calculation. I've seen this problem before (it went away but has returned):

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Gael Varoquaux
On Fri, Aug 15, 2008 at 02:59:43AM -0500, Stéfan van der Walt wrote: 2008/8/15 Robert Kern [EMAIL PROTECTED]: The devil is in the details. What exactly do you propose? When we discussed this last time, the participants more or less agreed that environment variables could cause more

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Pauli Virtanen
Fri, 15 Aug 2008 15:30:20 +0200, Gael Varoquaux wrote: On Fri, Aug 15, 2008 at 02:59:43AM -0500, Stéfan van der Walt wrote: [clip] 1) Expose a new interface to numpy, called numpy.api 2) If a certain environment variable is set, the numpy namespace is not populated, and numpy.api becomes

[Numpy-discussion] patch for new mgrid / ogrid functionality

2008-08-15 Thread David M. Kaplan
Hi, A while back, I sent some changes to index_tricks.py that would allow mgrid and ogrid to mesh things other than slices. For example: mgrid[['a','b'],[float,int],:3] [array([[['a', 'a', 'a'], ['a', 'a', 'a']], [['b', 'b', 'b'], ['b', 'b', 'b']]], dtype='|S1'),

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread David Cournapeau
On Thu, Aug 14, 2008 at 3:58 PM, Alan G Isaac [EMAIL PROTECTED] wrote: Two odd failures in test_print.py. Platform: Win XP SP3 on Intel T2600. Alan Isaac I got the fixes to make numpy buildable again with VS 2003, and the errors are mingw specific. Either a compiler bug or more likely a

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Jarrod Millman
On Fri, Aug 15, 2008 at 4:21 AM, Jon Wright [EMAIL PROTECTED] wrote: It seems the new release breaks matplotlib, for those pauvres who are using pre-compiled at least. If this means all C-modules compiled against numpy have to be recompiled, then this will make me very unhappy. Yes, the new

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
On Aug 15, 2008, at 4:38 PM, Pauli Virtanen wrote: I think you can still do something evil, like this: import os if os.environ.get('NUMPY_VIA_API', '0') != '0': from numpy.lib.fromnumeric import * ... But I'm not sure how many milliseconds must be gained

[Numpy-discussion] VS 2003 problems with cython-generated code

2008-08-15 Thread David Cournapeau
Hi, I noticed this morning that numpy 1.2 is not buildable with VS 2003 (the one you have to use for official python releases for at least python 2.5, and maybe 2.4). When we generate C code, both with internal code (numpy/core/code_generator) and with external tools (cython/pyrex for mtrand),

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Christopher Barker
Jarrod Millman wrote: NumPy 1.2.0b2 is now available. Please test this so that we can uncover any problems ASAP. Mac binary: https://cirl.berkeley.edu/numpy/numpy-1.2.0b2-py2.5-macosx10.5.dmg Ran 1715 tests in 12.671s OK (SKIP=1) OS-X 10.4.11 Dual G5 PPC Python version 2.5.2 (r252:60911,

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 10:41 AM, Andrew Dalke [EMAIL PROTECTED]wrote: On Aug 15, 2008, at 4:38 PM, Pauli Virtanen wrote: I think you can still do something evil, like this: import os if os.environ.get('NUMPY_VIA_API', '0') != '0': from numpy.lib.fromnumeric

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
I forgot to mention.. On Aug 15, 2008, at 9:00 AM, Travis E. Oliphant wrote: 1) Removing ctypeslib import * Can break code if somebody has been doing import numpy and then using numpy.ctypeslib * I'm fine with people needing to import numpy.ctypeslib to use the capability as long as we

Re: [Numpy-discussion] VS 2003 problems with cython-generated code

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 10:49 AM, David Cournapeau [EMAIL PROTECTED]wrote: Hi, I noticed this morning that numpy 1.2 is not buildable with VS 2003 (the one you have to use for official python releases for at least python 2.5, and maybe 2.4). When we generate C code, both with internal code

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Travis E. Oliphant
Andrew Dalke wrote: I forgot to mention.. On Aug 15, 2008, at 9:00 AM, Travis E. Oliphant wrote: 1) Removing ctypeslib import * Can break code if somebody has been doing import numpy and then using numpy.ctypeslib * I'm fine with people needing to import numpy.ctypeslib to use the

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Stéfan van der Walt
2008/8/15 Andrew Dalke [EMAIL PROTECTED]: I don't think it's enough. I don't like environmental variable tricks like that. My tests suggest: current SVN: 0.12 seconds my patch: 0.10 seconds removing some top-level imports: 0.09 seconds my patch and removing some additional

Re: [Numpy-discussion] VS 2003 problems with cython-generated code

2008-08-15 Thread David Cournapeau
On Fri, Aug 15, 2008 at 12:18 PM, Charles R Harris [EMAIL PROTECTED] wrote: Be careful if you break across lines. The gnu compilers will accept foo bar But for some others you need to use a line continuation. foo\ bar I don't put newlines: I really do foobar, to avoid this exact

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread David Cournapeau
On Fri, Aug 15, 2008 at 11:41 AM, Andrew Dalke [EMAIL PROTECTED] wrote: It's possible to clean up the code so this loop doesn't exist, and fix things so that fewer things are imported when some environment variable is set, but it doesn't look easy. Modules depend on other modules a bit too

[Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Travis E. Oliphant
Hello all, While we are on the subject of C-API changes, I've noticed that quite a few of the sub-classes of ndarray are constructed to basically add meta-information to the array. What if the base-class ndarray grew a dict object at it's end to hold meta information. Naturally, several

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 1:58 PM, Travis E. Oliphant [EMAIL PROTECTED]wrote: Hello all, While we are on the subject of C-API changes, I've noticed that quite a few of the sub-classes of ndarray are constructed to basically add meta-information to the array. What if the base-class ndarray

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 2:10 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Fri, Aug 15, 2008 at 1:58 PM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Hello all, While we are on the subject of C-API changes, I've noticed that quite a few of the sub-classes of ndarray are constructed

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Robert Kern
On Fri, Aug 15, 2008 at 15:15, Charles R Harris [EMAIL PROTECTED] wrote: On Fri, Aug 15, 2008 at 2:10 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Fri, Aug 15, 2008 at 1:58 PM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Hello all, While we are on the subject of C-API changes, I've

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Travis E. Oliphant
Robert Kern wrote: 3) Are the additional 4-8 bytes too expensive One of the problems with numarray was the time taken to allocate small arrays. Would adding a dictionary slow down the allocation of numpy arrays? No, I don't think so, not if we did nothing by default but set

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Travis E. Oliphant
Andrew Dalke wrote: Can that be made quicker? Not easily. lib is first imported in add_newdocs. Personally, I want to get rid of add_newdocs and move the docstrings into the correct locations. Where would that be, in the C-code? The reason for add_newdocs is to avoid writing

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 3:12 PM, Travis E. Oliphant [EMAIL PROTECTED]wrote: Robert Kern wrote: 3) Are the additional 4-8 bytes too expensive One of the problems with numarray was the time taken to allocate small arrays. Would adding a dictionary slow down the allocation of numpy

Re: [Numpy-discussion] min() of array containing NaN

2008-08-15 Thread Ryan May
Availability of the NaN functionality in a method of ndarray The last point is key. The NaN behavior is central to analyzing real data containing unavoidable bad values, which is the bread and butter of a substantial fraction of the user base. In the languages they're switching from,

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Christian Heimes
Robert Kern wrote: I think you could make the dictionary created lazily on the first getattr(). In order to make it work you have to reserve space for a PyObject* pointer for the instance dict somewhere in your type definition. It's going to increase the size of every object by 4 bytes on a

[Numpy-discussion] Questions about some of the random functions

2008-08-15 Thread Alan Jackson
Just a question - I'm gradually working through the distributions for the documentation marathon and I realised that there is a whole nest of them named standard-. For several (e.g., normal) they are just the regular distribution with all the parameters except size set to standard values.

[Numpy-discussion] [REVIEW] Update NumPy API format to support updates that don't break binary compatibility

2008-08-15 Thread Stéfan van der Walt
The current NumPy API number, stored as NPY_VERSION in the header files, needs to be incremented every time the NumPy C-API changes. The counter tells developers with exactly which revision of the API they are dealing. NumPy does some checking to make sure that it does not run against an old

Re: [Numpy-discussion] Questions about some of the random functions

2008-08-15 Thread Robert Kern
On Fri, Aug 15, 2008 at 17:26, Alan Jackson [EMAIL PROTECTED] wrote: Just a question - I'm gradually working through the distributions for the documentation marathon and I realised that there is a whole nest of them named standard-. For several (e.g., normal) they are just the regular

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Robert Kern
On Fri, Aug 15, 2008 at 17:31, Christian Heimes [EMAIL PROTECTED] wrote: Robert Kern wrote: I think you could make the dictionary created lazily on the first getattr(). In order to make it work you have to reserve space for a PyObject* pointer for the instance dict somewhere in your type

[Numpy-discussion] long double woes on win32

2008-08-15 Thread David Cournapeau
Hi, The test failures on windows with 1.2b2 are due to buggy long double behavior of mingw. My understanding is that on windows, long double is 8 bytes (that's the sizeof value returned by VS 2003), but mingw says 12 bytes. One solution would be forcing numpy to configure itself to handle long

[Numpy-discussion] Generalised ufuncs branch

2008-08-15 Thread Stéfan van der Walt
Hi all, I have moved the generalised ufuncs functionality off to a branch: http://svn.scipy.org/svn/numpy/branches/gen_ufuncs Please try it out and give us your feedback. We shall also pound on it at the sprint during SciPy'08, and thereafter decide how and when to best integrate it into

Re: [Numpy-discussion] long double woes on win32

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 5:43 PM, David Cournapeau [EMAIL PROTECTED]wrote: Hi, The test failures on windows with 1.2b2 are due to buggy long double behavior of mingw. My understanding is that on windows, long double is 8 bytes (that's the sizeof value returned by VS 2003), but mingw says 12

Re: [Numpy-discussion] Addition of a dict object to all NumPy objects

2008-08-15 Thread Christian Heimes
Robert Kern wrote: Yes, we know that. The concern I was addressing was the time overhead for creating the new dict object every time an ndarray gets instantiated. Most of these dict objects would be unused, so we would be wasting a substantial amount of time. If you push off the creation of

Re: [Numpy-discussion] Different results from repeated calculation, part 2

2008-08-15 Thread Xavier Gnata
Alok Singhal wrote: On 14/08/08: 10:20, Keith Goodman wrote: A unit test is attached. It contains three tests: In test1, I construct matrices x and y and then repeatedly calculate z = calc(x,y). The result z is the same every time. So this test passes. In test2, I construct matrices x

Re: [Numpy-discussion] C99 on windows

2008-08-15 Thread Christian Heimes
David Cournapeau wrote: The current trunk has 14 failures on windows (with mingw). 12 of them are related to C99 (see ticket 869). Can the people involved in recent changes to complex functions take a look at it ? I think this is high priority for 1.2.0 I'm asking just out of curiosity. Why

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
On Aug 15, 2008, at 6:41 PM, Andrew Dalke wrote: I don't think it's enough. I don't like environmental variable tricks like that. My tests suggest: current SVN: 0.12 seconds my patch: 0.10 seconds removing some top-level imports: 0.09 seconds my patch and removing some

Re: [Numpy-discussion] C99 on windows

2008-08-15 Thread Charles R Harris
On Fri, Aug 15, 2008 at 7:25 PM, Christian Heimes [EMAIL PROTECTED] wrote: David Cournapeau wrote: The current trunk has 14 failures on windows (with mingw). 12 of them are related to C99 (see ticket 869). Can the people involved in recent changes to complex functions take a look at it ? I

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Anne Archibald
2008/8/15 Andrew Dalke [EMAIL PROTECTED]: On Aug 15, 2008, at 6:41 PM, Andrew Dalke wrote: I don't think it's enough. I don't like environmental variable tricks like that. My tests suggest: current SVN: 0.12 seconds my patch: 0.10 seconds removing some top-level imports: 0.09

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
On Aug 15, 2008, at 11:18 PM, Travis E. Oliphant wrote: I've removed this loop. Are there other places in numpy.core that depend on numpy.lib? That fixed the loop I identified. I removed the import lib in add_newdocs.py and things imported fine. I then commented out the following lines

Re: [Numpy-discussion] NumPy 1.2.0b2 released

2008-08-15 Thread Andrew Dalke
On Aug 15, 2008, at 11:18 PM, Travis E. Oliphant wrote: Where would that be, in the C-code? The reason for add_newdocs is to avoid writing docstrings in C-code which is a pain. That was my thought. I could see that the code might useful during module development, where you don't want text

Re: [Numpy-discussion] C99 on windows

2008-08-15 Thread Christian Heimes
Charles R Harris wrote: I believe C99 was used as a guide to how complex corner cases involving +/-0, +/-inf, etc. should behave. However, it doesn't look possible to make that behaviour portable without a lot of work and it probably isn't worth the trouble. At the moment the failing tests

Re: [Numpy-discussion] Different results from repeated calculation, part 2

2008-08-15 Thread Andrew Dalke
On Aug 16, 2008, at 3:02 AM, Xavier Gnata wrote: abs(a-b)epsilon is the correct way to test that a and b are equal. But 1) the value of epsilon is algorithm dependent, 2) it may be that -epsilon1 a-b epsilon2 where the epsilons are not the same value, 3) the more valid test may be =