Re: [Numpy-discussion] chararray __mod__ behavior
On Fri, Jul 18, 2008 at 8:32 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: That looks like a bug to me. I would have expected at least one of the following to work: A % [[1, 2], [3, 4]] A % 1 A % (1, 2, 3, 4) and none of them do. I wouldn't expect the last one to work, since the shapes are different. The first two work if I use .flat to assign the result. Since this actually changes the existing behavior of chararray, I figured it deserved a tracker item: http://scipy.org/scipy/numpy/ticket/856 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
On Fri, 18 Jul 2008, Travis E. Oliphant wrote: It looks like with that added DECREF, the reference count leak is gone. I've looked at the latest head, and I agree that the problem is now solved. There is an important difference from my original solution: typecode is no longer reused after the finish label (instead it is always created anew). This makes all the difference in the world. I'm not actually convinced by the comment that's there now, which says /* typecode will be NULL */ but in truth it doesn't matter -- because of the correcly placed DECREF after the PyArray_Scalar calls the routine no longer owns typecode. If I can refer to my last message, I made the point that there wasn't a good invariant at the finish label -- we didn't know how many references to typecode we were responsible for at that point -- and I offered the solution to keep typecode. Instead you have chosen to recreate typecode, which I hadn't realised was just as good. This code is still horrible, but I don't think I want to try to understand it anymore. It'd be really nice (it'd make me feel a lot better) if you'd agree that my original patch was in fact correct. I'm not disputing the correcness of the current solution (except I think that typecode can end up being created twice, but who really cares?) but I've put a lot of effort into arguing my case, and the fact is my original patch was not wrong. Thank you. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building a better OSX install for 1.1.1
On Fri, Jul 18, 2008 at 3:37 PM, Charles R Harris [EMAIL PROTECTED] wrote: Since 1.1.1rc1 is coming out this Sunday, I'd like to know who is responsible for the OS X install improvements, if that is what they are. I don't know squat about them myself and don't run OS X. Chris Burns has been building the OS X installer for NumPy. He is looking into whether he can make a better Mac installer for the 1.1.1rc.1 release, which is why he is asking about whether anyone knows how to fix this. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
day pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
Accidental (virus?) post. Humblest apologies for the noise. Please ignore. Gary On Sat, 19 Jul 2008, Gary Strangman wrote: day pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck.Remember -- the Sunday pot-luck invite was a SHAM! The real party is on Saturday, and is not a pot-luck. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sat, Jul 19, 2008 at 7:13 AM, Pauli Virtanen [EMAIL PROTECTED] wrote: Hi all, Re: Ticket 854. I wrote tests for the branch cuts for all complex arc* functions in umathmodule. It turns out that all except arccosh were OK. The formula for arcsinh was written in a non-standard form with an unnecessary nc_neg, but this didn't affect the results. I also wrote tests for checking values of the functions at infs and nans. A patch for all of this is attached, with all currently non-passing tests marked as skipped. I'd like to commit this if there are no objections. Another thing I noticed is that the present implementations of the complex functions are naive, so they over- and underflow earlier than necessary: np.arcsinh(1e8) 19.113827924512311 np.arcsinh(1e8 + 0j) (inf-0j) np.arcsinh(-1e8 + 0j) (-19.113827924512311-0j) This particular thing in arcsinh occurs because of loss of precision in intermediate stages. (In the version in my patch this loss of precision is still present.) It would be nice to polish these up. BTW, are there obstacles to using the C99 complex functions when they are available? This would avoid quite a bit of drudgework... As an alternative, we can probably steal better implementations from Python's recently polished cmathmodule.c The main problem is determining when they are available and if they cover all the needed precisions. Since we will need standalone implementations on some platforms anyway, I am inclined towards stealing from cmathmodule.c if it offers improved code for some of the functions. *** Then comes a descent into pedantry: Numpy complex functions are not C99 compliant in handling of the signed zero, inf, and nan. I don't know whether we should comply with C99, but it would be nicer to have these handled in a controlled way rather than as a byproduct of the implementation chosen. Agreed. 1) The branch cuts for sqrt and arc* don't respect the negative zero: a = 1 + 0j np.sqrt(-a) 1j np.sqrt(-1 + 0j) 1j The branch cut of the logarithm however does: np.log(-a) -3.1415926535897931j np.log(-1 + 0j) 3.1415926535897931j All complex functions in the C99 standard respect the negative zero in their branch cuts. Do we want to follow? Hmm. I think so, to the extent we can. This might lead to some unexpected results, but that is what happens for arguments near the branch cuts. Do it and document it. I don't know how to check what Octave and Matlab do regarding this, since I haven't figured out how to place a negative zero in complex numbers in these languages. But at least in practice these languages appear not to respect the sign of zero. a = 1 + 0j log(-a) ans = 0.000 + 3.141592653589793i log(-1) ans = 0.000 + 3.141592653589793i 2) The numpy functions in general don't return C99 compliant results at inf or nan. I wrote up some tests for checking these. Do we want to fix these? I'd say yes. That way we can refer to the C99 standard to document numpy behavior. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
On Sat, Jul 19, 2008 at 7:13 AM, Pauli Virtanen [EMAIL PROTECTED] wrote: Hi all, Re: Ticket 854. I've backported the fixes to 1.1.x, so you had better commit these ;) Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
On Sat, Jul 19, 2008 at 8:57 AM, Michael Abbott [EMAIL PROTECTED] wrote: On Fri, 18 Jul 2008, Travis E. Oliphant wrote: It looks like with that added DECREF, the reference count leak is gone. I've looked at the latest head, and I agree that the problem is now solved. There is an important difference from my original solution: typecode is no longer reused after the finish label (instead it is always created anew). This makes all the difference in the world. I'm not actually convinced by the comment that's there now, which says /* typecode will be NULL */ but in truth it doesn't matter -- because of the correcly placed DECREF after the PyArray_Scalar calls the routine no longer owns typecode. If I can refer to my last message, I made the point that there wasn't a good invariant at the finish label -- we didn't know how many references to typecode we were responsible for at that point -- and I offered the solution to keep typecode. Instead you have chosen to recreate typecode, which I hadn't realised was just as good. This code is still horrible, but I don't think I want to try to understand it anymore. It'd be really nice (it'd make me feel a lot better) if you'd agree that my original patch was in fact correct. I'm not disputing the correcness of the current solution (except I think that typecode can end up being created twice, but who really cares?) but I've put a lot of effort into arguing my case, and the fact is my original patch was not wrong. Yep, the original patch looks good now. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ticket #842.
Stéfan van der Walt wrote: 2008/7/19 Charles R Harris [EMAIL PROTECTED]: In [2]: type(conjugate(array(8+7j))) Out[2]: type 'numpy.complex128' In [3]: type((array(8+7j))) Out[3]: type 'numpy.ndarray' So I think all that needs to be done is fix the return type conjugate if we agree that it should be an array. I think it should be an array. +1/2 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType
Michael Abbott wrote: I'm not actually convinced by the comment that's there now, which says /* typecode will be NULL */ but in truth it doesn't matter -- because of the correcly placed DECREF after the PyArray_Scalar calls the routine no longer owns typecode. I'm pretty sure that it's fine. If I can refer to my last message, I made the point that there wasn't a good invariant at the finish label -- we didn't know how many references to typecode we were responsible for at that point -- and I offered the solution to keep typecode. Instead you have chosen to recreate typecode, which I hadn't realised was just as good. I agree that this routine needs aesthetic improvement. I had hoped that someone would have improved the array scalars routines by now. I think a core issue is that to save a couple of lines of code, an inappropriate goto finish in the macro was used. This complicated the code more than the savings of a couple lines of code would justify. Really this code grew from a simple thing into a complicated thing as more features were added. This is a common issue that happens all over the place. This code is still horrible, but I don't think I want to try to understand it anymore. It'd be really nice (it'd make me feel a lot better) if you'd agree that my original patch was in fact correct. I'm not disputing the correcness of the current solution (except I think that typecode can end up being created twice, but who really cares?) but I've put a lot of effort into arguing my case, and the fact is my original patch was not wrong. From what I saw, I'm still not quite sure. Your description of reference counting was correct and it is clear you've studied the issue which is great, because there aren't that many people who understand reference counting on the C-level in Python anymore and it is still a useful skill. I'm hopeful that your description of reference counting will be something others can find and learn from. The reason I say I'm not sure is that I don't remember seeing a DECREF after the PyArray_Scalar in the obj = NULL part of the code in my looking at your patches.But, I could have just missed it. Regardless, that core piece was lost in my trying to figure out the other changes you were making to the code. From a generic reference counting point of view you did correctly emphasize the problem of having a reference count creation occur in an if-statement but a DECREF occur all the time in the finish: section of the code. It was really that statement: the fantasy that PyArray_Scalar steals a reference that tipped me off to what I consider one of the real problems to be.The fact that it was masked at the end of a long discussion about other reference counting and a stealing discussion that were not the core problem was distracting and ultimately not very helpful. I'm very impressed with your ability to follow these reference count issues. Especially given that you only started learning about the Python C-API a few months ago (if I remember correctly).I'm also very glad you are checking these corner cases which have not received the testing that they deserve.I hope we have not discouraged you too much from continuing to help. Your input is highly valued. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Branch cuts, inf, nan, C99 compliance
Pauli Virtanen wrote: Hi all, Re: Ticket 854. I wrote tests for the branch cuts for all complex arc* functions in umathmodule. It turns out that all except arccosh were OK. The formula for arcsinh was written in a non-standard form with an unnecessary nc_neg, but this didn't affect the results. I also wrote tests for checking values of the functions at infs and nans. Thanks for looking into these. These functions were contributed by Konrad Hinsen (IIRC) many years ago and I don't think they've really been reviewed since then. I'm all for using C99 when it is available and improving these functions with help from cmathmodule. IIRC, the cmathmodule was contributed by Konrad originally also. So +1 on C99 standardization. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Buildbot errors on Windows_XP.
Memmap problems. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Masked array fill_value
Hi, I just noticed this and found it surprising: In [8]: from numpy import ma In [9]: a = ma.array([1,2,3,4],mask=[False,False,True,False],fill_value=0) In [10]: a Out[10]: masked_array(data = [1 2 -- 4], mask = [False False True False], fill_value=0) In [11]: a[2] Out[11]: masked_array(data = --, mask = True, fill_value=1e+20) In [12]: np.__version__ Out[12]: '1.1.0' Is there a reason that the fill_value isn't inherited from the parent array? Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked array fill_value
Ryan May wrote: Hi, I just noticed this and found it surprising: In [8]: from numpy import ma In [9]: a = ma.array([1,2,3,4],mask=[False,False,True,False],fill_value=0) In [10]: a Out[10]: masked_array(data = [1 2 -- 4], mask = [False False True False], fill_value=0) In [11]: a[2] Out[11]: masked_array(data = --, mask = True, fill_value=1e+20) In [12]: np.__version__ Out[12]: '1.1.0' Is there a reason that the fill_value isn't inherited from the parent array? There was a thread about this a couple months ago, and Pierre GM explained it. I think the point was that indexing is giving you a new masked scalar, which is therefore taking the default mask value of the type. I don't see it as a problem; you can always specify the fill value explicitly when you need to. Eric Ryan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked array fill_value
Eric Firing wrote: Ryan May wrote: Hi, I just noticed this and found it surprising: In [8]: from numpy import ma In [9]: a = ma.array([1,2,3,4],mask=[False,False,True,False],fill_value=0) In [10]: a Out[10]: masked_array(data = [1 2 -- 4], mask = [False False True False], fill_value=0) In [11]: a[2] Out[11]: masked_array(data = --, mask = True, fill_value=1e+20) In [12]: np.__version__ Out[12]: '1.1.0' Is there a reason that the fill_value isn't inherited from the parent array? There was a thread about this a couple months ago, and Pierre GM explained it. I think the point was that indexing is giving you a new masked scalar, which is therefore taking the default mask value of the type. I don't see it as a problem; you can always specify the fill value explicitly when you need to. I thought it sounded familiar. You're right, it's not a big problem, it just seemed unintuitive. Thanks for the explaination. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked array fill_value
On Saturday 19 July 2008 18:41:22 Ryan May wrote: There was a thread about this a couple months ago, and Pierre GM explained it. I think the point was that indexing is giving you a new masked scalar, which is therefore taking the default mask value of the type. I don't see it as a problem; you can always specify the fill value explicitly when you need to. Actually, in that example, a[2] is THE masked scalar: that's a constant initialized when you import numpy.ma, it doesn't depend on the type. I thought it sounded familiar. You're right, it's not a big problem, it just seemed unintuitive. Thanks for the explaination. It is in a way, but it's needed for compatibility with older code. That way, you can test whether a value is masked by using: a[2] is masked Yeah, you could also check whether the mask if not nomask and whether the mask at this particular element is True, but it's a bit longer. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Backport r5452?
Robert, Is there any reason I shouldn't backport your build fixes? Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Backport r5452?
On Sat, Jul 19, 2008 at 21:51, Charles R Harris [EMAIL PROTECTED] wrote: Robert, Is there any reason I shouldn't backport your build fixes? Go ahead. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
