Re: [Numpy-discussion] Generalized ufuncs?
Robert Kern [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]...
On Sun, Aug 17, 2008 at 21:55, Anne Archibald
[EMAIL PROTECTED] wrote:
2008/8/17 Robert Kern [EMAIL PROTECTED]:
I suggested that we move it to a branch for the time being so we
can
play with it and come up with examples of its use. If you have
examples that you have already written, I would love to see them.
I,
for one, am amenable to seeing this in 1.2.0, but only if we push
back
the release by at least a few weeks.
This is not a worked example, but this is exactly what is needed to
make possible the arrays of matrices functions that were discussed
some time ago. For example, users frequently want to do something
like
multiply an array of matrices by an array of matrices; that is not
currently feasible without a very large temporary array (or a loop).
But I think you were looking for examples of code using the
interface,
to see whether it worked well.
I'll take what I can get. In order of increasing utility:
1. Descriptions of use cases (as above).
2. Mockups of the Python code demonstrating the use case (e.g.
nonfunctional Python code showing a potential generalized ufunc
with inputs and outputs).
3. The C code implementing the actual functionality for the use
case.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
Please find an example on inner1d in the following.
1. One use case for an inner function is described on
http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions and
http://thread.gmane.org/gmane.comp.python.numeric.general/17694.
(Another one would be the array of matrices usage mentioned above; we
have called this dot2d with signature (m,n),(n,p)-(m,p) on
http://scipy.org/scipy/numpy/ticket/887: here the matrix multiplication
would occur with respect to the last two dimensions.)
2. The mockup python code would be:
from numpy import *
N = 10
a = random.randn(3, 5, N)
b = random.randn(5, N)
# standard inner function
... inner(a, b).shape
(3, 5, 5)
# new ufunc inner1d with signature (i),(i)-(), satisfying
GeneralLoopingFunctions use case
... inner1d(a, b).shape
(3, 5)
3. Concrete implementation of inner1d in C:
/*
* This implements the function out = inner1d(in1, in2) with
* out[K] = sum_i { in1[K, i] * in2[K, i] }
* and multi-index K, as described on
* http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions
* and on http://projects.scipy.org/scipy/numpy/ticket/887.
*/
static void
DOUBLE_inner1d(char **args, intp *dimensions, intp *steps, void *func)
{
/* Standard ufunc loop length and strides. */
intp dn = dimensions[0];
intp s0 = steps[0];
intp s1 = steps[1];
intp s2 = steps[2];
intp n;
/* Additional loop length and strides for dimension i in
elementary function. */
intp di = dimensions[1];
intp i_s1 = steps[3];
intp i_s2 = steps[4];
intp i;
/* Outer loop: equivalent to standard ufuncs */
for (n = 0; n dn; n++, args[0] += s0, args[1] += s1, args[2] +=
s2) {
char *ip1 = args[0], *ip2 = args[1], *op = args[2];
/* Implement elementary function: out = sum_i { in1[i] * in2[i]
} */
npy_double sum = 0;
for (i = 0; i di; i++) {
sum += (*(npy_double *)ip1) * (*(npy_double *)ip2);
ip1 += i_s1;
ip2 += i_s2;
}
*(npy_double *)op = sum;
}
}
/* Actually create the ufunc object */
static PyUFuncGenericFunction inner1d_functions[] = { DOUBLE_inner1d };
static void *inner1d_data[] = { (void *)NULL };
static char inner1d_sigs[] = { PyArray_DOUBLE, PyArray_DOUBLE,
PyArray_DOUBLE };
static void
addInner1d(PyObject *dictionary) {
PyObject *f = PyUFunc_FromFuncAndDataAndSignature(inner1d_functions,
inner1d_data, inner1d_sigs, 1,
2, 1, PyUFunc_None, inner1d,
inner on the last dimension and
broadcast on the other dimensions,
0,
(i),(i)-());
PyDict_SetItemString(dictionary, inner1d, f);
}
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
Charles R Harris [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED]... On Sun, Aug 17, 2008 at 7:56 PM, Stéfan van der Walt [EMAIL PROTECTED]wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. That branch is here: http://[EMAIL PROTECTED]/svn/numpy/branches/gen_ufuncs For an earlier thread about using vector valued ufuncs for sorts and such -- and negative reactions to the suggestion -- go herehttp://thread.gmane.org/gmane.comp.python.numeric.general/20552/focus=20560. One of the major objections was how to call such functions with the ufunc machinery and the needed machinery for type promotions, sub classes, and all that nonsense. Are these dealt with in the patch? The current numpy code for all that is a bit of a mess anyway, and it would be nice to figure out some unified interface to call through and clean up the current code in the process. In fact, I've been making some preliminary efforts in that direction by formatting the current code and working through it. Also, do we also want reduce methods and such? I suspect there is still a lot of work to do to get this whole thing up and running. Chuck -- The good news is that the patch just uses of the existing code to deal with all the tricky issues (this is why the patch is so short). By the way, sort could be implemented with the proposed specifications, its signature would be (i)-(i). I agree that it would be nice if that code could be made somewhat clearer; however, I think that this task is orthogonal to the generalized ufuncs patch, because there is no code overlap. The way the suggested implementation basically works is to remove the core dimensions from the input/output arrays, and then have the existing code handle all the intricacies over the loop dimensions. Reduce methods are currently not supported (an error is raised). Therefore, the current patch does not forestall anything and the desired functionality can be added whenever it is clear what would be best. I do not think that it would makes sense to specify/implement all possible extensions, optimizations, concrete ufuncs, morphing of existing numpy functions to ufuncs, etc. at once; presumably it is much better to start with a small but extremely flexible specification of generalized ufuncs first. Best, Hansres ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
The good news is that the patch just uses of the existing code to deal with all the tricky issues (this is why the patch is so short). By the way, sort could be implemented with the proposed specifications, its signature would be (i)-(i). I agree that it would be nice if that code could be made somewhat clearer; however, I think that this task is orthogonal to the generalized ufuncs patch, because there is no code overlap. I agree with this. The way the suggested implementation basically works is to remove the core dimensions from the input/output arrays, and then have the existing code handle all the intricacies over the loop dimensions. Reduce methods are currently not supported (an error is raised). Therefore, the current patch does not forestall anything and the desired functionality can be added whenever it is clear what would be best. I do not think that it would makes sense to specify/implement all possible extensions, optimizations, concrete ufuncs, morphing of existing numpy functions to ufuncs, etc. at once; presumably it is much better to start with a small but extremely flexible specification of generalized ufuncs first. One of the key reasons I'm enthused about the patch is because it's so small. By enhancing the ufunc object and without changing the signature of the underlying function, the patch is able to implement the general description of a generalized ufunc. I think it is useful to evaluate whether or not a few more changes will allow more functionality with little cost, but I don't think it is worth holding up the patch hoping that the code will get cleaned-up (which all code needs according to somebody's definition of cleaning). -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Mon, Aug 18, 2008 at 10:13 AM, Travis E. Oliphant [EMAIL PROTECTED] wrote: The good news is that the patch just uses of the existing code to deal with all the tricky issues (this is why the patch is so short). By the way, sort could be implemented with the proposed specifications, its signature would be (i)-(i). I agree that it would be nice if that code could be made somewhat clearer; however, I think that this task is orthogonal to the generalized ufuncs patch, because there is no code overlap. I agree with this. The way the suggested implementation basically works is to remove the core dimensions from the input/output arrays, and then have the existing code handle all the intricacies over the loop dimensions. Reduce methods are currently not supported (an error is raised). Therefore, the current patch does not forestall anything and the desired functionality can be added whenever it is clear what would be best. I do not think that it would makes sense to specify/implement all possible extensions, optimizations, concrete ufuncs, morphing of existing numpy functions to ufuncs, etc. at once; presumably it is much better to start with a small but extremely flexible specification of generalized ufuncs first. One of the key reasons I'm enthused about the patch is because it's so small. By enhancing the ufunc object and without changing the signature of the underlying function, the patch is able to implement the general description of a generalized ufunc. I think it is useful to evaluate whether or not a few more changes will allow more functionality with little cost, but I don't think it is worth holding up the patch hoping that the code will get cleaned-up (which all code needs according to somebody's definition of cleaning). I think the plan is that 1.2.1 will come out before the end of the year and it would be reasonable to put the patch in there. As gen_ufuncs are currently unused there is no practical effect to waiting until after the 1.2 release. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. I wish everyone a great SciPy'08 conference and very much hope that you can include the proposed functionality in the forthcoming NumPy release. Best regards, Hansres ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 19:13, Engel, Hans-Andreas [EMAIL PROTECTED] wrote: I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. Not at all. You did everything right. Unfortunately, it comes at a slightly awkward time. We are just about to make a release, and this is a substantial feature that some of us developers think needs a little more consideration than we can give it if we were to put it into this release. It's only the timing that triggered the conflict, not anything you really had control over. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. My concern is not that the patch might be buggy or anything like that. Rather, this is a substantial feature, and one that affects the C API. It will be very difficult to remove it or modify it later if we find that it needs even a slight tweak to its design. Despite its use in other array languages, it's new to us, and I think we need to explore its use cases a little. numpy isn't PDL, and the way this feature integrates with the rest of numpy's semantics and idioms is important to figure out. It's possible that there are problems that it could solve with just a slight modification to the design. I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 6:13 PM, Engel, Hans-Andreas [EMAIL PROTECTED] wrote: I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. I wish everyone a great SciPy'08 conference and very much hope that you can include the proposed functionality in the forthcoming NumPy release. For those interested in the description posted with ticket #887http://scipy.org/scipy/numpy/ticket/887, here it is. The patch can be found with the ticket. I note that PyUFunc_FromFuncAndDataAndSignature is not put at the end of the ufunc_api_order.txt list and might break the current API, which has entries like #define PyUFunc_Type (*(PyTypeObject *)PyUFunc_API[0]) So we need to keep the PyUFunc_API entries in order. Chuck Generalized Universal Functions ¶http://scipy.org/scipy/numpy/ticket/887#GeneralizedUniversalFunctions There is a general need for looping over not only functions on scalars but also over functions on vectors (or arrays), as explained on http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions. We propose to realize this concept by generalizing the universal functions (ufuncs), and provide a C implementation that adds ~500 lines to the numpy code base. In current (specialized) ufuncs, the elementary function is limited to element-by-element operations, whereas the generalized version supports sub-array by sub-array operations. The Perl vector library PDL provides a similar functionality and its terms are re-used in the following. Each generalized ufunc has information associated with it that states what the core dimensionality of the inputs is, as well as the corresponding dimensionality of the outputs (the element-wise ufuncs have zero core dimensions). The list of the core dimensions for all arguments is called the signature of a ufunc. For example, the ufunc numpy.add has signature (),()-() defining two scalar inputs and one scalar output. Another example is (see the GeneralLoopingFunctionshttp://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctionspage) the function inner1d(a,b) with a signature of (i),(i)-(). This applies the inner product along the last axis of each input, but keeps the remaining indices intact. For example, where a is of shape (3,5,N) and b is of shape (5,N), this will return an output of shape (3,5). The underlying elementary function is called 3*5 times. In the signature, we specify one core dimension (i) for each input and zero core dimensions () for the output, since it takes two 1-d arrays and returns a scalar. By using the same name i, we specify that the two corresponding dimensions should be of the same size (or one of them is of size 1 and will be broadcasted). The dimensions beyond the core dimensions are called loop dimensions. In the above example, this corresponds to (3,5). The usual numpy broadcasting rules apply, where the signature determines how the dimensions of each input/output object are split into core and loop dimensions: 1. While an input array has a smaller dimensionality than the corresponding number of core dimensions, 1's are pre-pended to its shape. 2. The core dimensions are removed from all inputs and the remaining dimensions are broadcasted; defining the loop dimensions. 3. The output is given by the loop dimensions plus the output core dimensions. Definitions ¶ http://scipy.org/scipy/numpy/ticket/887#Definitions *Elementary Function:* Each ufunc consists of an elementary function that performs the most basic operation on the smallest portion of array arguments (e.g. adding two numbers is the most basic operation in adding two arrays). The
Re: [Numpy-discussion] Generalized ufuncs?
2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. That branch is here: http://[EMAIL PROTECTED]/svn/numpy/branches/gen_ufuncs Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 7:56 PM, Stéfan van der Walt [EMAIL PROTECTED]wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. That branch is here: http://[EMAIL PROTECTED]/svn/numpy/branches/gen_ufuncs For an earlier thread about using vector valued ufuncs for sorts and such -- and negative reactions to the suggestion -- go herehttp://thread.gmane.org/gmane.comp.python.numeric.general/20552/focus=20560. One of the major objections was how to call such functions with the ufunc machinery and the needed machinery for type promotions, sub classes, and all that nonsense. Are these dealt with in the patch? The current numpy code for all that is a bit of a mess anyway, and it would be nice to figure out some unified interface to call through and clean up the current code in the process. In fact, I've been making some preliminary efforts in that direction by formatting the current code and working through it. Also, do we also want reduce methods and such? I suspect there is still a lot of work to do to get this whole thing up and running. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 21:55, Anne Archibald [EMAIL PROTECTED] wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. I'll take what I can get. In order of increasing utility: 1. Descriptions of use cases (as above). 2. Mockups of the Python code demonstrating the use case (e.g. nonfunctional Python code showing a potential generalized ufunc with inputs and outputs). 3. The C code implementing the actual functionality for the use case. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 9:15 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Aug 17, 2008 at 21:55, Anne Archibald [EMAIL PROTECTED] wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. I'll take what I can get. In order of increasing utility: 1. Descriptions of use cases (as above). It is also possible to do sums, cumulative sums, and other such things. A generalization of the proposed ufuncs useful for these sorts of things would be mixed type arguments, which would also help in implementing such things as argsort and casting. This requires a different kind of infrastructure for defining and looking up the ufuncs, but I don't think of this as a complication, but rather a simplification as it would allow us to make good use of the code generator and introduce a certain uniformity to the implementation of numpy. Uniformity is good. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
Numpy 1.2 is for documentation, bug fixes, and getting the new testing framework in place. Discipline is called for if we are going to have timely releases. We also agreed to a change in the C-API (or at least did not object too loudly). I'm in favor of minimizing that sort of change. Why not wait until after the release then? The biggest reason is that the patch requires changing the C-API and we are already doing that for 1.2. I would rather not do it again for another 6 months at least. I don't think we should make the patch wait that long. Your code review is very much appreciated. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Thu, Aug 14, 2008 at 10:54 PM, Charles R Harris [EMAIL PROTECTED] wrote: Numpy 1.2 is for documentation, bug fixes, and getting the new testing framework in place. Discipline is called for if we are going to have timely releases. First, all your points are very valid. And I apologize for the role I played in this. Thanks for calling us on it. That said while you are correct that this release is mainly about documentation, bug fixes, and getting the new testing framework in place, there are several other things that have gone in. Their have been a few planned API changes and even a C-API change. Travis emailed me asking where we were on the beta release and whether we should discuss including this change on the list. I contacted Stefan and asked him if he could do me a huge favor and see if we could quickly apply the patch before making the beta release. My reasoning was that this looked very good and useful and just offered something new. Stefan was hesitant, but I persisted. He didn't like that it didn't have any tests, but I said if he put it in time for the beta he could add tests afterward. I wanted to make sure no new features got in after a beta. Also we are all ready requiring recompiling with this release, so I thought now would be a good time to add it. We is the numpy community, not you and Travis. Absolutely. There were several of us involved, not just Travis and Stefan. But that is no excuse. Stefan, David, Chris, and I have been trying very hard to get the beta out over the last few days and had started talking among ourselves since we were mostly just coordinating. Taking that over to feature adding was a mistake. Why not wait until after the release then? The motivation is that we are not allowing features in bugfix releases anymore. So it can't go in in 1.2.x if it isn't in 1.2.0. I also want to get several 1.2.x releases out. That means the earliest we could get it in is 1.3.0. But I would prefer not having to require recompiling extension code with every minor release. Sorry. This was handled poorly. But I think this would still be very useful and I would like to see it get in. We were planning on releasing a 1.2.0b3 early next week. But this is it, I promise. How about we work on it and see where we are early next week. If it doesn't look good, we can pull it. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. Where did you notice this? I didn't see it. python or numpy types. The use of inline and the local declaration of variables would also have been caught early in a code review. What do you mean by the local declaration of variables? -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
Travis E. Oliphant wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. Where did you notice this? I didn't see it. Are you referring to Stefan's patch to the Fu's _parse_signature code in r5654.This is a local function, I'm not sure why there is a concern. python or numpy types. The use of inline and the local declaration of variables would also have been caught early in a code review. What do you mean by the local declaration of variables? Never mind, I understand it's the mid-code declaration of variables (without a separate block defined) that Stefan fixed. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Fri, Aug 15, 2008 at 12:35 AM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Travis E. Oliphant wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. That would be fabulous. So far nobody has figured out how... Jarrod?? Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. Where did you notice this? I didn't see it. Are you referring to Stefan's patch to the Fu's _parse_signature code in r5654.This is a local function, I'm not sure why there is a concern. There probably isn't a problem, but the use of unsigned types in loop counters and such can lead to subtle errors, so when a signed type is changed to an unsigned type the code has to be audited to make sure there won't be any unintended consequences. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Aug 15, 2008, at 8:36 AM, Charles R Harris wrote: The inline keyword also tends to be gcc/icc specific, although it is part of the C99 standard. For reference, a page on using inline and doing so portably: http://www.greenend.org.uk/rjk/2003/03/inline.html Andrew [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Fri, Aug 15, 2008 at 12:45 AM, Andrew Dalke [EMAIL PROTECTED]wrote: On Aug 15, 2008, at 8:36 AM, Charles R Harris wrote: The inline keyword also tends to be gcc/icc specific, although it is part of the C99 standard. For reference, a page on using inline and doing so portably: http://www.greenend.org.uk/rjk/2003/03/inline.html Doesn't do the trick for compilers that aren't C99 compliant. And there are many of them. For gcc there are other options. -finline-functionsIntegrate all simple functions into their callers. The compiler heuristically decides which functions are simple enough to be worth integrating in this way. If all calls to a given function are integrated, and the function is declared static, then the function is normally not output as assembler code in its own right. Enabled at level -O3. -finline-functions-called-onceConsider all static functions called once for inlining into their caller even if they are not marked inline. If a call to a given function is integrated, then the function is not output as assembler code in its own right. Enabled if -funit-at-a-time is enabled. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Thu, Aug 14, 2008 at 9:05 PM, Charles R Harris [EMAIL PROTECTED] wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. It should work now. Let me know if you aren't getting them now. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Fri, Aug 15, 2008 at 1:04 AM, Jarrod Millman [EMAIL PROTECTED]wrote: On Thu, Aug 14, 2008 at 9:05 PM, Charles R Harris [EMAIL PROTECTED] wrote: Can we fix the ticket notification mailings some day? It's been almost four months now. It should work now. Let me know if you aren't getting them now. Thanks, it seems to be working now. What did you do? Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Fri, Aug 15, 2008 at 1:16 AM, Travis E. Oliphant [EMAIL PROTECTED] wrote: The biggest reason is that the patch requires changing the C-API and we are already doing that for 1.2. I would rather not do it again for another 6 months at least. I don't think we should make the patch wait that long. I understand the concern, but that should have been discussed I think. Changing C code affects most process wrt releases, not just people concerned with API stability. From my POV, recent C code caused me a lot of trouble wrt binaries building. If we keep changing C code during the beta, I won't be able to follow. The problem I see with any C (not necessarily C API) change is that they can break a lot of things. For example, I did not notice, but several generated code (umath stuff, mtrand) break Visual Studio compilation because of too long strings. If we accept changes in the C code during the beta phase, it just does not mean much to have beta. The point to have a time-based release is to enforce this kind of things; if we don't, then not only time-based release do not make sense, but they make those problem even worse (no benefit, and we rush things out which do not work). I see a lot of bugs in scipy/numpy, matplotlib problems in the last few days report on the ML and trac. Putting non bug fix-related C code will make this an ever-ending battle. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Generalized ufuncs?
Stefan, I notice that you have merged some new ufunc infrastructure. I think these sort of things should be discussed and reviewed on the list before being committed. Could you explain what the purpose of these patches is? The commit messages are rather skimpy. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Thu, Aug 14, 2008 at 22:45, Charles R Harris [EMAIL PROTECTED] wrote: Stefan, I notice that you have merged some new ufunc infrastructure. I think these sort of things should be discussed and reviewed on the list before being committed. Could you explain what the purpose of these patches is? The commit messages are rather skimpy. Stéfan happens to be in our offices this week, so he did discuss it with Travis, at least. This was actually contributed to us with extensive details from Wenjie Fu and Hans-Andreas Engel here: http://projects.scipy.org/scipy/numpy/ticket/887 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Thu, Aug 14, 2008 at 9:55 PM, Robert Kern [EMAIL PROTECTED] wrote: On Thu, Aug 14, 2008 at 22:45, Charles R Harris [EMAIL PROTECTED] wrote: Stefan, I notice that you have merged some new ufunc infrastructure. I think these sort of things should be discussed and reviewed on the list before being committed. Could you explain what the purpose of these patches is? The commit messages are rather skimpy. Stéfan happens to be in our offices this week, so he did discuss it with Travis, at least. This was actually contributed to us with extensive details from Wenjie Fu and Hans-Andreas Engel here: http://projects.scipy.org/scipy/numpy/ticket/887 Can we fix the ticket notification mailings some day? It's been almost four months now. Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. This is a risky sort of thing and needs to be checked. Nor is it clear we should use size_t instead of one of the python or numpy types. The use of inline and the local declaration of variables would also have been caught early in a code review. So I think in this case the patch should have been discussed and reviewed on the list. An internal discussion at Enthought doesn't serve the same purposel. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
Hi Charles 2008/8/14 Charles R Harris [EMAIL PROTECTED]: Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. This is a risky sort of thing and needs to be checked. Nor is it clear we should use size_t instead of one of the python or numpy types. The use of inline and the local declaration of variables would also have been caught early in a code review. So I think in this case the patch should have been discussed and reviewed on the list. An internal discussion at Enthought doesn't serve the same purposel. I apologise for not keeping the list up to date with the progress on this front. The patch is such a great contribution that I wanted it to become part of NumPy for 1.2b3. The idea was to merge it and, once done, report on the list. As is, I am still busy fixing some bugs on the Windows platform and integrating unit tests. I did, however, get Travis to review the patch beforehand, and we will keep reviewing the changes made until 1.2b3 goes out. The patch does not influence current NumPy behaviour in any way -- it simply provides hooks for general ufuncs, which can be implemented in the future. Thanks for your concern, Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Thu, Aug 14, 2008 at 11:45 PM, Stéfan van der Walt [EMAIL PROTECTED]wrote: Hi Charles 2008/8/14 Charles R Harris [EMAIL PROTECTED]: Re: the patch. I noticed the replacement of the signed type int by an unsigned size_t. This is a risky sort of thing and needs to be checked. Nor is it clear we should use size_t instead of one of the python or numpy types. The use of inline and the local declaration of variables would also have been caught early in a code review. So I think in this case the patch should have been discussed and reviewed on the list. An internal discussion at Enthought doesn't serve the same purposel. I apologise for not keeping the list up to date with the progress on this front. The patch is such a great contribution that I wanted it to become part of NumPy for 1.2b3. The idea was to merge it and, once Numpy 1.2 is for documentation, bug fixes, and getting the new testing framework in place. Discipline is called for if we are going to have timely releases. done, report on the list. Wrong way around. As is, I am still busy fixing some bugs on the Windows platform and integrating unit tests. I did, however, get Travis to review the patch beforehand, and we will keep reviewing the changes made until 1.2b3 goes out. We is the numpy community, not you and Travis. The patch does not influence current NumPy behaviour in any way -- it simply provides hooks for general ufuncs, which can be implemented in the future. Why not wait until after the release then? Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
