Re: [Numpy-discussion] C-API change for 1.2
2008/8/17 Charles R Harris [EMAIL PROTECTED]: I have nothing against extensions when they can be made to serve. If a dictionary gets added to ndarrays I hope it is done that way, likewise for generalized ufuncs. In the present case I think Travis wants to preserve the functionality while changing the name and type, and that doesn't really fit the extension model. But I might be wrong about that. I believe that is more or less accurate; Travis can confirm. With the new API version counter in place (it's not, btw -- still up for review), we should be able to keep the breakage to a minimum from here onwards. Unfortunately, I don't see an easy way to avoid it for the 1.2 release. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C-API change for 1.2
On Sun, Aug 17, 2008 at 12:51 AM, Charles R Harris [EMAIL PROTECTED] wrote: I have nothing against extensions when they can be made to serve. If a dictionary gets added to ndarrays I hope it is done that way, likewise for generalized ufuncs. But that's a totally different matter. You can add functionalities and not breaking API or ABI. I mean gtk 2 exists for maybe 7 or 8 years now, and a program linked against gtk 2.0.0 can run with gtk 2.12.0, but gtk 2.12.0 certainly is different than 2.0.0. Having a policy about API/ABI stability does not mean we can't change anything. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C-API change for 1.2
David Cournapeau wrote: Does that mean we will continue breaking the ABI from time to time during the 1.* cycle ? Jon Wright wrote: Can someone help me to understand me what is the compelling reason for this change? If it only means everyone recompiles, it is hard to see what we, as users, are gaining by doing that. As another NumPy user, and developer on another python project using Numeric/numpy at the C level, I had much the same thought. For our users who compile from source or use their Linux distribution's packages this isn't that big problem (the Linux packagers however may disagree!). However, to support Windows users, this means that in addition to providing one installation exe per version of python, we'd also need to provide separate versions for numpy 1.1 and 1.2. I am also not looking forward to the additional up front effort to reorganise our build system to cope with two versions of numpy. I imagine most 3rd party projects which use the NumPy at the C level will have similar views, and quite likely many of them are currently blissfully unaware of this impending API breakage, as I would have expect more comments. (I would have replied earlier but I had to track down which email address I was subscribed to this list with.) I really can't see why NumPy 1.2 is going to break the C-API compatibility for something that sounds so trivial as renaming the hasobject member of the PyArray_Descr structure to flags. This doesn't even add any new functionality! I do understand that the API will have to evolve at some point, and Stéfan van der Walt's version numbering scheme sounds like a sensible change to make when you do change the C-API. I'm just questioning the rational for breaking the C-API in NumPy 1.2 for what seems to me a trivial change. I don't know if this constitutes major opposition, but is keeping the same C-API for NumPy 1.2 unchanged still a possibility? Please? Thank you, Peter ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Aside from more operators needed, is there a consensus view among the developers? Taking a user's perspective, I see a short run and a long run. SR: I am very comfortable with adding dot versions of operators. I am not worried about reversing the Matlab/GAUSS meanings, but if others are very worried, we could append the dot instead of prepending it. LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 6:01 AM, Alan G Isaac [EMAIL PROTECTED] wrote: Aside from more operators needed, is there a consensus view among the developers? I don't think so, but given that pep 225 exists and is fully fleshed out, I guess it should be considered the starting point of the discussion for reference. This doesn't mean that modifications to it can't be suggested, but that I'm assuming python-dev will want that as the reference point. For something as big as this, they would definitely want to work off a real pep. Having said that, I think all ideas are fair game at this point. I personally would like to see it happen, but if not I'd like to see a final pronouncement on the matter rather than seeing pep 225 deferred forever. Taking a user's perspective, I see a short run and a long run. SR: I am very comfortable with adding dot versions of operators. I am not worried about reversing the Matlab/GAUSS meanings, but if others are very worried, we could append the dot instead of prepending it. LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. I'll be sure to list this as part of the received feedback. I'm personally not too crazy about unicode operators (at least not to the extent that Fortress went, where basically a special IDE would be required to write the code in any reasonable scenario). But I'm willing to change my mind, and I'm *definitely* acting as scribe here, so everything that is presented will be reported back. As we get more info I'll start a summary document, which will then be completed with 'live' feedback from the session at scipy next week. Thanks! Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
But would it be not-trivial to enter times ans alike unicode symbols within normal text editors? Otherwise it is a compelling proposition at first glance. Nadav. -הודעה מקורית- מאת: [EMAIL PROTECTED] בשם Alan G Isaac נשלח: א 17-אוגוסט-08 16:01 אל: Discussion of Numerical Python נושא: Re: [Numpy-discussion] Possible new multiplication operators for Python Aside from more operators needed, is there a consensus view among the developers? Taking a user's perspective, I see a short run and a long run. SR: I am very comfortable with adding dot versions of operators. I am not worried about reversing the Matlab/GAUSS meanings, but if others are very worried, we could append the dot instead of prepending it. LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Nadav Horesh wrote: But would it be not-trivial to enter times ans alike unicode symbols within normal text editors? Otherwise it is a compelling proposition at first glance. First, what is a normal text editor? Handling utf-8 seems pretty common these days. http://en.wikipedia.org/wiki/Unicode_input There are two separable issues: entry and display. Entry of unicode operators would presumably involve digraphs. http://en.wikipedia.org/wiki/Digraph_(computing) Display seems largely addressed by the STIX fonts: http://www.stixfonts.org/project.html fwiw, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Fernando Perez wrote: For something as big as this, they would definitely want to work off a real pep. What might be interesting, for those who want to experiment with this syntax, is to take my Python parser for Python (python4ply - http://www.dalkescientific.com/Python/python4ply.html ) and add support for the proposed syntax. It's only a grammar change and it shouldn't be that hard to change the syntax tree so that a proposed ~+ or whatever gets converted to the right method call. Python4ply generates PVM byte code, so the end result is code that works with the existing Python. You could even be adventurous and map things like a \power b into binary operators, perhaps named __op_power__ It's loads of fun to tweak the grammar. My tutorial even walks through how to add Perl-like syntax for regex creation and match operator, to allow for line in open(python_yacc.py): if line =~ m/def (?Pname\w+) *(?Pargs\(.*\)) *:/: print repr($1), repr($args) :) LR: It would be great to use unicode math operators. Though if you want to experiment with this .. I make neither guarantees nor warrantees about Unicode support in what I did. I don't even support the encoding hint that Python code can have. Andrew [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 11:10:11AM -0700, Fernando Perez wrote: LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. I'll be sure to list this as part of the received feedback. I'm personally not too crazy about unicode operators (at least not to the extent that Fortress went, where basically a special IDE would be required to write the code in any reasonable scenario). But I'm willing to change my mind, and I'm *definitely* acting as scribe here, so everything that is presented will be reported back. I am very much against unicode operators. I can see a huge amount of problems this will generate, for little gain. There are still some possibilities to use a plain ascii character (I can think of '^', '/', '$', '!', althought each one of these might lead to confusion, and they have a feeling of perl). Could we also go for multiple cahracter operators? Anybody care for '.*'? Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Gaël Varoquaux wrote: Anybody care for '.*'? That's border-line case, and probably on the bad idea side because 1.*2 already means something in normal Python. If added there would be a difference between 1.*2 and 1 .*2 This problem already exists. Consider 1 .__str__() '1' 1.__str__() File stdin, line 1 1.__str__() ^ SyntaxError: invalid syntax 1..__str__() '1.0' but it doesn't come up much because it's extremely rare for anyone to call numerical methods directly. Whereas people already write things like 2*x so writing 2.*x to mean piecewise but actually get object-wise seems likely. This is why I think being able to write experimental support for new syntax, and gain experience about it's advantages and disadvantages, could be useful and could provide additional sway for python-dev folks. Andrew [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 09:33:33PM +0200, Andrew Dalke wrote: Gaël Varoquaux wrote: Anybody care for '.*'? That's border-line case, and probably on the bad idea side because 1.*2 already means something in normal Python. If added there would be a difference between 1.*2 and 1 .*2 Good point. This was more of a joke than a serious proposition. The joke is that in Matlab, '.*' is a different operator than '*'. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 12:14 PM, Gael Varoquaux [EMAIL PROTECTED] wrote: On Sun, Aug 17, 2008 at 11:10:11AM -0700, Fernando Perez wrote: LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. I'll be sure to list this as part of the received feedback. I'm personally not too crazy about unicode operators (at least not to the extent that Fortress went, where basically a special IDE would be required to write the code in any reasonable scenario). But I'm willing to change my mind, and I'm *definitely* acting as scribe here, so everything that is presented will be reported back. I am very much against unicode operators. I can see a huge amount of problems this will generate, for little gain. There are still some possibilities to use a plain ascii character (I can think of '^', '/', '$', '!', althought each one of these might lead to confusion, and they have a feeling of perl). Could we also go for multiple cahracter operators? Anybody care for '.*'? Please read the pep first: the proposal is already for multichar operators, and the various alternatives are discussed in detail there. The .OP form is specifically addressed in the pep, so if you want to discuss that one, you need to cover the objections to it already raised in the pep. As I said before, this discussion shouldn't start from a blank slate: consider the pep 225 http://www.python.org/dev/peps/pep-0225/ as the starting point that python-dev will want for any useful discussion. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 7:01 AM, Alan G Isaac [EMAIL PROTECTED] wrote: Aside from more operators needed, is there a consensus view among the developers? Taking a user's perspective, I see a short run and a long run. SR: I am very comfortable with adding dot versions of operators. I am not worried about reversing the Matlab/GAUSS meanings, but if others are very worried, we could append the dot instead of prepending it. LR: It would be great to use unicode math operators. On this issue, Fortress is being foresightful. Accepting the times symbol would be a fairly small move for most users, since it is in the Latin 1 extension of ASCII. I kinda like the unicode idea because of the current dearth of usable symbols. In fact, my system is already using utf-8. [EMAIL PROTECTED] ~]$ locale LANG=en_US.utf8 LC_CTYPE=en_US.utf8 LC_NUMERIC=en_US.utf8 LC_TIME=en_US.utf8 LC_COLLATE=en_US.utf8 LC_MONETARY=en_US.utf8 LC_MESSAGES=en_US.utf8 LC_PAPER=en_US.utf8 LC_NAME=en_US.utf8 LC_ADDRESS=en_US.utf8 LC_TELEPHONE=en_US.utf8 LC_MEASUREMENT=en_US.utf8 LC_IDENTIFICATION=en_US.utf8 LC_ALL= And keymaps are not that hard to do in most editors. However... there are various unicode encodings and locales, often indicated by the first few bytes in a file. It's not clear to me that these encodings are universally implemented at this time and I would be loath to depend on them without some testing. Even so, it might be good to reserve a few symbols for future use as operators. The resistance to adding these operators might be less among Python developers because they are less visible, not multicharater, and won't conflict with current python usage. So at the least I think we should try to get some unicode symbols set aside and maybe several years from now we can start using them. Here is an interesting little article on unicode for unixhttp://eyegene.ophthy.med.umich.edu/unicode/. Here are some links to various symbolshttp://mindprod.com/jgloss/unicode.html#SYMBOLS. And here is a bit of unicode just so we can see how it looks for various folks. A = B⊛C Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Aug 17, 2008, at 9:38 PM, Charles R Harris wrote: And here is a bit of unicode just so we can see how it looks for various folks. A = B⊛C Or write B \circledast C ? (Or \oast?) Try using Google to search for that character. Andrew [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 1:58 PM, Andrew Dalke [EMAIL PROTECTED]wrote: On Aug 17, 2008, at 9:38 PM, Charles R Harris wrote: And here is a bit of unicode just so we can see how it looks for various folks. A = B⊛C Or write B \circledast C ? (Or \oast?) Try using Google to search for that character. That's what I sent, it's in utf-8. I'm thinking the circleddot looks better, but here a few others A = B⊛C #circled asterisk A = B⊙C #circled dot A = B⍟C #circled star A = B⊗C #circled times (tensor) A = B∙C #dot A = B⋆C #star Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Gael Varoquaux wrote: I am very much against unicode operators. I can see a huge amount of problems this will generate, for little gain. I actually basically like PEP 225, although I find @*, @+, etc more readable, and to provide the right visual emphasis. (Rather than ~*, ~+, etc.) Additionally, I do not think Unicode addresses the problem central to the PEP, as there is as far as I know no standard symbolic distinction between, say, elementwise multiplication and matrix multiplication. So I think the PEP is basically needed (perhaps with @ instead of ~). That said, what kind of problems do you have in mind? Cheers, Alan Isaac PS Here are the core unicode math operators: http://www.alanwood.net/unicode/mathematical_operators.html or if you do not have appropriate fonts you can try http://www.unicode.org/charts/PDF/U2200.pdf ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
Andrew Dalke wrote:
Or write B \circledast C ? (Or \oast?) Try using Google to search
for that character.
unicodedata.lookup('CIRCLED ASTERISK OPERATOR')
'⊛'
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 7:03 AM, Fernando Perez [EMAIL PROTECTED] wrote: Hi all, [ please keep all replies to this only on the numpy list. I'm cc'ing the scipy ones to make others aware of the topic, but do NOT reply on those lists so we can have an organized thread for future reference] In the Python-dev mailing lists, there were recently two threads regarding the possibility of adding to the language new multiplication operators (amongst others). This would allow one to define things like an element-wise and a matrix product for numpy arrays, for example: http://mail.python.org/pipermail/python-dev/2008-July/081508.html http://mail.python.org/pipermail/python-dev/2008-July/081551.html It turns out that there's an old pep on this issue: http://www.python.org/dev/peps/pep-0225/ which hasn't been ruled out, simply postponed. At this point it seems that there is room for some discussion, and obviously the input of the numpy/scipy crowd would be very welcome. I volunteered to host a BOF next week at scipy so we could collect feedback from those present, but it's important that those NOT present at the conference can equally voice their ideas/opinions. So I wanted to open this thread here to collect feedback. We'll then try to have the bof next week at the conference, and I'll summarize everything for python-dev. Obviously this doesn't mean that we'll get any changes in, but at least there's interest in discussing a topic that has been dear to everyone here. I like that Python is so easy to learn, so I hope no more operators and definitely not unicode operators will be added. And if, then only if they are really needed, which I think they are not. What I like on Python is that I can remember all it's syntax in my memory. The more operators, the more difficult this will be. There is some inconsistency though, for example one can override A() + A(), but one cannot override 1 + 1. This could (should) be fixed somehow. But apart from that, I very much like Python as it is now and I hope it will not become a bloated language. Ondrej ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Sun, Aug 17, 2008 at 17:00, Ondrej Certik [EMAIL PROTECTED] wrote: There is some inconsistency though, for example one can override A() + A(), but one cannot override 1 + 1. This could (should) be fixed somehow. This is getting off-topic, but I really hope that never changes. The difference between A.__add__ and int.__add__ is that you are defining A. Python defines int. If you mess with fundamental types like that, you will break other libraries and must confine yourself to just code that you've written. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Aug 17, 2008, at 10:35 PM, Christian Heimes wrote:
Andrew Dalke wrote:
Or write B \circledast C ? (Or \oast?) Try using Google to search
for that character.
unicodedata.lookup('CIRCLED ASTERISK OPERATOR')
'⊛'
I mean, go to Google and search for ⊛.
It finds no hits.
I didn't even know the name for that character. I had to copy and
paste it into a terminal window that didn't understand the encoding,
so it gave me \342\212\233. I then used Google and found a LaTeX
page listing octal Unicode escape characters for various LaTeX
definitions, those being \circledast which has \oast as an alias.
Those two terms I can search for. Not some sort of unknown-to-me
unicode character.
Andrew
[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Possible new multiplication operators for Python
On Aug 18, 2008, at 12:00 AM, Ondrej Certik wrote: There is some inconsistency though, for example one can override A() + A(), but one cannot override 1 + 1. This could (should) be fixed somehow. That will never, ever change in Python. There's no benefit to being able to redefine int.__add__ and doing so will break entirely too many assumptions. Here's one assumption - the C implementation does some simple constant folding: def f(): ... print 1+1 ... import dis dis.dis(f) 2 0 LOAD_CONST 2 (2) 3 PRINT_ITEM 4 PRINT_NEWLINE 5 LOAD_CONST 0 (None) 8 RETURN_VALUE With what you want that's not possible. Just think of the implementation difficulty. Are changes on the per-module or per-scope or per-thread level? And performance would be lousy (or require deep inferencing analysis) because every operation in C would need to go through the Python API just in case some fundamental definition like this was changed. Such a language is possible. I wouldn't be surprised if you could do it in Smalltalk and mostly do it in Ruby. But the general belief is that good design follows the open/closed principle: software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification - Bertrand Meyer, quoted by http://en.wikipedia.org/wiki/Open/closed_principle In Python, all types are closed for modification, and while classes are open for modification it's highly frowned upon to do so. The name monkey-patch sounds somewhat derisive for a reason. Andrew [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Problem with the mailing list?
Hi All, I received an email from Hans Andreas -- the gen-ufuncs guy -- and he is unable to post to the list even though subscribed. Anyone know what might be the problem? Please cc [EMAIL PROTECTED] if you reply to this post. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. I wish everyone a great SciPy'08 conference and very much hope that you can include the proposed functionality in the forthcoming NumPy release. Best regards, Hansres ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problem with the mailing list?
Charles R Harris [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED]... Hi All, I received an email from Hans Andreas -- the gen-ufuncs guy -- and he is unable to post to the list even though subscribed. Anyone know what might be the problem? Please cc [EMAIL PROTECTED] if you reply to this post. Chuck Thanks Chuck for following up on this -- it works now! (Initially I signed up a different address than the one I used for posting.) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalised ufuncs branch
Stéfan van der Walt [EMAIL PROTECTED] wrote in message news:[EMAIL
PROTECTED]...
Hi all,
I have moved the generalised ufuncs functionality off to a branch:
http://svn.scipy.org/svn/numpy/branches/gen_ufuncs
Please try it out and give us your feedback. We shall also pound on
it at the sprint during SciPy'08, and thereafter decide how and when
to best integrate it into NumPy.
Thanks to Wenjie Fu and Hans-Andreas Engel for taking the time to
think this issue through and to submit such a high-quality patch.
Regards
Stéfan
--
Motivated by Stefan's and Chuck's off-line comments, I have attached an example
implementation of a generalized ufunc:
/*
* This implements the function out = inner1d(in1, in2) with
* out[K] = sum_i { in1[K, i] * in2[K, i] }
* and multi-index K, as described on
* http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions
* and on http://projects.scipy.org/scipy/numpy/ticket/887.
*/
static void
DOUBLE_inner1d(char **args, intp *dimensions, intp *steps, void *func)
{
/* Standard ufunc loop length and strides. */
intp dn = dimensions[0];
intp s0 = steps[0];
intp s1 = steps[1];
intp s2 = steps[2];
intp n;
/* Additional loop length and strides for dimension i in elementary
function. */
intp di = dimensions[1];
intp i_s1 = steps[3];
intp i_s2 = steps[4];
intp i;
/* Outer loop: equivalent to standard ufuncs */
for (n = 0; n dn; n++, args[0] += s0, args[1] += s1, args[2] += s2) {
char *ip1 = args[0], *ip2 = args[1], *op = args[2];
/* Implement elementary function: out = sum_i { in1[i] * in2[i] } */
npy_double sum = 0;
for (i = 0; i di; i++) {
sum += (*(npy_double *)ip1) * (*(npy_double *)ip2);
ip1 += i_s1;
ip2 += i_s2;
}
*(npy_double *)op = sum;
}
}
/* Actually create the ufunc object */
static PyUFuncGenericFunction inner1d_functions[] = { DOUBLE_inner1d };
static void *inner1d_data[] = { (void *)NULL };
static char inner1d_sigs[] = { PyArray_DOUBLE, PyArray_DOUBLE, PyArray_DOUBLE };
static void
addInner1d(PyObject *dictionary) {
PyObject *f = PyUFunc_FromFuncAndDataAndSignature(inner1d_functions,
inner1d_data, inner1d_sigs, 1,
2, 1, PyUFunc_None, inner1d,
inner on the last dimension and broadcast
on the other dimensions,
0,
(i),(i)-());
PyDict_SetItemString(dictionary, inner1d, f);
}
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 19:13, Engel, Hans-Andreas [EMAIL PROTECTED] wrote: I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. Not at all. You did everything right. Unfortunately, it comes at a slightly awkward time. We are just about to make a release, and this is a substantial feature that some of us developers think needs a little more consideration than we can give it if we were to put it into this release. It's only the timing that triggered the conflict, not anything you really had control over. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. My concern is not that the patch might be buggy or anything like that. Rather, this is a substantial feature, and one that affects the C API. It will be very difficult to remove it or modify it later if we find that it needs even a slight tweak to its design. Despite its use in other array languages, it's new to us, and I think we need to explore its use cases a little. numpy isn't PDL, and the way this feature integrates with the rest of numpy's semantics and idioms is important to figure out. It's possible that there are problems that it could solve with just a slight modification to the design. I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [ANNOUNCE] EPD with Py2.5 v4.0.3001 Beta1 now available
Hello, Thanks to heroic efforts by Chris Galvan this weekend, and significant efforts by the team that finalized ETS 3.0.0 this week, we've been able to publish public beta releases of EPD with Py2.5 v4.0.30001 Beta1 for Windows and Mac OS X today. I've uploaded them to the downloads website and updated the EPD product pages to provide download links for the public. You can find the link to the betas here: http://www.enthought.com/products/epddownload.php Please give them a try and report any bugs to the EPD Trac site at https://svn.enthought.com/epd. In this release, EPD has been updated to include ETS 3.0.0, NumPy 1.1.1, IPython 0.9.beta, Matplotlib 0.98.1, Sphinx 0.4.2, pyhdf 0.8, VTK 5.0.4, wxPython 2.8.8.1, and many more updated projects. There are a few issues known at this time, but remember these are our first beta release of this version: * The included documentation hasn't been updated to the current versions of the third-party libraries. * Some of the product branding is not up-to-date with regard to the product name change to EPD with Py2.5, nor with the version number of 4.0.30001 Beta 1. -- Dave ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 6:13 PM, Engel, Hans-Andreas [EMAIL PROTECTED] wrote: I am sorry that our submission http://projects.scipy.org/scipy/numpy/ticket/887 has created some annoyance; presumably we have taken the Make contributions (e.g. code patches), (...) by submitting a 'ticket' on the Trac pages linked below on http://scipy.org/Developer_Zone somewhat too literally. It was great to receive very positive responses to our patch and to receive a very timely review; this is encouraging for submitting code to the numpy repository. I would like to add that the proposed change is not that arbitrary; it is a well-established concept -- it is outlined on the GeneralLoopingFunctions wiki page, and it is a prominent concept of perl's PDL vector library. Of course, there is still room for arguing about details. The fact that no explicit generalized ufuncs are provided, should in my opinion not be an argument why not to include the change in the 1.2.0 release. Writing extension libraries that implement such generalized ufuncs, while being able to use the standard numpy distribution, would certainly be very valuable. Furthermore, the risk for including the proposed patch in 1.2.0 is very low: existing functionality is not touched. (Except the glitch we had by declaring variables in a gcc-way.) For standard ufuncs, it should be straightforward to see that the patch does not modify the behavior at all. I wish everyone a great SciPy'08 conference and very much hope that you can include the proposed functionality in the forthcoming NumPy release. For those interested in the description posted with ticket #887http://scipy.org/scipy/numpy/ticket/887, here it is. The patch can be found with the ticket. I note that PyUFunc_FromFuncAndDataAndSignature is not put at the end of the ufunc_api_order.txt list and might break the current API, which has entries like #define PyUFunc_Type (*(PyTypeObject *)PyUFunc_API[0]) So we need to keep the PyUFunc_API entries in order. Chuck Generalized Universal Functions ¶http://scipy.org/scipy/numpy/ticket/887#GeneralizedUniversalFunctions There is a general need for looping over not only functions on scalars but also over functions on vectors (or arrays), as explained on http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions. We propose to realize this concept by generalizing the universal functions (ufuncs), and provide a C implementation that adds ~500 lines to the numpy code base. In current (specialized) ufuncs, the elementary function is limited to element-by-element operations, whereas the generalized version supports sub-array by sub-array operations. The Perl vector library PDL provides a similar functionality and its terms are re-used in the following. Each generalized ufunc has information associated with it that states what the core dimensionality of the inputs is, as well as the corresponding dimensionality of the outputs (the element-wise ufuncs have zero core dimensions). The list of the core dimensions for all arguments is called the signature of a ufunc. For example, the ufunc numpy.add has signature (),()-() defining two scalar inputs and one scalar output. Another example is (see the GeneralLoopingFunctionshttp://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctionspage) the function inner1d(a,b) with a signature of (i),(i)-(). This applies the inner product along the last axis of each input, but keeps the remaining indices intact. For example, where a is of shape (3,5,N) and b is of shape (5,N), this will return an output of shape (3,5). The underlying elementary function is called 3*5 times. In the signature, we specify one core dimension (i) for each input and zero core dimensions () for the output, since it takes two 1-d arrays and returns a scalar. By using the same name i, we specify that the two corresponding dimensions should be of the same size (or one of them is of size 1 and will be broadcasted). The dimensions beyond the core dimensions are called loop dimensions. In the above example, this corresponds to (3,5). The usual numpy broadcasting rules apply, where the signature determines how the dimensions of each input/output object are split into core and loop dimensions: 1. While an input array has a smaller dimensionality than the corresponding number of core dimensions, 1's are pre-pended to its shape. 2. The core dimensions are removed from all inputs and the remaining dimensions are broadcasted; defining the loop dimensions. 3. The output is given by the loop dimensions plus the output core dimensions. Definitions ¶ http://scipy.org/scipy/numpy/ticket/887#Definitions *Elementary Function:* Each ufunc consists of an elementary function that performs the most basic operation on the smallest portion of array arguments (e.g. adding two numbers is the most basic operation in adding two arrays). The
Re: [Numpy-discussion] Generalized ufuncs?
2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. That branch is here: http://[EMAIL PROTECTED]/svn/numpy/branches/gen_ufuncs Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 7:56 PM, Stéfan van der Walt [EMAIL PROTECTED]wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. That branch is here: http://[EMAIL PROTECTED]/svn/numpy/branches/gen_ufuncs For an earlier thread about using vector valued ufuncs for sorts and such -- and negative reactions to the suggestion -- go herehttp://thread.gmane.org/gmane.comp.python.numeric.general/20552/focus=20560. One of the major objections was how to call such functions with the ufunc machinery and the needed machinery for type promotions, sub classes, and all that nonsense. Are these dealt with in the patch? The current numpy code for all that is a bit of a mess anyway, and it would be nice to figure out some unified interface to call through and clean up the current code in the process. In fact, I've been making some preliminary efforts in that direction by formatting the current code and working through it. Also, do we also want reduce methods and such? I suspect there is still a lot of work to do to get this whole thing up and running. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 21:55, Anne Archibald [EMAIL PROTECTED] wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. I'll take what I can get. In order of increasing utility: 1. Descriptions of use cases (as above). 2. Mockups of the Python code demonstrating the use case (e.g. nonfunctional Python code showing a potential generalized ufunc with inputs and outputs). 3. The C code implementing the actual functionality for the use case. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalized ufuncs?
On Sun, Aug 17, 2008 at 9:15 PM, Robert Kern [EMAIL PROTECTED] wrote: On Sun, Aug 17, 2008 at 21:55, Anne Archibald [EMAIL PROTECTED] wrote: 2008/8/17 Robert Kern [EMAIL PROTECTED]: I suggested that we move it to a branch for the time being so we can play with it and come up with examples of its use. If you have examples that you have already written, I would love to see them. I, for one, am amenable to seeing this in 1.2.0, but only if we push back the release by at least a few weeks. This is not a worked example, but this is exactly what is needed to make possible the arrays of matrices functions that were discussed some time ago. For example, users frequently want to do something like multiply an array of matrices by an array of matrices; that is not currently feasible without a very large temporary array (or a loop). But I think you were looking for examples of code using the interface, to see whether it worked well. I'll take what I can get. In order of increasing utility: 1. Descriptions of use cases (as above). It is also possible to do sums, cumulative sums, and other such things. A generalization of the proposed ufuncs useful for these sorts of things would be mixed type arguments, which would also help in implementing such things as argsort and casting. This requires a different kind of infrastructure for defining and looking up the ufuncs, but I don't think of this as a complication, but rather a simplification as it would allow us to make good use of the code generator and introduce a certain uniformity to the implementation of numpy. Uniformity is good. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
