On Wed, Jul 14, 2010 at 9:09 PM, Robert Bradshaw
<rober...@math.washington.edu> wrote:
> On Wed, Jul 14, 2010 at 11:23 AM, Dag Sverre Seljebotn
> <da...@student.matnat.uio.no> wrote:
>> On 07/14/2010 06:01 PM, Kurt Smith wrote:
>>> On Wed, Jul 14, 2010 at 5:15 AM, Fernando Perez<fperez....@gmail.com>
>>> wrote:
>>>
>>>> On Tue, Jul 13, 2010 at 10:30 PM, Kurt Smith<kwmsm...@gmail.com> wrote:
>>>>
>>>>> Unfortunately, for some compilers I've used, array expressions like:
>>>>>
>>>>> array1 = array2 + array3**2
>>>>>
>>>>> Are slower than the 'do' version with explicit indexing -- that is to
>>>>> say, slower than they should be. Apparently the compilers create
>>>>> temporary arrays, exactly analogous to how numpy does it (although
>>>>> many times faster and with more opportunities for optimization).
>>>>>
>>>> as a minor note, blitz++ handles this with expression templates, and
>>>> that's what weave uses for array manipulations. So one gets very
>>>> numpy-like arrays in c++ that have also efficient behavior in many
>>>> expressions.
>>>>
>>>> Not saying that we don't also want 'inline fortran', just that weave's
>>>> inline/blitz support is actually quite powerful.
>>>>
>>> Yes, it is quite powerful. It will take some doing to get equivalent
>>> support for all that weave has to offer. I'm hoping to allow
>>> something like this, modulo syntactic stuff:
>>>
>>> def cy_weave_example(arr):
>>> src = \
>>> '''
>>> for(npy_intp i=0; i<arr_shape[0]; i++) {
>>> for(npy_intp j=0; j<arr_shape[1]; j++) {
>>> arr[i,j] = i * j;
>>> }
>>> }
>>> '''
>>> cyweave.inline(src, ['arr'])
>>>
>>> The "arr[i,j]" syntax will take some work, and I'm still thinking
>>> about it. It's not valid C, obviously. One possibility would be to
>>> do some simple preprocessing to transform into something like
>>> "arr[__2D_idx(i, j, arr_shape[1])]" where "__2D_idx(...)" is a
>>> generated macro which is something like:
>>>
>>> #define __2D_idx(i, j, s1) ((i) * (s1) + (j))
>>>
>>> If we were to use the C syntax "arr[i][j]", it would require a weave
>>> recompile whenever the shape of "arr" changes, which would be
>>> annoying. The "arr[i,j]" would preserve the array-like feel and would
>>> be doable. Perhaps there are pitfalls in doing a poor-man's bracket
>>> overloading -- I'm certainly open to suggestions.
>>>
>>
>> Isn't there's enough tools going in this direction already? This would
>> very closely duplicate what is aimed for (or already exists) in Cython,
>> and not be backwards compatible with Weave...
>>
>> As long as it is not something people are already using anyway, why not
>> allow Cython code within the string? I think you can do pretty much the
>> same as what you propose for C currently, and more future possibilities
>> then seem open (such as array expressions/first class arrays).
>
> That was my first thought too--what advantage is there to inlining C
> into the mix? Fortran I can see more of a usecase for, though even
> then I hope that Cython will be able to handle most of those usecases
> as well.
>
> Though I can see the advantages of weave, I think it'd be even better
> if one didin't have to switch back and forth between completely
> unrelated syntaxes/types/etc. Do people use weave because there are
> certain things that are more natural to express in Fortran (and/or C),
> or is it primarily a convenient way to inline compiled code? Still
> waiting for someone to put up a wiki page on what they like from weave
> and what should go into Cython.
I'll get to it soon; but this thread serves to get the ideas out there
in raw form.
My impressions are from using weave a bit in the past and from those
who have mentioned interest in keeping it alive with a Cython
transplant (primarily Fernando). I welcome Fernando (& others) to
chime in with the other killer features that weave offers. And
critics of the weave 'approach' are more than weclome, too :-)
(Note: I've only used weave, but much of what I say applies to blitz, too.)
1) For an existing function that has to be sped up, weave has a lower
barrier to usage than Cython, even with pyximport. (pyximport doesn't
enable an equivalent functionality as weave, so it isn't exactly
germane). It is very nice to just drop into C/C++ in the middle of a
function and have things compiled & run automatically. This is in
large part because weave allows incremental usage wherever needed on a
function-by-function basis, whereas Cython's smallest compilation unit
is the module.
2) I liked weave because you know you're getting to the metal with no
more than 5-10 lines of setup code, and with absolutely no fuss with
external tools or having to create a setup.py. This may sound lazy,
but it really is nice not to have to deal with these things.
Sometimes numpy array expressions aren't the most natural way to
express something, and you just want to cut through it all and get a
simple, fast C 'for' loop in there and move on, but cythonizing would
be too big of a step, or impossible if you have an inner function
somewhere else in the module, for example. Weave scratches the itch
very nicely.
3) The C-source-in-a-string is pretty ugly, I admit. Weave is very
far on the "practicality" side of the "practicality vs. purity" scale.
It is a small benefit, though, to have the C/C++ code quarantined to
one place in a function, and to always know exactly what to expect
from each part of your code -- python remains python, C remains C.
The fluidity of Cython is a tremendous advantage, but it does require
some care to be sure you're typing everything that needs to be typed,
turning off boundschecking when it's safe, etc. I tend to use 'cython
-a' or inspecting the generated source to be sure I'm "totally C"
fairly often. Weave gives this to you automatically. In short, with
weave, I don't have to pay attention as much to get the same degree of
performance. The tradeoff is ugly code.
4) Dag mentioned allowing Cython code in the source string. This
certainly has merit and I'm somewhat in favor of it. It would avoid
reinventing the wheel for the array indexing stuff, and would have a
more pythonic feel. But it would negate some of the benefits of
having "just C" in the source string.
6) Up until now I've been assuming that this functionality will be
implemented as an external tool. Perhaps you (Robert, Dag, others)
have thoughts on getting some subset of weave's features in Cython
itself, namely the function-level compilation of a chunk of code.
Robert mentions this below with a @cython.compile decorator. And the
'cython.eval(<cython source>)' has all the essential features of
weave/blitz right there, including the ugliness ;-)
So, remaining issues:
-) Essential features of weave/blitz that I haven't mentioned.
-) Is this thing better off as an external tool, or integrated in cython itself?
If the source string is cython code, then it's better as an
internal-to-cython tool. If C, then an external tool would suffice.
-) I'll summarize this, make it less verbose, and put it on the wiki.
Kurt
>
> I think an
>
> @cython.compile
> def foo(...):
> ...
>
> could go a long way here, and even
>
> ...
> cython.eval("""
> [valid cython code]
> """)
>
> could have its merits too (though it's not near as clean.
>
> - Robert
> _______________________________________________
> Cython-dev mailing list
> Cython-dev@codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
_______________________________________________
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev
