Re: RFC 272 (v1) Arrays: transpose()
Jeremy Howard wrote: So where is mv(), you ask? If you use the 'reorder' syntax, but don't specify all of the dimensions in the list ref, then the remaining dimensions are added in order: That sounds good. I'd say why not also allow the mv syntax? It is syntactically different from the others, may be the least often used variant but then there are N+1 ways to do it ;) But no strong feelings either way... Christian
Re: RFC 272 (v1) Arrays: transpose()
Karl Glazebrook wrote: the arguments to reshape should be sizes not last elements (i.e. N's not N-1's). Yup, it's simple: size (N) vs index range (0..N-1) How does this sound? Logical and consistent ;) Christian
Re: Implementing RFC 272
Buddha Buck wrote: When I heard about transpose() (as well as reshape(), etc), I was concerned about the time it would take to execute these complex operations. To wit, naively, I believed that a lot of data shuffling would be necessary. If I understand your message correctly this is exactly what PDL, NumPY, etc do right now and is also described in one of my messages I sent yesterday (Re: RFC 272 (v1) Arrays: transpose()) about implementing aliasing ;) Christian
Re: RFC 231 (v1) Data: Multi-dimensional arrays/hashes and slices
Ilya Zakharevich wrote: On Fri, Sep 22, 2000 at 05:24:55PM -0400, Karl Glazebrook wrote: It's now boiling down to a matter of opinion and we'll have to agree to differ. Of course I use array arithmetic all the time as a heavy PDL user. ...Do you say you are confused by using vectors (=scalars) instead of arrays? I'm not having a problem with that personally but *many* users of PDL have complained about being confused by this. They assume ndim == array == perl array. Christian
Re: RFC 272 (v1) Arrays: transpose()
Jeremy Howard wrote:
Karl Glazebrook wrote:
you should look at the PDL mv() and xchg() methods
and factor this into your thinking!
Actually, the RFC is based on PDL's xchg()! I forgot to document using
negative numbers to count from the last dimension--I'll add that into the
next version. Are there any other differences with xchg() that you think
would be useful?
I haven't used mv() before, but now I look at it I can see it's pretty
interesting. Is this used much? If we add it to the RFC, do you think we'd
want a separate function, or add another arg to transpose:
transpose([$a,$b], 0, @arr); # xchg
transpose([$a,$b], 1, @arr); # mv
How about (if perl6 allows passing arrays implicitly by reference
without arglist flattening)
transpose @arr, $a, $b; # xchg
transpose @arr, {$a = $b}; # mv
transpose @arr, [0,3,4,1,2]; # PDL reorder
Aliasing for bounded typed arrays is simple:
each array has a block of data (void *) and vectors of shape (int[]),
strides (int[]) and offset (int). To get element arr[i;j;k] Perl
accesses into the typecast block
((type *) data)[offset+i*stride[0]+j*stride[1]+k*stride[2]]
When creating an alias only the shape, strides and offset are
manipulated which tells the new alias how to interpret the same block of
data differently. This kind of implementation is used in PDL, NumPy and
GSL AFAIK.
In PDL we have found it useful to have a method turning the alias into a
being of its own ($pdl-sever, oops undocumented).
Christian
Re: RFC 207 (v1) Array: Efficient Array Loops
Reading through the examples left me wondering about some
technicalities:
@t[|i;|j] = @a[|j;|i]; # transpose 2-d @a
Written like this it would require that @a is exact 2-dim, i.e. it would
not just swap the first two dims of any n-dim array? I suppose if I'd
want that I'd write
@t[|i;|j;] = @a[|j;|i;]; # trailing ';' implies there might be
trailing dims
#compute pairwise sum, pairwise product, pairwise difference...
@sum = @a[|i;|j;|k;|l] + @b[|i;|j;|k;|l];
@prod= @a[|i;|j;|k;|l] * @b[|i;|j;|k;|l];
@diff= @a[|i;|j;|k;|l] - @b[|i;|j;|k;|l];
Hm, not sure if I am missing the point of these examples. Is that any
different from the elementwise '+','*','-' apart from being possibly
limited to 4D arrays?
#print lots of stuff to the screen.
sub foo { print join(',',@_),"\n"; return 0; }
$zero[|i;|j;|k;|l;|m;|n;|o;|p] = foo(|i,|j,|k,|l,|m,|n,|o,|p);
Should that be '$zero' or '@zero'?
# Sneaky way to generate dot-product:
my $dotproduct;
{ my @temp[|i] = $dotproduct += $a[|i] * $b[|i]; }
Hm, how can this work with lazy evaluation? How is Perl supposed to know
that @temp should be transiently created to increment $dotproduct as a
side effect? Also, doesn't the above syntax seem to be conflicting with
perl context rules since it essentially contains a statement
@arr = $scalar
Christian
Re: RFC 206 (v1) Array: @#arr for getting the dimensions of an array
Perl6 RFC Librarian wrote:
This RFC proposes using @#array, analogous to $#array, to get the list of
upper bounds for a multidimensional array @array. The length of @#array
would indicate the dimensionality of @array.
That's fine. This RFC does not seem to touch on the question what
$#array should be for a multidim array (e.g., $#array == $#{flatten @a}
or rather undef). Or is that dealt with in another RFC?
Christian
Re: RFC 82 (v3) Arrays: Apply operators element-wise in a list context
Nathan Wiger wrote: what people would want to use the ops for, and it's also more usable to us non-PDLers. I'd like to suggest that it is not a very good idea to start dividing the world into PDLers and non-PDLers. There are a multitude of reasons but I am not keen to go into details. Christian
Re: RFC 82 (v3) Arrays: Apply operators element-wise in a list context
Nathan Wiger wrote: This RFC proposes that operators in a list context should be applied element-wise to the elements of their arguments: @d = @b * @c; # Returns (2,8,18) If the lists are not of equal length, an error is raised. I've been watching this RFC for a while. I would hesitate to change the default behavior of * and other operators in so radical a sense, especially since it would create unexpected error conditions. I think these operations should remain scalar. I disagree. You end up with a situation where some @a * @b; are in scalar context, some not. And what happens when you say @a = (1,2,2); @b = tie Matrix (1,2,3); @a*@b; Who wins? From the user point of view a 1D multi-dim array will look just the same as a normal perl list in *all* respects in a would-be perl6. I bet money that users will be highly confused if some arrays enforce scalar context in such operations, others don't. It is *much* clearer to have a clear break with perl5 in this respect (even if code needs to be rewritten). A p52p6 could replace all occurences of @a*@b with scalar(@a)*scalar(@b). Christian
Re: Some PDL issues (was Re: Test)
Dan Sugalski wrote: to make foo and bar 5x5x5 matricies that you casn multiply to get baz then, well, say it. If that means you need to define a way to provide overridden operators in the Matrix package, then go for it and say that. Let the -internals folks worry about the Weird Magic needed to implement what you want. (This is a variation of my standard "Don't get hung up on the implementation, that's my job" speech) Although there is one point that is crucial in a useful implementation -- packed and typed for the reasons I mentioned before. Since we are on language-data speed and efficiency on large chunks of data is essential and can not be completely left to druids/wizards and their weird magic ;) Christian
Re: New variable type: matrix
Karl Glazebrook wrote: The key from my point of view is to have enough syntactical hooks in the core so that using it is not like wading through treacle. Hence the PDL RFCs - especially on [] overloading and ranges. The Numerical Python people seem to have accomplished this, and we can't let them win!! :-) Hear, hear! It would be nice to relax some of the idealistic ideas of what must not be touched or changed and let us do our dirty tricks. And did I say that any extra keystrokes should be avoided at all cost ;-) Christian
Re: PDL-P: Re: RFC 115 (v1) Default methods for objects
"David L. Nicol" wrote: =head1 TITLE Default methods for objects Title does not match description. How about Overloadable bracketing or Brackets as Methods or Syntax as Overloadable Methods Good point! Will be part of v2. Christian
Re: PDL-P: Re: PDL and perl6
Tuomas Lukka wrote: Do we really need PDL objects in perl 6? How about we investigate building compact arrays into the language? We're already talking about how we can add typing to the language--I would have thought that if these semantics make it in, we could look to optimise how numeric types are stored. I think we should look to remove the distinction between 'stuff that works in PDL' and 'stuff that works in perl'. To do this I guess we need an RFC proposing compact array storage for numeric data. Most of the rest of PDL core is already proposed in one form or another. Stuff like SLATEC support still belongs in external modules, of course. Again, sorry that I can only offer support from the sidelines and not take up the task of writing an RFC. One VERY important thing to remember about the compact data types that PDL deals with is that of smart pointers: e.g. a slice or a diagonal of a matrix is not a copy: it's a smart reference to the originals, and changing the values there will change the originals. This functionality is crucial to have for a good numerical language. So if compact arrays go in, please make sure that implementing such smart pointer arrays is possible, efficient and easy. I agree with Tuomas' assessment. We would certainly love to dispense with the need for PDL if perl6 offered something along these lines. But PDL is much more than an efficient memory representation for typed N-D arrays. Above Jeremy suggests that most of the rest of the PDL core is already proposed in one form or another. Is that really true? There are a number of absolute conerstones of functionality/semantics that are implemented in the current PDL Core which any replacement *must* have or improve upon to be really useful for us: PDL threading, PDL dataflow + smart references (slicing), the PP code generator to generate all the glue code for PDL functions and library interfaces. This crucial functionality (that I don't see in any RFCs yet) currently takes about 1 lines of C and XS code for the core + ~7000 lines of perl for the code generator (which will probably have to be rewritten as well). Without those features we will still need an object that does what piddles do now and our own core. There is nothing wrong with thinking about a rewrite of our core, a tighter integration with the perl core and also getting some input from principal perl gurus on how to throw in some more syntactical sugar. However, I can't see the current PDL porters taking a large role in implementing these things for a new perl simply since they are all *very busy* with their own projects and can't invest the weeks of coding as Tuomas did when he almost single-handedly wrote the current core (we are still struggling with just *documenting* what PDL currently does). So I am not sure how realistic it would be to just impose this task on the perl6 implementers. I mean, has anybody active in the perl6 thrust seriously worked with PDL yet and come across those features that I am alluding to? PDL currently supports affine transformations but other kinds of transforms would be interesting. But to compile to C there need to be few types, unless a JIT compiler is included in the plans for perl 6. And yes, the current PDL could certainly do with extension of its features to objects representing sparse N-D arrays, non-rectangular arrays, etc. So my (personal) feeling is/was that from the PDL porters view suggestions for syntactical changes that make integration of PDL in perl an easier job (rather than a full PDL replacement) are currently our most pragmatic contribution to the perl6 design effort. Christian
Re: PDL-P: Re: PDL and perl6
Jeremy Howard wrote: Make sure you read the interesting RFCs from Damian Conway on related issues: * Built-ins: min() and max() functions and acceptors * Built-ins: reduce() function Couldn't see these. * Data structures: Semi-finite (lazy) lists * Subroutines: higher order functions Found those (RFC2324) * Subroutines: lazy evaluation of argument lists * Superpositions: vector operations via superpositions Couldn't see those either. Could you refer to the actual RFC #s, please? The excellent news is that Dan Sugalski (who's heading up the internals team) has promised that he'd make sure that the optimisations we need would be seen to. In particular, we talked about making sure that the following creates only one loop and no array copies: $a = sum(@b*@c+@d) ...and I did point out that @b et al might actually be columns of a sparse matrix accessed through an iterator. Sounds interesting. From our point of view one question is: assuming that there still is a need for compact numerical objects (PDLs) as opposed to conventional perl arrays (for reasons of efficiency and memory consumption) will those compile time optimizations also be accessible to PDL objects? Anyway, all I can say is, give it a go! Start cross-posting your thoughts to perl6-language, and pop over to dev.perl.org to see what RFCs are around. I'm sure you want to keep up the discussion with PDL-P, but if it's about perl 6, why not have it on perl6-language where everyone can follow it? Sure. One problem we AFAIK have on pdl-porters is that all developers I know of use PDL and perl for real work in their day jobs and are hard pressed to deal with all the mailing list traffic, upcoming problems and RFC writing just in the time frame that seems available. Christian
