I don't know how your model would work, but in the model I was
thinking of, a meta array with metarank 2 3 and metashape 2 2 2,:9 9 9
would contain 35 integers.
How many integers would the meta array contain in your model?
Thanks,
--
Raul
On Thu, Mar 3, 2022 at 3:06 PM 'Pascal Jasmin' via Programming
<programm...@jsoftware.com> wrote:
>
> when "subarrays" are all the same rank and there are x such items then shape
> is x , subarrayShape.
>
> exotic/table shapes would seem to fit with the arrays of arrays exotic
> concept.
>
>
>
>
>
>
> On Thursday, March 3, 2022, 02:52:14 p.m. EST, Raul Miller
> <rauldmil...@gmail.com> wrote:
>
>
>
>
>
> Conceptually, I think an "array" whose "shape's rank" is higher than 1
> should represent a collection of arrays of the same rank.
>
> --
> Raul
>
> On Thu, Mar 3, 2022 at 11:50 AM 'Pascal Jasmin' via Programming
> <programm...@jsoftware.com> wrote:
> >
> > I'm ok with the 0s, but another way to have 2 shapes (referring to 2
> > arrays) of different sizes ie. one of 3 2 2 the other shape 4, would be
> > that the "shape of the shapes" is a "by 1 table":
> >
> > 3
> > 1
> >
> > so
> >
> > (3 ,:1) $ 3 2 2 4 would be the shape of i. (3 ,:1) $ 3 2 2 4 or ((3 ,:1)
> > $ 3 2 2 4) $ i.16
> >
> > I'm ok with this as a formal definition for arrays of arrays, but I would
> > definitely create a verb named shape to ease with typing and even
> > processing/reshaping such that
> >
> > ((3 ,:1) $ 3 2 2 4) $ i.16
> >
> > would be equivalent to
> >
> > (3 2 2 ,: 4) shape i.16
> >
> > ie. in table shapes, trailing 0s (fills) are equivalent to axes not being
> > present. I don't think this loses very much in expressibility, though I
> > understand that it creates an extra computation step to properly understand
> > a "shape item" with trailing 0s.
> >
> >
> > On Thursday, March 3, 2022, 11:01:57 a.m. EST, Henry Rich
> > <henryhr...@gmail.com> wrote:
> >
> >
> >
> >
> >
> > I like this, except for the trailing-0 bit. I would say having data with
> > trailing 0 shape is 'common' rather than 'ultra-rare'.
> >
> > Henry Rich
> >
> > On 3/3/2022 10:15 AM, 'Pascal Jasmin' via Programming wrote:
> > > I don't have a good grasp of order, but a different practical
> > > interpretation of non-list shapes:
> > >
> > > 1. You mentioned distinguishing atoms from lists of 1 element. J already
> > > manages this well, and making an atomic shape result would not be of any
> > > help, IMO.
> > >
> > > 2. A table shape could be a code for ragged arrays, or arrays of arrays.
> > > The following shape:
> > >
> > > 3 2 2
> > > 4 0 0
> > >
> > > Would be 2 arrays, the first is shape 3 2 2, and the 2nd is shape 4.
> > > Trailing 0s would be fills. This precludes the ultra rare need of needing
> > > an "embedded" shape 4 0 or shape 4 0 0 in your data.
> > >
> > > A list of 3 strings/lists with individual lengths of 2 3 4 would have the
> > > shape:
> > >
> > > 2
> > > 3
> > > 4
> > >
> > > If there is an extention of the meaning of shape, I'd prefer it be in
> > > this direction rather than ultra high dimensional math that hurts my head.
> > >
> > >
> > > On Thursday, March 3, 2022, 05:13:27 a.m. EST, Elijah Stone
> > > <elro...@elronnd.net> wrote:
> > >
> > >
> > >
> > >
> > >
> > > A little thought exercise I embarked upon.
> > > Probably not very clearly expressed, and certainly of no practical use,
> > > but I think there are some interesting ideas:
> > >
> > > Why must the result of $y always be a vector? What would happen if we let
> > > it be a higher-ranked array? What would that _mean_?
> > >
> > > Well, say that 2 3 4-:$y. That means that y is a mapping from arrays x
> > > that have the same shape as 2 3 4 (and whose elements are natural and
> > > bounded by 2 3 4). In other words, if (x -:&$ 2 3 4) *. -. 0 e. x<2 3 4
> > > then (<x){y is well-defined as an atom. (I am intentially ignoring the
> > > possibility of sentences such as ((<1 1){y) ((<,1){y) ((<1){y) and so
> > > on; consider these sentences ill-formed for the time being.)
> > > Generalising: if (x -:&$ $y) *. -. 0 e. x<$y then (<x){y is well-defined
> > > as an atom.
> > >
> > > But this sentence makes no restrictions whatsoever on the rank of x! So
> > > it would seem that we've discovered a way to make a y whose shape is an
> > > array of any rank. But we must be careful. Really, all we've constructed
> > > is a strange entity y--call it an array-as-we-do-not-know-it--and, on
> > > arrays-as-we-do-not-know-them, we have defined that: $y is well-defined,
> > > and evaluates to an array-as-we-know-it. x{y is well-defined for some
> > > subset of arrays-as-we-know-them x, and evaluates to an
> > > atom-as-we-know-it.
> > >
> > > I will call this strange creature a 4th-order array. And all
> > > arrays-as-we-know-them are 3rd-order (this includes scalars, vectors, and
> > > higher-rank arrays). A 4th-order array has a shape which is a 3rd-order
> > > array, and maps from 3rd-order arrays to atoms. This suggests the
> > > existence of lower-order arrays, and higher-order ones.
> > >
> > > Following the same logic, then, a 3rd-order array would map 2nd-order
> > > arrays to atoms, and its shape would be a 2nd-order array. But--for
> > > 3rd-order y, the x in (<x){y for 3rd-order y is currently also a 3rd-order
> > > array, and $y is currently also 3rd-order.
> > >
> > > Or is it, really? We've already established that $y must always be a
> > > vector for ordinary, familiar, 3rd-order y; and that x must be too for
> > > (<x){y. (A vector is a 3rd-order array y which satisfies (,1)-:$$y.) I
> > > propose that there is another strange creature, which is a 2nd-order
> > > array, or _list_. This is no less strange than 4th-order array, and j
> > > does not have them. A list is very similar to a vector, though, in that
> > > it is an ordered sequence of atoms. And there is an obvious 1:1 mapping
> > > between vectors and lists. Therefore, when you write (<x){y, and x is a
> > > vector, it is first _converted_ to the corresponding list, and that list
> > > is used to index y. Similarly, when you write $y, and y is a third-order
> > > array, its shape is actually a list; that list is _converted_ to the
> > > corresponding vector before being returned.
> > >
> > > Then, fairly straightforwardly, we can define the 1st-order array, or
> > > _atom_, another strange creature. Atoms are used to index lists, and the
> > > shape of a list is an atom. And every atom (1st-order array) also has a
> > > counterpart in familiar 3rd-order land, as a _scalar_. (A scalar is a
> > > 3rd-order array y which satisfies ((,0)$0)-:$y.)
> > >
> > > The obvious difference between a vector and a list is that, if we want
> > > element 5 of y, and y is a list, we write (<5){y; but if y is a vector, we
> > > must write (<,5){y. But to make the distinction that way misses the
> > > point, I think. A 2nd-order array is a particular sort of array which is
> > > restricted in that it can only contain unidirectional sequences of atoms.
> > > A vector is of a different sort of array, which is less restricted, but it
> > > happens to belong to a particular subset which maps to lists. (The same
> > > relationship holds for scalars and atoms.)
> > >
> > > Now, imagine a hypothetical version of j which could represent arrays of
> > > orders other than 3. It is difficult to talk about such a j, because--for
> > > instance--there might be an atom value 3 and a scalar value 3, and these
> > > would be distinct; if I simply write '3', it is ambiguous which is
> > > referred to. So I will err on the side of verbosity rather than
> > > ambiguity.
> > >
> > > One thing immediately becomes clear: the result of $y in such a j should
> > > be the _actual_ shape of y; that is, it should have order one lower than
> > > y. What, then is the shape of an atom? It might be an error, but I
> > > should rather make it the _unit_. The unit is the singular value which is
> > > an order-0 array. The unit is also its own shape. Then, we can also say
> > > that an order n array is one which requires n applications of $ to reach a
> > > fixed point. Personally, I think a fixed point of a unit is nicer than a
> > > fixed point of ,1 (which is what we currently have, with all results
> > > squashed into 3rd-order arrays), but this is a point of style.
> > >
> > > Another nice consistency that results is that the idiom $$y evaluates to
> > > an atom when y is a 3rd-order array. That is, it gives you the rank of y,
> > > rather than a 1-element vector containing the rank.
> > >
> > > Another observation: this provides a way to relate j to k; the difference
> > > between them is one of degree, not kind. In k, everything is a list or a
> > > scalar; that is, all values have order 1 or order 2. In j, all values
> > > have order 3. (There is another difference, which is that k has nests
> > > where j has boxes, but this is orthogonal.)
> > >
> > > We can think of 3rd-order arrays as functions; for instance, an array of
> > > rank 4 is like a function of 4 parameters. If we index such an array
> > > using a list of length 4, that is like applying the corresponding
> > > function, where each element of the list is associated with the
> > > corresponding argument. By convention, these functions are curried.
> > > Even if y has rank 4, it is legal in j to write (<1 2 0){y, even though 1
> > > 2 0 has length 3. (Previously, I said I was considering this illegal;
> > > now, I am defining it again.) This works by currying y; producing an
> > > array which has its first 3 arguments fixed (to 1, 2, and 0,
> > > respectively), requiring only 1 more; because it only requires 1 more
> > > argument, we say it has rank 1. This works well enough for 3rd-order
> > > arrays, but what about higher-order ones? When is something a valid
> > > 'partial index', and what are the 'first arguments'?
> > >
> > > A better definition comes from , . If p is a valid index into y (as
> > > defined in the 3rd paragraph), then (<p){y applies the indexing procedure
> > > as usual. Otherwise, it is the array for which (<q){(<p){y is equivalent
> > > to (<p,q){y. (This only holds if there is q for which, given p and y, p,q
> > > is a valid index into y; otherwise it is an index error.) This is
> > > adequate to describe existing indexing behaviour for 3rd-order arrays .
> > > And it works without a hitch for 4th-order arrays. For higher-order
> > > arrays, it is simply necessary to extend , to work on higher-order arrays;
> > > given such an extension, that, too will work without a hitch (and probably
> > > (?) give sensible results).
> > >
> > > I think the foregoing definition goes most of the way towards explaining
> > > why we may index arrays using scalars; it is more than a convenience
> > > feature, and the scalar is not simply converted to a vector. (Going the
> > > rest of the way would require a complete and general model of , .)
> > >
> > > Now, some questions:
> > >
> > > 1. It there any practical value to this scheme? In some respects, an
> > > array with a non-list shape just seems like an array with a list shape,
> > > and bells on. But I am a low-dimensional creature, and perhaps if I had
> > > more use for very high-dimensional arrays, I would find it useful to
> > > structure my axes (and then structure the structure of my axes, and so on,
> > > leading to arrays of ever higher orders).
> > >
> > > The simplest 4th-order array which is 'interesting' has shape 2 2$2; that
> > > is, it corresponds roughly to a 4-dimensional array; that is rare enough.
> > > The simplest 5th-order array which is 'interesting' has shape (2 2$2)$2,
> > > that is 16-dimensional-ish. I generally have little use for arrays of
> > > such high dimension (4 is reasonable, 16 is quite unlikely), and have not
> > > found a linear organization stifling or problematic.
> > >
> > > The jump from the unit to the atom seems most significant, as it allows
> > > you to communicate any information at all; the unit carries no
> > > information. Next in significance is the jump from atoms to lists, which
> > > permit the communication of inherently _structured_ information. This is
> > > good enough for many applications (and k gets along just fine); moving to
> > > 3rd-order arrays simply increases the degree of structure available, which
> > > can be useful in some (ok, many) situations. So it would not be
> > > surprising if the added benefit of moving to higher orders continued to
> > > peter out, such that 3 hit a good sweet spot.
> > >
> > > 2. Is there any theoretical value to this scheme, or anything interesting
> > > which can be described only using it? I used it to explain a number of
> > > small inconsistencies, but that is marginal at best.
> > >
> > > Is there any existing mathematical precedent? I know that what j calls
> > > arrays the literature calls tensors. So I googled for 'higher-order
> > > tensors', but only found information on what we would call higher-_rank_
> > > arrays, not what I distinguished here as higher-_order_ arrays. I also
> > > searched for non-cartesian coordinate systems, but could not find anything
> > > where the components of the coordinates were not sequential and flat.
> > >
> > > 3. Are the different orders of arrays qualitatively different? E.G. a
> > > 3rd-order array can approximate both a 1st-order array (using a scalar)
> > > and a 2nd-order array (using a vector); but a 2nd-order array can not
> > > approximate a 1st-order array. (This is why, to my understanding, a
> > > scalar is an array in j, but it is not a list in k.) (On the other hand,
> > > 3rd-order arrays can not represent the unit.) I think all higher-order
> > > arrays can approximate any 3rd-order array; but do they gain any new
> > > powers?
> > >
> > > 4. I called arrays of order 0, 1, and 2 units, atoms, and lists,
> > > respectively. What should arrays of order 3 be called? Perhaps 'spaces'?
> > > Is there a sensible name for 4th- or 5th-order arrays?
> > >
> > > Is the name 'shape' still sensible if extended to different orders?
> > >
> > > 5. How would the j primitives work if extended to work on arrays of order
> > > other than 3? What are the rules of conformability? Can you add an atom
> > > to a scalar? A list to a matrix? A list to a vector? When can you
> > > catenate two 5th-order arrays? How does rank work on 4th-order arrays?
> > >
> > > 6. Is there a good intuition for higher-order arrays? I can more or less
> > > imagine what 4th-dimensional space is like (though I can't visualize it),
> > > but I have no idea what a 'space' of shape 2 2$y is like, even though it
> > > has the same number of 'degrees of freedom'.
> > >
> > > Is there a well-defined notion of distance defined in higher-order
> > > 'space'? If so, is it still scalar?
> > >
> > >
> > > Thoughts?
> > >
> > > -E
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> >
> > --
> > This email has been checked for viruses by AVG.
> > https://www.avg.com
>
> >
> >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
