Yes that lines up with my thoughts on how this could work. As has already
been suggested this is essentially the structure used by
'general/misc/inverted' which then already models how some of the J
primitives would operate on it.
The bit that is missing, as you point out is the toolset to elegantly link
the boxed list of columns to the list of column names. Or alternatively
link a name with each boxed column (sounds a bit like a dictionary?)
If in some future universe Dataframes became a new datatype in J then, in
my ideal world, rather than needing (tfrom=: <@[ {&.> ]) to retrieve items
at the provided indicies, we would just be able to use ({) etc. Not sure
how feasible that is!
On Tue, Feb 1, 2022 at 1:48 PM Raul Miller <rauldmil...@gmail.com> wrote:
> I think a dataframe, in J would basically be a boxed list of columns,
> and some associated list of column names.
>
> It's the toolset we would build up to work with such a thing that
> would make it useful.
>
> (And maybe Jd is such a toolset? I do not know -- I have not motivated
> myself to try Jd yet...)
>
> --
> Raul
>
> On Mon, Jan 31, 2022 at 7:44 PM Vanessa McHale <vamch...@gmail.com> wrote:
> >
> > I’ve tried Jd, it’s equivalent to pandas I think (and about as
> performant) though it’s persistent (being a database).
> >
> > q/k is faster, I think because it’s ordered by default - maybe something
> like ordered dataframes could be implemented in J?
> >
> > Cheers,
> > Vanessa McHale
> >
> > > On Jan 30, 2022, at 8:21 PM, Ric Sherlock <tikk...@gmail.com> wrote:
> > >
> > > Yes, I've been thinking that a Dataframes equivalent in J would be
> useful.
> > > Most things are already possible with J's arrays, but conceptually
> > > DataFrames are well understood by many now, and they make it easy to
> work
> > > with datasets as named fields.
> > > I've spent a reasonable amount of time working with Pandas, but have
> > > recently been using Polars (Rust backend with Python bindings) which
> really
> > > shines for larger datasets. Performance (especially read/write) is
> awesome,
> > > and the LazyFrames which optimise your query/analysis plan make a big
> > > difference too.
> > > I haven't taken enough time to explore it, but maybe Jd is the starting
> > > point in this space for J?
> > >
>
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
