[DISCUSS][DataFusion] Table time travel support

2023-08-17 Thread Marko Grujic
Hi all! I'm wondering what people think of a possibility to extend DataFusion so as to accommodate time-travel querying? This would work well with the new table formats, particularly Iceberg and Delta Lake, where table versioning is at the core of the protocol. You can see some details in the

Re: Sort a Table In C++?

2023-08-17 Thread Li Jin
Aha thanks both! I ended up using the Acero example to do it. On Thu, Aug 17, 2023 at 5:45 PM Antoine Pitrou wrote: > > Or you can simply call the "sort_indices" compute function: > https://arrow.apache.org/docs/cpp/compute.html#sorts-and-partitions > > > Le 17/08/2023 à 23:20, Ian Cook a écrit

Re: Sort a Table In C++?

2023-08-17 Thread Antoine Pitrou
Or you can simply call the "sort_indices" compute function: https://arrow.apache.org/docs/cpp/compute.html#sorts-and-partitions Le 17/08/2023 à 23:20, Ian Cook a écrit : Li, Here's a standalone C++ example that constructs a Table and executes an Acero ExecPlan to sort it:

Re: Sort a Table In C++?

2023-08-17 Thread Ian Cook
Li, Here's a standalone C++ example that constructs a Table and executes an Acero ExecPlan to sort it: https://gist.github.com/ianmcook/2aa9aa82e61c3ea4405450b93cf80fbc Ian On Thu, Aug 17, 2023 at 4:50 PM Li Jin wrote: > > Hi, > > I am writing some C++ test and found myself in need for an c++

Sort a Table In C++?

2023-08-17 Thread Li Jin
Hi, I am writing some C++ test and found myself in need for an c++ function to sort an arrow Table. Before I go around implementing one myself, I wonder if there is already a function that does that? (I searched the doc but didn’t find one). There is function in Acero can do it but I didn’t find

Re: [DISCUSS][Format] Starting the draft implementation of the ArrayView array format

2023-08-17 Thread Pedro Eugenio Rocha Pedreira
Hi all, Getting back to this thread as I realize there were a few unanswered questions. Adding a bit more context on the rationale and usage of ArrayViews in Velox, and the importance to standardize it: re: Why do we need it? We use ArrayViews for two main reasons. First, for efficient

[DISCUSS] Proposal to add VariableShapeTensor Canonical Extension Type

2023-08-17 Thread Rok Mihevc
Hey all! Besides the recently added FixedShapeTensor [1] canonical extension type there appears to be a need for an already proposed VariableShapeTensor [2]. VariableShapeTensor would store tensors of variable shapes but uniform number of dimensions, dimension names and dimension permutations.

Mailing list threading improvements

2023-08-17 Thread Christofer Dutz
TL;DR: We’re updating how auto-generated email from Github will be threaded on your mailing lists. If you want to keep the old defaults, details are below. We’re pleased to let you know that we’re tweaking the way that auto- generated email from Github will appear on your mailing lists. This will