ite/rel/logical/LogicalTableFunctionScan.html
> Can i model a mapPartitions T -> U as ^ ?
>
> On Thu, Mar 4, 2021 at 12:06 PM Rui Wang wrote:
>
> > I feel like the mapPartitions can be implemented as a SELECT + GROUP BY,
> > where GROUP BY is to partition the data, then per partition
Yes there is definitely some similarity to groupby
What is this used for:
https://calcite.apache.org/javadocAggregate/org/apache/calcite/rel/logical/LogicalTableFunctionScan.html
Can i model a mapPartitions T -> U as ^ ?
On Thu, Mar 4, 2021 at 12:06 PM Rui Wang wrote:
> I fee
I feel like the mapPartitions can be implemented as a SELECT + GROUP BY,
where GROUP BY is to partition the data, then per partition computation is
handled by the SELECT.
-Rui
On Thu, Mar 4, 2021 at 11:56 AM Debajyoti Roy wrote:
> Thanks again Julian.
>
> Since, mapPartitions
Thanks again Julian.
Since, mapPartitions is really a specialized map would it be best to model
it as a SELECT (similar to functions inside an expression) ?
Barring cases where h > h' and mapPartitions acts like a filter.
On Thu, Mar 4, 2021 at 11:41 AM Julian Hyde wrote:
> SQL has equiv
data). mapPartitions is in this category. Of course a physical implementation
of one of SQL’s logical operators might use mapPartitions.
Julian
> On Mar 4, 2021, at 10:44 AM, Debajyoti Roy wrote:
>
> Thanks for the responses, adding some more color below.
>
> Spark's API adopte
hard to model in terms of standard relational operators.
Let me take one example of mapPartitions.
mapPartitions( T -> U ):
w columns and h rows can turn into totally different w' != w columns and h'
!= h rows. Since processing happens per partition, this API is a great
choice for vectori
I searched for mapPartitions and flatMapGroupsWithState, and it looks as if you
are talking about Apache Spark operations. Can you give some examples of
typical queries that would use these operations?
It’s possible that these operations accomplish things that are not possible
1 at 10:12 PM Debajyoti Roy wrote:
> Hi All,
>
> For operators like filter, join, union, aggregate, window the
> logical RelNode choices are obvious within org.apache.calcite.rel.logical
> package.
>
> However, for more complex applications that require operations like
> map
Hi All,
For operators like filter, join, union, aggregate, window the
logical RelNode choices are obvious within org.apache.calcite.rel.logical
package.
However, for more complex applications that require operations like
mapPartitions, flatMapGroupsWithState, etc. what would be some choices
