Sorabh Hamirwasia created DRILL-6635:
----------------------------------------
Summary: PartitionLimit for Lateral/Unnest
Key: DRILL-6635
URL: https://issues.apache.org/jira/browse/DRILL-6635
Project: Apache Drill
Issue Type: Task
Components: Execution - Relational Operators
Affects Versions: 1.14.0
Reporter: Sorabh Hamirwasia
Assignee: Sorabh Hamirwasia
With batch processing changes in Lateral/Unnest the limit/TopN clause within
Lateral-Unnest subquery will not work as expected since it will impose
limit/TopN across RowId's. We need a new mechanism to apply these operators at
rowId boundary.
For now we are planning to add support for only limit and hence need to have a
new operator PartitionLimit which will get the partitionColumn on which the
limit should be imposed. This will currently only support queries between
lateral and unnest.
For TopN we can still achieve that using combination of Sort and Partition
Limit and later we can figure out how to address it directly within TopN or is
it needed at all. Since the number of rows across EMIT boundary on which SORT
will operate should not be big enough and mostly be done in memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
