[
https://issues.apache.org/jira/browse/DRILL-6498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6498:
---------------------------------
Reviewer: Parth Chandra
> Support for EMIT outcome in ExternalSortBatch
> ---------------------------------------------
>
> Key: DRILL-6498
> URL: https://issues.apache.org/jira/browse/DRILL-6498
> Project: Apache Drill
> Issue Type: Task
> Components: Execution - Relational Operators
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
> Priority: Major
> Fix For: 1.14.0
>
>
> With Lateral and Unnest if Sort is present in the sub-query, then it needs to
> handle the EMIT outcome correctly. This means when a EMIT is received then
> perform the Sort operation on the records buffered so far and produce output
> with it. After EMIT Sort should refresh it's state and again work on next
> batches of incoming record unless an EMIT is seen again.
> For first cut Sort will not support spilling in the subquery between Lateral
> and Unnest since spilling is very unlikely. The worst case that can happen is
> that Lateral will get a batch with only 1 row of data because of repeated
> type column data size being too big. In that case Unnest will produce 1
> output batch only and Sort or other blocking operators anyways needs enough
> memory to at least hold 1 incoming batch. So in ideal cases spilling should
> not happen. But if there is a operator between Sort and Unnest which
> increases the data size then Sort might be in a situation to spill but thats
> not a common case for now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
