[
https://issues.apache.org/jira/browse/DRILL-229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805781#comment-13805781
]
Ben Becker commented on DRILL-229:
----------------------------------
Created reviewboard
> Build a Merging Recevier
> ------------------------
>
> Key: DRILL-229
> URL: https://issues.apache.org/jira/browse/DRILL-229
> Project: Apache Drill
> Issue Type: New Feature
> Reporter: Jacques Nadeau
> Assignee: Ben Becker
> Attachments: DRILL-229.patch
>
>
> Build a merging receiver operator which combines a number of incoming batches
> into a single outgoing batch.
> h3. Overview
> Incoming batches are individually presorted before reaching this operator, so
> a priority queue is built with each value to determine which batch contains
> the next record. When the next value is removed from the priority queue, the
> underlying record is copied from the underlying ValueVectors to the outgoing
> VectorContainer.
> The comparator for the priority queue is generated based on the supplied
> LogicalExpression (e.g. a single ValueVectorReadExpression).
> h3. Example
> The following example illustrates a distributed count operation, where each
> remote fragment counts a subset of the data and the root fragment produces a
> sum of each count aggregate.
> h4. Data Flow
> {noformat}
> RecordReader
> |
> +-> Sort
> |
> +-> StreamingAggregate(COUNT)
> |
> +-> MergingPartitionExchange
> |
> +-> StreamingAggregate(SUM)
> |
> +-> UnionExchange
> |
> +-> Screen
> {noformat}
> h4. Control Flow
> {noformat}
> Root Fragment
> -------------
> Screen
> |
> +->UnionExchange
> | | |
> | | +->AggSum
> | | |
> | | +->MergingReceiver
> | |
> | +--->AggSum
> | |
> | +->MergingReceiver
> |
> +----->AggSum
> |
> +->MergingReceiver
> ...
> Remote Fragment
> ---------------
> PartitionSender
> |
> +->AggCount
> |
> +->Sort
> |
> +->Reader
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
