[
https://issues.apache.org/jira/browse/DRILL-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5231:
-------------------------------
Description:
If one takes a careful look at a query profile, one finds that the JSON
representation of each operator contains entries for operations that do not
appear in the text plan. The following is an example. First, an excerpt of the
text representation of the plan:
{code}
...
02-01 UnorderedMuxExchange
03-01 Project
03-02 HashAgg
03-03 Project
03-04 Project
03-05 HashJoin
03-07 Project
03-08 Project
03-09 Project
03-10 HashJoin
03-12 Project
03-14 Scan
03-11 Project
03-13 BroadcastExchange
...
04-04 Scan
03-06 BroadcastExchange
05-01 Project
...
{code}
Notice that there is no 03-00 in the above list. But, this operator *does*
exist in the profile JSON.
Next, the output of a small program to match up operator profile elements in
JSON with the text plan elements. (The plan presents operators in ascending
order; the profile presents them in descending order):
{code}
3-0-14 - Def: Scan / Prof: PARQUET_ROW_GROUP_SCAN
3-0-12 - Def: Project / Prof: PROJECT
3-0-13 - Def: BroadcastExchange / Prof: UNORDERED_RECEIVER
3-0-11 - Def: Project / Prof: PROJECT
3-0-10 - Def: HashJoin / Prof: HASH_JOIN
3-0-9 - Def: Project / Prof: PROJECT
3-0-8 - Def: Project / Prof: PROJECT
3-0-7 - Def: Project / Prof: PROJECT
3-0-6 - Def: BroadcastExchange / Prof: UNORDERED_RECEIVER
3-0-5 - Def: HashJoin / Prof: HASH_JOIN
3-0-4 - Def: Project / Prof: PROJECT
3-0-3 - Def: Project / Prof: PROJECT
3-0-2 - Def: HashAgg / Prof: HASH_AGGREGATE
3-0-1 - Def: Project / Prof: PROJECT
3-0-0 - Def: Sender / Prof: SINGLE_SENDER
{code}
In the above, we see that 03-00 does exist and is a Single Sender. (FWIW, the
notation is major, minor, operator, name of operator from the Plan, name of the
operator from the operator profile resolved via the operator type.)
The query plan emitted into the profile should include these elements so it is
clear what is expected to happen at the top of each major fragment.
was:
If one takes a careful look at a query profile, one finds that the JSON
representation of each operator contains entries for operations that do not
appear in the text plan. The following is an example. First, an excerpt of the
text representation of the plan:
{code}
...
02-01 UnorderedMuxExchange
03-01 Project
03-02 HashAgg
03-03 Project
03-04 Project
03-05 HashJoin
03-07 Project
03-08 Project
03-09 Project
03-10 HashJoin
03-12 Project
03-14 Scan
03-11 Project
03-13 BroadcastExchange
{code}
Notice that there is no 03-00 or 03-06 in the above list. But, these operators
*do* exist in the profile JSON.
Next, the output of a small program to match up operator profile elements in
JSON with the text plan elements. (The plan presents operators in ascending
order; the profile presents them in descending order):
{code}
3-0-14 - Def: Scan / Prof: PARQUET_ROW_GROUP_SCAN
3-0-12 - Def: Project / Prof: PROJECT
3-0-13 - Def: Unknown / Prof: UNORDERED_RECEIVER
3-0-11 - Def: Unknown / Prof: PROJECT
3-0-10 - Def: HashJoin / Prof: HASH_JOIN
3-0-9 - Def: Project / Prof: PROJECT
3-0-8 - Def: Project / Prof: PROJECT
3-0-7 - Def: Project / Prof: PROJECT
3-0-6 - Def: Unknown / Prof: UNORDERED_RECEIVER
3-0-5 - Def: HashJoin / Prof: HASH_JOIN
3-0-4 - Def: Project / Prof: PROJECT
3-0-3 - Def: Project / Prof: PROJECT
3-0-2 - Def: HashAgg / Prof: HASH_AGGREGATE
3-0-1 - Def: Project / Prof: PROJECT
3-0-0 - Def: Sender / Prof: SINGLE_SENDER
{code}
In the above, we see that 03-06 does exist, it is an Unordered Receiver.
Similarly, 03-00 exists and is a Single Sender. (FWIW, the notation is major,
minor, operator, name of operator from the Plan, name of the operator from the
operator profile resolved via the operator type.)
The query plan emitted into the profile should include these elements so it is
clear what is expected to happen at the top and bottom of each major fragment.
> Query profiles omit some operations from text plan
> --------------------------------------------------
>
> Key: DRILL-5231
> URL: https://issues.apache.org/jira/browse/DRILL-5231
> Project: Apache Drill
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.9.0
> Reporter: Paul Rogers
> Priority: Minor
>
> If one takes a careful look at a query profile, one finds that the JSON
> representation of each operator contains entries for operations that do not
> appear in the text plan. The following is an example. First, an excerpt of
> the text representation of the plan:
> {code}
> ...
> 02-01 UnorderedMuxExchange
> 03-01 Project
> 03-02 HashAgg
> 03-03 Project
> 03-04 Project
> 03-05 HashJoin
> 03-07 Project
> 03-08 Project
> 03-09 Project
> 03-10 HashJoin
> 03-12 Project
> 03-14 Scan
> 03-11 Project
> 03-13 BroadcastExchange
> ...
> 04-04 Scan
> 03-06 BroadcastExchange
> 05-01 Project
> ...
> {code}
> Notice that there is no 03-00 in the above list. But, this operator *does*
> exist in the profile JSON.
> Next, the output of a small program to match up operator profile elements in
> JSON with the text plan elements. (The plan presents operators in ascending
> order; the profile presents them in descending order):
> {code}
> 3-0-14 - Def: Scan / Prof: PARQUET_ROW_GROUP_SCAN
> 3-0-12 - Def: Project / Prof: PROJECT
> 3-0-13 - Def: BroadcastExchange / Prof: UNORDERED_RECEIVER
> 3-0-11 - Def: Project / Prof: PROJECT
> 3-0-10 - Def: HashJoin / Prof: HASH_JOIN
> 3-0-9 - Def: Project / Prof: PROJECT
> 3-0-8 - Def: Project / Prof: PROJECT
> 3-0-7 - Def: Project / Prof: PROJECT
> 3-0-6 - Def: BroadcastExchange / Prof: UNORDERED_RECEIVER
> 3-0-5 - Def: HashJoin / Prof: HASH_JOIN
> 3-0-4 - Def: Project / Prof: PROJECT
> 3-0-3 - Def: Project / Prof: PROJECT
> 3-0-2 - Def: HashAgg / Prof: HASH_AGGREGATE
> 3-0-1 - Def: Project / Prof: PROJECT
> 3-0-0 - Def: Sender / Prof: SINGLE_SENDER
> {code}
> In the above, we see that 03-00 does exist and is a Single Sender. (FWIW, the
> notation is major, minor, operator, name of operator from the Plan, name of
> the operator from the operator profile resolved via the operator type.)
> The query plan emitted into the profile should include these elements so it
> is clear what is expected to happen at the top of each major fragment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
