[
https://issues.apache.org/jira/browse/DRILL-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16227349#comment-16227349
]
Vitalii Diravka commented on DRILL-5822:
----------------------------------------
[~Paul.Rogers]
Here is the issue of unnecessary sorting of columns for query with the
following conditions: using wildcard in the query and ORDER BY clause, and when
this is planned into multiple fragments ("alter session set
`planner.slice_target`=1;").
The issue is connected to adding canonicalizing the schemas of input batches
for Merging Receiver in DRILL-847. But this approach is outdated since for now
in the process of loading batches in the RecordBatchLoader the new batch with
same columns (SchemaPaths) but other ordering of them is perceived with same
schema as the previous batch has:
[All fields from the last batch is a hashMap
structure|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L90]
and [when new batch appears the columns are just removed from the old one by
the
key|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L102].
So the schemaChange flag still equals to false. And then [the schema will
built|https://github.com/apache/drill/blob/fe79a633a3da8b4f6db50454fde64c30c73233bb/exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java#L138].
Here is only the issue that RecordBatchLoader permutes column order for the
above case. And it was described in the jira ticket created by you DRILL-5828
and can be fixed there.
So my changes fix the current issue but not fully cover the requirements from
your comment. Will It be reasonably if that changes will be done in context of
DRILL-5828?
> The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1
> doesn't preserve column order
> -----------------------------------------------------------------------------------------------------------
>
> Key: DRILL-5822
> URL: https://issues.apache.org/jira/browse/DRILL-5822
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.11.0
> Reporter: Prasad Nagaraj Subramanya
> Assignee: Vitalii Diravka
> Fix For: 1.12.0
>
>
> Columns ordering doesn't preserve for the star query with sorting when this
> is planned into multiple fragments.
> Repro steps:
> 1) {code}alter session set `planner.slice_target`=1;{code}
> 2) ORDER BY clause in the query.
> Scenarios:
> {code}
> 0: jdbc:drill:zk=local> alter session reset `planner.slice_target`;
> +-------+--------------------------------+
> | ok | summary |
> +-------+--------------------------------+
> | true | planner.slice_target updated. |
> +-------+--------------------------------+
> 1 row selected (0.082 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +--------------+----------+--------------+------------------------------------------------------+
> | n_nationkey | n_name | n_regionkey | n_comment
> |
> +--------------+----------+--------------+------------------------------------------------------+
> | 0 | ALGERIA | 0 | haggle. carefully final deposits
> detect slyly agai |
> +--------------+----------+--------------+------------------------------------------------------+
> 1 row selected (0.141 seconds)
> 0: jdbc:drill:zk=local> alter session set `planner.slice_target`=1;
> +-------+--------------------------------+
> | ok | summary |
> +-------+--------------------------------+
> | true | planner.slice_target updated. |
> +-------+--------------------------------+
> 1 row selected (0.091 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +------------------------------------------------------+----------+--------------+--------------+
> | n_comment | n_name |
> n_nationkey | n_regionkey |
> +------------------------------------------------------+----------+--------------+--------------+
> | haggle. carefully final deposits detect slyly agai | ALGERIA | 0
> | 0 |
> +------------------------------------------------------+----------+--------------+--------------+
> 1 row selected (0.201 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
