[
https://issues.apache.org/jira/browse/IMPALA-10970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451572#comment-17451572
]
ASF subversion and git services commented on IMPALA-10970:
----------------------------------------------------------
Commit b0b6e11f53ce4a43423bdd14297cc6624d344760 in impala's branch
refs/heads/master from guojingfeng
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b0b6e11 ]
IMPALA-10970: Fix criterion for classifying coordinator only query
This patch fixes a bug in the criterion which decided whether a query
can be considered as a coordinator only query. It did not consider
the possibility of parallel plans and ended up mis-classifying some
queries as coordinator only queries.
This classification was used during scheduling when dedicated
coordinators and executor groups are used and allowed coordinator
queries to be scheduled only on the coordinator even in the absence
of healthy executor groups.
As a result of this bug, queries classified wrongly ended up with
error code: NO_REGISTERED_BACKENDS.
Testing:
- Add new mt_dop test case for functional_query and pass
- Ran and passed custom_cluster/test_coordinators, test_executor_groups
Change-Id: Icaaf1f1ba7a976122b4d37bd675e6d8181dc8700
Reviewed-on: http://gerrit.cloudera.org:8080/17937
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Bikramjeet Vig <[email protected]>
> Coordinator only query judgement logic should concern separated join build
> execution
> ------------------------------------------------------------------------------------
>
> Key: IMPALA-10970
> URL: https://issues.apache.org/jira/browse/IMPALA-10970
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.0.0
> Reporter: guojingfeng
> Assignee: guojingfeng
> Priority: Major
>
> IMPALA-8830 introduce the ability of executing trivail queries regardless the
> healthness of executor groups. Coordinatory only query judgment logic
> controls that whether a quer is trivail query or not. It take the query
> ExecRequest and check the plan only contains a single unpartitioned fragment.
> But join builder is scheduled at a separated fragment and colocated with join
> node after IMPALA-4224, in this case a query plan will contain two
> PlenExecInfo. I think *the coorinator only judgment logic should also check
> the jon build side* of the total execution plan.
> Here is a bad case with query option MT_DOP>0:
> {code:java}
> SELECT
> STRAIGHT_JOIN * // Add straight_join hint to disable optimization
> FROM
> ( SELECT '20210831' AS ts
> UNION ALL SELECT '20210901' AS ts
> UNION ALL SELECT '20210902' AS ts
> UNION ALL SELECT '20210903' AS ts ) t1
> CROSS JOIN tpch.lineitem t2
> LIMIT 100;
> {code}
> In debug build this query will fail and got core dump at hit the DECHECK at
> /be/src/scheduling/scheduler.cc#L162
> {code:java}
> DCHECK(executor_config.group.NumExecutors() > 0 || exec_at_coord);
> {code}
> In release build this query will fail with message "Scheduling for executor
> group: empty group
> (using coordinator only) with 0 executors"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
