[
https://issues.apache.org/jira/browse/IMPALA-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956558#comment-16956558
]
ASF subversion and git services commented on IMPALA-8997:
---------------------------------------------------------
Commit 96143cfcf11f2a455a0f2511da67dc2e6b69d7f3 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=96143cf ]
IMPALA-8997: auto fallback to mt_dop=0
Add a temporary --mt_dop_auto_fallback to allow a graceful transition to
using mt_dop for workloads. When this flag is set, DML queries and joins
that would otherwise fail with an error when run with mt_dop > 0 fall
back to running with mt_dop = 0. This means that a user can set mt_dop
for their queries and it will only take effect when supported.
The behaviour generally does not change when this flag is not set,
with a couple of exceptions:
* I made mt_dop automatic for compute stats on all file formats
* mt_dop is allowed for single node plans with inserts. The
quirky validatePlan() logic previously disallowed this but
allowed joins in single node plans.
The checks added by this patch can be removed safely once mt_dop is
supported by default for all queries.
This includes some cleanup:
* isDmlStmt() was stale and incorrectly implemented.
* Various TreeNode methods did not return instances of subclasses of
the requested class, which was strange. This fix is required to
make 'contains(JoinNode.class)' work correctly. I checked the
callsites of the fixed functions and none of them would be affected
by this change because they specified a terminal class without
any subclasses.
I didn't actually use this fix in the end (I had to write a custom
tree traversal in hasUnsupportedMtDopJoin()), but figured I would
leave the improvement in here.
Testing:
Add some basic functional tests ensuring that the fallback takes
effect.
Run basic join and insert tests with this flag enabled.
Change-Id: Ie0d73d8744059874293697c8e104891a10dba04d
Reviewed-on: http://gerrit.cloudera.org:8080/14344
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Allow graceful fallback to mt_dop=0
> -----------------------------------
>
> Key: IMPALA-8997
> URL: https://issues.apache.org/jira/browse/IMPALA-8997
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> We should provide a way to specify mt_dop and have Impala automatically fall
> back to non-mt_dop for queries where it isn't supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
