[
https://issues.apache.org/jira/browse/IMPALA-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451571#comment-17451571
]
ASF subversion and git services commented on IMPALA-4166:
---------------------------------------------------------
Commit 6f80a1f16015b628a31548bdbecd61a554fc0d85 in impala's branch
refs/heads/master from skyyws
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6f80a1f ]
IMPALA-11021: Fix bug when query contains illegal predicate hints
Currently Impala support predicate hint: ALWAYS_TRUE, we can use this
hint after where keyword. If we use illegal hints carelessly, query
will throw IllegalStateException which is not expected. Query should
return normal results with a warning instead of a exception. This is
due to the condition check in Analyzer.addWarning().
After create TExecRequest and initialize it, Impala will get warnings
from 'GlobalState.warnings', and 'GlobalState.warningsRetrieved' will
be set to 'true' then. But after this, Impala will substitute predicate
by clone(), and analyze new predicate in later phase. New predicate
analyze will add hint warning to 'GlobalState.warnings', but failed and
throw IllegalStateException due to 'globalState_.warningsRetrieved'
check failed which is expected as 'false'.
This check is added in IMPALA-4166, I removed original condition check
and added a new check to ensure that all warnings for new/substituted
predicates are already exists in 'globalState_.warnings'. And this will
also avoiding exception caused by illegal hints.
Testing:
- Added new fe tests in 'AnalyzeStmtsTest'
Change-Id: Id719bc4280c811456333eb4b4ec5bc9cb8bae128
Reviewed-on: http://gerrit.cloudera.org:8080/18040
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Introduce SORT BY clause in CREATE TABLE statement
> --------------------------------------------------
>
> Key: IMPALA-4166
> URL: https://issues.apache.org/jira/browse/IMPALA-4166
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Affects Versions: Impala 2.2, Impala 2.3.0, Impala 2.5.0, Impala 2.4.0,
> Impala 2.6.0, Impala 2.7.0
> Reporter: Alexander Behm
> Assignee: Lars Volker
> Priority: Major
> Labels: ramp-up, usability
> Fix For: Impala 2.9.0
>
>
> This issue is intended as a usability improvement for IMPALA-4163 where the
> SORT BY columns can be specified directly in the table definition like this:
> {code}
> CREATE TABLE t (day INT, hour INT)
> PARTITIONED BY (year INT, month INT)
> SORT BY (day, hour);
> {code}
> The above table creation has the effect that all inserts into the table have
> an implicit "sortby(day,hour)" plan hint applied. See IMPALA-4163 for details
> on the hint.
> Just like with the "sortby" hint the SORT BY clause can only contain
> non-partition columns for HDFS tables and non-primary key columns for Kudu
> tables.
> This has the following benefits:
> * Users will not have to remember to put the sortby hint in all insert
> statements.
> * The SORT BY columns are a physical design choice, so it makes sense to
> store them as part of the table metadata.
> * This is a convenience feature. It has the same effect as the sortby() hint
> for INSERT statements, but doesn't require the user to remember to include
> the hint for every INSERT statement.
> Challenges:
> * The Hive Metastore has no SORT BY concept, so we'll need to store the
> information in the generic TBLPROPERTIES map.
> * No other engines (Hive, Spark) will understand this table property. That
> means that data written by those engines will require an explicit sorting
> hint (as far as that's available).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
