[
https://issues.apache.org/jira/browse/IMPALA-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647859#comment-17647859
]
Quanlong Huang commented on IMPALA-10792:
-----------------------------------------
I can still reproduce this using the master branch (compiled with CDP hive)
running on apache-hive-4.0.0-alpha-2.
The accessType that Impala gets is 0, which is treated as NONE:
{code:java}
public static String getTableAccessType(Table msTbl) {
Preconditions.checkNotNull(msTbl);
switch (msTbl.getAccessType()) {
case ACCESSTYPE_READONLY:
return "READONLY";
case ACCESSTYPE_WRITEONLY:
return "WRITEONLY";
case ACCESSTYPE_READWRITE:
return "READWRITE";
case ACCESSTYPE_NONE:
default: // <-- 0 goes here
return "NONE";
}
}
{code}
[https://github.com/apache/impala/blob/4.2.0/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java#L338-L351]
Note that 0 is not a valid value of accessType. The valid values are [1, 2, 4,
8]:
{code:java}
const byte ACCESSTYPE_NONE = 1;
const byte ACCESSTYPE_READONLY = 2;
const byte ACCESSTYPE_WRITEONLY = 4;
const byte ACCESSTYPE_READWRITE = 8;
{code}
[https://github.com/apache/hive/blob/36f5d91acb0fac00a5d46049bd45b744fe9aaab6/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift#L32-L35]
Impala gets an invalid accessType value due to thrift definition
incompatibility. In hive_metastore.thrift of apache-hive-4.0.0-alpha-2,
'accessType' is the 22-th field of 'Table':
{code:java}
struct Table {
...
22: optional byte accessType,
{code}
[https://github.com/apache/hive/blob/rel/release-4.0.0-alpha-2/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift#L628]
However, in CDP hive, it's the 23-th field of 'Table':
{code:java}
struct Table {
...
23: optional byte accessType,{code}
${IMPALA_HOME}/toolchain/cdp_components-34919320/hive-3.1.3000.7.2.16.0-272/standalone-metastore/src/main/thrift/hive_metastore.thrift
Thus, if compiled with CDP hive (i.e. USE_APACHE_HIVE=false), Impala can't get
the correct accessType from Apache Hive4. This is an incompatibility issue
between CDP Hive client and Apache Hive Metastore.
> The query failed because the access type is: NONE
> -------------------------------------------------
>
> Key: IMPALA-10792
> URL: https://issues.apache.org/jira/browse/IMPALA-10792
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.0.0
> Environment: hadoop3.0.0-cdh6.3.1\apache.hive4.0
> Reporter: zhi tang
> Priority: Major
>
> Details of the exception:
> I0712 11:29:18.289659 365471 impala-server.cc:1267] Found local timezone
> "Asia/Shanghai".
> I0712 11:29:18.292014 365471 impala-server.cc:1324]
> 8d4741015f8278d5:9c74184200000000] Registered query
> query_id=8d4741015f8278d5:9c74184200000000
> session_id=b046fccaef8bb439:08128e50acebc8a4
> I0712 11:29:18.292245 365471 Frontend.java:1618]
> 8d4741015f8278d5:9c74184200000000] Analyzing query: select * from
> tangzhi_test db: default
> I0712 11:29:18.292516 365471 FeSupport.java:315]
> 8d4741015f8278d5:9c74184200000000] Requesting prioritized load of table(s):
> default.tangzhi_test
> I0712 11:29:22.751842 365367 ImpaladCatalog.java:225] Adding:
> TABLE:default.tangzhi_test version: 12 size: 1404
> I0712 11:29:22.751950 365367 ImpaladCatalog.java:225] Adding:
> CATALOG_SERVICE_ID version: 12 size: 60
> I0712 11:29:22.752007 365367 ImpaladCatalog.java:250] Adding 1 partition(s):
> HDFS_PARTITION:default.tangzhi_test:, version=12, size=203
> I0712 11:29:22.752832 365367 ImpaladCatalog.java:534] Applied incremental
> table updates on 0 existing partitions of table default.tangzhi_test: added 1
> new partitions, deleted 0 stale partitions.
> I0712 11:29:22.752916 365367 impala-server.cc:2022] Catalog topic update
> applied with version: 12 new min catalog object version: 2
> I0712 11:29:22.753744 365471 BaseAuthorizationChecker.java:112]
> 8d4741015f8278d5:9c74184200000000] Authorization check took 4461 ms
> I0712 11:29:22.753932 365471 jni-util.cc:286]
> 8d4741015f8278d5:9c74184200000000]
> org.apache.impala.common.AnalysisException: Operations not supported. Table
> default.tangzhi_test access type is: NONE
> at
> org.apache.impala.analysis.Analyzer.ensureTableSupported(Analyzer.java:343)
> at
> org.apache.impala.analysis.Analyzer.checkTableCapability(Analyzer.java:299)
> at org.apache.impala.analysis.BaseTableRef.analyze(BaseTableRef.java:67)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:83)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:278)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:265)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:258)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:481)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:445)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1659)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1625)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1595)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:162)
> I0712 11:29:22.753962 365471 status.cc:129]
> 8d4741015f8278d5:9c74184200000000] AnalysisException: Operations not
> supported. Table default.tangzhi_test access type is: NONE
> @ 0xd6c673
> @ 0x159e889
> @ 0x13353fb
> @ 0x19e6feb
> @ 0x13a4651
> @ 0x13b6772
> @ 0x14b432d
> @ 0x14b5538
> @ 0x145d4da
> @ 0x13c14a7
> @ 0xcc9a95
> @ 0x1128116
> @ 0x111bff8
> @ 0x1664981
> @ 0x1665dfa
> @ 0x1ea2161
> @ 0x7fd34a849ea4
> @ 0x7fd34735f9fc
> I0712 11:29:22.754181 365471 impala-server.cc:1408]
> 8d4741015f8278d5:9c74184200000000] UnregisterQuery():
> query_id=8d4741015f8278d5:9c74184200000000
> I0712 11:29:22.754487 365355 impala-server.cc:1440] Query successfully
> unregistered: query_id=8d4741015f8278d5:9c74184200000000
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
