[
https://issues.apache.org/jira/browse/IMPALA-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661358#comment-16661358
]
Tim Armstrong commented on IMPALA-6062:
---------------------------------------
The below commit added a hack to insert quotes to the error messages
commit 783de170c934be52109eccd88aac91e5a017e1ce
Author: Philip Zeyliger <[email protected]>
Date: Fri Mar 2 16:16:06 2018 -0800
IMPALA-4277: Support multiple versions of Hadoop ecosystem
Adds support for building against two sets of Hadoop ecosystem
components. The control variable is IMPALA_MINICLUSTER_PROFILE_OVERRIDE,
which can either be set to 2 (for Hadoop 2, Hive 1, and so on) or 3 (for
Hadoop 3, Hive 2, and so on).
We intend (in a trivial follow-on change soon) to make 3 the new default
and to explicitly deprecate 2, but this change only does not switch the
default yet. We support both to facilitate a smoother transition, but
support will be removed soon in the Impala 3.x line.
The switch is done at build time, following the pattern from IMPALA-5184
(build fe against both Hive 1 & 2 APIs). Switching back and forth
requires running 'cmake' again. Doing this at build-time avoids
complicating the Java code with classloader configuration.
There are relatively few incompatible APIs. This implementation
encapsulates that by extracting some Java code into
fe/src/compat-minicluminicluster-profile-{2,3}. (This follows the
pattern established by IMPALA-5184, but, to avoid a proliferation
of directories, I've moved the Hive files into the same tree.)
pattern from IMPALA-5184 (build fe against both Hive 1 & 2 APIs). I
consolidated the Hive changes into the same directory structure.
For Maven, I introduced Maven "profiles" to handle the two cases where
the dependencies (and exclusions) differ. These are driven by the
$IMPALA_MINICLUSTER_PROFILE environment variable.
For Sentry, exception class names changed. We work around this by adding
"isSentry...(Exception)" methods with two different implementations.
Sentry is also doing some odd shading, whereby some exceptions are
"sentry.org.apache.sentry..."; we handle both. Similarly, the mechanism
to create a SentryAuthProvider is slightly different. The easiest way to
see the differences is to run:
diff -u
fe/src/compat-minicluster-profile-{2,3}/java/org/apache/impala/util/SentryUtil.java
diff -u
fe/src/compat-minicluster-profile-{2,3}/java/org/apache/impala/authorization/SentryAuthProvider.java
....
Testing: Ran core & exhaustive tests with both profiles.
Cherry-picks: not for 2.x.
Change-Id: I7a2ab50331986c7394c2bbfd6c865232bca975f7
Reviewed-on: http://gerrit.cloudera.org:8080/9716
Reviewed-by: Alex Behm <[email protected]>
Tested-by: Impala Public Jenkins
> Frontend tests need to handle hadoop 3 error messages
> -----------------------------------------------------
>
> Key: IMPALA-6062
> URL: https://issues.apache.org/jira/browse/IMPALA-6062
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.10.0
> Reporter: Joe McDonnell
> Priority: Critical
> Fix For: Impala 3.0
>
>
> In HADOOP-13605, the error message for a bad filesystem has changed.
> Previously, the code threw an IOException:
> {code:java}
> throw new IOException("No FileSystem for scheme: " + scheme);
> {code}
> Now it throws an UnsupportedFileSystemException and quotes the scheme.
> {code:java}
> throw new UnsupportedFileSystemException("No FileSystem for scheme "
> + "\"" + scheme + "\"");
> {code}
> This causes the following frontend test failures when running on hadoop 3:
> org.apache.impala.analysis.AnalyzeAuthStmtsTest
> org.apache.impala.analysis.AnalyzeDDLTest
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
