[
https://issues.apache.org/jira/browse/IMPALA-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837380#comment-16837380
]
ASF subversion and git services commented on IMPALA-8369:
---------------------------------------------------------
Commit 17daa6efb9c3c5c6fbd0908f2176b99d8498a250 in impala's branch
refs/heads/master from Todd Lipcon
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=17daa6e ]
IMPALA-8369 (part 2): Hive 3: switch to Tez-on-YARN execution
This switches away from Tez local mode to tez-on-YARN. After spending a
couple of days trying to debug issues with Tez local mode, it seemed
like it was just going to be too much of a lift.
This patch switches on the starting of a Yarn RM and NM when
USE_CDP_HIVE is enabled. It also switches to a new yarn-site.xml with a
minimized set of configurations, generated by the new python templating.
In order for everything to work properly I also had to update the Hadoop
dependency to come from CDP instead of CDH when using CDP Hive.
Otherwise, the classpath of the launched Tez containers had conflicting
versions of various Hadoop classes which caused tasks to fail.
I verified that this fixes concurrent query execution by running queries
in parallel in two beeline sessions. With local mode, these queries
would periodically fail due to various races (HIVE-21682). I'm also able
to get farther along in data loading.
Change-Id: If96064f271582b2790a3cfb3d135f3834d46c41d
Reviewed-on: http://gerrit.cloudera.org:8080/13224
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Todd Lipcon <[email protected]>
> Impala should be able to interoperate with Hive 3.1.0
> -----------------------------------------------------
>
> Key: IMPALA-8369
> URL: https://issues.apache.org/jira/browse/IMPALA-8369
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
> Labels: impala-acid
>
> Currently, Impala only works with Hive 2.1.1. Since Hive 3.1.0 has been
> released for a while it would be good to add support for Hive 3.1.0 (HMS
> 3.1.0). This patch will focus on ability to connect to HMS 3.1.0 and run
> existing tests. It will not focus on adding support for newer features like
> ACID in Hive 3.1.0 which can be taken up as separate JIRA.
> It would be good to make changes to Impala source code such that it can work
> with both Hive 2.1.0 and Hive 3.1.0 without the need to create a separate
> branch. However, this should be a aspirational goal. If we hit a blocker we
> should investigate alternative approaches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
