[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=528819&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528819
]
ASF GitHub Bot logged work on HIVE-24484:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Dec/20 16:06
Start Date: 28/Dec/20 16:06
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #1742:
URL: https://github.com/apache/hive/pull/1742#issuecomment-751765866
@wangyum Stuck.
There are two big issues here:
1. Hive integration tests fire up Druid, Kafka, HDFS, LLAP, etc. all in the
same JVM and their 3rd party dependencies are all over the place. Using a
higher version of a dependency breaks one product, but using a lower version
breaks the other. To make this work well, there probably needs to be a way to
launch each service in their own JVM class loader. In lieu of that, I've been
trying to move the ball closer to the goal post and getting dependencies closer
together.
https://github.com/apache/druid/pull/10683
HIVE-24542
2. In HDFS 3.3.0, Hadoop team introduced `ProtobufRpcEngine2` in addition to
`ProtobufRpcEngine` (sigh). Some of the Hive LLAP stuff is using this Hadoop
Protobuf RPC engine (`ProtobufRpcEngine`). There's some `static` logic in the
protocol engines that prohibits loading both RPC engines into the same JVM at
the same time, I'm not sure why. HDFS was migrated to `ProtobufRpcEngine2`.
So, again, in the integration tests, when the HDFS mini cluster is loaded,
version 2 of the RPC engine is loaded into the JVM. When LLAP is later loaded,
it fails to start because version 1 cannot be registered at the same time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 528819)
Time Spent: 50m (was: 40m)
> Upgrade Hadoop to 3.3.0
> -----------------------
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
