Hey,
>> In another email thread
(https://lists.apache.org/thread/sxcrcf4v9j630tl9domp0bn4m33bdq0s) Sun Chao mentioned
that other projects (Spark,
>> Iceberg and Trino/Presto) are still depending on old Hive, because the exec-core jar has been removed, and the exec jar contains unshaded versions of various
dependencies. Until this is fixed, they can not upgrade to a newer version of Hive, so I would like to add this as a blocker for Hive 4.0.0 release.
>> @Chao Sun: Could you help us find the jira for this issue, or file a new one?
I was thinking about this and I think this is a bit unfair...say project X is using Hive 2.3's core jar; should "we" the Hive community do all the work to run their project
with Hive 4? I don't think so.
What if some project is not interested in upgrading? Should we really put
efforts into thing even in that case?
The best middle ground idea I was able to come up so far was to ask for a broken development branch set up to run with some 4.0.0-alpha-X release where we can start fixing
the shading issues they might face together.
In this case they will be already ready to go upgrading their Hive; and if they also able to run tests/etc: as a bonus we will get early pre-integration feedback(s)...which
will be valuable for both them and us.
What do you guys think?
Are there any other options?
cheers,
Zoltan
On 5/11/22 7:33 AM, Chao Sun wrote:
Thanks for reminding me, Peter. There is
https://issues.apache.org/jira/browse/HIVE-25317 but that's for Hive
2.3 and is mostly for the Spark use case. I just created
https://issues.apache.org/jira/browse/HIVE-26220 and marked it as a
blocker.
On Tue, May 10, 2022 at 10:01 PM Peter Vary <pv...@cloudera.com> wrote:
In another email thread
(https://lists.apache.org/thread/sxcrcf4v9j630tl9domp0bn4m33bdq0s) Sun Chao
mentioned that other projects (Spark,
Iceberg and Trino/Presto) are still depending on old Hive, because the
exec-core jar has been removed, and the exec jar contains unshaded versions of
various dependencies. Until this is fixed, they can not upgrade to a newer
version of Hive, so I would like to add this as a blocker for Hive 4.0.0
release.
@Chao Sun: Could you help us find the jira for this issue, or file a new one?
Any more blockers?
Thanks,
Peter
On Fri, Apr 29, 2022, 13:46 Peter Vary <pv...@cloudera.com> wrote:
Hi Team,
With Zoltan Haindrich, we have been brainstorming about the next steps after
the 4.0.0-alpha-1 release.
We come up with the following plan:
- Define a desired scope for the 4.0.0 release
- Release minimally quarterly - create alpha release(s) until the scope is
reached
- If the scope is reached - create a beta release
- For fixes - create a beta release
- If we are satisfied with the quality of the release then we can release the
Hive 4.0.0
- Keep up with the quarterly release cadence
Until now we collected the following items which could be part of the scope:
- Java 11 upgrade (minimally)
- Hadoop 3.3 (needed to the Java 11 upgrade)
- Full Iceberg integration (Read, Write, Delete, Update, Merge)
- Clean up the HMS API interface (deprecate old methods which are already
released, remove unreleased methods which have not been released yet,
use/create methods with Request objects as parameters instead of Context
objects)
We might want to collect information about the usage of specific modules, and
might deprecate some based on the feedback (remove them from the release or at
least mark them deprecated), so we can reduce the project complexity based on
the info. Some features which pooped up:
- HCatalog
- WebHCat
- Pig integration
- ??
We would be interested on any feedback for this plan / scope / deprecation.
Feel free to suggest any additions or removals from these lists, or even
propose an entirely different plan.
Also if you would like to take over specific tasks, feel free to grab it, and
start working on it or start discussing it.
Thanks,
Peter
