Hi, All.
+1 for 1.6.3 RC1.
BTW, although this is not a regression, Apache ORC 1.6.0 ~ 1.6.3rc1 doesn't
work with all Apache Hive (2.x/3.x/4.0.0-SNAPSHOT) releases due to the
breaking change by ORC-520.
ORC-520: Fix file merging for column encryption
https://github.com/apache/orc/pull/423
I'm wondering if there is a roadmap to recover the relationship or not.
The reason why I asked about this at this release vote is that Apache Spark
3.0+ became to fail with Apache ORC 1.6.x since Jan 2020.
Let me give you some background on this.
Apache Spark 3.0 has two Hive profiles.
- Hive 1.2 profile with unofficial Apache Hive 1.2.1 forked artifacts.
This profile depends on Hive 1.2 and Apache ORC no-hive jar.
- Hive 2.3 profile with official Apache Hive 2.3.7 artifacts.
This profile depends on Hive 2.3.7 and Apache ORC jar. (not no-hive jar)
The following is the compatibility timeline until now.
1.6.0 2019/08/30 This didn't work with Spark 3.0 for both Hive
profiles due to the unit test failure.
1.6.1 2019/10/25 This didn't work with Spark 3.0 for both Hive
profiles due to the unit test failure.
1.6.2 2019/11/21 This worked with Spark 3.0 with Hive 1.2.1 default
profile while it fails Hive 2.3.7 profile
2020/01/09 Apache Spark 3.0 switch to Apache Hive 2.3.7
profile by default. (SPARK-30034)
1.6.3 2020/04/24 By default, this will not work with Spark 3.0. With
Hive 1.2.1 profile, it still works.
Bests,
Dongjoon.
On Thu, Apr 23, 2020 at 5:16 PM Owen O'Malley <[email protected]>
wrote:
> All,
>
> Should we release the following artifacts as ORC 1.6.3?
>
> tar: http://home.apache.org/~omalley/orc-1.6.3/
> tag: https://github.com/apache/orc/releases/tag/release-1.6.3rc1
> jiras: https://issues.apache.org/jira/projects/ORC/versions/12346545
>
> Thanks!
>
