[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-594774176 Closing it as we need to use regular orc This is an automated message from the Apache Git

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-591177652 thank you @omalley and @dongjoon-hyun! btw, are we concerned that hive-common shipped with hive 2.3.6 and hive-storage-api 2.6.0 used by orc 1.5.9

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-589229742 @dongjoon-hyun So, hive 2.3 depends on apache orc instead of using orc embedded in hive, which means we will need to pull in regular orc instead of

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-587232690 @dongjoon-hyun thank you for looking into it. How did sql/hive work with hive 1.2 and orc nohive? Does sql/hive also use hive's orc when hive 1.2 was used?

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-586549374 thank you @dongjoon-hyun ! This is an automated message from the Apache Git Service. To

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-586398279 @dongjoon-hyun @wangyum do you happen to know what happened with https://github.com/apache/spark/pull/27536#issuecomment-585042303? Seems in hive module,

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-585043898 taking

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-585042303 Also, the error cause was ``` Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-585041496 hmm. We need to keep hive-storage-api. But I will need to check why we hit the runtime exception. Somehow we used hive-storage-api's VectorizedRowBatch

[GitHub] [spark] yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive

yhuai commented on issue #27536: [SPARK-30784] Use ORC nohive URL: https://github.com/apache/spark/pull/27536#issuecomment-584898607 oh hive-storage-api still gets pulled in. Let me check. This is an automated message from