[
https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057875#comment-17057875
]
Panagiotis Garefalakis edited comment on HIVE-23014 at 3/12/20, 12:33 PM:
--------------------------------------------------------------------------
[~petertoth] thanks for reporting this.
It seems that the particular regression is observed on Single Column Scans
>From Wide Columns:
[https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267]
Did you observe similar ReaderImpl behaviour in other cases? The only change
was Hive version from 1.2.1 to 2.3.6 ?
was (Author: pgaref):
[~petertoth] thanks for reporting this.
It seems that the particular regression is observed on Single Column Scans
>From Wide Columns:
[https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267]
Did you observe similar ReaderImpl behaviour in other cases?
> ORC reading performance
> -----------------------
>
> Key: HIVE-23014
> URL: https://issues.apache.org/jira/browse/HIVE-23014
> Project: Hive
> Issue Type: Bug
> Components: ORC
> Affects Versions: 2.3.6
> Reporter: Peter Toth
> Priority: Major
>
> Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version.
> Some of the ORC reading benchmark shows that there is a huge performance
> difference in ORC reading between the 2 versions. I measured that
> {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar
> is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.
> I'm not sure if more recent Hive versions still suffer from this performance
> regression.
> Please see some details here: SPARK-30565
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
