[
https://issues.apache.org/jira/browse/SPARK-32532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-32532:
------------------------------------
Assignee: Apache Spark
> Improve ORC read/write performance on nested structs and array of structs
> -------------------------------------------------------------------------
>
> Key: SPARK-32532
> URL: https://issues.apache.org/jira/browse/SPARK-32532
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Muhammad Samir Khan
> Assignee: Apache Spark
> Priority: Major
>
> Have some improvements for ORC file format to reduce time taken when
> reading/writing nested/array'd structs. Using benchmarks in SPARK-32531 was
> able to improve performance on branch-3.0 as follows (measurements in
> seconds):
> Read:
> Nested Structs: 184 -> 44
> Array of Struct: 66 -> 15
> Write
> Nested Structs: 543 -> 39
> Array of Struct: 330 -> 37
> Will be putting up the PR soon with the changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
