[
https://issues.apache.org/jira/browse/HUDI-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17490960#comment-17490960
]
Harsha Gudladona commented on HUDI-3391:
----------------------------------------
Adding some more context on the dependency fixes.
Original issue on the parquet lib was reported here:
https://issues.apache.org/jira/browse/PARQUET-1441 and fixed in
[https://github.com/apache/parquet-mr/pull/560] - which is available in 1.11.x
On the hudi mvn deps, it appears that the parquet version prop is set to
[1.10.1|https://github.com/apache/hudi/blob/master/pom.xml#L100]. however its
overridden for the spark context build
[here][[https://github.com/apache/hudi/blob/master/pom.xml#L1589]]
I think, updating the version in the parent pom global prop should fix it for
presto as well?
> presto and hive beeline fails to read MOR table w/ 2 or more array fields
> -------------------------------------------------------------------------
>
> Key: HUDI-3391
> URL: https://issues.apache.org/jira/browse/HUDI-3391
> Project: Apache Hudi
> Issue Type: Task
> Components: reader-core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Critical
> Fix For: 0.11.0
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> We have an issue reported by user
> [here|[https://github.com/apache/hudi/issues/2657].] Looks like w/ 0.10.0 or
> later, spark datasource read works, but hive beeline does not work. Even
> spark.sql (hive table) querying works as well.
> Another related ticket:
> [https://github.com/apache/hudi/issues/3834#issuecomment-997307677]
>
> Steps that I tried:
> [https://gist.github.com/nsivabalan/fdb8794104181f93b9268380c7f7f079]
> From beeline, you will encounter below exception
> {code:java}
> Failed with exception
> java.io.IOException:org.apache.hudi.org.apache.avro.SchemaParseException:
> Can't redefine: array {code}
> All linked ticket states that upgrading parquet to 1.11.0 or greater should
> work. We need to try it out w/ latest master and go from there.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
