[
https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039904#comment-17039904
]
Ismaël Mejía edited comment on SPARK-26346 at 2/19/20 10:54 AM:
----------------------------------------------------------------
It seems this issue is blocked because Spark is having dependency issues
because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x)
and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues
from API changes on Avro. I proposed a patch for Hive with the hope that it
gets backported into Hive 2.x to unlock the Avro 1.9 upgrade on Spark (and
transitively this issue) but so far I have not received any attention from the
Hive community, for more info HIVE-21737 and [my multiples call for attention
in the Hive
ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].]
If someone can ping somebody in the Hive community that can help this advance
that would be great!
was (Author: iemejia):
It seems this issue is blocked because Spark is having dependency issues
because of its dependency on Hive. Hive depends on both Avro 1.7 (for Hive 1.x)
and Avro 1.8 (for Hive 2.x). Parquet depends on Avro 1.9. So there are issues
from API changes on Avro. I proposed a patch for Hive with the hope that it
gets backported into Hive 2.x but so far I have not received any attention from
the Hive community, for more info HIVE-21737 and [my multiples call for
attention in the Hive
ML|[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E].]
If someone can ping somebody in the Hive community that can help this advance
that would be great!
> Upgrade parquet to 1.11.1
> -------------------------
>
> Key: SPARK-26346
> URL: https://issues.apache.org/jira/browse/SPARK-26346
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Yuming Wang
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
