[
https://issues.apache.org/jira/browse/PARQUET-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558442#comment-17558442
]
ASF GitHub Bot commented on PARQUET-2158:
-----------------------------------------
steveloughran commented on code in PR #976:
URL: https://github.com/apache/parquet-mr/pull/976#discussion_r905967944
##########
pom.xml:
##########
@@ -76,7 +76,7 @@
<jackson-databind.version>2.13.2.2</jackson-databind.version>
<japicmp.version>0.14.2</japicmp.version>
<shade.prefix>shaded.parquet</shade.prefix>
- <hadoop.version>2.10.1</hadoop.version>
+ <hadoop.version>3.2.0</hadoop.version>
Review Comment:
I was being unambitious. move to this, the oldest 3.x release working on
java11 ensures that anything else on a version >= to this should link properly.
if you do want to be more current, well, spark is on 3.3.3, hive is trying
to move to 3.3.x and I will be doing a 3.3.4 release in a week's time, which is
just some security changes mostly of relevance to servers
> Upgrade Hadoop dependency to version 3.2.0
> ------------------------------------------
>
> Key: PARQUET-2158
> URL: https://issues.apache.org/jira/browse/PARQUET-2158
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Affects Versions: 1.13.0
> Reporter: Steve Loughran
> Priority: Major
>
> Parquet still builds against Hadoop 2.10. This is very out of date and does
> not work with java 11, let alone later releases.
> Upgrading the dependency to Hadoop 3.2.0 makes the release compatible with
> java 11, and lines up with active work on HADOOP-18287, _Provide a shim
> library for modern FS APIs_
> This will significantly speed up access to columnar data, especially in
> cloud stores.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
