[
https://issues.apache.org/jira/browse/PARQUET-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332566#comment-17332566
]
Gabor Szadovszky commented on PARQUET-2035:
-------------------------------------------
[~cas], I am not sure about the original purpose of shading. Usually, shading
is implemented if a component relies on a specific version of dependency and do
not want to add conflicts in a large ecosystem like Hadoop.
Even though java8 is already EOL it does not mean we cannot limit our source to
be compatible with java8 and still built for that. Most of the environments
where parquet-mr is used use java11 already but only for runtime. AFAIK, the
whole Hadoop ecosystem is still stuck on java8 talking about the source level.
So this is not about supporting java9 or java11 but supporting the the modules
feature of java.
As I've said, I don't have any experience with java modules so I am not sure
about the actual problem and what would be the best fix for it.
Base on the original commit we shade fastutils since a while. PARQUET-1529 was
only about keeping it shaded in all of our modules where it is used. By
reverting PARQUET-1529 we would end up conflicting other fastutil versions in
the Hadoop ecosystem.
We need someone who have experience with java modules to contribute here. It
also seems a tough issue as we can hardly implement unit tests for such changes.
> Java module import error due to shaded package
> shaded.parquet.it.unimi.dsi.fastutil
> -----------------------------------------------------------------------------------
>
> Key: PARQUET-2035
> URL: https://issues.apache.org/jira/browse/PARQUET-2035
> Project: Parquet
> Issue Type: Bug
> Components: parquet-avro
> Affects Versions: 1.12.0, 1.11.1
> Reporter: Maxim Kolesnikov
> Priority: Major
> Attachments: parquet-example.zip
>
>
> *Description:*
> Due to collision of shaded packages
> {code:java}
> shaded.parquet.it.unimi.dsi.fastutil{code}
> in
> {code:java}
> org.apache.parquet:parquet-avro{code}
> and
> {code:java}
> org.apache.parquet:parquet-column{code}
> it is not possible to use both these dependencies within a modularized java
> project at the same time.
>
>
> *How to reproduce:*
> * create a maven project with dependency
> org.apache.parquet:parquet-avro:1.11.1
> * declare java module that requires both parquet.avro and parquet.column
> * run
> {code:java}
> mvn compile{code}
>
> *Expected behaviour:*
> Project should compile without errors.
>
> *Actual behaviour:*
> Project fails with compilation errors:
>
> {code:java}
> [ERROR] the unnamed module reads package shaded.parquet.it.unimi.dsi.fastutil
> from both parquet.column and parquet.avro
> ...{code}
>
>
> *Reproducible example* (same code as in the attached zip file):
> https://github.com/xCASx/parquet-example
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
