[
https://issues.apache.org/jira/browse/PARQUET-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333047#comment-17333047
]
Maxim Kolesnikov edited comment on PARQUET-2035 at 4/27/21, 9:46 AM:
---------------------------------------------------------------------
Yes, the original ticket and follow up conversation is about modules feature,
there was no claim that Java 9 is not supported.
I'm not sure what quick solution options I have, if any. For that I need to dig
deeper.
Though the long term solution would be to get rid of split package, as java
modules [doesn't allow
those|http://openjdk.java.net/projects/jigsaw/spec/sotms/#readability].
I see 4 ways of doing this:
* Use unique names for shaded packages, e.g.
{{shaded.parquet.{color:#ff0000}avro{color}.it.unimi.dsi.fastutil}},
{{shaded.parquet.{color:#ff0000}column{color}.it.unimi.dsi.fastutil}}. May be
not an option, as it may break application logic due to presence of multiple
instances of the same classes.
* Get rid of dependency on {{fastutil}} in any two out of three modules that
are currently shading it. That may require significant code refactoring and may
be not feasible. At least in {{parquet-hadoop}} it seems to be used in a single
place for some performance optimisation.
* Achieve consistency of {{fastutil}} across hadoop projects. After that no
shading is needed. Would be ideal, but probably even less feasible solution.
* Create a new module, e.g. {{parquet-fastutil}} that would contain only the
shaded library. Add this module to transitive non shaded dependencies that have
dependency on {{fastutil}}: {{parquet-avro}}, {{parquet-column}},
{{parquet-hadoop}}.
What do you think? Do you see any other options?
was (Author: cas):
Yes, the original ticket and follow up conversation is about modules feature,
there was no claim that Java 9 is not supported.
I'm not sure what quick solution options I have, if any. For that I need to dig
deeper.
Though the long term solution would be to get rid of split package, as java
modules [doesn't allow
those|http://openjdk.java.net/projects/jigsaw/spec/sotms/#readability].
I see 4 ways of doing this:
* Use unique names for shaded packages, e.g.
{{shaded.parquet.{color:#FF0000}avro{color}.it.unimi.dsi.fastutil}},
{{shaded.parquet.{color:#FF0000}column{color}.it.unimi.dsi.fastutil}}. Probably
not an option, as it may break application logic due to presence of multiple
instances of the same classes.
* Get rid of dependency on {{fastutil}} in any two out of three modules that
are currently shading it. That may require significant code refactoring and may
be not feasible. At least in {{parquet-hadoop}} it seems to be used in a single
place for some performance optimisation.
* Achieve consistency of {{fastutil}} across hadoop projects. Would be ideal,
but probably even less feasible solution.
* Create a new module, e.g. {{parquet-fastutil}} that would contain only the
shaded library. Add this module to transitive non shaded dependencies that have
dependency on {{fastutil}}: {{parquet-avro}}, {{parquet-column}},
{{parquet-hadoop}}.
What do you think? Do you see any other options?
> Java module import error due to shaded package
> shaded.parquet.it.unimi.dsi.fastutil
> -----------------------------------------------------------------------------------
>
> Key: PARQUET-2035
> URL: https://issues.apache.org/jira/browse/PARQUET-2035
> Project: Parquet
> Issue Type: Bug
> Components: parquet-avro
> Affects Versions: 1.12.0, 1.11.1
> Reporter: Maxim Kolesnikov
> Priority: Major
> Attachments: parquet-example.zip
>
>
> *Description:*
> Due to collision of shaded packages
> {code:java}
> shaded.parquet.it.unimi.dsi.fastutil{code}
> in
> {code:java}
> org.apache.parquet:parquet-avro{code}
> and
> {code:java}
> org.apache.parquet:parquet-column{code}
> it is not possible to use both these dependencies within a modularized java
> project at the same time.
>
>
> *How to reproduce:*
> * create a maven project with dependency
> org.apache.parquet:parquet-avro:1.11.1
> * declare java module that requires both parquet.avro and parquet.column
> * run
> {code:java}
> mvn compile{code}
>
> *Expected behaviour:*
> Project should compile without errors.
>
> *Actual behaviour:*
> Project fails with compilation errors:
>
> {code:java}
> [ERROR] the unnamed module reads package shaded.parquet.it.unimi.dsi.fastutil
> from both parquet.column and parquet.avro
> ...{code}
>
>
> *Reproducible example* (same code as in the attached zip file):
> https://github.com/xCASx/parquet-example
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
