[
https://issues.apache.org/jira/browse/FLINK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555369#comment-17555369
]
Yang Wang commented on FLINK-28016:
-----------------------------------
[~chesnay] I appreciate for your detailed explanation. Just like what you have
said, the non-relocation transitive dependencies of flink-kubernetes will
bundled into the flink-dist again when using maven 3.3.9. The following output
could prove this.
{code:java}
$ ~/Downloads/apache-maven-3.3.9/bin/mvn clean install -DskipTests
$ unzip -l build-target/lib/flink-dist-1.16-SNAPSHOT.jar | grep
'dk.brics.automaton'
0 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/
17519 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Automaton.class
2626 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/AutomatonMatcher.class
282 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/AutomatonProvider.class
8324 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/BasicAutomata.class
11065 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/BasicOperations.class
21514 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Datatypes.class
839 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/DatatypesAutomatonProvider.class
390 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$IntPair.class
956 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$StateList.class
1181 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$StateListNode.class
7440 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations.class
1506 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp$1.class
1964 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp$Kind.class
13840 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp.class
4523 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RunAutomaton.class
2587 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/ShuffleOperations$ShuffleConfiguration.class
7519 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/ShuffleOperations.class
11175 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/SpecialOperations.class
3566 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/State.class
1079 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StatePair.class
830 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations$1.class
3051 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations$State.class
4562 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations.class
2232 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Transition.class
1247 06-16-2022 19:40
org/apache/flink/kubernetes/shaded/dk/brics/automaton/TransitionComparator.class
0 06-16-2022 19:40 dk/brics/automaton/
14740 06-16-2022 19:40 dk/brics/automaton/Automaton.class
2486 06-16-2022 19:40 dk/brics/automaton/AutomatonMatcher.class
212 06-16-2022 19:40 dk/brics/automaton/AutomatonProvider.class
7202 06-16-2022 19:40 dk/brics/automaton/BasicAutomata.class
9422 06-16-2022 19:40 dk/brics/automaton/BasicOperations.class
20524 06-16-2022 19:40 dk/brics/automaton/Datatypes.class
699 06-16-2022 19:40
dk/brics/automaton/DatatypesAutomatonProvider.class
320 06-16-2022 19:40
dk/brics/automaton/MinimizationOperations$IntPair.class
676 06-16-2022 19:40
dk/brics/automaton/MinimizationOperations$StateList.class
901 06-16-2022 19:40
dk/brics/automaton/MinimizationOperations$StateListNode.class
6430 06-16-2022 19:40 dk/brics/automaton/MinimizationOperations.class
1334 06-16-2022 19:40 dk/brics/automaton/RegExp$1.class
1728 06-16-2022 19:40 dk/brics/automaton/RegExp$Kind.class
11770 06-16-2022 19:40 dk/brics/automaton/RegExp.class
4001 06-16-2022 19:40 dk/brics/automaton/RunAutomaton.class
2038 06-16-2022 19:40
dk/brics/automaton/ShuffleOperations$ShuffleConfiguration.class
6089 06-16-2022 19:40 dk/brics/automaton/ShuffleOperations.class
9255 06-16-2022 19:40 dk/brics/automaton/SpecialOperations.class
3041 06-16-2022 19:40 dk/brics/automaton/State.class
764 06-16-2022 19:40 dk/brics/automaton/StatePair.class
763 06-16-2022 19:40 dk/brics/automaton/StringUnionOperations$1.class
2737 06-16-2022 19:40
dk/brics/automaton/StringUnionOperations$State.class
3796 06-16-2022 19:40 dk/brics/automaton/StringUnionOperations.class
1987 06-16-2022 19:40 dk/brics/automaton/Transition.class {code}
However, when I use maven 3.8.5 to do the same thing. I find that the
non-relocation dependencies are *NOT* bundled into the flink-dist uber jar.
{code:java}
$ ~/Downloads/apache-maven-3.8.5/bin/mvn clean install -DskipTests
$ unzip -l build-target/lib/flink-dist-1.16-SNAPSHOT.jar | grep
'dk.brics.automaton'
0 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/
17519 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Automaton.class
2626 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/AutomatonMatcher.class
282 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/AutomatonProvider.class
8324 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/BasicAutomata.class
11065 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/BasicOperations.class
21514 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Datatypes.class
839 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/DatatypesAutomatonProvider.class
390 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$IntPair.class
956 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$StateList.class
1181 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations$StateListNode.class
7440 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/MinimizationOperations.class
1506 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp$1.class
1964 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp$Kind.class
13840 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RegExp.class
4523 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/RunAutomaton.class
2587 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/ShuffleOperations$ShuffleConfiguration.class
7519 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/ShuffleOperations.class
11175 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/SpecialOperations.class
3566 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/State.class
1079 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StatePair.class
830 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations$1.class
3051 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations$State.class
4562 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/StringUnionOperations.class
2232 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/Transition.class
1247 06-17-2022 11:13
org/apache/flink/kubernetes/shaded/dk/brics/automaton/TransitionComparator.class
{code}
Please note that I completely build all the modules at the same time for both
above testing. I am not sure what I have missed in the testing.
> Support Maven 3.3+
> ------------------
>
> Key: FLINK-28016
> URL: https://issues.apache.org/jira/browse/FLINK-28016
> Project: Flink
> Issue Type: Technical Debt
> Components: Build System
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Major
> Fix For: 1.16.0
>
>
> We are currently de-facto limited to Maven 3.2.5 because our packaging relies
> on the shade-plugin modifying the dependency tree at runtime when bundling
> dependencies, which is no longer possible on Maven 3.3+.
> Being locked in to such an old Maven version isn't a good state to be in, and
> the contributor experience suffers as well.
> I've been looking into removing this limitation by explicitly marking every
> dependency that we bundle as {{optional}} in the poms, which really means
> {{non-transitive}}. This ensures that the everything being bundled by one
> module is not visible to other modules. Some tooling to capture developer
> mistakes were also written.
> Overall this is actually quite a nice change, as it makes things more
> explicit and reduces inconsistencies (e.g., the dependency plugin results are
> questionable if the shade-plugin didn't run!); and it already highlighted
> several problems in Flink.
> This change will have no effect on users or the released poms, because the
> dependency-reduced poms will be generated as before and remove all modified
> dependencies.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
