Yikf commented on code in PR #43436:
URL: https://github.com/apache/spark/pull/43436#discussion_r1372554800
##########
connector/connect/client/jvm/pom.xml:
##########
@@ -124,6 +124,10 @@
<include>io.grpc.**</include>
</includes>
</relocation>
+ <relocation>
Review Comment:
The current Apache Spark includes two versions of Guava, which are (The
historical reasons are not clear to me. @LuciferYang may know some background):
1.
https://github.com/apache/spark/blob/8cdcfd262f9fd46fb9a8e1ceb0bccefe452582bd/pom.xml#L203
2.
https://github.com/apache/spark/blob/8cdcfd262f9fd46fb9a8e1ceb0bccefe452582bd/pom.xml#L292
The current Apache Spark has shading in the following three places for these
two versions:
1. for guava.version
https://github.com/apache/spark/blob/8cdcfd262f9fd46fb9a8e1ceb0bccefe452582bd/common/network-common/pom.xml#L125-L129
2. for connect.guava.version
https://github.com/apache/spark/blob/8cdcfd262f9fd46fb9a8e1ceb0bccefe452582bd/connector/connect/server/pom.xml#L310-L316
3. for conflicting versions, the connect-common does not override the
parent's shade plugin, so it is relocated to
'${spark.shade.packageName}.guava'. However, the Guava version of
connect-common is connect.guava.version, which results in two versions of the
same package in the classpath.
https://github.com/apache/spark/blob/8cdcfd262f9fd46fb9a8e1ceb0bccefe452582bd/connector/connect/common/pom.xml#L50-L61
Based on the above, it will lead to:
- The server with the connect profile has version 'guava.version' of
'${spark.shade.packageName}.guava'.
- The server with the connect profile has version 'connect.guava.version' of
'${spark.shade.packageName}.guava'.
- The server with the connect profile has version 'connect.guava.version' of
'${spark.shade.packageName}.connect.guava'.
- The server with the connect profile does not have version
'connect.guava.version' of 'failureaccess'
This PR plans to make modifications following this line of thought, we
should use connect-common shade guava, connect-client and connect-server shade
connect-common to make connect-server and connect-client use the same guava
dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
