Tim Tim created SPARK-51174:
-------------------------------
Summary: Deployment of Spark 4.0+ fails on the module
common/network-common
Key: SPARK-51174
URL: https://issues.apache.org/jira/browse/SPARK-51174
Project: Spark
Issue Type: Bug
Components: Deploy
Affects Versions: 4.0.0, 4.1.0
Environment: Ubuntu20.04, JDK17
Reporter: Tim Tim
"Deployment of Spark 4.0+ fails on the module {{common/network-common}} as
described below:
To reproduce the issue, use the following CLI command:
bash
./build/mvn -DskipTests --settings=local-maven-repo-settings.xml clean deploy
This issue first occurred after the PR was merged:
[https://github.com/apache/spark/pull/49765]. The problem disappears if I
revert that PR.
The failure is not observed in Spark 3.0+ versions; it only occurs in Spark
4.0+ versions."
```
./build/mvn -DskipTests --settings=local-maven-repo-settings.xml clean deploy
....
*[**INFO**]* *Downloaded* *from snapshots:
https:*{*}//test.example/maven-repo/org/bouncycastle/bcprov-jdk18on/1.80/bcprov-jdk18on-1.80.jar
(8.7 MB at 3.2 MB/s){*}
*[**INFO**]* *Downloading* *from snapshots:
https:*{*}//test.example/maven-repo/org/bouncycastle/bcpkix-jdk18on/1.80/bcpkix-jdk18on-1.80.jar{*}
*[**INFO**]* *Downloaded* *from snapshots:
https:*{*}//test.example/maven-repo/org/bouncycastle/bcpkix-jdk18on/1.80/bcpkix-jdk18on-1.80.jar
(1.1 MB at 2.6 MB/s){*}
{*}[{*}*INFO*{*}]
------------------------------------------------------------------------{*}
*[*{*}INFO{*}*]* *Reactor* ** *Summary* ** *for* ** *Spark* ** *Project* **
*Parent* ** *POM* ** *4.1**.**0**-*{*}SNAPSHOT{*}*:*
*[*{*}INFO{*}*]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Parent* ** *POM*
*...........................* *SUCCESS* *[* *22.795* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Tags*
*.................................* *SUCCESS* *[* *29.952* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Sketch*
*...............................* *SUCCESS* *[* *20.523* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Common* ** *Utils*
*.........................* *SUCCESS* *[* *33.674* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Local* ** *DB*
*.............................* *SUCCESS* *[* *22.940* *s]*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Networking*
*...........................* *FAILURE* *[* *6.459* *s]*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Shuffle* ** *Streaming* ** *Service*
*............* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Variant*
*..............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Unsafe*
*...............................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Shims*
*........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Launcher*
*.............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Core*
*.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *ML* ** *Local* ** *Library*
*.....................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *GraphX*
*...............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Streaming*
*............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *SQL* ** *API*
*..............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Catalyst*
*.............................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *SQL*
*..................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *ML* ** *Library*
*...........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Tools*
*................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Hive*
*.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Common*
*.......................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Avro*
*.........................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Protobuf*
*.....................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *REPL*
*.................................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Server*
*.......................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *Connect* ** *Client*
*.......................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Project* ** *YARN* ** *Shuffle* ** *Service*
*.................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *YARN*
*.................................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Kubernetes*
*...........................* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Assembly*
*.............................* *SKIPPED*
*[*{*}INFO{*}*]* *Kafka* ** *0.10**+* *Token* ** *Provider* ** *for* **
*Streaming* *...........* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Integration* ** *for* ** *Kafka* ** *0.10*
*...................* *SKIPPED*
*[*{*}INFO{*}*]* *Kafka* ** *0.10**+* *Source* ** *for* ** *Structured* **
*Streaming* *........* *SKIPPED*
{*}[{*}*INFO*{*}]{*} *Spark* ** *Project* ** *Examples*
*.............................* *SKIPPED*
*[*{*}INFO{*}*]* *Spark* ** *Integration* ** *for* ** *Kafka* ** *0.10* **
*Assembly* *..........* *SKIPPED*
{*}[{*}*INFO*{*}]
------------------------------------------------------------------------{*}
*[*{*}INFO{*}*]* *BUILD* ** *FAILURE*
{*}[{*}*INFO*{*}]
------------------------------------------------------------------------{*}
{*}[{*}*INFO*{*}]{*} *Total* *time:* *02*{*}:{*}*17* *min*
{*}[{*}*INFO*{*}]{*} *Finished* *at:*
*2025*{*}-{*}*02*{*}-12T02:{*}*13*{*}:45Z{*}
{*}[{*}*INFO*{*}]
------------------------------------------------------------------------{*}
{*}[{*}*ERROR*{*}]{*} *Failed* \{*}to execute goal on project
spark-network-common_2.{*}*13*{*}:{*} *Could* *not resolve dependencies* *for*
\{*}project
org.{*}*apache*{*}.{*}*spark*{*}:spark-network-common_2.{*}*13*{*}:jar:{*}*4.1*{*}.{*}*0*{*}-{*}*SNAPSHOT*
{*}[{*}*ERROR*{*}] dependency:
org.{*}*bouncycastle*{*}:bcutil-jdk18on:jar:{*}*1.80* *(test)*
{*}[{*}*ERROR*{*}] {*} *Could* \{*}not transfer artifact
org.{*}*bouncycastle*{*}:bcutil-jdk18on:jar:{*}*1.80* *from/to* *central*
\{*}(https:{*}*//test.example/maven-repo): status code: 401, reason phrase:
(401)*
{*}[{*}*ERROR*{*}] -> [{*}*Help* ** *1*{*}]{*}
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *To* *see the full stack trace of the errors, re-run*
*Maven* *with the -e* *switch*{*}.{*}
{*}[{*}*ERROR*{*}]{*} *Re*{*}-run{*} *Maven* \{*}using the -{*}*X* ** *switch*
*to enable full debug logging.*
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *For* *more information about the errors and possible
solutions, please read the following articles:*
*[**ERROR**] [**Help* ** *1**]
http:*{*}//cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException{*}
*[*{*}ERROR{*}*]*
{*}[{*}*ERROR*{*}]{*} *After* *correcting the problems, you can resume the
build with the command*
{*}[{*}*ERROR*{*}] mvn <args> -rf :spark-network-common_2.{*}*13*
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
