LuciferYang commented on PR #36321:
URL: https://github.com/apache/spark/pull/36321#issuecomment-1107827369
@srowen I update pr title and pr description, without this pr there are
compile error on `Appleslicon/MacOs` as follows:
```
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.8:run (unpack) on project
spark-network-yarn_2.12: An Ant BuildException has occured: Warning: Could not
find file
/spark-source/common/network-yarn/target/exploded/META-INF/native/libnetty_transport_native_epoll_x86_64.so
to copy.
[ERROR] around Ant part ...<move
file="/spark-source/common/network-yarn/target/exploded/META-INF/native/libnetty_transport_native_epoll_x86_64.so"
tofile="/spark-source/common/network-yarn/target/exploded/META-INF/native/lib${spark.shade.native.packageName}_netty_transport_native_epoll_x86_64.so"/>...
@ 6:340 in /spark-source/common/network-yarn/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn <args> -rf :spark-network-yarn_2.12
```
due to `maven-antrun-plugin` configured in `spark-network-yarn` module as
follows:
```
<execution>
<id>unpack</id>
<phase>package</phase>
<configuration>
<target>
<echo message="Shade netty native libraries to
${spark.shade.native.packageName}" />
<unzip src="${shuffle.jar}"
dest="${project.build.directory}/exploded/" />
<move
file="${project.build.directory}/exploded/META-INF/native/libnetty_transport_native_epoll_x86_64.so"
tofile="${project.build.directory}/exploded/META-INF/native/lib${spark.shade.native.packageName}_netty_transport_native_epoll_x86_64.so"
/>
<move
file="${project.build.directory}/exploded/META-INF/native/libnetty_transport_native_kqueue_x86_64.jnilib"
tofile="${project.build.directory}/exploded/META-INF/native/lib${spark.shade.native.packageName}_netty_transport_native_kqueue_x86_64.jnilib"
/>
<jar destfile="${shuffle.jar}"
basedir="${project.build.directory}/exploded" />
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
```
and this pr also fixes compilation problems:
```
[INFO] Reactor Summary for Spark Project Parent POM 3.4.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 3.654
s]
[INFO] Spark Project Tags ................................. SUCCESS [ 6.412
s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 7.344
s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 6.002
s]
[INFO] Spark Project Networking ........................... SUCCESS [ 10.175
s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.930
s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 8.863
s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 5.580
s]
[INFO] Spark Project Core ................................. SUCCESS [01:53
min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 22.776
s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 28.497
s]
[INFO] Spark Project Streaming ............................ SUCCESS [ 46.242
s]
[INFO] Spark Project Catalyst ............................. SUCCESS [03:12
min]
[INFO] Spark Project SQL .................................. SUCCESS [03:03
min]
[INFO] Spark Project ML Library ........................... SUCCESS [01:49
min]
[INFO] Spark Project Tools ................................ SUCCESS [ 5.811
s]
[INFO] Spark Project Hive ................................. SUCCESS [01:24
min]
[INFO] Spark Project REPL ................................. SUCCESS [ 17.332
s]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 20.261
s]
[INFO] Spark Project YARN ................................. SUCCESS [ 36.544
s]
[INFO] Spark Project Mesos ................................ SUCCESS [ 29.431
s]
[INFO] Spark Project Kubernetes ........................... SUCCESS [ 32.766
s]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [ 37.870
s]
[INFO] Spark Ganglia Integration .......................... SUCCESS [ 7.186
s]
[INFO] Spark Project Hadoop Cloud Integration ............. SUCCESS [ 12.707
s]
[INFO] Spark Project Assembly ............................. SUCCESS [ 6.827
s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 16.881
s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [ 22.341
s]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [ 34.514
s]
[INFO] Spark Kinesis Integration .......................... SUCCESS [ 22.891
s]
[INFO] Spark Project Examples ............................. SUCCESS [ 42.790
s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [ 9.503
s]
[INFO] Spark Avro ......................................... SUCCESS [ 31.137
s]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 14.853
s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 20:39 min
[INFO] Finished at: 2022-04-24T20:00:19+08:00
[INFO]
------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
