hasnain-db opened a new pull request, #43289: URL: https://github.com/apache/spark/pull/43289
### What changes were proposed in this pull request? Exclude a `LICENSE` file in favor of an existing `LICENSE.txt` with the same contents to fix the build for network-yarn ### Why are the changes needed? Making a distribution build using the following command currently fails: ``` ./dev/make-distribution.sh --tgz -Phive -Phive-thriftserver -Pyarn ``` Failures look like ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run (unpack) on project spark-network-yarn_2.13: An Ant BuildException has occured: Error while expanding /Users/hasnain.lakhani/spark/common/network-yarn/target/scala-2.13/spark-4.0.0-SNAPSHOT-yarn-shuffle.jar [ERROR] java.nio.file.FileSystemException: /Users/hasnain.lakhani/spark/common/network-yarn/target/exploded/META-INF/license/LICENSE.aix-netbsd.txt: Not a directory [ERROR] around Ant part ...<unzip src="/Users/hasnain.lakhani/spark/common/network-yarn/target/scala-2.13/spark-4.0.0-SNAPSHOT-yarn-shuffle.jar" dest="/Users/hasnain.lakhani/spark/common/network-yarn/target/exploded/" />... @ 5:198 in /Users/hasnain.lakhani/spark/common/network-yarn/target/antrun/build-main.xml [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run (unpack) on project spark-network-yarn_2.13: An Ant BuildException has occured: Error while expanding /Users/hasnain.lakhani/spark/common/network-yarn/target/scala-2.13/spark-4.0.0-SNAPSHOT-yarn-shuffle.jar java.nio.file.FileSystemException: /Users/hasnain.lakhani/spark/common/network-yarn/target/exploded/META-INF/license/LICENSE.aix-netbsd.txt: Not a directory around Ant part ...<unzip src="/Users/hasnain.lakhani/spark/common/network-yarn/target/scala-2.13/spark-4.0.0-SNAPSHOT-yarn-shuffle.jar" dest="/Users/hasnain.lakhani/spark/common/network-yarn/target/exploded/" />... @ 5:198 in /Users/hasnain.lakhani/spark/common/network-yarn/target/antrun/build-main.xml ``` After much trial and error (I added a debugging section for the next person who runs into this) I realized the problem is that inside the jar, the META-inf folder looks like this: ``` META-INF/LICENSE META-INF/LICENSE.txt META-INF/license/ META-INF/license/LICENSE.aix-netbsd.txt META-INF/license/LICENSE.boringssl.txt META-INF/license/LICENSE.mvn-wrapper.txt META-INF/license/LICENSE.tomcat-native.txt ``` Since the filesystem on macosx is case insensitive, this causes issues. Unzipping this fails, as we unzip the file first, then do not create the directory, and fail to unzip the files after that. This PR proposes to fix it by removing the `LICENSE` file and keeping the `license` folder, which lets the unzipping proceed. The contents of the `LICENSE` file and`LICENSE.txt` are the same so that should be fine I hope license wise - we are still showing the same license in the output jar file. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI I reran ``` ./dev/make-distribution.sh --tgz -Phive -Phive-thriftserver -Pyarn ``` and it succeeded ### Was this patch authored or co-authored using generative AI tooling? No ### Appendix: debugging I was able to debug the contents of the jar using `jar tf $jar` and print out the files using `unzip -q -c $jar $path`. To make iterations go faster I went to `/Users/hasnain.lakhani/spark/common/network-yarn/target/antrun` and edited the ant file by hand then ran `ant -f build-main.xml` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
