holdenk commented on code in PR #57427:
URL: https://github.com/apache/spark/pull/57427#discussion_r3769358895
##########
dev/make-distribution.sh:
##########
@@ -240,7 +255,32 @@ if [ "$MAKE_PIP" == "true" ]; then
pushd "$SPARK_HOME/python" > /dev/null
# Delete the egg info file if it exists, this can cache older setup files.
rm -rf pyspark.egg-info || echo "No existing egg info file, skipping
deletion"
+ # Ship the Apache LICENSE and NOTICE inside the PySpark source distributions
+ # (see MANIFEST.in). These are removed again after the sdists are built.
+ #
+ # The classic pyspark sdist bundles the assembly jars
(packaging/classic/setup.py
+ # builds a deps/jars symlink farm), so it ships the binary LICENSE/NOTICE
that
+ # enumerate the bundled third-party jars' licenses, mirroring the binary
+ # distribution above. The connect and client sdists bundle no jars and ship
the
+ # plain source LICENSE/NOTICE.
+ if [ -e "$SPARK_HOME/LICENSE-binary" ]; then
+ cp "$SPARK_HOME/LICENSE-binary" LICENSE
+ cp "$SPARK_HOME/NOTICE-binary" NOTICE
+ else
+ cp "$SPARK_HOME/LICENSE" LICENSE
+ cp "$SPARK_HOME/NOTICE" NOTICE
+ fi
python3 setup.py sdist
+ # Guard against regressions: every PySpark sdist must contain LICENSE and
NOTICE
+ # at the package root. The missing files were only caught by a Spark 4.2.0
RC1
+ # vote -1 (SPARK-57393); fail the release build here instead of at vote time.
+ for f in dist/pyspark*.tar.gz; do
+ listing=$(tar tzf "$f")
+ for required in LICENSE NOTICE; do
+ grep -qE "^[^/]+/$required\$" <<< "$listing" || \
+ { echo "ERROR: $f is missing $required at the package root"; exit 1; }
+ done
+ done
Review Comment:
Good catch, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
