Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22084#discussion_r209480769
--- Diff: dev/make-distribution.sh ---
@@ -188,6 +190,23 @@ if [ -f
"$SPARK_HOME"/common/network-yarn/target/scala*/spark-*-yarn-shuffle.jar
cp
"$SPARK_HOME"/common/network-yarn/target/scala*/spark-*-yarn-shuffle.jar
"$DISTDIR/yarn"
fi
+# Only copy external jars if built
+if [ -f
"$SPARK_HOME"/external/avro/target/spark-avro_${SCALA_VERSION}-${VERSION}.jar
]; then
+ cp
"$SPARK_HOME"/external/avro/target/spark-avro_${SCALA_VERSION}-${VERSION}.jar
"$DISTDIR/external/jars/"
+fi
+if [ -f
"$SPARK_HOME"/external/kafka-0-10/target/spark-streaming-kafka-0-10_${SCALA_VERSION}-${VERSION}.jar
]; then
+ cp
"$SPARK_HOME"/external/kafka-0-10/target/spark-streaming-kafka-0-10_${SCALA_VERSION}-${VERSION}.jar
"$DISTDIR/external/jars/"
--- End diff --
I didn't because the jars/ dir doesn't contain assemblies... well, it's
kind of the contents of an assembly. I assume the end-user here is someone who
wants to just consume the integration, to add to an app. The assembly would
pull back in all of Spark (?)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
