Emil Ejbyfeldt created SPARK-38502: -------------------------------------- Summary: Distribution with hadoop-provided and log4j2 Key: SPARK-38502 URL: https://issues.apache.org/jira/browse/SPARK-38502 Project: Spark Issue Type: Bug Components: Build Affects Versions: 3.3.0 Reporter: Emil Ejbyfeldt
Currently building spark 3.3.0-SNAPSHOT using `./dev/make-distribution.sh --tgz --name hadoop-provided-test -Phadoop-provided -Pyarn` script will build a package that does not included log4j2. Trying to run spark-submit with the latest hadoop release 3.3.2 and this build will result in ``` $ spark-submit run-example org.apache.spark.examples.SparkPi Error: Unable to initialize main class org.apache.spark.deploy.SparkSubmit Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Filter ``` Since log4j2 is not found. So I believe the maven build settings needs to be tweaked so that log4j2 is included in the spark distribution. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org