shane knapp created SPARK-22325:
-----------------------------------
Summary: SPARK_TESTING env variable breaking non-spark builds on
amplab jenkins
Key: SPARK-22325
URL: https://issues.apache.org/jira/browse/SPARK-22325
Project: Spark
Issue Type: Bug
Components: Build, Project Infra
Affects Versions: 2.2.0
Environment: riselab jenkins, all workers (ubuntu & centos)
Reporter: shane knapp
Priority: Critical
in the riselab jenkins master config, the SPARK_TESTING environment variable is
set to 1 and applied to all workers.
see:
https://amplab.cs.berkeley.edu/jenkins/view/RISELab%20Infra/job/testing-foo/9/console
(the 'echo 1' is actually 'echo $SPARK_TESTING')
this is problematic, as some of our lab builds are attempting to run pyspark as
part of the build process, and the hard-coded checks for SPARK_TESTING in the
setup scripts are causing hard failures.
see:
https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/2440/HADOOP_VERSION=2.6.2,SCALAVER=2.11,SPARK_VERSION=2.2.0,label=centos/consoleFull
i would strongly suggest that we do the following:
* remove the SPARK_TESTING environment variable declaration in the jenkins
config
* add the environment variable to each spark build config in github:
https://github.com/databricks/spark-jenkins-configurations/
* add the environment variable to SparkPullRequstBuilder and
NewSparkPullRequestBuilder
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
