[ https://issues.apache.org/jira/browse/SPARK-19592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Armin Braun updated SPARK-19592: -------------------------------- Affects Version/s: 2.2.0 > Duplication in Test Configuration Relating to SparkConf Settings Should be > Removed > ---------------------------------------------------------------------------------- > > Key: SPARK-19592 > URL: https://issues.apache.org/jira/browse/SPARK-19592 > Project: Spark > Issue Type: Improvement > Components: Tests > Affects Versions: 2.1.0, 2.2.0 > Environment: Applies to all Environments > Reporter: Armin Braun > Priority: Minor > > This configuration for Surefire, Scalatest is duplicated in the parent POM as > well as the SBT build. > While this duplication cannot be removed in general it can at least be > removed for all system properties that simply result in a SparkConf setting I > think. > Instead of having lines like > {code} > <spark.ui.enabled>false</spark.ui.enabled> > {code} > twice in the pom.xml > and once in SBT as > {code} > javaOptions in Test += "-Dspark.ui.enabled=false", > {code} > it would be a lot cleaner to simply have a > {code} > var conf: SparkConf > {code} > field in > {code} > org.apache.spark.SparkFunSuite > {code} > that has SparkConf set up with all the shared configuration that > `systemProperties` currently provide. Obviously this cannot be done straight > away given that > many subclasses of the parent suit do this, so I think it would be best to > simply add a method to the parent that provides this configuration for now > and start refactoring away duplication in other suit setups from there step > by step until the sys properties can be removed from the pom and sbt.build. > This makes the build a lot easier to maintain and makes tests more readable > by making the environment setup more explicit in the code. > (also it would allow running more tests straight from the IDE which is always > a nice thing imo) -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org