Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/16451#discussion_r94564162
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala ---
@@ -339,10 +339,15 @@ class HiveSparkSubmitSuite
private def runSparkSubmit(args: Seq[String]): Unit = {
val sparkHome = sys.props.getOrElse("spark.test.home",
fail("spark.test.home is not set!"))
val history = ArrayBuffer.empty[String]
- val commands = Seq("./bin/spark-submit") ++ args
+ val sparkSubmit = if (Utils.isWindows) {
+ new File("..\\..\\bin\\spark-submit.cmd").getAbsolutePath
+ } else {
+ new File("../../bin/spark-submit").getAbsolutePath
+ }
+ val commands = Seq(sparkSubmit) ++ args
val commandLine = commands.mkString("'", "' '", "'")
- val builder = new ProcessBuilder(commands: _*).directory(new
File(sparkHome))
--- End diff --
`ProcessBuilder.directory` seems not changing the working directory on
Windows. I verified this with the codes below:
```scala
import scala.io.Source
import java.lang.ProcessBuilder
import java.io.File
val sparkHome = "your-spark-home"
val process = new ProcessBuilder(".\\bin\\spark-submit.cmd").directory(new
File(sparkHome)).start()
process.waitFor()
Source.fromInputStream(process.getInputStream()).getLines().mkString("\n")
```
This code path resembles `org.apache.spark.deploy.SparkSubmitSuite` and the
test codes there already use relative paths.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
