shaneknapp commented on issue #25423: [SPARK-28701][test-java11][k8s] adding java11 support for pull request builds URL: https://github.com/apache/spark/pull/25423#issuecomment-521761223 > I meant both Maven and SBT need the same treatment, to use Java 11 for tests (only). The commented-out part here would only affect SBT. Rather than set env variables, I'd add `-Djava.version=11` to the command line, perhaps in `run_scala_tests`. And this is where you'd set `JAVA_HOME`, IMHO. (Probably we get away with just this as everything before will be Java 8, and nothing else runs after tests.) > got it... i think my commit c4ddeff covers this for the PRB. > Yes I'm wondering if we have some duplication or conflict between what is the Jenkins job config, the bash commands it runs, and what is in `dev/run-tests.py`. I'd imagine we'd prefer to keep the complexity in the latter script. i honestly don't think there's any really bad duplication or conflict here. the only "bad" stuff going on is really how many times we set environment variables... > I can't see Jenkins right now but I do recall it has a fair bit of logic including a `mvn` build command. But then I wonder why the `run-tests.py` script appears to also build Spark. I may well be missing something but is that redundant? It may not be, like, maybe it's in order to be able to run style tests first or something? But at the least it makes maintenance harder. > the PRB only explicitly executes two things: 1) `dev/run-tests-jenkins` (which is a wrapper for `run-tests-jenkins.py`)[1] 2) `build/sbt unsafe/test > /dev/null` [1] -- `dev/run-tests-jenkins` launches `dev/run-tests` (which is also a wrapper for `run-tests.py`. btw `dev/run-tests` definitely builds spark (see the `build_apache_spark` function). > We need not solve that right now, just want to make sure we're making the right changes in the right places before deciding if this works or not. absolutely. as you can tell, maintenance is already a nightmare. :)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
