mwalenia commented on a change in pull request #10878: Implement precommit
portability test to run on java 11
URL: https://github.com/apache/beam/pull/10878#discussion_r386954174
##########
File path: runners/google-cloud-dataflow-java/examples/build.gradle
##########
@@ -97,6 +97,42 @@ task verifyPortabilityApi() {
dependsOn verifyFnApiWorker
}
+def passedDockerImageName = project.findProperty('passedDockerImageName')
+def java8Home = project.findProperty('java8Home')
+def java11Home = project.findProperty('java11Home')
+
+task runFnApiTest(type: Test) {
+ def dataflowWorkerJar = project.findProperty('dataflowWorkerJar') ?:
project(":runners:google-cloud-dataflow-java:worker").shadowJar.archivePath
+ with commonConfig(dataflowWorkerJar, passedDockerImageName,
["--experiments=${fnapiExperiments}"])
+ useJUnit {
+ excludeCategories 'org.apache.beam.sdk.testing.StreamingIT'
+ }
+}
+
+task verifyFnApiWorkerJava11(type: Test) {
+ dependsOn ":runners:google-cloud-dataflow-java:worker:shadowJar"
+ dependsOn ":runners:google-cloud-dataflow-java:buildAndPushDockerContainer"
+ doLast {
+ exec {
+ workingDir "${rootProject.projectDir}"
+ commandLine "./gradlew", "${project.path}:runFnApiTest",
"-Pjava8Home=${java8Home}", "-PcompileWithJava8",
"-PpassedDockerImageName=${dockerImageName}", "-x shadowJar", "-x
shadowTestJar", "-x jar", "-x testJar", "-x classes", "-x testClasses",
"-PdisableSpotlessCheck=true", "-Dorg.gradle.java.home=${java11Home}"
Review comment:
@lukecwik
> Anything that relies on independent gradle executions needs to perform all
the artifact copying/moving between executions to ensure that output isn't
overwritten/corrupted between Gradle runs.
This is why the second Gradle command is started with explicit exclusions of
all compilation tasks. The worst-case scenario is that somehow the built
sources aren't good enough and the task fails.
We researched multiple ways of doing what is done in this PR - in a
nutshell, Gradle doesn't seem to be well suited for JVM manipulations in
subsequent stages of a task. Best we could do was a hack like this one or like
in
[job_PostCommit_Java_ValidatesRunner_Dataflow_Java11.groovy](https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_Java11.groovy).
If you know of a way of such manipulation that would be more elegant, please
let us know - we're painfully aware that this is suboptimal and error-prone.
Still, to my knowledge it is still the only way to make it work.
Also tagging @kennknowles - if you know something, please share :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
