[
https://issues.apache.org/jira/browse/BEAM-5931?focusedWorklogId=162086&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-162086
]
ASF GitHub Bot logged work on BEAM-5931:
----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Nov/18 19:38
Start Date: 02/Nov/18 19:38
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #6916: [BEAM-5931] Update
nexmark performance test with dataflow worker jar
URL: https://github.com/apache/beam/pull/6916
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/java/testing/nexmark/build.gradle
b/sdks/java/testing/nexmark/build.gradle
index 328a16912db..1944d638285 100644
--- a/sdks/java/testing/nexmark/build.gradle
+++ b/sdks/java/testing/nexmark/build.gradle
@@ -24,15 +24,22 @@ description = "Apache Beam :: SDKs :: Java :: Nexmark"
// When running via Gradle, this property can be used to pass commandline
arguments
// to the nexmark launch
def nexmarkArgsProperty = "nexmark.args"
-def nexmarkArgs = project.hasProperty(nexmarkArgsProperty) ?
- project.getProperty(nexmarkArgsProperty).split() : []
// When running via Gradle, this property sets the runner dependency
def nexmarkRunnerProperty = "nexmark.runner"
-def nexmarkRunnerDependency = (project.hasProperty(nexmarkRunnerProperty)
- ? project.getProperty(nexmarkRunnerProperty)
- : ":beam-runners-direct-java")
+def nexmarkRunnerDependency = project.findProperty(nexmarkRunnerProperty)
+ ?: ":beam-runners-direct-java"
def shouldProvideSpark = ":beam-runners-spark".equals(nexmarkRunnerDependency)
+def isDataflowRunner =
":beam-runners-google-cloud-dataflow-java".equals(nexmarkRunnerDependency)
+
+if (isDataflowRunner) {
+ /*
+ * We need to rely on manually specifying these evaluationDependsOn to
ensure that
+ * the following projects are evaluated before we evaluate this project.
This is because
+ * we are attempting to reference a property from the project directly.
+ */
+ evaluationDependsOn(":beam-runners-google-cloud-dataflow-java-legacy-worker")
+}
configurations {
// A configuration for running the Nexmark launcher directly from Gradle,
which
@@ -98,7 +105,21 @@ if (shouldProvideSpark) {
// -Pnexmark.args
// Specify the command line for invoking org.apache.beam.sdk.nexmark.Main
task run(type: JavaExec) {
+ def nexmarkArgsStr = project.findProperty(nexmarkArgsProperty) ?: ""
+
+ if (isDataflowRunner) {
+ dependsOn
":beam-runners-google-cloud-dataflow-java-legacy-worker:shadowJar"
+
+ def dataflowWorkerJar = project.findProperty('dataflowWorkerJar') ?:
project(":beam-runners-google-cloud-dataflow-java-legacy-worker").shadowJar.archivePath
+ // Provide job with a customizable worker jar.
+ // With legacy worker jar, containerImage is set to empty (i.e. to use the
internal build).
+ // More context and discussions can be found in PR#6694.
+ nexmarkArgsStr = nexmarkArgsStr +
+ " --dataflowWorkerJar=${dataflowWorkerJar} " +
+ " --workerHarnessContainerImage="
+ }
+
main = "org.apache.beam.sdk.nexmark.Main"
classpath = configurations.gradleRun
- args nexmarkArgs
+ args nexmarkArgsStr.split()
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 162086)
Time Spent: 1h 50m (was: 1h 40m)
> Rollback PR/6899
> ----------------
>
> Key: BEAM-5931
> URL: https://issues.apache.org/jira/browse/BEAM-5931
> Project: Beam
> Issue Type: Task
> Components: beam-model, runner-dataflow
> Reporter: Luke Cwik
> Assignee: Ruoyun Huang
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> To rollback this change, one must either:
> 1) Update nexmark / perf test framework to use Dataflow worker jar.
> This requires adding the
> {code}
> "--dataflowWorkerJar=${dataflowWorkerJar}",
> "--workerHarnessContainerImage=",
> {code}
> when running the tests.
> OR
> 2) Update the dataflow worker image with code that contains the rollback of
> PR/6899 and then rollback PR/6899 in Github with the updated Dataflow worker
> image.
> #1 is preferable since we will no longer have tests running that don't use a
> Dataflow worker jar built from Github HEAD.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
