[
https://issues.apache.org/jira/browse/BEAM-2146?focusedWorklogId=496098&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496098
]
ASF GitHub Bot logged work on BEAM-2146:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Oct/20 18:31
Start Date: 06/Oct/20 18:31
Worklog Time Spent: 10m
Work Description: tysonjh commented on a change in pull request #12870:
URL: https://github.com/apache/beam/pull/12870#discussion_r500511794
##########
File path: runners/google-cloud-dataflow-java/build.gradle
##########
@@ -151,33 +161,34 @@ def commonExcludeCategories = [
// For the following test tasks using legacy worker, set
workerHarnessContainerImage to empty to
// make Dataflow pick up the non-versioned container image, which handles a
staged worker jar.
-task validatesRunnerLegacyWorkerTest(type: Test) {
- group = "Verification"
- dependsOn
":runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar"
-
- systemProperty "beamTestPipelineOptions", JsonOutput.toJson([
- "--runner=TestDataflowRunner",
- "--project=${dataflowProject}",
- "--region=${dataflowRegion}",
- "--tempRoot=${dataflowValidatesTempRoot}",
- "--dataflowWorkerJar=${dataflowLegacyWorkerJar}",
- "--workerHarnessContainerImage=",
- ])
-
- // Increase test parallelism up to the number of Gradle workers. By default
this is equal
- // to the number of CPU cores, but can be increased by setting
--max-workers=N.
- maxParallelForks Integer.MAX_VALUE
- classpath = configurations.validatesRunner
- testClassesDirs =
files(project(":sdks:java:core").sourceSets.test.output.classesDirs) +
- files(project(project.path).sourceSets.test.output.classesDirs)
- useJUnit {
- includeCategories 'org.apache.beam.sdk.testing.ValidatesRunner'
- commonExcludeCategories.each {
- excludeCategories it
+def createValidatesRunnerTest = { name, pipelineOptions=legacyPipelineOptions,
disabledTests=[] ->
Review comment:
I meant the name of this method specifically, not the tasks. This method
has a dependsOn dependency with the legacy worker, so it will always be for the
legacy worker and the naming change seems to make this less clear. Unless of
course, in the future, this method could be used for new runners (e.g. runner
v2)?
I was considering if it would possible to pull out the `dependsOn
...legacy-worker:shadowJar` (by passing it in as a parameter, specifying the
dependency in some other scope, or some other injection mechanism) and then
reusing the `createValidatesRunnerTest` method for the runner v2 by passing in
`pipelineOptions + ["--experiments=use_runner_v2"]`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 496098)
Time Spent: 1h 20m (was: 1h 10m)
> Continuously run ValidatesRunner suite on Dataflow forcing streaming mode
> -------------------------------------------------------------------------
>
> Key: BEAM-2146
> URL: https://issues.apache.org/jira/browse/BEAM-2146
> Project: Beam
> Issue Type: New Feature
> Components: runner-dataflow
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
> Priority: P2
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{ValidatesRunner}} test suite works on Dataflow with streaming mode
> forced, and just needs CI setup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
