[
https://issues.apache.org/jira/browse/BEAM-6148?focusedWorklogId=170756&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-170756
]
ASF GitHub Bot logged work on BEAM-6148:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Nov/18 18:25
Start Date: 29/Nov/18 18:25
Worklog Time Spent: 10m
Work Description: aaltay closed pull request #7158: [BEAM-6148] ptest
support arbitrary runners
URL: https://github.com/apache/beam/pull/7158
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/go/pkg/beam/testing/ptest/ptest.go
b/sdks/go/pkg/beam/testing/ptest/ptest.go
index 9557e3dd88b3..008591b08ee8 100644
--- a/sdks/go/pkg/beam/testing/ptest/ptest.go
+++ b/sdks/go/pkg/beam/testing/ptest/ptest.go
@@ -18,9 +18,14 @@ package ptest
import (
"context"
+ "flag"
+ "os"
+ "testing"
"github.com/apache/beam/sdks/go/pkg/beam"
- "github.com/apache/beam/sdks/go/pkg/beam/runners/direct"
+
+ // ptest uses the direct runner to execute pipelines by default.
+ _ "github.com/apache/beam/sdks/go/pkg/beam/runners/direct"
)
// TODO(herohde) 7/10/2017: add hooks to verify counters, logs, etc.
@@ -53,8 +58,42 @@ func CreateList2(a, b interface{}) (*beam.Pipeline,
beam.Scope, beam.PCollection
return p, s, beam.CreateList(s, a), beam.CreateList(s, b)
}
+// Runner is a flag that sets which runner pipelines under test will use.
+//
+// The test file must have a TestMain that calls Main or MainWithDefault
+// to function.
+var Runner = flag.String("runner", "direct", "Pipeline runner.")
+
// Run runs a pipeline for testing. The semantics of the pipeline is expected
// to be verified through passert.
func Run(p *beam.Pipeline) error {
- return direct.Execute(context.Background(), p)
+ return beam.Run(context.Background(), *Runner, p)
+}
+
+// Main is an implementation of testing's TestMain to permit testing
+// pipelines on runners other than the direct runner.
+//
+// To enable this behavior, _ import the desired runner, and set the flag
accordingly.
+//
+// func TestMain(m *testing.M) {
+// ptest.Main(m)
+// }
+//
+func Main(m *testing.M) {
+ MainWithDefault(m, "direct")
+}
+
+// MainWithDefault is an implementation of testing's TestMain to permit testing
+// pipelines on runners other than the direct runner, while setting the default
+// runner to use.
+func MainWithDefault(m *testing.M, runner string) {
+ if !flag.Parsed() {
+ flag.Parse()
+ }
+ // Override if not otherwise set.
+ if *Runner == "direct" {
+ *Runner = runner
+ }
+ beam.Init()
+ os.Exit(m.Run())
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 170756)
Time Spent: 0.5h (was: 20m)
> Support Go "Unit" tests on arbitrary runners
> --------------------------------------------
>
> Key: BEAM-6148
> URL: https://issues.apache.org/jira/browse/BEAM-6148
> Project: Beam
> Issue Type: New Feature
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There's no clear path to testing pipelines on runners other than the direct
> runner. It should be possibly to "redirect" tests to use a runner of choice.
> This would enable more "testy" ValidatesRunner tests in Go.
>
> In particular, users should need to at least _ import the runner they want,
> and be able to set a flag.
> The tricky bit is ensuring beam.Init is called so that each individual test
> can convert to WorkerMode when it's spun up as a SDK harness. This can be
> done by having a TestMain.
> ptest should provide convenience functions to help with this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
