junaiddshaukat opened a new pull request, #39362:
URL: https://github.com/apache/beam/pull/39362
Adds `TestKafkaStreamsRunner`, a `PipelineRunner` that `TestPipeline`-based
suites -- including Beam's `@ValidatesRunner` tests -- can dispatch to via
`--runner`. Part of #18479, second step toward the @ValidatesRunner suite
(after the metrics PR #39341).
- Fills in the options the translation needs (a unique `applicationId`, the
EMBEDDED harness) on the pipeline's options, translates and drives the
topology to quiescence through `TopologyTestDriver`, and returns a terminal
`PipelineResult` exposing the harness metrics.
- `TestPipeline.run()` verifies through those metrics that every `PAssert`
actually ran and succeeded (`verifyPAssertsSucceeded` counts
`PAssert.SUCCESS_COUNTER`) -- this is what the metrics PR was for.
- A failing `PAssert` fails the run with the assertion text in the error.
Over
the Fn API a DoFn failure travels as an error string rather than a Java
`AssertionError` object, so the metrics count check is what catches
silently
dropped assertions (e.g. a watermark bug suppressing the assertion DoFn).
Tests: `containsInAnyOrder` on `Create` (PAssert's GBK + Flatten bootstrap
materialization, no side inputs), an assertion on `Count.perElement()`
(Combine.perKey works today via its GBK + ParDo expansion in the harness),
and
the failing-assertion case.
Next up: the `validatesRunner` gradle task enabling Beam's suite (with a
sickbay list), following the Flink `createValidatesRunnerTask` pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
