Eugene Kirpichov created BEAM-3261:
--------------------------------------
Summary: Apex runner does not detect pipeline failure
Key: BEAM-3261
URL: https://issues.apache.org/jira/browse/BEAM-3261
Project: Beam
Issue Type: Bug
Components: runner-apex
Reporter: Eugene Kirpichov
Assignee: Thomas Weise
Priority: Blocker
I was looking at https://github.com/apache/beam/pull/4074/files and asked
myself "Don't we already have a ValidatesRunner test for this?"
Turns out we do:
https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ViewTest.java#L793
I ran this test with TestApexRunner, and I observed the same exception as in
https://stackoverflow.com/questions/46982827/error-when-using-side-input-withsideinputs-method-not-accepting-kv-type-as-inp?noredirect=1#comment81040223_46982827
, however the test passed.
It seems that it is passing because ApexRunnerResult.waitUntilFinish() detects
only PAssert assertion errors, but not any other errors:
https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/apex/src/main/java/org/apache/beam/runners/apex/ApexRunnerResult.java#L62
This seems very problematic, as it means that 1) the status of Apex
ValidatesRunner tests cannot be trusted: if they fail in any other way other
than a PAssert error, the failure will be undetected, and 2) in general, a user
can not trust a successful pipeline.run() from Apex runner.
For tests in particular, some other TestXRunner's guard against such a failure
mode by verifying not only that there were no assertion failures, but also that
all assertions succeeded - using metrics: e.g.
https://github.com/apache/beam/blob/3b79b6298e84711528b5ad1302200cb8acbac07e/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/TestDataflowRunner.java#L248
. Fixing this would be optimal, but meanwhile, the runner should at least fail
the pipeline in case of error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
