Jozef Vilcek created BEAM-6484:
----------------------------------
Summary: Spark's pipeline result throws exceptions
Key: BEAM-6484
URL: https://issues.apache.org/jira/browse/BEAM-6484
Project: Beam
Issue Type: Bug
Components: runner-spark
Reporter: Jozef Vilcek
Assignee: Amit Sela
Spark's pipeline result seems to allow throwing Exceptions from
`waitUntilFinish()` methods.
[https://github.com/apache/beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L101]
Documentation on PipelineResult does not mention this possibility. As a result,
when I am running pipelines from Scio, it does swallow such exceptions because
if does not expect them
[https://github.com/spotify/scio/issues/1617]
Should the Spark Runner just log those exceptions and update it's state of is
documentation about `PipelineResult.waitUntilFinish()` not up to date?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
