[
https://issues.apache.org/jira/browse/BEAM-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218444#comment-15218444
]
Davor Bonaci commented on BEAM-153:
-----------------------------------
I think it makes sense for streaming pipelines too.
This certainly can be generalized, but it would be easier to make it
runner-specific for now. Service is the one who can do TTL on a job -- hacking
around BlockingDataflowPipelineRunner is not a particularly great solution.
> Support timeout in runner API
> -----------------------------
>
> Key: BEAM-153
> URL: https://issues.apache.org/jira/browse/BEAM-153
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Eugene Kirpichov
>
> Some users want to make sure that their pipeline doesn't run longer than X
> minutes (e.g. because sometimes it runs longer than that due to bugs, and in
> that case they'd rather auto-cancel it than incur the costs).
> The runner API should have a timeout option, so that if the pipeline isn't in
> a terminal state by then, it is automatically cancelled.
> Naturally, this only applies to batch pipelines.
> A simple way to implement this for a blocking runner (such as
> BlockingDataflowPipelineRunner) is a wrapper of the sort "start pipeline, and
> cancel it after timeout" inside run(). For a non-blocking runner this will
> require support on the underlying execution environment side.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
