[
https://issues.apache.org/jira/browse/BEAM-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmet Altay resolved BEAM-5529.
-------------------------------
Resolution: Fixed
Fix Version/s: 2.8.0
> Dataflow runner raises AssertionError if job takes > 50 seconds to go from
> PENDING to RUNNING
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-5529
> URL: https://issues.apache.org/jira/browse/BEAM-5529
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.6.0
> Reporter: Joar Wandborg
> Assignee: Ahmet Altay
> Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> To reproduce:
> - Try to run a Pipeline that takes longer than 50 seconds to go from
> JOB_STATE_PENDING to JOB_STATE_RUNNING using "with Pipeline() as pipeline:"
> (automatically executes {{pipeline.run().wait_until_finish()}} via
> {{Pipeline.__exit__}}) on Dataflow.
> - Your script will raise "AssertionError: Job did not reach to a terminal
> state after waiting indefinitely." (from
> [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L1152-L1153])
> I think the issue is that the block at
> [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L147-L160]
> unintentionally exits if it takes > 50 seconds for a job to go from
> JOB_STATE_PENDING to JOB_STATE_RUNNING.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
