jaketf commented on pull request #9590:
URL: https://github.com/apache/airflow/pull/9590#issuecomment-666470087
Good point on predictability.
Especially for those migrating from the old BQ Operator contract (of always
submit new job) it will be most predictable to have the same contract by
default.
I think parameterizing a `reattach_states: Set[str]` which defaults to empty
set feels really flexible this way user can say reattach to `{"PENDING",
"RUNNING"}` or `{"PENDING","RUNNING", "DONE"}`.
Unfortunately, I think we'd need yet another parameter for
`retry_on_reattach_to_error: bool` default to `True` (but won't have any affect
unless `"DONE" in self.reattach_states).
For context I believe this is the enum of job states
- PENDING
- RUNNING
- DONE
https://cloud.google.com/bigquery/docs/reference/auditlogs/rest/Shared.Types/JobState
This notably does not distinguish between SUCCEEDED and ERROR but that
requires checking the errorResult in jobstatus
https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobStatus
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
