Lee-W commented on PR #39050:
URL: https://github.com/apache/airflow/pull/39050#issuecomment-2060209595
> Thanks @Lee-W , also @Lee-W one more question, we use deferrable airflow
operator, initially we use runnow operator, somehow in deferrable operator, we
always get same exception `Exception: Job run failed with terminal state:
{'life_cycle_state': 'INTERNAL_ERROR', 'result_state': 'FAILED',
'state_message': 'Task FLAT_CREATION_pl_cafe_inventory_balance failed with
message: Workload failed, see run output for details. This caused all
downstream tasks to get skipped.'}` using `context.get('exception')`, but same
was working with exact job error from databricks, is this also fixed or is it
expected behaviour in deferrable mode or can I fix it, just want to know
whether you have any context for same or not
Might need more information to understand why it fails 🤔 I guess the
triggerer component works fine. Otherwise, you won't be able to see this
message.
[This](https://github.com/apache/airflow/blob/57c6a4bfdcee03ac11d7e1aff167a91453acceb8/airflow/providers/databricks/hooks/databricks.py#L409)
is how it works under the hook. Maybe we could test it directly and see
whether it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
