uranusjr commented on issue #47832: URL: https://github.com/apache/airflow/issues/47832#issuecomment-2728399542
OK so there are three separate problems here. First, we don’t support syntax like `quotes=raw_quotes` in the arguments (you’re supposed to just reference the asset by argument name like in pytest), so this should fail on parse time. We need to add a check in the dag parser. Second, when you try to reference an asset that does not exist (here `quote` because of the previous point), AssetOperations does not handle 404 correct and thus crash the entire task. We should handle this more gracefully (although it should probably still make the task fail). Third (the exception in Rahul’s comment), when a task is failed, the scheduler calls `_handle_failure`, which tries to get the ti context, but during the ti context creation process it tries to use the supervisor. The supervisor is not available in the scheduler, and the entire thing crumbles down. I’m not sure how to handle this one though. Should this handler be moved into the task runner? Or should we add some logic so the supervisor is usable in the scheduler (but actually just goes to the db directly instead)? Need some input from the sdk folks here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
