johnhoran opened a new pull request, #60778: URL: https://github.com/apache/airflow/pull/60778
When running kubernetes pod operator in deferred mode we have implemented support for most callbacks, however the progress_callback one was still missing. This PR implements that callback from the triggerer, along with a fallback to the operator if any logs were missing from what the triggerer reports. When the callbacks were first added, the original commiter noted in https://github.com/apache/airflow/pull/35714 that > support is blocked by the way of serializing the trigger kwargs, where all the provided types and methods are serialized as a string, and it's a bit difficult to recreate the original object without a custom serializer or impacting the event loop (for example provide the class path and import it in the triggerer) We could probably do something a lot more complicated to serialize/deserialize the callbacks, but as the callbacks need to be static anyway, I'm just putting their module path as a string and importing when de-serializing the trigger There might be a need to do something more complicated, but I think this is better than nothing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
