mobuchowski opened a new pull request, #41690: URL: https://github.com/apache/airflow/pull/41690
We've moved to a model where DagRun level events utilize ProcessPoolExecutor to send data, to not do this on the direct scheduler process. This PR adds more guards to that model: - do not send/pickle whole DagRun model to process, since serializing only selected attributes is cheaper and less error prone - add notification on ProcessPoolExecutor complete - this allows listener to log potential serialization failures. - guard whole listener with BaseException checks - we don't want errors to bubble to scheduler. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
