dstandish opened a new pull request, #40867: URL: https://github.com/apache/airflow/pull/40867
Shipping messages to TI logs, which unfortunately in many cases means uploading to s3, is inefficient, and not really something the scheduler should be doing. Instead we can use an old but underutilized feature: the Log table. I have to do some trickery to save log events from executors, namely store them in a queue and consume them in the scheduler loop. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
