potiuk commented on issue #34013: URL: https://github.com/apache/airflow/issues/34013#issuecomment-1704411889
For me it looks like something indeed takes a lot of memory for the workers but what it would be - hard to say. Is it possible to run your workers with memray (https://github.com/bloomberg/memray) ? It would be great if it could produce flamechart with peak memory usage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
