mobuchowski commented on issue #44984: URL: https://github.com/apache/airflow/issues/44984#issuecomment-2556807745
The solution should be to _not_ serialize any operators. Not exactly sure where the operator is coming from (I don't think it's this: https://github.com/apache/airflow/blob/490b5e816b804f338b0eb97f240ae874d4e15810/providers/src/airflow/providers/openlineage/utils/utils.py#L529). If we need something from operator, we should select the properties we need and pass them explicitely, rather than relying on pickle. Regarding ThreadPoolExecutor, we've switched from that solution since it caused even worse issues: https://github.com/apache/airflow/pull/39235 @paul-laffon-dd do you know where it's coming from? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
