paul-laffon-dd commented on issue #44984: URL: https://github.com/apache/airflow/issues/44984#issuecomment-2556775523
Thanks @kacpermuda From my understanding of the issue there is one argument of the [dag_started](https://github.com/apache/airflow/blob/providers-openlineage/1.14.0/providers/src/airflow/providers/openlineage/plugins/adapter.py#L326) that is trying to serialize an operator where the `get_parse_time_mapped_ti_count` has already been computed and cached. it's unclear to me which facet is holding this operator What do you think of switching to a `ThreadPoolExecutor` instead of a [ProcessPoolExecutor](https://github.com/apache/airflow/blob/providers-openlineage/1.14.0/providers/src/airflow/providers/openlineage/plugins/listener.py#L398) ? This would eliminate the need for serialization while still allowing asynchronous execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
