amoghrajesh commented on PR #58992: URL: https://github.com/apache/airflow/pull/58992#issuecomment-3625470596
> Should not serialzation be moved to `shared` - that sounds like a most reasonable place to have it - then we will be able to use it anywhere. Is there a particular reason we want to do this weird dance of whether task-sdk is installed or not? Yes. There is a reason to do it actually. This is actually splitting serialization properly as it should rightfully be and it separates things rightly towards the future direction. So, serialization is in two parts (somewhat messy, but let me still explain): 1. Serialization related to DAG: this should certainly be in airflow-core and it is going to continue to be there (everything but serde in here: https://github.com/apache/airflow/tree/main/airflow-core/src/airflow/serialization) 2. Serde: this is a module that should ONLY belong at execution time, and I think moving serde out of `airflow.serialization` is on the right track in a reorganised way. The rest of the package (except serde) is to deal with dag serialisation, and should probably be renamed/moved to reflect that (later by me). So this loading of task-sdk will only happen when we try to deserialise an object relevant to `airflow.sdk` which was pushed during task execution anyways (so task sdk is installed) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
