dstandish commented on issue #19096: URL: https://github.com/apache/airflow/issues/19096#issuecomment-953329485
OK so, I found the source of the the problem but need to look into it more to see what we need to do. Serialization blows up here because it's not json encodable: https://github.com/apache/airflow/blob/main/airflow/models/serialized_dag.py#L97 And the reason why is the set is not serialized but left as a set. And the reason why this happens might be here: https://github.com/apache/airflow/blob/main/airflow/serialization/serialized_objects.py#L420 It doesn't use `cls._serialize`. We could change it to do so, but then we wouldn't be able to deserialize it back to set. I am not sure why we have a separate process for params. Maybe we can somehow unify them. In any case I have to look into this more to see how we might be able to ensure that params are properly serialized. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
