casassg commented on pull request #12569:
URL: https://github.com/apache/airflow/pull/12569#issuecomment-735959212
`Cloudpickle can only be used to send objects between the exact same version
of Python.`
I tried to find if there's a compilation of compatible versions or how often
does it break (aka does it need to be on re-release).
Regarding your code example, it would indeed not work. But you can teach
users to make sure to only use scoped variables. So it comes off as flexibility
to create complex functions vs compatibility between different environments. I
believe each option has its issue.
My view is that you more often will find yourself wanting to re-execute old
DAGs (after upgrading) than want to make complex functions with weird scopes.
The alternative is to allow both options and let user decide what's used
underneath.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
