Hi we were just reminded of the fact that we still pickle executor config (instead of force it to be json
(or at least airlfow-json-serializer-serializable) Pickling occasionally causes trouble. It did with k8s exec pod overrides when unpickling across airflow versions (and we resolved this by running through airflow serializer). We've seen it with other weird things that users do too. There are 3 paths 1. force users to use only valid json. so e.g. they would no longer be able to supply raw k8s objects as executor config (something that we currently support in pod override) 2. less extreme, we just run the whole config through the airlfow json serializer. this will handle k8s objects, and, various other things, but not everything that users would toss in there 3. do nothing and leave it so pod override is json encoded first then the whole thing is pickled IIRC What say you?
