#2 is what I’d go for — that way it’ll “natively” work easily for things like v1.Pod spec etc which is quite common when using KubeExec — not supporting this would involve a lot of changes to DAGs that I think is easy for us to avoid.
-ash > On 25 Feb 2025, at 14:18, Daniel Standish > <daniel.stand...@astronomer.io.INVALID> wrote: > > Hi we were just reminded of the fact that we still pickle executor config > (instead of force it to be json > > (or at least airlfow-json-serializer-serializable) > > Pickling occasionally causes trouble. It did with k8s exec pod overrides > when unpickling across airflow versions (and we resolved this by running > through airflow serializer). We've seen it with other weird things that > users do too. > > There are 3 paths > > 1. force users to use only valid json. so e.g. they would no longer be able > to supply raw k8s objects as executor config (something that we currently > support in pod override) > 2. less extreme, we just run the whole config through the airlfow json > serializer. this will handle k8s objects, and, various other things, but > not everything that users would toss in there > 3. do nothing and leave it so pod override is json encoded first then the > whole thing is pickled IIRC > > What say you? --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
