#2 for me too. On Tue, Feb 25, 2025 at 3:23 PM Ash Berlin-Taylor <a...@apache.org> wrote:
> #2 is what I’d go for — that way it’ll “natively” work easily for things > like v1.Pod spec etc which is quite common when using KubeExec — not > supporting this would involve a lot of changes to DAGs that I think is easy > for us to avoid. > > -ash > > > On 25 Feb 2025, at 14:18, Daniel Standish > <daniel.stand...@astronomer.io.INVALID> wrote: > > > > Hi we were just reminded of the fact that we still pickle executor config > > (instead of force it to be json > > > > (or at least airlfow-json-serializer-serializable) > > > > Pickling occasionally causes trouble. It did with k8s exec pod overrides > > when unpickling across airflow versions (and we resolved this by running > > through airflow serializer). We've seen it with other weird things that > > users do too. > > > > There are 3 paths > > > > 1. force users to use only valid json. so e.g. they would no longer be > able > > to supply raw k8s objects as executor config (something that we currently > > support in pod override) > > 2. less extreme, we just run the whole config through the airlfow json > > serializer. this will handle k8s objects, and, various other things, but > > not everything that users would toss in there > > 3. do nothing and leave it so pod override is json encoded first then the > > whole thing is pickled IIRC > > > > What say you? > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
