Prior to 2.2.0, you could use non-json-serializable params in a dag. Here's
an example with `set`:

@dag.task(params={"a": {1, 2, 3}, "b": [3, 4, 5]})
def set_param(intersection):
    print(intersection)


set_param("{{ params.a.intersection(params.b).pop() }}")

In 2.2.0 this was broken, and in 2.2.2rc1 it *should be* fixed
<https://github.com/apache/airflow/pull/19267>.

There's a problem though.  An important part of params is the ability to
override them when triggering a dag from the web UI or CLI.  But params
supplied through either mechanism should be json-serializable.

So on one hand we allow arbitrary params, and on the other hand we require
them to be json serializable.  We can see how this causes a problem in the
above example: if you provide `a` as a `list` in the UI, it won't have the
method `intersection`.

So the question: should we deprecate non-json-serializable params?

My feeling is yes.  But I'm not sure how widely this flexibility is used in
the wild.

Reply via email to