MatrixManAtYrService opened a new issue #20559: URL: https://github.com/apache/airflow/issues/20559
### Apache Airflow version 2.2.3 (latest released) ### What happened I wanted to improve the [Params concepts doc](https://airflow.apache.org/docs/apache-airflow/stable/concepts/params.html) to show how one can reference the params from a task. While doing so, I tried to run that DAG by clicking "Trigger" (i.e. I didn't opt to modify the params first). Then I saw this: ``` Oops. Something bad has happened. ... Python version: 3.9.9 Airflow version: 2.2.3+astro.1 Node: df9204fff6e6 ------------------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/airflow/models/param.py", line 62, in __init__ jsonschema.validate(self.value, self.schema, format_checker=FormatChecker()) File "/usr/local/lib/python3.9/site-packages/jsonschema/validators.py", line 934, in validate raise error jsonschema.exceptions.ValidationError: 'NoValueSentinel' is too long Failed validating 'maxLength' in schema: {'maxLength': 4, 'minLength': 2, 'type': 'string'} On instance: 'NoValueSentinel' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python3.9/site-packages/airflow/www/auth.py", line 51, in decorated return func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/www/decorators.py", line 72, in wrapper return f(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper return func(*args, session=session, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/www/views.py", line 1650, in trigger dag = current_app.dag_bag.get_dag(dag_id) File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper return func(*args, session=session, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/models/dagbag.py", line 186, in get_dag self._add_dag_from_db(dag_id=dag_id, session=session) File "/usr/local/lib/python3.9/site-packages/airflow/models/dagbag.py", line 261, in _add_dag_from_db dag = row.dag File "/usr/local/lib/python3.9/site-packages/airflow/models/serialized_dag.py", line 180, in dag dag = SerializedDAG.from_dict(self.data) # type: Any File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 947, in from_dict return cls.deserialize_dag(serialized_obj['dag']) File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 861, in deserialize_dag v = {task["task_id"]: SerializedBaseOperator.deserialize_operator(task) for task in v} File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 861, in <dictcomp> v = {task["task_id"]: SerializedBaseOperator.deserialize_operator(task) for task in v} File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 641, in deserialize_operator v = cls._deserialize_params_dict(v) File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 459, in _deserialize_params_dict op_params[k] = cls._deserialize_param(v) File "/usr/local/lib/python3.9/site-packages/airflow/serialization/serialized_objects.py", line 439, in _deserialize_param return class_(**kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/models/param.py", line 64, in __init__ raise ValueError(err) ValueError: 'NoValueSentinel' is too long Failed validating 'maxLength' in schema: {'maxLength': 4, 'minLength': 2, 'type': 'string'} On instance: 'NoValueSentinel' ``` I started with the dag in the docs, but ended up making some tweaks. Here's how it was when I saw the error. ``` from airflow import DAG from airflow.models.param import Param from airflow.operators.python import PythonOperator from datetime import datetime with DAG( "my_dag", start_date=datetime(1970, 1, 1), schedule_interval=None, params={ # a int param with default value "int_param": Param(10, type="integer", minimum=0, maximum=20), # a mandatory str param "str_param": Param(type="string", minLength=2, maxLength=4), # a param which can be None as well "dummy": Param(type=["null", "number", "string"]), # no data or type validations "old": "old_way_of_passing", # no data or type validations "simple": Param("im_just_like_old_param"), "email": Param( default="[email protected]", type="string", format="idn-email", minLength=5, maxLength=255, ), }, ) as the_dag: def print_these(*params): for param in params: print(param) PythonOperator( task_id="ref_params", python_callable=print_these, op_args=[ # you can modify them in jinja templates "{{ params.int_param + 10 }}", # or just leave them as-is "{{ params.str_param }}", "{{ params.dummy }}", "{{ params.old }}", "{{ params.simple }}", "{{ params.email }}", ], ) ``` ### What you expected to happen If there's a required parameter, and I try to trigger a dagrun, I should get a friendly warning explaining what I've done wrong. ### How to reproduce Run the dag above ### Operating System docker / debian ### Versions of Apache Airflow Providers n/a ### Deployment Astronomer ### Deployment details `astro dev start` Dockerfile: ``` FROM quay.io/astronomer/ap-airflow:2.2.3-onbuild ``` ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
