[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for
newly added fields in BaseOperator
URL: https://github.com/apache/airflow/pull/7162#discussion_r367413700
##
File path: tests/serialization/test_dag_serialization.py
##
@@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self):
dag_params: set = set(dag_schema.keys()) - ignored_keys
self.assertEqual(set(DAG.get_serialized_fields()), dag_params)
+def test_no_new_fields_added_to_base_operator(self):
+"""
+This test verifies that there are no new fields added to BaseOperator.
And reminds that
+tests should be added for it.
+"""
+base_operator = BaseOperator(task_id="10")
+fields = base_operator.__dict__
+self.assertEqual({'_dag': None,
+ '_downstream_task_ids': set(),
+ '_inlets': [],
+ '_log': base_operator.log,
+ '_outlets': [],
+ '_upstream_task_ids': set(),
+ 'depends_on_past': False,
+ 'do_xcom_push': True,
+ 'email': None,
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'end_date': None,
+ 'execution_timeout': None,
+ 'executor_config': {},
+ 'inlets': [],
+ 'max_retry_delay': None,
+ 'on_execute_callback': None,
+ 'on_failure_callback': None,
+ 'on_retry_callback': None,
+ 'on_success_callback': None,
+ 'outlets': [],
+ 'owner': 'airflow',
+ 'params': {},
+ 'pool': 'default_pool',
+ 'priority_weight': 1,
+ 'queue': 'default',
+ 'resources': None,
+ 'retries': 0,
+ 'retry_delay': timedelta(0, 300),
+ 'retry_exponential_backoff': False,
+ 'run_as_user': None,
+ 'sla': None,
+ 'start_date': None,
+ 'subdag': None,
+ 'task_concurrency': None,
+ 'task_id': '10',
+ 'trigger_rule': 'all_success',
+ 'wait_for_downstream': False,
+ 'weight_rule': 'downstream'}, fields,
+ """
+!!!
+
+ ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY
+
+ Some fields were added to the BaseOperator! Please add them to the list above
and make sure that
+ you add support for DAG serialization - you should add the field to
+ `airflow/serialization/schema.json` - they should have correct type defined
there.
+
+ Note that we do not support versioning yet so you should only add optional
fields. We do not support
Review comment:
```suggestion
Note that we do not support versioning yet so you should only add optional
fields.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for
newly added fields in BaseOperator
URL: https://github.com/apache/airflow/pull/7162#discussion_r367413700
##
File path: tests/serialization/test_dag_serialization.py
##
@@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self):
dag_params: set = set(dag_schema.keys()) - ignored_keys
self.assertEqual(set(DAG.get_serialized_fields()), dag_params)
+def test_no_new_fields_added_to_base_operator(self):
+"""
+This test verifies that there are no new fields added to BaseOperator.
And reminds that
+tests should be added for it.
+"""
+base_operator = BaseOperator(task_id="10")
+fields = base_operator.__dict__
+self.assertEqual({'_dag': None,
+ '_downstream_task_ids': set(),
+ '_inlets': [],
+ '_log': base_operator.log,
+ '_outlets': [],
+ '_upstream_task_ids': set(),
+ 'depends_on_past': False,
+ 'do_xcom_push': True,
+ 'email': None,
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'end_date': None,
+ 'execution_timeout': None,
+ 'executor_config': {},
+ 'inlets': [],
+ 'max_retry_delay': None,
+ 'on_execute_callback': None,
+ 'on_failure_callback': None,
+ 'on_retry_callback': None,
+ 'on_success_callback': None,
+ 'outlets': [],
+ 'owner': 'airflow',
+ 'params': {},
+ 'pool': 'default_pool',
+ 'priority_weight': 1,
+ 'queue': 'default',
+ 'resources': None,
+ 'retries': 0,
+ 'retry_delay': timedelta(0, 300),
+ 'retry_exponential_backoff': False,
+ 'run_as_user': None,
+ 'sla': None,
+ 'start_date': None,
+ 'subdag': None,
+ 'task_concurrency': None,
+ 'task_id': '10',
+ 'trigger_rule': 'all_success',
+ 'wait_for_downstream': False,
+ 'weight_rule': 'downstream'}, fields,
+ """
+!!!
+
+ ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY
+
+ Some fields were added to the BaseOperator! Please add them to the list above
and make sure that
+ you add support for DAG serialization - you should add the field to
+ `airflow/serialization/schema.json` - they should have correct type defined
there.
+
+ Note that we do not support versioning yet so you should only add optional
fields. We do not support
Review comment:
```suggestion
Note that we do not support versioning yet so you should only add optional
fields.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for
newly added fields in BaseOperator
URL: https://github.com/apache/airflow/pull/7162#discussion_r367413148
##
File path: tests/serialization/test_dag_serialization.py
##
@@ -543,6 +543,68 @@ def test_dag_serialized_fields_with_schema(self):
dag_params: set = set(dag_schema.keys()) - ignored_keys
self.assertEqual(set(DAG.get_serialized_fields()), dag_params)
+def test_no_new_fields_added_to_base_operator(self):
+"""
+This test verifies that there are no new fields added to BaseOperator.
And reminds that
+tests should be added for it.
+"""
+base_operator = BaseOperator(task_id="10")
+fields = base_operator.__dict__
+self.assertEqual({'_dag': None,
+ '_downstream_task_ids': set(),
+ '_inlets': [],
+ '_log': base_operator.log,
+ '_outlets': [],
+ '_upstream_task_ids': set(),
+ 'depends_on_past': False,
+ 'do_xcom_push': True,
+ 'email': None,
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'end_date': None,
+ 'execution_timeout': None,
+ 'executor_config': {},
+ 'inlets': [],
+ 'max_retry_delay': None,
+ 'on_execute_callback': None,
+ 'on_failure_callback': None,
+ 'on_retry_callback': None,
+ 'on_success_callback': None,
+ 'outlets': [],
+ 'owner': 'airflow',
+ 'params': {},
+ 'pool': 'default_pool',
+ 'priority_weight': 1,
+ 'queue': 'default',
+ 'resources': None,
+ 'retries': 0,
+ 'retry_delay': timedelta(0, 300),
+ 'retry_exponential_backoff': False,
+ 'run_as_user': None,
+ 'sla': None,
+ 'start_date': None,
+ 'subdag': None,
+ 'task_concurrency': None,
+ 'task_id': '10',
+ 'trigger_rule': 'all_success',
+ 'wait_for_downstream': False,
+ 'weight_rule': 'downstream'}, fields,
+ """
+!!!
+
+ ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY
+
+ Some fields were added to the BaseOperator! Please add them to the list above
and make sure that
+ you add support for DAG serialization - you should add the field to
+ `airflow/serialization/schema.json` - they should have correct type defined
there.
+
+ Note that we do not support versioning yet so you should only add optional
fields. We do not support
+ versioning yet so you should make sure all fields added to the BaseOperator
should be optional.
Review comment:
Ah yeah :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for
newly added fields in BaseOperator
URL: https://github.com/apache/airflow/pull/7162#discussion_r367100569
##
File path: tests/serialization/test_dag_serialization.py
##
@@ -543,6 +543,66 @@ def test_dag_serialized_fields_with_schema(self):
dag_params: set = set(dag_schema.keys()) - ignored_keys
self.assertEqual(set(DAG.get_serialized_fields()), dag_params)
+def test_no_new_fields_added_to_base_operator(self):
+"""
+This test verifies that there are no new fields added to BaseOperator.
And reminds that
+tests should be added for it.
+"""
+base_operator = BaseOperator(task_id="10")
+fields = base_operator.__dict__
+self.assertEqual({'_dag': None,
+ '_downstream_task_ids': set(),
+ '_inlets': [],
+ '_log': base_operator.log,
+ '_outlets': [],
+ '_upstream_task_ids': set(),
+ 'depends_on_past': False,
+ 'do_xcom_push': True,
+ 'email': None,
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'end_date': None,
+ 'execution_timeout': None,
+ 'executor_config': {},
+ 'inlets': [],
+ 'max_retry_delay': None,
+ 'on_execute_callback': None,
+ 'on_failure_callback': None,
+ 'on_retry_callback': None,
+ 'on_success_callback': None,
+ 'outlets': [],
+ 'owner': 'airflow',
+ 'params': {},
+ 'pool': 'default_pool',
+ 'priority_weight': 1,
+ 'queue': 'default',
+ 'resources': None,
+ 'retries': 0,
+ 'retry_delay': timedelta(0, 300),
+ 'retry_exponential_backoff': False,
+ 'run_as_user': None,
+ 'sla': None,
+ 'start_date': None,
+ 'subdag': None,
+ 'task_concurrency': None,
+ 'task_id': '10',
+ 'trigger_rule': 'all_success',
+ 'wait_for_downstream': False,
+ 'weight_rule': 'downstream'}, fields,
+ """
+!!!
+
+ ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY
+
+ Some fields were added to the BaseOperator! Please add them to the list above
and make sure that
+ you add support for DAG serialization - you should add the field to
+ `airflow/serialization/schema.json` and add it in
`serialized_simple_dag_ground_truth` above
Review comment:
Similar rules as for protobuf. I will modify the message. We cannot test a
lot of that fully automatically yet but we can at least provide the right
message.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for newly added fields in BaseOperator
potiuk commented on a change in pull request #7162: [AIRFLOW-6557] Add test for
newly added fields in BaseOperator
URL: https://github.com/apache/airflow/pull/7162#discussion_r366367086
##
File path: tests/serialization/test_dag_serialization.py
##
@@ -543,6 +543,66 @@ def test_dag_serialized_fields_with_schema(self):
dag_params: set = set(dag_schema.keys()) - ignored_keys
self.assertEqual(set(DAG.get_serialized_fields()), dag_params)
+def test_no_new_fields_added_to_base_operator(self):
+"""
+This test verifies that there are no new fields added to BaseOperator.
And reminds that
+tests should be added for it.
+"""
+base_operator = BaseOperator(task_id="10")
+fields = base_operator.__dict__
+self.assertEqual({'_dag': None,
+ '_downstream_task_ids': set(),
+ '_inlets': [],
+ '_log': base_operator.log,
+ '_outlets': [],
+ '_upstream_task_ids': set(),
+ 'depends_on_past': False,
+ 'do_xcom_push': True,
+ 'email': None,
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'end_date': None,
+ 'execution_timeout': None,
+ 'executor_config': {},
+ 'inlets': [],
+ 'max_retry_delay': None,
+ 'on_execute_callback': None,
+ 'on_failure_callback': None,
+ 'on_retry_callback': None,
+ 'on_success_callback': None,
+ 'outlets': [],
+ 'owner': 'airflow',
+ 'params': {},
+ 'pool': 'default_pool',
+ 'priority_weight': 1,
+ 'queue': 'default',
+ 'resources': None,
+ 'retries': 0,
+ 'retry_delay': timedelta(0, 300),
+ 'retry_exponential_backoff': False,
+ 'run_as_user': None,
+ 'sla': None,
+ 'start_date': None,
+ 'subdag': None,
+ 'task_concurrency': None,
+ 'task_id': '10',
+ 'trigger_rule': 'all_success',
+ 'wait_for_downstream': False,
+ 'weight_rule': 'downstream'}, fields,
+ """
+!!!
+
+ ACTION NEEDED! PLEASE READ THIS CAREFULLY AND CORRECT TESTS CAREFULLY
+
+ Some fields were added to the BaseOperator! Please add them to the list above
and make sure that
+ you add support for DAG serialization - you should add the field to
+ `airflow/serialization/schema.json` and add it in
`serialized_simple_dag_ground_truth` above
Review comment:
I think then we need to decide what to do in this case and tell @lokeshlal
what to do in this case: https://github.com/apache/airflow/pull/7162 (and
update this description).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
