potiuk commented on issue #20974:
URL: https://github.com/apache/airflow/issues/20974#issuecomment-1019547859
Full stack trace showing unfurling of the Context object just before failing:
```
[2022-01-23, 19:07:26 UTC] {process_utils.py:164} INFO - Executing cmd:
/usr/local/bin/python -m virtualenv /tmp/venv279duyub --system-site-packages
[2022-01-23, 19:07:26 UTC] {process_utils.py:168} INFO - Output:
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - created virtual
environment CPython3.8.12.final.0-64 in 140ms
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - creator
CPython3Posix(dest=/tmp/venv279duyub, clear=False, no_vcs_ignore=False,
global=True)
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - seeder
FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle,
via=copy, app_data_dir=/root/.local/share/virtualenv)
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - added seed
packages: pip==21.3.1, setuptools==60.2.0, wheel==0.37.1
[2022-01-23, 19:07:26 UTC] {process_uti[2022-01-23, 19:07:26 UTC]
{process_utils.py:164} INFO - Executing cmd: /usr/local/bin/python -m
virtualenv /tmp/venv279duyub --system-site-packages
[2022-01-23, 19:07:26 UTC] {process_utils.py:168} INFO - Output:
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - created virtual
environment CPython3.8.12.final.0-64 in 140ms
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - creator
CPython3Posix(dest=/tmp/venv279duyub, clear=False, no_vcs_ignore=False,
global=True)
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - seeder
FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle,
via=copy, app_data_dir=/root/.local/share/virtualenv)
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - added seed
packages: pip==21.3.1, setuptools==60.2.0, wheel==0.37.1
[2022-01-23, 19:07:26 UTC] {process_utils.py:172} INFO - activators
BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[2022-01-23, 19:07:26 UTC] {process_utils.py:164} INFO - Executing cmd:
/tmp/venv279duyub/bin/pip install -r /tmp/venv279duyub/requirements.txt
[2022-01-23, 19:07:26 UTC] {process_utils.py:168} INFO - Output:
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'execution_date' from the template is deprecated and will be removed in a
future version. Please use 'data_interval_start' or 'logical_date' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_ds' from the template is deprecated and will be removed in a future
version. Please use '{{ data_interval_end | ds }}' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_ds_nodash' from the template is deprecated and will be removed in a
future version. Please use '{{ data_interval_end | ds_nodash }}' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_execution_date' from the template is deprecated and will be removed in a
future version. Please use 'data_interval_end' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_execution_date' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_execution_date_success' from the template is deprecated and will be
removed in a future version. Please use 'prev_data_interval_start_success'
instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'tomorrow_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'tomorrow_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'yesterday_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'yesterday_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {taskinstance.py:1702} ERROR - Task failed with
exception
Traceback (most recent call last):
File "/opt/airflow/airflow/models/taskinstance.py", line 1325, in
_run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow/airflow/models/taskinstance.py", line 1451, in
_execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow/airflow/models/taskinstance.py", line 1507, in
_execute_task
result = execute_callable(context=context)
File "/opt/airflow/airflow/operators/python.py", line 390, in execute
return super().execute(context=serializable_context)
File "/opt/airflow/airflow/operators/python.py", line 169, in execute
return_value = self.execute_callable()
File "/opt/airflow/airflow/operators/python.py", line 425, in
execute_callable
self._write_args(input_filename)
File "/opt/airflow/airflow/operators/python.py", line 461, in _write_args
self.pickling_library.dump({'args': self.op_args, 'kwargs':
self.op_kwargs}, file)
TypeError: cannot pickle 'module' object
[2022-01-23, 19:07:29 UTC] {taskinstance.py:1267} INFO - Marking task as
FAILED. dag_id=venv_op_not_accepting_context_kwarg, task_id=test,
execution_date=20220123T190725, start_date=20220123T190726,
end_date=20220123T190729
[2022-01-23, 19:07:29 UTC] {standard_task_runner.py:89} ERROR - Failed to
execute job 7 for task test
Traceback (most recent call last):
File "/opt/airflow/airflow/task/task_runner/standard_task_runner.py", line
85, in _start_by_fork
args.func(args, dag=self.dag)
File "/opt/airflow/airflow/cli/cli_parser.py", line 49, in command
return func(*args, **kwargs)
File "/opt/airflow/airflow/utils/cli.py", line 98, in wrapper
return f(*args, **kwargs)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 346, in
task_run
_run_task_by_selected_method(args, dag, ti)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 154, in
_run_task_by_selected_method
_run_raw_task(args, ti)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 227, in
_run_raw_task
ti._run_raw_task(
File "/opt/airflow/airflow/utils/session.py", line 71, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow/airflow/models/taskinstance.py", line 1325, in
_run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow/airflow/models/taskinstance.py", line 1451, in
_execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow/airflow/models/taskinstance.py", line 1507, in
_execute_task
result = execute_callable(context=context)
File "/opt/airflow/airflow/operators/python.py", line 390, in execute
return super().execute(context=serializable_context)
File "/opt/airflow/airflow/operators/python.py", line 169, in execute
return_value = self.execute_callable()
File "/opt/airflow/airflow/operators/python.py", line 425, in
execute_callable
self._write_args(input_filename)
File "/opt/airflow/airflow/operators/python.py", line 461, in _write_args
self.pickling_library.dump({'args': self.op_args, 'kwargs':
self.op_kwargs}, file)
TypeError: cannot pickle 'module' object
[2022-01-23, 19:07:29 UTC] {local_task_job.py:156} INFO - Task exited with
return code 1
[2022-01-23, 19:07:29 UTC] {local_task_job.py:273} INFO - 0 downstream tasks
scheduled from follow-on schedule checkls.py:172} INFO - activators
BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[2022-01-23, 19:07:26 UTC] {process_utils.py:164} INFO - Executing cmd:
/tmp/venv279duyub/bin/pip install -r /tmp/venv279duyub/requirements.txt
[2022-01-23, 19:07:26 UTC] {process_utils.py:168} INFO - Output:
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'execution_date' from the template is deprecated and will be removed in a
future version. Please use 'data_interval_start' or 'logical_date' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_ds' from the template is deprecated and will be removed in a future
version. Please use '{{ data_interval_end | ds }}' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_ds_nodash' from the template is deprecated and will be removed in a
future version. Please use '{{ data_interval_end | ds_nodash }}' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'next_execution_date' from the template is deprecated and will be removed in a
future version. Please use 'data_interval_end' instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_execution_date' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'prev_execution_date_success' from the template is deprecated and will be
removed in a future version. Please use 'prev_data_interval_start_success'
instead.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'tomorrow_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'tomorrow_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'yesterday_ds' from the template is deprecated and will be removed in a future
version.
[2022-01-23, 19:07:29 UTC] {logging_mixin.py:115} WARNING -
/opt/***/***/utils/context.py:157 AirflowContextDeprecationWarning: Accessing
'yesterday_ds_nodash' from the template is deprecated and will be removed in a
future version.
[2022-01-23, 19:07:29 UTC] {taskinstance.py:1702} ERROR - Task failed with
exception
Traceback (most recent call last):
File "/opt/airflow/airflow/models/taskinstance.py", line 1325, in
_run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow/airflow/models/taskinstance.py", line 1451, in
_execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow/airflow/models/taskinstance.py", line 1507, in
_execute_task
result = execute_callable(context=context)
File "/opt/airflow/airflow/operators/python.py", line 390, in execute
return super().execute(context=serializable_context)
File "/opt/airflow/airflow/operators/python.py", line 169, in execute
return_value = self.execute_callable()
File "/opt/airflow/airflow/operators/python.py", line 425, in
execute_callable
self._write_args(input_filename)
File "/opt/airflow/airflow/operators/python.py", line 461, in _write_args
self.pickling_library.dump({'args': self.op_args, 'kwargs':
self.op_kwargs}, file)
TypeError: cannot pickle 'module' object
[2022-01-23, 19:07:29 UTC] {taskinstance.py:1267} INFO - Marking task as
FAILED. dag_id=venv_op_not_accepting_context_kwarg, task_id=test,
execution_date=20220123T190725, start_date=20220123T190726,
end_date=20220123T190729
[2022-01-23, 19:07:29 UTC] {standard_task_runner.py:89} ERROR - Failed to
execute job 7 for task test
Traceback (most recent call last):
File "/opt/airflow/airflow/task/task_runner/standard_task_runner.py", line
85, in _start_by_fork
args.func(args, dag=self.dag)
File "/opt/airflow/airflow/cli/cli_parser.py", line 49, in command
return func(*args, **kwargs)
File "/opt/airflow/airflow/utils/cli.py", line 98, in wrapper
return f(*args, **kwargs)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 346, in
task_run
_run_task_by_selected_method(args, dag, ti)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 154, in
_run_task_by_selected_method
_run_raw_task(args, ti)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 227, in
_run_raw_task
ti._run_raw_task(
File "/opt/airflow/airflow/utils/session.py", line 71, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow/airflow/models/taskinstance.py", line 1325, in
_run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow/airflow/models/taskinstance.py", line 1451, in
_execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow/airflow/models/taskinstance.py", line 1507, in
_execute_task
result = execute_callable(context=context)
File "/opt/airflow/airflow/operators/python.py", line 390, in execute
return super().execute(context=serializable_context)
File "/opt/airflow/airflow/operators/python.py", line 169, in execute
return_value = self.execute_callable()
File "/opt/airflow/airflow/operators/python.py", line 425, in
execute_callable
self._write_args(input_filename)
File "/opt/airflow/airflow/operators/python.py", line 461, in _write_args
self.pickling_library.dump({'args': self.op_args, 'kwargs':
self.op_kwargs}, file)
TypeError: cannot pickle 'module' object
[2022-01-23, 19:07:29 UTC] {local_task_job.py:156} INFO - Task exited with
return code 1
[2022-01-23, 19:07:29 UTC] {local_task_job.py:273} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
