potiuk commented on issue #17045: URL: https://github.com/apache/airflow/issues/17045#issuecomment-886231579
I run your example using local executor and it all works fine (see below). The problem you most likely experienced was problem with retrieving task logs. You need to make sure you have `AIRFLOW__WEBSERVER__SECRET_KEY` the same for workers and webserver - then your logs will be retrieved by the webserver https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key I am closing it now - please re-open if the issue is not fixed (but I am quite sure it will be). ``` *** Reading local file: /root/airflow/logs/monitor/validate_parameters/2021-07-25T17:02:55.522373+00:00/1.log [2021-07-25, 17:02:57 UTC] {taskinstance.py:913} INFO - Dependencies all met for <TaskInstance: monitor.validate_parameters 2021-07-25T17:02:55.522373+00:00 [queued]> [2021-07-25, 17:02:57 UTC] {taskinstance.py:913} INFO - Dependencies all met for <TaskInstance: monitor.validate_parameters 2021-07-25T17:02:55.522373+00:00 [queued]> [2021-07-25, 17:02:57 UTC] {taskinstance.py:1104} INFO - -------------------------------------------------------------------------------- [2021-07-25, 17:02:57 UTC] {taskinstance.py:1105} INFO - Starting attempt 1 of 1 [2021-07-25, 17:02:57 UTC] {taskinstance.py:1106} INFO - -------------------------------------------------------------------------------- [2021-07-25, 17:02:57 UTC] {taskinstance.py:1124} INFO - Executing <Task(PythonOperator): validate_parameters> on 2021-07-25T17:02:55.522373+00:00 [2021-07-25, 17:02:57 UTC] {standard_task_runner.py:52} INFO - Started process 1353 to run task [2021-07-25, 17:02:57 UTC] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'monitor', 'validate_parameters', '2021-07-25T17:02:55.522373+00:00', '--job-id', '2', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/test_dag_failure.py', '--cfg-path', '/tmp/tmpook1fgsn', '--error-file', '/tmp/tmp45ahny1g'] [2021-07-25, 17:02:57 UTC] {standard_task_runner.py:77} INFO - Job 2: Subtask validate_parameters [2021-07-25, 17:02:57 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: monitor.validate_parameters 2021-07-25T17:02:55.522373+00:00 [running]> on host 19f13fbfde00 [2021-07-25, 17:02:57 UTC] {taskinstance.py:1257} INFO - Exporting the following env vars: AIRFLOW_CTX_DAG_OWNER=boss_admin AIRFLOW_CTX_DAG_ID=monitor AIRFLOW_CTX_TASK_ID=validate_parameters AIRFLOW_CTX_EXECUTION_DATE=2021-07-25T17:02:55.522373+00:00 AIRFLOW_CTX_DAG_RUN_ID=manual__2021-07-25T17:02:55.522373+00:00 [2021-07-25, 17:02:57 UTC] {taskinstance.py:1458} ERROR - Task failed with exception Traceback (most recent call last): File "/opt/airflow/airflow/models/taskinstance.py", line 1185, in _run_raw_task self._prepare_and_execute_task_with_callbacks(context, task) File "/opt/airflow/airflow/models/taskinstance.py", line 1288, in _prepare_and_execute_task_with_callbacks result = self._execute_task(context, task_copy) File "/opt/airflow/airflow/models/taskinstance.py", line 1318, in _execute_task result = task_copy.execute(context=context) File "/opt/airflow/airflow/operators/python.py", line 150, in execute return_value = self.execute_callable() File "/opt/airflow/airflow/operators/python.py", line 161, in execute_callable return self.python_callable(*self.op_args, **self.op_kwargs) File "/files/dags/test_dag_failure.py", line 24, in validate_parameters raise ParameterValidationException(reason='job_identifier should not be empty') unusual_prefix_f60d67b5f19a88c1f508e2153f05719e51032699_test_dag_failure.ParameterValidationException [2021-07-25, 17:02:57 UTC] {taskinstance.py:1132} INFO - Marking task as FAILED. dag_id=monitor, task_id=validate_parameters, execution_date=20210725T170255, start_date=20210725T170257, end_date=20210725T170257 [2021-07-25, 17:02:57 UTC] {local_task_job.py:152} INFO - Task exited with return code 1 [2021-07-25, 17:02:57 UTC] {test_dag_failure.py:31} INFO - Failed to load task run error [2021-07-25, 17:02:57 UTC] {logging_mixin.py:109} INFO - NoneType: None [2021-07-25, 17:02:57 UTC] {local_task_job.py:255} INFO - 0 downstream tasks scheduled from follow-on schedule check ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
