rodrigo-morais edited a comment on issue #16881:
URL: https://github.com/apache/airflow/issues/16881#issuecomment-876388007
Pods before to deploy a new scheduler
```
NAME READY
STATUS RESTARTS AGE
airflow-scheduler-5c989d8478-kvllt 1/1
Running 0 3m57s
airflow-web-74f56b9df-hvpzz 2/2
Running 0 3m57s
ascheckcountssfvspgbusinessdb.89bf993a26f3431fa07667de82954239 1/1
Running 0 40s
ascheckcountssfvspgcontentdb.8cf4f8e0ae794b3dbc9b471c9aef4532 1/1
Running 0 40s
dbt-docs-647db48b7d-vpj4p 1/1
Running 0 3m57s
```
Pods after to deploy the scheduler
```
NAME READY
STATUS RESTARTS AGE
airflow-scheduler-5c989d8478-kvllt 1/1
Running 0 4m6s
airflow-scheduler-776f94956b-s4dfv 0/1
ContainerCreating 0 5s
airflow-web-5c667d85db-kfcx6 0/2
ContainerCreating 0 2s
airflow-web-74f56b9df-hvpzz 2/2
Running 0 4m6s
ascheckcountssfvspgbusinessdb.89bf993a26f3431fa07667de82954239 1/1
Running 0 49s
ascheckcountssfvspgcontentdb.8cf4f8e0ae794b3dbc9b471c9aef4532 1/1
Running 0 49s
dbt-docs-647db48b7d-vpj4p 1/1
Running 0 4m6s
```
Pods after the current scheduler have been terminated and the new one has
been created
```
NAME
READY STATUS RESTARTS AGE
airflow-scheduler-776f94956b-s4dfv
1/1 Running 0 3m6s
airflow-web-5c667d85db-kfcx6
2/2 Running 0 3m3s
ascheckcountssfvspgcontentdb.44eee838b6754b0aa0c8e6288a6195a2
0/1 Error 0 2m16s
ascheckcountssfvspgcontentdb.8cf4f8e0ae794b3dbc9b471c9aef4532
0/1 Completed 0 3m50s
dbt-docs-647db48b7d-vpj4p
1/1 Running 0 7m7s
```
<details><summary>Pod with error</summary>
```log
Running <TaskInstance: as_check_counts_sf_vs_pg.contentdb
2021-07-08T11:42:09.208095+00:00 [queued]> on host
ascheckcountssfvspgcontentdb.44eee838b6754b0aa0c8e6288a6195a2
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line
40, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line
91, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 237, in task_run
_run_task_by_selected_method(args, dag, ti)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 64, in _run_task_by_selected_method
_run_task_by_local_task_job(args, ti)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 120, in _run_task_by_local_task_job
run_job.run()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/base_job.py",
line 237, in run
self._execute()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/local_task_job.py",
line 133, in _execute
self.heartbeat()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/base_job.py",
line 218, in heartbeat
self.heartbeat_callback(session=session)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py",
line 67, in wrapper
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/local_task_job.py",
line 207, in heartbeat_callback
ti._run_finished_callback(error=error) # pylint:
disable=protected-access
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1367, in _run_finished_callback
task.on_failure_callback(context)
```
</details>
<details><summary>Pod with scheduler</summary>
```log
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:850:
DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and
airflow_home in the config file is deprecated. Please use only the AIRFLOW_HOME
environment variable and remove the config file entry.
warnings.warn(msg, category=DeprecationWarning)
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-07-08 11:44:16,768] {dagrun.py:429} ERROR - Marking run <DagRun
as_check_counts_sf_vs_pg @ 2021-07-08 11:42:09.208095+00:00:
manual__2021-07-08T11:42:09.208095+00:00, externally triggered: True> failed
[2021-07-08 11:44:26,873] {kubernetes_executor.py:202} ERROR - Event:
ascheckcountssfvspgcontentdb.44eee838b6754b0aa0c8e6288a6195a2 Failed
```
</details>
<details><summary>Log of the task</summary>
```log
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1137, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1311, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1341, in _execute_task
result = task_copy.execute(context=context)
File "/usr/local/airflow/plugins/operators/jw_pg_vs_sf_count_check.py",
line 65, in execute
cur_pg.execute(sql)
File "/usr/local/lib/python3.8/encodings/utf_8.py", line 15, in decode
def decode(input, errors='strict'):
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1266, in signal_handler
raise AirflowException("Task received SIGTERM signal")
airflow.exceptions.AirflowException: Task received SIGTERM signal
[2021-07-08 13:16:42,544] {taskinstance.py:1524} INFO - Marking task as
FAILED. dag_id=as_check_counts_sf_vs_pg, task_id=contentdb,
execution_date=20210708T114209, start_date=20210708T131622,
end_date=20210708T131642
[2021-07-08 13:16:42,601] {process_utils.py:66} INFO - Process
psutil.Process(pid=149, status='terminated', exitcode=1, started='13:16:21')
(149) terminated with exit code 1
```
The UI:
The previous and these logs don't have any error in the DB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
