ranga2crazyy opened a new issue, #29400:
URL: https://github.com/apache/airflow/issues/29400
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
While changing the DAG's task instance(any) status as below, Airflow UI
throws "Network error". But task instance's status does change though there is
an error. It happens every time when one tries to change the TI status.
Not limited to Geo location.
Success to Failed OR Failed to Success
### What you think should happen instead
There should be no error pop up in UI
Enabled debug and analysed NGINX and gunicorn logs bit not useful. I could
see HTTP post request in gunicorn logs to change the status as failed.
> 127.0.0.1 - - [07/Feb/2023:09:36:51 +0000] "POST /failed HTTP/1.1" 302 217
"<truncated>task_id=load" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) App
leWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
> 127.0.0.1 - - [07/Feb/2023:09:36:52 +0000] "POST /failed HTTP/1.1" 302 217
"<truncated>&task_id=load" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
debug logs from nginx error log file
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http upstream request:
"/failed?"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http upstream process header
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 malloc: 0000561549221F00:4096
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 recv: eof:0, avail:1
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 recv: fd:18 617 of 4096
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 **http proxy status 302 "302
FOUND"**
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header: "Server:
gunicorn"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header: "Date:
Tue, 07 Feb 2023 09:36:52 GMT"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"Connection: close"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"Content-Type: text/html; charset=utf-8"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"Content-Length: 217"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"Location: <Airflow_DNS>/home"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"X-Robots-Tag: noindex, nofollow"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header:
"Set-Cookie:
session=9ccd9591-1c77-4751-8f3a-9742d7965b1c.FaWqilm3H_PHBYT6s4Zbms42M-Q;
Expires=Tue, 07-Feb-2023 10:06:52 GMT; HttpOnly; Path=/; SameSite=Lax"
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy header done
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 HTTP/1.1 302 FOUND
> Server: nginx/1.14.1
> Date: Tue, 07 Feb 2023 09:36:52 GMT
> Content-Type: text/html; charset=utf-8
> Content-Length: 217
> Connection: keep-alive
> Location: <Airflow_DNS>/home
> X-Robots-Tag: noindex, nofollow
> Set-Cookie:
session=9ccd9591-1c77-4751-8f3a-9742d7965b1c.FaWqilm3H_PHBYT6s4Zbms42M-Q;
Expires=Tue, 07-Feb-2023 10:06:52 GMT; HttpOnly; Path=/; SameSite=Lax
>
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 write new buf t:1 f:0
000056154925AC08, pos 000056154925AC08, size: 409 file: 0, size: 0
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http write filter: l:0 f:0
s:409
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http cacheable: 0
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http proxy filter init s:302
h:0 c:0 l:217
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 http upstream process
upstream
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 pipe read upstream: 0
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 pipe preread: 217
> 2023/02/07 09:36:52 [debug] 1162293#0: *84162 pipe buf free s:0 t:1 f:0
0000561549221F00, pos 0000561549222090, size: 217 file: 0, size: 0
### How to reproduce
Env details provided in deployment section. this error has not been observed
in older airflow version which we used(2.2.1 & 3.6).
1. Login to UI.
2. Nav to any DAG.
3. try to change any task instance's status. Error will pop up down bottom
4. Click on confirm and again error reappears.
5. Cancel and refresh the page, status would have changed as intended.
### Operating System
RHEL 8.7 (Ootpa)
### Versions of Apache Airflow Providers
Providers info
apache-airflow-providers-ftp | 2.1.2
apache-airflow-providers-http | 2.1.2
apache-airflow-providers-imap | 2.2.3
apache-airflow-providers-microsoft-mssql | 2.1.1
apache-airflow-providers-postgres | 3.0.0
apache-airflow-providers-snowflake | 2.5.1
apache-airflow-providers-sqlite | 2.1.3
### Deployment
Virtualenv installation
### Deployment details
Apache Airflow
**version | 2.3.2**
executor | CeleryExecutor
task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
sql_alchemy_conn | postgresql
AWS EC2 System info
OS | Linux
architecture | x86_64
python_version | 3.9.13 (main, Nov 9 2022, 13:16:24) [GCC 8.5.0 20210514
(Red Hat 8.5.0-15)]
python_location | /usr/bin/python3.9
nginx version: nginx/1.14.1
gunicorn==20.1.0
gunicorn worker class = sync
NGINX conf
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name <DNS> ;
ssl_certificate <truncated>
ssl_certificate_key <truncated>
location / {
proxy_pass http://localhost:8080;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
### Anything else
Every time it occurs.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
