alkismavridis commented on issue #56641:
URL: https://github.com/apache/airflow/issues/56641#issuecomment-3460632269
@wjddn279 hello! So:
- We run about 120 dags per day (representing about 140 Tasks). All of the
tasks are Python Operators who wrap some tiny logic around an SSH Operator call.
- Right, the memory drops represent restarts. As a hacky workaround to the
memory leak, we introduced a script that monitors the memory in a loop and
restarts it when: it is above the threshold of 75% AND no tasks are running at
the moment.
- Sure! Here it is:
```
$ smem -s pss
PID User Command Swap USS PSS
RSS
1 default /usr/bin/dumb-init -- /entr 0 92 114
1296
19975 default /bin/sh -c airflow jobs che 0 112 157
1628
19982 default bash 0 1700 1760
3836
19990 default /usr/bin/python3 /usr/bin/s 0 13804 13892
16412
2656 default airflow worker -- LocalExec 0 26560 42727
172020
2654 default airflow worker -- LocalExec 0 26676 42847
172220
2645 default airflow worker -- LocalExec 0 26524 43545
171808
2644 default airflow worker -- LocalExec 0 26752 43761
171948
2668 default airflow worker -- LocalExec 0 34172 45770
170444
2676 default airflow worker -- LocalExec 0 35144 46674
170588
2680 default airflow worker -- LocalExec 0 36364 47880
172264
856 default airflow worker -- LocalExec 0 26012 54588
170760
26 default /usr/python/bin/python3.12 0 14440 55555
154716
857 default airflow worker -- LocalExec 0 27564 56139
172308
858 default airflow worker -- LocalExec 0 34768 58567
169944
25 default airflow serve-logs 0 20496 61855
162556
9885 default airflow worker -- LocalExec 0 39616 75244
178656
10202 default airflow worker -- LocalExec 0 41056 75405
180696
10384 default airflow worker -- LocalExec 0 41392 75658
180696
9919 default airflow worker -- LocalExec 0 42316 76601
178660
14485 default airflow worker -- LocalExec 0 96232 100210
171112
13584 default airflow worker -- LocalExec 0 96372 100278
171112
12679 default airflow worker -- LocalExec 0 97780 101180
171112
6250 default airflow worker -- LocalExec 0 99760 101764
170236
7154 default airflow worker -- LocalExec 0 99828 101829
170380
7512 default airflow worker -- LocalExec 0 100668 102598
170248
5359 default airflow worker -- LocalExec 0 100660 102654
171148
6 default /usr/python/bin/python3.12 0 96628 102855
200664
8745 default airflow worker -- LocalExec 0 100804 103257
172260
1752 default airflow worker -- LocalExec 0 101600 103493
170828
8066 default airflow worker -- LocalExec 0 102092 104039
171880
3561 default airflow worker -- LocalExec 0 102140 104078
171952
11108 default airflow worker -- LocalExec 0 101412 104390
178860
11755 default airflow worker -- LocalExec 0 101728 104593
178864
10848 default airflow worker -- LocalExec 0 101804 105288
180696
8980 default airflow worker -- LocalExec 0 102216 106644
178720
4466 default airflow worker -- LocalExec 0 105272 107104
171108
8758 default airflow worker -- LocalExec 0 105852 108276
177248
10128 default airflow worker -- LocalExec 0 107792 110238
180660
19981 default /usr/python/bin/python3.12 0 129024 130981
158464
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
