wjddn279 commented on issue #56641:
URL: https://github.com/apache/airflow/issues/56641#issuecomment-3460906655
@alkismavridis
The memory limit needs to be increased. Since the scheduler spawns 32 worker
subprocesses, there is a significant memory load. I have observed that the
memory of each worker process gradually increases over time and then stabilizes
at a certain level. (This is likely due to the parent process’s memory being
copy-on-write, but I am still investigating the root cause.)
I recommend setting the restart threshold to 5 GB. It would also be helpful
to increase the memory limit and use smem to check whether the memory of each
worker process continues to grow over time. By my measurements, there is almost
no change in memory usage at the following levels.
```
PID User Command Swap USS PSS
RSS
1 airflow /usr/bin/dumb-init -- /entr 0 84 110
888
12079 airflow /bin/bash 0 1736 1835
3484
1212609 airflow /usr/bin/python3 /usr/bin/s 0 14144 14268
16000
54 airflow airflow serve-logs 0 101836 107213
175100
52 airflow airflow worker -- LocalExec 0 125588 128036
201980
26 airflow airflow worker -- LocalExec 0 125652 128097
201996
53 airflow airflow worker -- LocalExec 0 125664 128112
202056
38 airflow airflow worker -- LocalExec 0 125928 128369
202288
36 airflow airflow worker -- LocalExec 0 126588 129008
202788
40 airflow airflow worker -- LocalExec 0 126736 129162
203028
47 airflow airflow worker -- LocalExec 0 126764 129190
203064
50 airflow airflow worker -- LocalExec 0 126820 129245
203112
31 airflow airflow worker -- LocalExec 0 126960 129385
203240
46 airflow airflow worker -- LocalExec 0 127048 129473
203336
44 airflow airflow worker -- LocalExec 0 127092 129516
203376
39 airflow airflow worker -- LocalExec 0 127112 129535
203380
41 airflow airflow worker -- LocalExec 0 127180 129602
203440
35 airflow airflow worker -- LocalExec 0 127192 129615
203460
42 airflow airflow worker -- LocalExec 0 127228 129639
203312
34 airflow airflow worker -- LocalExec 0 127440 129861
203684
29 airflow airflow worker -- LocalExec 0 127484 129905
203728
43 airflow airflow worker -- LocalExec 0 127540 129960
203772
32 airflow airflow worker -- LocalExec 0 127772 130190
203976
33 airflow airflow worker -- LocalExec 0 127832 130251
204040
49 airflow airflow worker -- LocalExec 0 127848 130268
204060
27 airflow airflow worker -- LocalExec 0 127852 130270
204020
28 airflow airflow worker -- LocalExec 0 127852 130270
204016
51 airflow airflow worker -- LocalExec 0 127852 130272
204076
30 airflow airflow worker -- LocalExec 0 127856 130274
204040
48 airflow airflow worker -- LocalExec 0 127864 130284
204076
37 airflow airflow worker -- LocalExec 0 127876 130295
204076
45 airflow airflow worker -- LocalExec 0 127900 130318
204068
7 airflow /usr/python/bin/python3.12 0 301160 313669
398636
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
