vatsrahul1001 opened a new pull request, #76016: URL: https://github.com/apache/airflow/pull/76016
The triggerer forks `serve_logs` from its main process. Under the `fork` multiprocessing start method the child shares the parent's pages copy-on-write; when the main process later runs a full (gen2) GC, it writes to every live object's header and splits those COW-shared pages into private copies — the pod's anonymous memory steps up with no new allocation. This mirrors the same `gc.freeze()`-before-fork fix already applied to the other long-lived forkers: - Executors / LocalExecutor: #58365 - Dag processor manager: #60505 - Celery worker: #62212 `gc.freeze()` moves already-allocated objects into the permanent generation so a later GC does not scan or write to them, keeping those pages shared; `gc.unfreeze()` runs right after the fork so the parent's own GC behaviour is unchanged. The effect applies only under the `fork` start method — a no-op for `forkserver`/`spawn`, where children re-import instead of sharing. > Draft: `gc.freeze()` placement is still being validated against the soak / local repro. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
