jason810496 commented on code in PR #68875:
URL: https://github.com/apache/airflow/pull/68875#discussion_r3459254918
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -76,6 +76,37 @@ core:
type: string
example: ~
default: "LocalExecutor"
+ mp_start_method:
+ description: |
+ The ``multiprocessing`` start method used by long-lived components
that spawn worker
+ processes (the scheduler's ``LocalExecutor``, the triggerer, and the
api-server). Must be
+ one of the values returned by `multiprocessing.get_all_start_methods()
+
<https://docs.python.org/3/library/multiprocessing.html#multiprocessing.get_all_start_methods>`__
+ on your platform (typically ``fork``, ``forkserver`` or ``spawn``).
When unset (the
+ default) the platform default is used.
+
+ Python 3.14 changed the Unix default from ``fork`` to ``forkserver``.
``forkserver`` and
+ ``spawn`` children re-import Airflow instead of sharing it
copy-on-write, which can sharply
+ increase the resident memory of the scheduler and triggerer. Set this
to ``fork`` to restore
+ the pre-3.14 behaviour (note: forking a multi-threaded process is not
always safe), or keep
+ ``forkserver`` and set ``[core] mp_forkserver_preload`` to recover
most of the sharing.
+
+ This can be overridden per component by setting ``mp_start_method`` in
the ``[scheduler]``,
+ ``[triggerer]``, ``[dag_processor]`` or ``[api]`` section.
Review Comment:
```suggestion
``[triggerer]`` or ``[dag_processor]`` section.
```
##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -76,6 +76,37 @@ core:
type: string
example: ~
default: "LocalExecutor"
+ mp_start_method:
+ description: |
+ The ``multiprocessing`` start method used by long-lived components
that spawn worker
+ processes (the scheduler's ``LocalExecutor``, the triggerer, and the
api-server). Must be
Review Comment:
```suggestion
processes (the scheduler's ``LocalExecutor`` and the triggerer).
Must be
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
