turbaszek commented on a change in pull request #11467:
URL: https://github.com/apache/airflow/pull/11467#discussion_r503550725
##########
File path: docs/scheduler.rst
##########
@@ -65,3 +65,72 @@ If you want to use 'external trigger' to run future-dated
execution dates, set `
This only has effect if your DAG has no ``schedule_interval``.
If you keep default ``allow_trigger_in_future = False`` and try 'external
trigger' to run future-dated execution dates,
the scheduler won't execute it now but the scheduler will execute it in the
future once the current date rolls over to the execution date.
+
+Running more than one scheduler
+-------------------------------
+
+Airflow 2.0 is the first release that officially supports running more one
scheduler concurrently -- both for
+performance reasons and for resiliency.
+
+Overview
+""""""""
+
+The HA scheduler is written to take advantage of the existing metadata
database. This was primarily done for
+operational simplicity: every component already has to speak to this DB, and
by not using direct communication
+or consensus algorithm between schedulers (Raft, Paxos, etc.) nor another
consensus tool (Apache Zookeeper, or
+Consul for instance) we have kept the "operational surface area" to a minimum.
Review comment:
```suggestion
Consul for instance) we have kept the "operational surface area" to a
minimum. For more information on architectural choices check `Airflow
Improvement Proposal 15
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651>`_.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
