ashb commented on a change in pull request #10956:
URL: https://github.com/apache/airflow/pull/10956#discussion_r500609191
##########
File path: airflow/config_templates/config.yml
##########
@@ -1562,6 +1562,41 @@
type: string
example: ~
default: "512"
+ - name: use_row_level_locking
+ description: |
+ Should the scheduler issue `SELECT ... FOR UPDATE` in relevant queries.
+ If this is set to False then you should not run more than a single
+ scheduler at once
+ version_added: 2.0.0
+ type: boolean
+ example: ~
+ default: "True"
+ - name: max_dagruns_to_create_per_loop
+ description: |
+ This changes the number of dags that are locked by each scheduler when
+ creating dag runs. One possible reason for setting this lower is if you
+ have huge dags and are running multiple schedules, you won't want one
+ scheduler to do all the work.
+
+ Default: 10
+ example: ~
+ version_added: 2.0.0
+ type: string
+ default: ~
+ - name: max_dagruns_per_loop_to_schedule
+ description: |
+ How many DagRuns should a scheduler exaimne (and lock) when scheduling
+ and queuing tasks. Increasing this limit will allow more throughput for
+ smaller DAGs but will likely slow down throughput for larger (>500
+ tasks for example.) DAGs. Setting this to high when using multiple
Review comment:
:face_with_head_bandage: Working late never pays off. I'll fix this in
the morning :laughing:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
