hussein-awala commented on issue #33212: URL: https://github.com/apache/airflow/issues/33212#issuecomment-1670416513
As @raphaelauv proposed, you can use the conf `trigerrer.default_capacity` to limit the number of active trigger in a triggerrer, and the total number of active triggers is the number of trigerrer instances * `trigerrer.default_capacity`. Once the task is deferred, it won't consume any more resources, waiting for its trigger to complete, so IMHO limiting the number of deferred tasks is pointless. However, what matters is the number of active triggers which, although light, could put some pressure on the triggerer service. Could you explain your needs? Is it to reduce the pressure on the scheduler and the workers or to reduce it on an external system? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
