potiuk commented on PR #25101: URL: https://github.com/apache/airflow/pull/25101#issuecomment-1187596104
> My understanding of the purpose of `parallelism` was that it was supposed to be used to limit strain on workers, not the scheduler. > > By "queues" do you mean pools? I think your interpretation makes sense for the current implementation. Your proposed solution would mean that users would primarily control worker resources by using pools and control scheduler resources by what is currently known as parallelism, correct? IMO this makes sense, including moving parallelism from `core` to `scheduler`. @ReadytoRocc recently asked a very similar question. > > @collinmcnulty FYI Oh yeah. I meant "pool" of course, sorry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
