alkismavridis opened a new issue, #57495: URL: https://github.com/apache/airflow/issues/57495
### Apache Airflow version 3.1.1 ### If "Other Airflow 2/3 version" selected, which one? _No response_ ### What happened? When trying to use LocalExecutor with parallelism=0, which, as mentioned [in the docs](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/local.html) should mean "unlimited Parallelism, I get the following error: ``` Traceback (most recent call last): File "/home/airflow/.local/bin/airflow", line 7, in <module> sys.exit(main()) ^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line 55, in main args.func(args) File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 49, in command return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 114, in wrapper return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py", line 52, in scheduler run_command_with_daemon_option( File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/daemon_utils.py", line 86, in run_command_with_daemon_option callback() File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py", line 55, in <lambda> callback=lambda: _run_scheduler_job(args), ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/scheduler_command.py", line 43, in _run_scheduler_job run_job(job=job_runner.job, execute_callable=job_runner._execute) File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py", line 100, in wrapper return func(*args, session=session, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/job.py", line 368, in run_job return execute_job(job, execute_callable=execute_callable) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/job.py", line 397, in execute_job ret = execute_callable() ^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/scheduler_job_runner.py", line 1085, in _execute for executor in self.job.executors: ^^^^^^^^^^^^^^^^^^ File "/usr/python/lib/python3.12/functools.py", line 998, in __get__ val = self.func(instance) ^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/job.py", line 164, in executors return ExecutorLoader.init_executors() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/executor_loader.py", line 222, in init_executors loaded_executor = cls.load_executor(executor_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/executor_loader.py", line 273, in load_executor executor = executor_cls() ^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/local_executor.py", line 154, in __init__ super().__init__(parallelism=parallelism) File "/home/airflow/.local/lib/python3.12/site-packages/airflow/executors/base_executor.py", line 178, in __init__ raise ValueError("parallelism is set to 0 or lower") ValueError: parallelism is set to 0 or lower ``` The implementations of BaseExecutor and LocalExecutor also seem to contradict each other. Base executor forbids the value zero. [Code](https://github.com/apache/airflow/blob/15e0b7501d9a66491597f77626de639403effa77/airflow-core/src/airflow/executors/base_executor.py#L177). But LocalExecutor seems to give special meaning to the value zero. [Code](https://github.com/apache/airflow/blob/15e0b7501d9a66491597f77626de639403effa77/airflow-core/src/airflow/executors/local_executor.py#L193) ### What you think should happen instead? Parallelism=0 should be a legit value and should mean, "Unlimited Parallelism", as mentioned in the [Docs](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/local.html) ### How to reproduce Use LocalExecutor and code.parallelism=0. Restart your scheduler ### Operating System Linux ### Versions of Apache Airflow Providers 3.1.1 ### Deployment Docker-Compose ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
