jason810496 opened a new pull request, #47189: URL: https://github.com/apache/airflow/pull/47189
closes: #46978 ## Why By introducing `CliDefinitionLoader`, we can extend `cli_parser` to support **provider module-level** CLI commands. > Additionally, as a next step, we could decouple the get_cli_commands method from both Executor and AuthManager. This would address the root cause of the current CLI slowdown and potentially improve CLI performance by 3–4x, as shown in the benchmark: #46789. ## How This PR only add `CliDefinitionLoader` and **provider module-level** (only for kubernetes now) CLI commands. ## Benchmark **Environment:** - MacOS, M2 Chip, 16G RAB, In breeze container. - Command:`breeze --python 3.9 shell --answer n --executor KubernetesExecutor` - Benchmark Command:`/files/time.sh airflow --help` ( [benchmark script: time.sh](https://gist.github.com/jason810496/11b208fb964e0c490a6cd82f7d1882f7) ) Run | Original (seconds) | With CliDefinitionLoader (seconds) | Difference (seconds) -- | -- | -- | -- 1 | 3.88124 | 4.12564 | +0.24440 2 | 3.35908 | 3.60187 | +0.24279 3 | 3.38379 | 3.58633 | +0.20254 4 | 3.39347 | 3.59239 | +0.19892 5 | 3.36010 | 3.76085 | +0.40075 Average | 3.47554 | 3.73340 | +0.25786 As https://github.com/apache/airflow/issues/46978#issuecomment-2679446808 pointed out, using `ProviderManager` adds an overhead of approximately **200ms to 300ms** when discovering all available providers. <!-- Please keep an empty line above the dashes. --> --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
