blag opened a new pull request #22613: URL: https://github.com/apache/airflow/pull/22613
<!-- Thank you for contributing! Please make sure that your code changes are covered with tests. And in case of new features or big changes remember to adjust the documentation. Feel free to ping committers for the review! In case of existing issue, reference it using one of the following: closes: #ISSUE related: #ISSUE How to write a good git commit message: http://chris.beams.io/posts/git-commit/ --> ## PR status ## **RFC** - gauging interest and feedback before continuing This PR converts the `airflow db` CLI to using the Click and Rich frameworks with [`rich-click`](https://pypi.org/project/rich-click/) (note that I refer to the combination here as just "Click"). It adds a **temporary** `airflow-ng` console script entry point to enable easier/better comparisons between the existing `airflow` console script entry point and the `airflow-ng` one. We will need to convert all the others subcommands, and I don't know of a good way to do so except converting them all at the same time. #### Reverse Compatibility #### I believe we can largely, if not entirely, keep reverse compatibility with the previous argparse-based CLI. However, if we can't completely maintain reverse compatibility, this might be a better change to implement in Airflow 3.0. ## Motivation ## While argparse is quite extensive in what it supports, third party frameworks like Click have faster development cycles and therefore have implemented 21st century features that argparse does not, and likely will not, support. Things like a simple Python API, automatic coloring for `--help` output, use of non-symbolic ASCII characters, and a better understanding of common CLI author needs are all available with Click but aren't easily supported in argparse. With the Rich framework, it is trivial for command authors to output text formatted with colors and other font effects. The existing argparse implementation is a [bit of a mess with `ARG_` variables](https://github.com/apache/airflow/blob/c58b5e583bcad83048fdee72bc9ddf6c3a7b4229/airflow/cli/cli_parser.py#L180) for DRY, and diving into the CLI commands isn't very easy. This Click implementation also creates some examples for future conversions: * a globally reusable option (the `click_yes` decorator) * some options that are reused within a single module (`click_revision`, `click_version`, `click_from_revision`, `click_from_version`, `click_show_sql_only`) * a decorator that checks for issues with common options between the `db upgrade` and `db downgrade` subcommands All other options are implemented as decorators for for individual functions, illustrating how to add options to subcommands in many different ways that might make sense. You can also define options that apply to command groups. Since most options are bespoke to their subcommand, it makes sense to me that they should be defined closer to where they are used instead of all in a single file. Adding options to a command via decorators makes this very easy to do. ## Comparison ## We get some things "for free" by converting over to Click instead of argparse. #### Formatting #### * Note the descriptions correctly rewrapping in the `airflow db upgrade --help` example as opposed to the existing argparse implementation. * The different sections of command output are rendered in their own ASCII boxes, including command errors. #### Colors! #### Click automatically colorizes output without any additional hinting. * Note how the use of colors separates pieces of information like: - The long-form option - The short-form option - The type of the option value * Colors also draw your attention to the most important part of command output in a context-dependent manner - Note how the names of the different subcommands are in bright blue, and the error output is easily recognizable in a red ASCII box. - Some warnings (not shown) are also displayed in yellow text to differentiate them from normal output and error output, although this really depends on when the warnings are emitted at import time. ### `airflow db --help` ### <details><summary>Click to expand before/after</summary> #### Before ####  #### After ####  </details> ### `airflow db upgrade --help` ### <details><summary>Click to expand before/after</summary> #### Before ####  #### After ####  </details> ### `airflow db upgrade --hep` (misspelled option) ### <details><summary>Click to expand before/after</summary> #### Before ####  #### After ####  </details> ## Future Work ## #### Command Grouping #### We can further separate the `airflow` subcommands into different groups so they are rendered into ASCII boxes with their group label (see the "Groups and sorting" section of the [rich-click documentation](https://pypi.org/project/rich-click/)). #### Styling Customization #### Some Airflow vendors may customize the `airflow` command. The rich-click library allows for extensive style customization. #### Click Goodies #### * The Click API makes it easier to [test the CLI](https://click.palletsprojects.com/en/8.1.x/testing/) programmatically entirely within Python. * [Autocomplete with Click](https://click.palletsprojects.com/en/8.1.x/shell-completion/) looks like it would be pretty easy to implement. * [Progress bars](https://click.palletsprojects.com/en/8.1.x/utils/#showing-progress-bars) - imagine launching a DAG via the CLI and then viewing a progress bar while it finishes. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
