It would be great to see Ruff more. It performs well too over other linters. Plus any time we save during the migration process would be helpful to users.
On Fri, Nov 22, 2024 at 1:27 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Agree. Very good direction! > > On Thu, Nov 21, 2024 at 5:00 PM Julian LaNeve <jul...@astronomer.io.invalid > > > wrote: > > > This is awesome to see - I wrote the first rule in Ruff for Airflow, and > > it was super easy to do despite not having written any Rust prior. > > > > The Ruff maintainers are also incredibly responsive and helpful. I think > > Airflow’s one of the largest projects they work with, and they have us > > listed on the Ruff README. > > > > > On Nov 21, 2024, at 10:44 AM, Pierre Jeambrun <pierrejb...@gmail.com> > > wrote: > > > > > > That's a nice one, looking forward to it :) > > > > > > On Thu, Nov 21, 2024 at 12:55 PM Tzu-ping Chung > <t...@astronomer.io.invalid > > > > > > wrote: > > > > > >> Hi all, > > >> > > >> I am calling a LAZY CONSENSUS to officially anoint Ruff as the tool to > > >> implement lint rules that detect potential incompatibilities in > existing > > >> user code (including DAG files and plugins) that may break in Airflow > 3. > > >> The rationale is > > >> > > >> * Ruff provides the best set of features for our needs according to my > > >> research, most importantly import resolution, and also a nice CLI “for > > >> free”. > > >> > > >> * While a rule in Ruff must be implemented in Rust, which can be a > > hurdle > > >> to most Airflow contributors, it does not actually require a very deep > > >> understanding to Rust, but more toward the Python AST parser that Ruff > > >> provides (which is very similar to Python’s built-in ast). > > >> > > >> * We intend to only implement lint rules, not “fixes” (which would > allow > > >> Ruff to rewrite offending files automatically. This reduces the > learning > > >> curve even further. > > >> > > >> * Most Airflow contributors likely don’t know Python’s ast library > very > > >> well to begin with anyway, so the learning curve would not be much > > flatter > > >> even if we choose a Python-based tool. > > >> > > >> * We would not be the first Python library to use Ruff for this—Numpy > > also > > >> uses Ruff to detect potential incompatibilities for Numpy 2.0. We are > > also > > >> not the first to create Airflow rules in Ruff either—it already > contains > > >> one Airflow-specific rule. > > >> > > >> If we are to implement an auto fixer, it can still be rewritten in > > Python. > > >> With Ruff handling most of the user interaction (including generating > > >> proper error messages and a clear error code), the fixer can have a > > >> relatively simplistic CLI that can be more easily implemented with > just > > >> argparse and print() (or a little drop of Rich if we feel fancy). > > >> > > >> I have created a POC on generating lint errors against creating an > > Airflow > > >> DAG without using the 'schedule' argument correctly for Airflow 3: > > >> https://github.com/uranusjr/ruff/pull/1 > > >> > > >> If no one objects, the vote will pass three days from now, at 12:00 > > (noon) > > >> UTC, 24 November 2024. I intend to formally submit the rule > implemented > > >> above to Ruff, and start contributing other rules to Ruff. A list of > > rule > > >> candidates can be found in Wei’s thread “[Discuss] Airflow 2 to 3 > > migration > > >> rules” sent to this list earlier. > > >> > > >> Thanks all, > > >> TP > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > >> For additional commands, e-mail: dev-h...@airflow.apache.org > > >> > > >> > > > > > -- Bugra Ozturk
