On 15 Jul 2024, at 14:03, Bruno Oliveira wrote: > Hi Florian, > > I find this plugin very useful, in fact I think it would make sense for this > to be in pytest-core given that `--lf` already exists.
I wouldn't mind having it in core. I just wanted people to easily try it first and maybe improve it before inclusion. Like pytest-warnings was done initially. Using ``config._do_configure`` would also be a non-issue if it is in core. > What if the --lf option can optionally accept a file name, which when given, > will read the set of failed tests from that file instead of reading it from > the cache. This way you can just save the `.pytest_cache/lastfailed` file > from CI, and reuse it locally: > > pytest --lf=lastfailed-file-from-ci > > Seems like the changes would be minimal for that to happen. Adding a json parser for that file would be trivial and it would also remove my gripe with the ``--restore-lf`` name. I don't think it is possible or wise to overwrite ``--lf`` from the plugin though. Also changing ``--lf`` to have an optional filename might cause problems with invocation if file paths are involved. Could be a backward compatibility issue. How would one access ``.pytest_cache/lastfailed`` from GitHub Actions for example? Just copy'n'paste from the log output seems more intuitive for most people, so the parsing should be kept. I'm not sure how user-friendly it is to explain how to copy ``.pytest_cache/lastfailed``, so I think ``--dump-lf`` would still be useful. > I know this is not exactly the feedback you wanted, but thought I would throw > the idea in there for your consideration. Totally valid feedback. Thanks! Regards, Florian > > Cheers, > Bruno > > On Mon, Jul 15, 2024, at 2:29 AM, Florian Schulze wrote: >> Hi! >> >> I wrote https://github.com/fschulze/pytest-failed and would like to get some >> feedback on a few things before I publish it. >> >> What is it? >> >> The plugin adds ``--dump-lf`` and ``--restore-lf`` options. >> >> My use case is two fold for test suites with runtimes of several minutes: >> >> 1. when tests fail in CI, I can copy the summary from the report and paste >> it into a file like ``failed.txt`` then re-run just those tests with >> ``--restore-lf=failed.txt`` >> >> 2. when fixing a bug I often fix only some of tests at the first attempt, >> then refactor the fix and want to re-run all initially failing tests. With >> this plugin I only run the full test suite once, then run with >> ``--dump-lf=failed.txt`` and can restore the initial bunch of tests with >> ``--restore-lf=failed.txt`` whenever I like, without ever re-running the >> full test suite. >> >> This is also super useful with ``git bisect`` and likely many other >> use-cases. >> >> The ``=`` in the options seems to be required, as otherwise the filename >> might be interpreted as ``[file_or_dir]`` by pytest in some forms of >> invokation. >> >> My questions: >> >> In >> [``pytest_cmdline_main``](https://github.com/fschulze/pytest-failed/blob/main/src/pytest_failed/plugin.py#L44) >> I use ``config._do_configure``, which is internal API. Is there a better >> way? >> >> Anyone have suggestions for better names of the plugin itself and its >> options? I think ``--dump-lf`` is ok, but I don't really like >> ``--restore-lf``. >> >> When reading the list of test names it knows about the pytest report summary >> syntax, are there any other formats that would be useful to support for >> cut'n'paste from CI? >> >> If you try the plugin, please give feedback on whether you found it useful >> and understand how it works. >> >> Regards, >> Florian Schulze >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev@python.org >> https://mail.python.org/mailman/listinfo/pytest-dev >> _______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
