Re: Re-run only tests that failed last time

[aRkadeFR]

Great tool, thanks :)

On 03/24/2015 11:22 PM, Russell Keith-Magee wrote:

Hi Gergely,

One option is to use a test suite GUI tool, like cricket:

http://pybee.org/cricket

Cricket lets you view your entire test suite as a tree, select subsets 
of the suite to run, see test results as the suite executes, and 
re-run the failures from the previous execution.


Yours,
Russ Magee %-)


On Tue, Mar 24, 2015 at 6:27 PM, Gergely Polonkai > wrote:


Hello,

I have a pretty extended test suite for my application, which can
run for about 15 minutes. Now when I introduce an error, I will be
notified of it pretty late, and a one-liner fix needs another 15
minutes to check. Of course if only one of my tests fail, I can
add the name of the test case as a parameter to manage.py test,
but in case of many failures it’s pretty hard to do the same.

Is there a way to run only the tests that failed last time, even
if I have to install a separate app/module for that? A quick
Google search gave me no usable results.

Best,
Gergely
-- 
You received this message because you are subscribed to the Google

Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To post to this group, send email to django-users@googlegroups.com
.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/CACczBU%2B-SR3uASD_eQc_c%2BKFp%3DvCgaWga6pfUgA%2BJHr5HgkCHQ%40mail.gmail.com

.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com 
.
To post to this group, send email to django-users@googlegroups.com 
.

Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAJxq84-hZn2VOe_dHRS63BXTmFJ8h3pbdsuKgQE1uFJo6csBZw%40mail.gmail.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55127937.3090406%40arkade.info.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Gergely Polonkai]

Hello Carl,

thanks, I will take a look at this, too!

Best,
Gergely

On 24 March 2015 at 23:33, Carl Meyer  wrote:

> Hi Gergely,
>
> On 03/24/2015 04:27 AM, Gergely Polonkai wrote:
> > I have a pretty extended test suite for my application, which can run
> > for about 15 minutes. Now when I introduce an error, I will be notified
> > of it pretty late, and a one-liner fix needs another 15 minutes to
> > check. Of course if only one of my tests fail, I can add the name of the
> > test case as a parameter to manage.py test, but in case of many failures
> > it’s pretty hard to do the same.
> >
> > Is there a way to run only the tests that failed last time, even if I
> > have to install a separate app/module for that? A quick Google search
> > gave me no usable results.
>
> I use py.test [1], another test runner which has plugins that offer this
> feature. The pytest-django [2] plugin provides the necessary integration
> to make py.test able to run a normal Django test suite. Then the
> pytest-cache [3] plugin offers an ``--lf`` (last-failed) option to run
> only the tests that failed in the last run, and the pytest-xdist [4]
> plugin offers the even-more-advanced ``--looponfail`` option, which "run
> your tests repeatedly in a subprocess. After each run py.test waits
> until a file in your project changes and then re-runs the previously
> failing tests. This is repeated until all tests pass after which again a
> full run is performed."
>
> If you're interested in advanced Python testing, I think py.test is near
> the top of the list of tools you should at least take for a spin.
>
> Carl
>
> [1] http://pytest.org/latest/
> [2] https://pypi.python.org/pypi/pytest-django/
> [3] https://pypi.python.org/pypi/pytest-cache
> [4] https://bitbucket.org/pytest-dev/pytest-xdist
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/5511E627.8090409%40oddbird.net
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CACczBUL4JS184G5Fx_kLQx59i2FVwhgcnp5K0nMD5uHGh3sppQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Carl Meyer]

Hi Gergely,

On 03/24/2015 04:27 AM, Gergely Polonkai wrote:
> I have a pretty extended test suite for my application, which can run
> for about 15 minutes. Now when I introduce an error, I will be notified
> of it pretty late, and a one-liner fix needs another 15 minutes to
> check. Of course if only one of my tests fail, I can add the name of the
> test case as a parameter to manage.py test, but in case of many failures
> it’s pretty hard to do the same.
> 
> Is there a way to run only the tests that failed last time, even if I
> have to install a separate app/module for that? A quick Google search
> gave me no usable results.

I use py.test [1], another test runner which has plugins that offer this
feature. The pytest-django [2] plugin provides the necessary integration
to make py.test able to run a normal Django test suite. Then the
pytest-cache [3] plugin offers an ``--lf`` (last-failed) option to run
only the tests that failed in the last run, and the pytest-xdist [4]
plugin offers the even-more-advanced ``--looponfail`` option, which "run
your tests repeatedly in a subprocess. After each run py.test waits
until a file in your project changes and then re-runs the previously
failing tests. This is repeated until all tests pass after which again a
full run is performed."

If you're interested in advanced Python testing, I think py.test is near
the top of the list of tools you should at least take for a spin.

Carl

[1] http://pytest.org/latest/
[2] https://pypi.python.org/pypi/pytest-django/
[3] https://pypi.python.org/pypi/pytest-cache
[4] https://bitbucket.org/pytest-dev/pytest-xdist

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5511E627.8090409%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Re-run only tests that failed last time

[Gergely Polonkai]

Hello Russ,

I've already installed the juno test runner, but a graphical tool cannot
hurt locally  I'll check it out tomorrow, thanks for the link!

Best,
Gergely
On 24 Mar 2015 23:22, "Russell Keith-Magee"  wrote:

> Hi Gergely,
>
> One option is to use a test suite GUI tool, like cricket:
>
> http://pybee.org/cricket
>
> Cricket lets you view your entire test suite as a tree, select subsets of
> the suite to run, see test results as the suite executes, and re-run the
> failures from the previous execution.
>
> Yours,
> Russ Magee %-)
>
>
> On Tue, Mar 24, 2015 at 6:27 PM, Gergely Polonkai 
> wrote:
>
>> Hello,
>>
>> I have a pretty extended test suite for my application, which can run for
>> about 15 minutes. Now when I introduce an error, I will be notified of it
>> pretty late, and a one-liner fix needs another 15 minutes to check. Of
>> course if only one of my tests fail, I can add the name of the test case as
>> a parameter to manage.py test, but in case of many failures it’s pretty
>> hard to do the same.
>>
>> Is there a way to run only the tests that failed last time, even if I
>> have to install a separate app/module for that? A quick Google search gave
>> me no usable results.
>>
>> Best,
>> Gergely
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/CACczBU%2B-SR3uASD_eQc_c%2BKFp%3DvCgaWga6pfUgA%2BJHr5HgkCHQ%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAJxq84-hZn2VOe_dHRS63BXTmFJ8h3pbdsuKgQE1uFJo6csBZw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CACczBULt18gAbJT1-1OWEd2Xy%2Bsk-201%3D%2Bs1EYgJ9zveyo-Uaw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Russell Keith-Magee]

Hi Gergely,

One option is to use a test suite GUI tool, like cricket:

http://pybee.org/cricket

Cricket lets you view your entire test suite as a tree, select subsets of
the suite to run, see test results as the suite executes, and re-run the
failures from the previous execution.

Yours,
Russ Magee %-)


On Tue, Mar 24, 2015 at 6:27 PM, Gergely Polonkai 
wrote:

> Hello,
>
> I have a pretty extended test suite for my application, which can run for
> about 15 minutes. Now when I introduce an error, I will be notified of it
> pretty late, and a one-liner fix needs another 15 minutes to check. Of
> course if only one of my tests fail, I can add the name of the test case as
> a parameter to manage.py test, but in case of many failures it’s pretty
> hard to do the same.
>
> Is there a way to run only the tests that failed last time, even if I have
> to install a separate app/module for that? A quick Google search gave me no
> usable results.
>
> Best,
> Gergely
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CACczBU%2B-SR3uASD_eQc_c%2BKFp%3DvCgaWga6pfUgA%2BJHr5HgkCHQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAJxq84-hZn2VOe_dHRS63BXTmFJ8h3pbdsuKgQE1uFJo6csBZw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Gergely Polonkai]

@Erik: On the CI side, certainly, I will run all the tests upon each
commit. I want to utilize this thing only on my developer machine, so I can
do a quick run after I think I fixed a problem. Naturally, if those failing
tests are green, I will run the whole suite again.

Separating/tweaking my current tests is not really an option; the whole
project is pretty big, with some large apps (yes, those must be cut in
smaller pieces, but that’s not today’s work yet). It will happen, though,
but thanks for pointing it out!

@Ramiro Morales: This juno test runner seems promising, thank you for the
link!

On 24 March 2015 at 15:08, Ramiro Morales  wrote:

> On Tue, Mar 24, 2015 at 7:27 AM, Gergely Polonkai 
> wrote:
> > Hello,
> >
> > I have a pretty extended test suite for my application, which can run for
> > about 15 minutes. Now when I introduce an error, I will be notified of it
> > pretty late, and a one-liner fix needs another 15 minutes to check. Of
> > course if only one of my tests fail, I can add the name of the test case
> as
> > a parameter to manage.py test, but in case of many failures it’s pretty
> hard
> > to do the same.
> >
> > Is there a way to run only the tests that failed last time, even if I
> have
> > to install a separate app/module for that? A quick Google search gave me
> no
> > usable results.
>
> Mine found:
>
> https://pypi.python.org/pypi/django-juno-testrunner
>
> http://nose.readthedocs.org/en/latest/plugins/testid.html#looping-over-failed-tests
> plus https://github.com/django-nose/django-nose
>
> --
> Ramiro Morales
> @ramiromorales
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAO7PdF_-Km0%2BHC9jcJ%3DfvFJEJZqQoA_Xp_uSL6r0cTCSXjhN%2Bg%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CACczBU%2B3gm7COWJnJqCd80u12Upk%3DhFnfjpZGF7D1MnJfMb9WQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Ramiro Morales]

On Tue, Mar 24, 2015 at 7:27 AM, Gergely Polonkai  wrote:
> Hello,
>
> I have a pretty extended test suite for my application, which can run for
> about 15 minutes. Now when I introduce an error, I will be notified of it
> pretty late, and a one-liner fix needs another 15 minutes to check. Of
> course if only one of my tests fail, I can add the name of the test case as
> a parameter to manage.py test, but in case of many failures it’s pretty hard
> to do the same.
>
> Is there a way to run only the tests that failed last time, even if I have
> to install a separate app/module for that? A quick Google search gave me no
> usable results.

Mine found:

https://pypi.python.org/pypi/django-juno-testrunner
http://nose.readthedocs.org/en/latest/plugins/testid.html#looping-over-failed-tests
plus https://github.com/django-nose/django-nose

-- 
Ramiro Morales
@ramiromorales

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAO7PdF_-Km0%2BHC9jcJ%3DfvFJEJZqQoA_Xp_uSL6r0cTCSXjhN%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Erik Cederstrand]

> Den 24/03/2015 kl. 11.27 skrev Gergely Polonkai :
> 
> I have a pretty extended test suite for my application, which can run for 
> about 15 minutes. Now when I introduce an error, I will be notified of it 
> pretty late, and a one-liner fix needs another 15 minutes to check. Of course 
> if only one of my tests fail, I can add the name of the test case as a 
> parameter to manage.py test, but in case of many failures it’s pretty hard to 
> do the same.
> 
> Is there a way to run only the tests that failed last time, even if I have to 
> install a separate app/module for that? A quick Google search gave me no 
> usable results.

Only re-running failed tests partly defeats the purpose of testing. Your fix 
for failing test case A may introduce a failure in test case B.

Instead, I would look into either tweaking you very slow test so they run 
faster, or splitting up your test cases into fast and slow tests, so you can 
run your slow tests less frequently depending on some setting.

Erik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/4A592356-7DB3-44B5-8ACC-332B4811E40D%40cederstrand.dk.
For more options, visit https://groups.google.com/d/optout.

Re: Re-run only tests that failed last time

[Andreas Kuhne]

If you have created packages for your tests, you can run all tests in a
package. If the tests aren't in the same package, it won't help you however.

Regards,

Andréas

2015-03-24 11:27 GMT+01:00 Gergely Polonkai :

> Hello,
>
> I have a pretty extended test suite for my application, which can run for
> about 15 minutes. Now when I introduce an error, I will be notified of it
> pretty late, and a one-liner fix needs another 15 minutes to check. Of
> course if only one of my tests fail, I can add the name of the test case as
> a parameter to manage.py test, but in case of many failures it’s pretty
> hard to do the same.
>
> Is there a way to run only the tests that failed last time, even if I have
> to install a separate app/module for that? A quick Google search gave me no
> usable results.
>
> Best,
> Gergely
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CACczBU%2B-SR3uASD_eQc_c%2BKFp%3DvCgaWga6pfUgA%2BJHr5HgkCHQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CALXYUbnx9KRCi%2BNSCbLsUp3Hym9V5DWcBSFuw%3D5-cK5HTwMyjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re-run only tests that failed last time

[Gergely Polonkai]

Hello,

I have a pretty extended test suite for my application, which can run for
about 15 minutes. Now when I introduce an error, I will be notified of it
pretty late, and a one-liner fix needs another 15 minutes to check. Of
course if only one of my tests fail, I can add the name of the test case as
a parameter to manage.py test, but in case of many failures it’s pretty
hard to do the same.

Is there a way to run only the tests that failed last time, even if I have
to install a separate app/module for that? A quick Google search gave me no
usable results.

Best,
Gergely

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CACczBU%2B-SR3uASD_eQc_c%2BKFp%3DvCgaWga6pfUgA%2BJHr5HgkCHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.