Re: Adding an option to re-test only failed tests

[Dougal Matthews]

2009/10/2 Javier Guerra 
>
>
> - if there's no 'failure record' run all
> - if there's some record, first test those that have failed the last time
>   - if they still fail, stop there
>   - if there's no further failures, rerun the whole set
>
>
+1, that sounds like a good approach to me.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

> - if there's no 'failure record' run all
> - if there's some record, first test those that have failed the last time
>   - if they still fail, stop there
>   - if there's no further failures, rerun the whole set

That's a pretty cool idea.  I haven't seen this kind of behavior
before but it makes total sense.  I think it's the way everyone does
it anyway, it would just take the manual command line changes out of
the process.  I wonder what you would call such an option.  ./bin/test
--smartfail?

Rob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Javier Guerra]

On Fri, Oct 2, 2009 at 5:10 AM, Harro  wrote:
>
> Sounds like a bad plan, what if by fixing the failed test you break
> another one?

while testing, when i found some not-obvious test failure and i have
to run the test repeatedly, i try to run just this one until it
passes. then i rerun the whole set.

it would be very nice to have a flat to run just the ones that failed
last time.  after that, rerun the whole thing.

in the (very rare) case that fixing a test breaks another, it would be
caught by the full run at the end, maybe there could be an option so
that:

- if there's no 'failure record' run all
- if there's some record, first test those that have failed the last time
  - if they still fail, stop there
  - if there's no further failures, rerun the whole set

that seems the easiest use, while still not running unrelated tests
again and again.

-- 
Javier

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Harro]

Sounds like a bad plan, what if by fixing the failed test you break
another one?


On Sep 29, 6:03 pm, Rob Madole  wrote:
> I've been using nose for our tests, and one of the features that I
> really like is the ability to run the tests again but filter only the
> ones that caused a problem.
>
> I'm thinking it would look something like this
>
> ./manage.py test --failed
>
> Does this sound worthwhile to anybody?
>
> Rob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Simon Willison]

On Oct 2, 12:09 pm, Ned Batchelder  wrote:
> I would think the same logic applies to Django.  Nose needs to work with
> lots of different projects, so they can't own the Django details, since
> by that logic they'd also own the TurboGears logic, the Pylons logic,
> the Twisted logic, etc...  But the Nose plugin API isn't changing much
> now, so Django could include a nose plugin which would only have to
> change when Django details change.  And we'd get all sorts of nose
> features without having to implement them in the Django code.

That seems to make sense. I rather like the gesture here as well -
shipping plugins that get Django to work well with other parts of the
Python ecosystem sends a powerful message that Django wants to work
well with other parts of the Python ecosystem.

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Ned Batchelder]

Rob Madole wrote:
>> From the point of view of encouraging the usage of nose, either would
>> work fine. I think this is fits in to the conversation at DjangoCon
>> about how we should go about encouraging Django users to explore the
>> wider Python ecosystem. The important thing is that we can have some
>> official (or semi-official) documentation somewhere saying "to access
>> enhanced test running commands, install nose and drop this string in
>> to your TEST_RUNNER setting." The string itself could point at
>> dango.something or nose.something, the important thing from my point
>> of view is that they don't have to grab a test runner script from
>> another third party, then figure out where to put it within their
>> project. If nose don't want to ship the test runner, I'd be fine with
>> putting it in django.contrib somewhere.
>> 
>
> It's hard to get people to write tests.  It's hard to get them to
> document their work.  I wouldn't care if it's in Nose or Django that
> has the test runner, just as long as doing an "easy_install nose" and
> changing the TEST_RUNNER is all a user has to do to make the switch.
> I just can't imagine the Nose guys including a Django test runner in
> their base install.  Maybe I'm wrong.
>
>   
I've been wrestling with a similar issue in coverage.py:  As I update 
coverage.py and break the nose coverage plugin, how should it get 
fixed?  The nose developers don't seem to have the cycles to update the 
coverage plugin today.  I'm coming around to the conclusion that 
coverage.py should ship the nose coverage plugin, because coverage.py is 
changing faster than the nose plugin API is. 

I would think the same logic applies to Django.  Nose needs to work with 
lots of different projects, so they can't own the Django details, since 
by that logic they'd also own the TurboGears logic, the Pylons logic, 
the Twisted logic, etc...  But the Nose plugin API isn't changing much 
now, so Django could include a nose plugin which would only have to 
change when Django details change.  And we'd get all sorts of nose 
features without having to implement them in the Django code.

--Ned.
http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

> From the point of view of encouraging the usage of nose, either would
> work fine. I think this is fits in to the conversation at DjangoCon
> about how we should go about encouraging Django users to explore the
> wider Python ecosystem. The important thing is that we can have some
> official (or semi-official) documentation somewhere saying "to access
> enhanced test running commands, install nose and drop this string in
> to your TEST_RUNNER setting." The string itself could point at
> dango.something or nose.something, the important thing from my point
> of view is that they don't have to grab a test runner script from
> another third party, then figure out where to put it within their
> project. If nose don't want to ship the test runner, I'd be fine with
> putting it in django.contrib somewhere.

It's hard to get people to write tests.  It's hard to get them to
document their work.  I wouldn't care if it's in Nose or Django that
has the test runner, just as long as doing an "easy_install nose" and
changing the TEST_RUNNER is all a user has to do to make the switch.
I just can't imagine the Nose guys including a Django test runner in
their base install.  Maybe I'm wrong.

Simon's comment about documentation makes me think there needs to be
some detailed docs about how you would use this (beyond the "change
your TEST_RUNNER").  The Nose docs are good, but they don't have the
same flavor that the Django docs do.  The audience is different I
think.  If this is really going to be easy, I'd like to see docs about
how to use --pdb and --failed and --coverage built right into the
Django testing docs.

This may sunset or compete with two other tickets that are slotted for
1.2 though:
http://code.djangoproject.com/ticket/4501 (via Nose's coverage
support)
http://code.djangoproject.com/ticket/11613 (you can use --pdb and --
pdb-failures to get a similar behavior)

Rob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Simon Willison]

On Sep 30, 5:47 am, Russell Keith-Magee 
wrote:
> I'm yet to be convinced that Nose should be the default test runner
> for the simple reason that it doesn't come out of the box with Python.
> However, I agree that using Nose with Django should be as painless as
> possible. I included the TEST_RUNNER setting specifically for this
> reason.

Agreed - adding nose as a dependency for running unit tests would make
people less, not more likely to run them.

> Then there is the issue of whether Django should be responsible for
> shipping a Nose-compatible test runner, or whether that is Nose's
> responsibility. Historically, I've been of the opinion that it should
> be Nose's responsibility to ship a test runner, but I'm open to other
> opinions on this.

>From the point of view of encouraging the usage of nose, either would
work fine. I think this is fits in to the conversation at DjangoCon
about how we should go about encouraging Django users to explore the
wider Python ecosystem. The important thing is that we can have some
official (or semi-official) documentation somewhere saying "to access
enhanced test running commands, install nose and drop this string in
to your TEST_RUNNER setting." The string itself could point at
dango.something or nose.something, the important thing from my point
of view is that they don't have to grab a test runner script from
another third party, then figure out where to put it within their
project. If nose don't want to ship the test runner, I'd be fine with
putting it in django.contrib somewhere.

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Russell Keith-Magee]

On Wed, Sep 30, 2009 at 3:47 AM, Simon Willison  wrote:
>
> On Sep 29, 7:34 pm, Rob Madole  wrote:
>> TEST_RUNNER = 'django.contrib.test.nose.run_tests'
>>
>> There might be some futzy bits to make that actually work, but I think
>> it'd doable.
>
> I'd love to see this working. Obviously this would work just as well
> implemented as an external project - but if it's as useful as it
> sounds I'd personally be up for including it in core, if only to
> promote the usage of nose amongst Django developers (and hence help
> weaken the impression that Django doesn't integrate well enough with
> the wider Python community).

I'm yet to be convinced that Nose should be the default test runner
for the simple reason that it doesn't come out of the box with Python.
However, I agree that using Nose with Django should be as painless as
possible. I included the TEST_RUNNER setting specifically for this
reason.

I haven't seen any reports in Trac that the capabilities of
TEST_RUNNER aren't suitable for the task, and I've seen several
snippets and blog posts indicating that Nose can be used with Django.
If I'm incorrect on this, please let me know (or better still, open a
ticket in Trac).

Then there is the issue of whether Django should be responsible for
shipping a Nose-compatible test runner, or whether that is Nose's
responsibility. Historically, I've been of the opinion that it should
be Nose's responsibility to ship a test runner, but I'm open to other
opinions on this.

I'm slightly hesitant to start building extra features like --failfast
into Django's testing tools, especially when tools like Nose already
cover this territory. Django isn't a test framework tool, and I'm
hesitant to support proposals that  try to turn it into one.

Russ %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Jeff Balogh]

On Tue, Sep 29, 2009 at 8:56 PM, Rob Madole  wrote:
>
> I'll see if I can talk Jeff into adding what he's got as a start to
> this.  It looks solid to me.
>
> Ticket and patches forthcoming...

The nose test-runner that I'm currently using is at
http://gist.github.com/197593.  (The code I point to in the blog has
gotten more convoluted than I'd like; it turns out that can be a
problem if you're pointing at a vcs repo.)

For the most part, switching test runners is simple since Django
exposes all the setup and teardown functions.  The hacky part is
passing extra arguments to nose:  Django doesn't recognize nose
arguments so it complains and dies if you try something like
`./manage.py test --pdb-fail`.  Since I just wanted to get something
working, I pass all arguments after a '--' on the command line to nose
and Django happily ignores them.  If something like nose_runner is
shipped with Django it should run normally and be able to show the
same help screen that `nose --help` shows.

It was one of those "I'll come back and fix it later" sort of things. ;)

Cheers,
jeff

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

I'll see if I can talk Jeff into adding what he's got as a start to
this.  It looks solid to me.

Ticket and patches forthcoming...

On Sep 29, 2:47 pm, Simon Willison  wrote:
> On Sep 29, 7:34 pm, Rob Madole  wrote:
>
> > TEST_RUNNER = 'django.contrib.test.nose.run_tests'
>
> > There might be some futzy bits to make that actually work, but I think
> > it'd doable.
>
> I'd love to see this working. Obviously this would work just as well
> implemented as an external project - but if it's as useful as it
> sounds I'd personally be up for including it in core, if only to
> promote the usage of nose amongst Django developers (and hence help
> weaken the impression that Django doesn't integrate well enough with
> the wider Python community).
>
> Cheers,
>
> Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Simon Willison]

On Sep 29, 7:34 pm, Rob Madole  wrote:
> TEST_RUNNER = 'django.contrib.test.nose.run_tests'
>
> There might be some futzy bits to make that actually work, but I think
> it'd doable.

I'd love to see this working. Obviously this would work just as well
implemented as an external project - but if it's as useful as it
sounds I'd personally be up for including it in core, if only to
promote the usage of nose amongst Django developers (and hence help
weaken the impression that Django doesn't integrate well enough with
the wider Python community).

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

http://blog.jeffbalogh.org/post/57653515/nose-test-runner-for-django

It's certainly been done and doesn't require changes to Django.

On Sep 29, 1:34 pm, Rob Madole  wrote:
> Ok, --failfast would be nice too :D, I think I remember seeing a
> ticket on that.  So make that 4 features from nose...
>
> Which would be great if the test is third or fourth in the stack.  If
> it's the last test in 50, it would loose it's effectiveness.
>
> I know, I know.  If you are running 50 tests you can reduce that down
> to the module that is causing the problem.
>
> Maybe time would be better spent making the use of nose really super
> easy.
>
> In settings.py:
>
> TEST_RUNNER = 'django.contrib.test.nose.run_tests'
>
> There might be some futzy bits to make that actually work, but I think
> it'd doable.
>
> Eh?
>
> Rob
>
> On Sep 29, 1:23 pm, Rob Madole  wrote:
>
>
>
> > Yep, I use the pdb stuff too.  That would be handy.
>
> > The way this works in nose is through the testid plugin. Typically you
> > do this:
>
> > nosetests --with-id --failed
>
> > This will create a file called .noseids in the current working
> > directory.
>
> > You can make it use something else by saying:
>
> > nosetests --with-id --id-file=/somewhere/else/.noseids --failed
>
> > As far as storing the data of which test failed for Django, I'm not
> > sure what the *best* approach would be.  Ned Batchelder's coverage
> > module does a similar thing.  It keeps a .coverage file in the root I
> > think.  Maybe just call ours .failedtests.  Kinda gross, and not my
> > first choice, but it would work.
>
> > Or, perhaps use Python's tempfile module.  But I'm not sure how to
> > grab a hold of the temp file again for the second pass through (maybe
> > tempfile.NamedTemporaryFile but this has problems on some platforms
> > according to the docs).
>
> > On one hand, I can see this argument: If you are adding 3 features
> > from nose, why not just use nose.  But setting up nose and Django to
> > use it as the test runner isn't trivial the last time I checked.
> > We're using buildout to ease the pain.
>
> > Thanks for the input.
>
> > Rob
>
> > On Sep 29, 12:58 pm, Simon Willison  wrote:
>
> > > On Sep 29, 5:03 pm, Rob Madole  wrote:
>
> > > > I've been using nose for our tests, and one of the features that I
> > > > really like is the ability to run the tests again but filter only the
> > > > ones that caused a problem.
>
> > > > I'm thinking it would look something like this
>
> > > > ./manage.py test --failed
>
> > > > Does this sound worthwhile to anybody?
>
> > > I don't understand how this works - does it persist some indication of
> > > which tests failed somewhere? If so, where?
>
> > > If we're talking about features from nose, the two I'd really like in
> > > Django's test runner are --pdb and --pdb-failures:
>
> > > --pdb = when an error occurs, drop straight in to the interactive
> > > debugger
> > > --pdb-failures = when a test assertion fails, drop in to the debugger
>
> > > Cheers,
>
> > > Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

Ok, --failfast would be nice too :D, I think I remember seeing a
ticket on that.  So make that 4 features from nose...

Which would be great if the test is third or fourth in the stack.  If
it's the last test in 50, it would loose it's effectiveness.

I know, I know.  If you are running 50 tests you can reduce that down
to the module that is causing the problem.

Maybe time would be better spent making the use of nose really super
easy.

In settings.py:

TEST_RUNNER = 'django.contrib.test.nose.run_tests'

There might be some futzy bits to make that actually work, but I think
it'd doable.

Eh?

Rob

On Sep 29, 1:23 pm, Rob Madole  wrote:
> Yep, I use the pdb stuff too.  That would be handy.
>
> The way this works in nose is through the testid plugin. Typically you
> do this:
>
> nosetests --with-id --failed
>
> This will create a file called .noseids in the current working
> directory.
>
> You can make it use something else by saying:
>
> nosetests --with-id --id-file=/somewhere/else/.noseids --failed
>
> As far as storing the data of which test failed for Django, I'm not
> sure what the *best* approach would be.  Ned Batchelder's coverage
> module does a similar thing.  It keeps a .coverage file in the root I
> think.  Maybe just call ours .failedtests.  Kinda gross, and not my
> first choice, but it would work.
>
> Or, perhaps use Python's tempfile module.  But I'm not sure how to
> grab a hold of the temp file again for the second pass through (maybe
> tempfile.NamedTemporaryFile but this has problems on some platforms
> according to the docs).
>
> On one hand, I can see this argument: If you are adding 3 features
> from nose, why not just use nose.  But setting up nose and Django to
> use it as the test runner isn't trivial the last time I checked.
> We're using buildout to ease the pain.
>
> Thanks for the input.
>
> Rob
>
> On Sep 29, 12:58 pm, Simon Willison  wrote:
>
>
>
> > On Sep 29, 5:03 pm, Rob Madole  wrote:
>
> > > I've been using nose for our tests, and one of the features that I
> > > really like is the ability to run the tests again but filter only the
> > > ones that caused a problem.
>
> > > I'm thinking it would look something like this
>
> > > ./manage.py test --failed
>
> > > Does this sound worthwhile to anybody?
>
> > I don't understand how this works - does it persist some indication of
> > which tests failed somewhere? If so, where?
>
> > If we're talking about features from nose, the two I'd really like in
> > Django's test runner are --pdb and --pdb-failures:
>
> > --pdb = when an error occurs, drop straight in to the interactive
> > debugger
> > --pdb-failures = when a test assertion fails, drop in to the debugger
>
> > Cheers,
>
> > Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Rob Madole]

Yep, I use the pdb stuff too.  That would be handy.

The way this works in nose is through the testid plugin. Typically you
do this:

nosetests --with-id --failed

This will create a file called .noseids in the current working
directory.

You can make it use something else by saying:

nosetests --with-id --id-file=/somewhere/else/.noseids --failed

As far as storing the data of which test failed for Django, I'm not
sure what the *best* approach would be.  Ned Batchelder's coverage
module does a similar thing.  It keeps a .coverage file in the root I
think.  Maybe just call ours .failedtests.  Kinda gross, and not my
first choice, but it would work.

Or, perhaps use Python's tempfile module.  But I'm not sure how to
grab a hold of the temp file again for the second pass through (maybe
tempfile.NamedTemporaryFile but this has problems on some platforms
according to the docs).

On one hand, I can see this argument: If you are adding 3 features
from nose, why not just use nose.  But setting up nose and Django to
use it as the test runner isn't trivial the last time I checked.
We're using buildout to ease the pain.

Thanks for the input.

Rob

On Sep 29, 12:58 pm, Simon Willison  wrote:
> On Sep 29, 5:03 pm, Rob Madole  wrote:
>
> > I've been using nose for our tests, and one of the features that I
> > really like is the ability to run the tests again but filter only the
> > ones that caused a problem.
>
> > I'm thinking it would look something like this
>
> > ./manage.py test --failed
>
> > Does this sound worthwhile to anybody?
>
> I don't understand how this works - does it persist some indication of
> which tests failed somewhere? If so, where?
>
> If we're talking about features from nose, the two I'd really like in
> Django's test runner are --pdb and --pdb-failures:
>
> --pdb = when an error occurs, drop straight in to the interactive
> debugger
> --pdb-failures = when a test assertion fails, drop in to the debugger
>
> Cheers,
>
> Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Waylan Limberg]

On Tue, Sep 29, 2009 at 1:58 PM, Simon Willison  wrote:
>
> On Sep 29, 5:03 pm, Rob Madole  wrote:
>> I've been using nose for our tests, and one of the features that I
>> really like is the ability to run the tests again but filter only the
>> ones that caused a problem.
>>
>> I'm thinking it would look something like this
>>
>> ./manage.py test --failed
>>
>> Does this sound worthwhile to anybody?
>
> I don't understand how this works - does it persist some indication of
> which tests failed somewhere? If so, where?
>

My recollection is yes, nose persists which tests passed/failed for
the previous run, but I don't recall where.

Personally, I've never used this (except to see how it works in nose)
because I'm always concerned that while a recent change may fix a
failing test, it could cause some other test to fail that previously
passed. So I'm going to run the whole test suite anyway - or at least
all the tests for that module, etc.

Alex suggestion of --failfast seems like a much more useful way to
shorten test runs.

-- 

\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Alex Gaynor]

On Tue, Sep 29, 2009 at 1:58 PM, Simon Willison  wrote:
>
> On Sep 29, 5:03 pm, Rob Madole  wrote:
>> I've been using nose for our tests, and one of the features that I
>> really like is the ability to run the tests again but filter only the
>> ones that caused a problem.
>>
>> I'm thinking it would look something like this
>>
>> ./manage.py test --failed
>>
>> Does this sound worthwhile to anybody?
>
> I don't understand how this works - does it persist some indication of
> which tests failed somewhere? If so, where?
>
> If we're talking about features from nose, the two I'd really like in
> Django's test runner are --pdb and --pdb-failures:
>
> --pdb = when an error occurs, drop straight in to the interactive
> debugger
> --pdb-failures = when a test assertion fails, drop in to the debugger
>
> Cheers,
>
> Simon
> >
>

If we're throwing off testing ponies: --failfast.  Runs the full test
suite as normal but as soon as it hits a failure it spits out the fail
and ends.  Unladen Swallow added this to their test suite and it's a
huge help in terms of not running the full test suite to see what the
failure was.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

[Simon Willison]

On Sep 29, 5:03 pm, Rob Madole  wrote:
> I've been using nose for our tests, and one of the features that I
> really like is the ability to run the tests again but filter only the
> ones that caused a problem.
>
> I'm thinking it would look something like this
>
> ./manage.py test --failed
>
> Does this sound worthwhile to anybody?

I don't understand how this works - does it persist some indication of
which tests failed somewhere? If so, where?

If we're talking about features from nose, the two I'd really like in
Django's test runner are --pdb and --pdb-failures:

--pdb = when an error occurs, drop straight in to the interactive
debugger
--pdb-failures = when a test assertion fails, drop in to the debugger

Cheers,

Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Adding an option to re-test only failed tests

[Rob Madole]

I've been using nose for our tests, and one of the features that I
really like is the ability to run the tests again but filter only the
ones that caused a problem.

I'm thinking it would look something like this

./manage.py test --failed

Does this sound worthwhile to anybody?

Rob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---