Re: test discovery

[Chris Wilson]

Hi Carl,

On Sat, 18 May 2013, Carl Meyer wrote:

I don't think this should be fixed in the test runner itself; in 
general, file-path test labels _should_ be interpreted as relative to 
wherever you are running the tests from.


But it should be fixed in the 
test_runner.test_discover_runner.DiscoverRunnerTest.test_file_path test 
- that test apparently needs to isolate itself better by setting the CWD 
for the duration of the test, or something similar. Mind filing a bug? I 
should be able to take a look soon.


Done, thanks: https://code.djangoproject.com/ticket/20449

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Chris,

On May 18, 2013, at 8:34 AM, Chris Wilson  wrote:

> On Sat, 18 May 2013, Chris Wilson wrote:
> 
>> I think Travis is unhappy about something in this commit. Any ideas?
>> 
>> ==
>> ERROR: test_file_path (test_runner.test_discover_runner.DiscoverRunnerTest)
>> --
>> Traceback (most recent call last):
>> File 
>> "/home/travis/build/aptivate/django/tests/test_runner/test_discover_runner.py",
>>  line 65, in test_file_path
>>   ["test_discovery_sample/"],
>> File "/home/travis/build/aptivate/django/django/test/runner.py", line 63, in 
>> build_suite
>>   tests = self.test_loader.loadTestsFromName(label)
>> File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
>>   module = __import__('.'.join(parts_copy))
>> ImportError: Import by filename is not supported.
>> 
>> 
> 
> It seems to be checking if a file exists, using a relative path, in 
> django/test/runner.py:65:
> 
>if not os.path.exists(label_as_path):
>tests = self.test_loader.loadTestsFromName(label)
> 
> And that will behave differently depending if you run your tests from inside 
> the tests directory like this:
> 
>./runtests.py -v2 --settings=test_postgres_nogis test_runner
> 
> Or from the parent directory, as Travis currently does, like this:
> 
>python -Wall tests/runtests.py --selenium --verbosity 2 \
>--settings=django_settings
> 
> I can work around it in Travis by changing working directory before running 
> the tests, but is it worth fixing this in the test runner, perhaps using an 
> absolute path based on __file__?

I don't think this should be fixed in the test runner itself; in general, 
file-path test labels _should_ be interpreted as relative to wherever you are 
running the tests from. 

But it should be fixed in the 
test_runner.test_discover_runner.DiscoverRunnerTest.test_file_path test - that 
test apparently needs to isolate itself better by setting the CWD for the 
duration of the test, or something similar. Mind filing a bug? I should be able 
to take a look soon.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Chris,

On May 18, 2013, at 9:42 AM, Chris Wilson  wrote:
> Another odd behaviour of the new test runner. This runs the tests, but fails 
> to add the test app to INSTALLED_APPS, so they all fail because their tables 
> are not created:
> 
>PYTHONPATH=.. python -Wall runtests.py --selenium --verbosity 2 \
>--settings=tests.travis_configs.test_postgres_nogis \
>tests.transactions.tests.AtomicInsideTransactionTests
> 
> Changing the spec to remove the initial "tests." results in the app being 
> loaded and the tests run:
> 
>PYTHONPATH=.. python -Wall runtests.py --selenium --verbosity 2 \
>--settings=tests.travis_configs.test_postgres_nogis \
>transactions.tests.AtomicInsideTransactionTests
> 
> Is this expected behaviour? It's rather counter-intuitive to me.

The initial "tests." prefix shouldn't be used, and wasn't intended to be 
supported. There is no __init__.py in the tests/ directory, and when running 
runtests.py the tests/ directory itself is placed on sys.path, so every module 
in there is a top-level module when running tests. I'm actually surprised 
unittest itself even runs the tests when specified that way, given the lack of 
__init__.py.

That said, if this turns out to to be something that trips a lot of people up, 
it would be trivial to add a special-case in runtests.py to look for and remove 
the initial "tests." in provided test labels.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Chris Wilson]

Hi all,

On Fri, 10 May 2013, Carl Meyer wrote:

I merged this patch tonight. Thanks to everyone who contributed! Now 
let's see how the CI servers feel about it...


I think Travis is unhappy about something in this commit. Any ideas?

==
ERROR: test_file_path 
(test_runner.test_discover_runner.DiscoverRunnerTest)

--
Traceback (most recent call last):
  File 
"/home/travis/build/aptivate/django/tests/test_runner/test_discover_runner.py", 
line 65, in test_file_path

["test_discovery_sample/"],
  File "/home/travis/build/aptivate/django/django/test/runner.py", line 
63, in build_suite

tests = self.test_loader.loadTestsFromName(label)
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in 
loadTestsFromName

module = __import__('.'.join(parts_copy))
ImportError: Import by filename is not supported.



Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Daniel Lindsley]

Russ & Carl,


I'm interested in this one, so I sketched up a potential avenue tonight 
(https://github.com/toastdriven/django/compare/check_setup). I actually 
implemented Russ' setup first, but ran into a couple issues, so I pared it 
down into an "opt-in" scheme. If there's a feeling like this is a 
worthwhile start, I'd be happy to flesh it out further, adding tests docs, 
refinements, etc. If not, no big deal.


Daniel



On Friday, May 10, 2013 8:32:38 PM UTC-7, Carl Meyer wrote:
>
> On Thursday, May 9, 2013 8:15:02 PM UTC-4, Russell Keith-Magee wrote:
>
>> On Thu, May 9, 2013 at 11:30 PM, Carl Meyer <ca...@oddbird.net> wrote:
>>
>>> Hi Russ,
>>>
>>> On 05/09/2013 04:51 AM, Russell Keith-Magee wrote:
>>> > Great work Carl and Preston! (And everyone else who had a hand in the
>>> > patch - I know this has been kicking around for a while now)
>>>
>>> Thanks for the reminder; I should have mentioned that Mahdi Yusuf did
>>> the initial draft patch, based on Jannis' django-discover-runner, which
>>> was based on code I showed on a slide in my "Django and Testing" talk at
>>> PyCon 2012; it all comes full circle :-)
>>>
>>> > I agree with Jacob's tests aren't production sensitive, so making a 
>>> fast
>>> > transition to the new test discovery has less impact.
>>> >
>>> > However, we know from experience that it doesn't matter how much we 
>>> flag
>>> > this change in the release notes, they won't be read, and *someone* is
>>> > going to get stung. In an ideal world, it would be good to be able to
>>> > warn people who upgrade that their test suite may not be run as 
>>> expected.
>>> >
>>> > We've had a proposal kicking around for a while to add a management
>>> > command that does an "upgrade check". I'm wondering if this might be an
>>> > opportune time to dig into this some more.
>>> >
>>> > To be clear, this isn't something I consider to be a merge blocker. I'd
>>> > be happy to see 1.6 released with a fast transition to the new test
>>> > runner. It would just be nice icing on the cake if we can get an 
>>> upgrade
>>> > validator included in the same release.
>>>
>>> It wouldn't be too hard to write something that tried test discovery
>>> using the old runner and using the new runner and did a simple
>>> comparison of test counts to warn you if they differ.
>>>
>>> I'm not personally likely to work on this "upgrade checks" framework,
>>> but if it does happen I'd be happy to contribute this check to it.
>>
>>  
>> I thought about that approach, but my immediate reaction was that it will 
>> end up producing more false positives than helpful feedback. Even the 
>> simple case -- myapp.TestCase -> myapp.tests.TestCase -- is going to 
>> require some name munging to work out if the old test and the new test are 
>> actually the same one. Since this is for a short-lived transitional tool, 
>> the effort involved in getting it right exceeds the value gained by having 
>> it, IMHO.
>>
>
> I wasn't anticipating trying to actually match up individual tests by 
> name/path, just trying both runners and seeing whether the test counts 
> match. On second thought, though, I realize that contrib / third-party apps 
> will mean this check would alert "missing tests!" for pretty much everyone, 
> so it would need to be smarter to be useful, and you're right that's likely 
> not worth it.
>  
>
>> I'd prefer something much simpler:
>>
>>  * On syncdb/validate, check for a marker somewhere in user space. 
>>
>>  * If the marker isn't present, do a check for likely problems. In this 
>> case, look for the value of TEST_RUNNER; if it's set to the new default 
>> value, display a warning about the change in test discovery format. Add 
>> similar checks for other setting changes, or warnings about features that 
>> have been fully deprecated (e.g., the final transition of the {% url %} 
>> behavior)
>>
>>  * Provide a management command to set marker. 
>>
>>  * On the next syncdb/validate, the marker is present, so the install is 
>> verified as being "updated", and no warnings are displayed.
>>
>> This leaves the question of where to put the marker. Some initial ideas:
>>
>>  - a file on disk (e.g., an .updated file next to the settings file)
>>  - a new setting (VERIFIED_VERSION = "1.5")
>>  - a key-value tabl

Re: test discovery

[Florian Apolloner]

On Saturday, May 11, 2013 2:38:23 PM UTC+2, Carl Meyer wrote:
>
> No good reason, just an oversight I think. If that's all that's needed to 
> make the CI happy, feel free to change it, should be a simple fix.
>

Perfect, pushed a fix, let's see what jenkins says. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Florian,

On May 11, 2013, at 7:51 AM, Florian Apolloner  wrote:

> Hi Carl,
> 
> before I read all the tickets etc; why does runtests.py not use the 
> TEST_RUNNER from settings.py (see 
> https://github.com/django/django/commit/9012833af857e081b515ce760685b157638efcef#L60L149)?
>  We'd need that for jenkins to produce xml files as output.

No good reason, just an oversight I think. If that's all that's needed to make 
the CI happy, feel free to change it, should be a simple fix.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Florian Apolloner]

Hi Carl,

before I read all the tickets etc; why does runtests.py not use the 
TEST_RUNNER from settings.py (see 
https://github.com/django/django/commit/9012833af857e081b515ce760685b157638efcef#L60L149)?
 
We'd need that for jenkins to produce xml files as output.

Thanks,
Florian 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Florian Apolloner]

Not good, at least our Jenkins runner which should generate xml output 
doesn't like it :/

On Saturday, May 11, 2013 5:36:55 AM UTC+2, Carl Meyer wrote:
>
> I merged this patch tonight. Thanks to everyone who contributed! Now let's 
> see how the CI servers feel about it...
>
> Carl
>
> On Wednesday, May 8, 2013 5:00:56 PM UTC-4, Carl Meyer wrote:
>>
>> Preston Timmons and I have been working the last several weeks to get 
>> the test discovery branch (#17365) ready for merge, and I think we now 
>> have a pull request ready for consideration: 
>> https://github.com/django/django/pull/1050 
>>
>> Brief summary of the features, changes, and open questions: 
>>
>> * You can now place tests in any file matching the pattern 'test*.py', 
>> anywhere in your codebase, rather than only in tests.py and models.py 
>> modules of an INSTALLED_APP. The filename pattern is customizable via 
>> the --pattern argument to "manage.py test". 
>>
>> * When you run "manage.py test" with no arguments, tests are discovered 
>> and run in all subdirectories (recursively) of the current working 
>> directory. (This means that django.contrib and other third-party app 
>> tests are no longer run by default). 
>>
>> * When you pass arguments to "manage.py test", they are now full Python 
>> dotted module paths. So if you have a "myproject.myapp" app, and its 
>> tests.py contains "SomeTestCase", you would now run that single test 
>> case via "manage.py myproject.myapp.tests.SomeTestCase" rather than 
>> "manage.py myapp.SomeTestCase". This is longer, but allows more control 
>> when an app's tests are split into multiple files, and allows for tests 
>> to be placed anywhere you like. 
>>
>> * Doctests are no longer discovered by default; you will need to 
>> explicitly integrate them with your test suite as recommended in the 
>> Python docs: http://docs.python.org/2/library/doctest.html#unittest-api 
>>
>> * Tests in models.py and tests/__init__.py will no longer be discovered 
>> (as those don't match the default filename pattern). 
>>
>> * The old test runner, and Django's extensions to the doctest module, 
>> are deprecated and will be removed in Django 1.8; they could of course 
>> be packaged separately if some people would like to continue using them. 
>>
>> Open question: how to handle the transition? 
>>
>> Jacob has suggested that back-compat breaks in test-running are not as 
>> serious as in production code, and that we should just switch to the new 
>> test runner by default in Django 1.6. This is what the pull request 
>> currently does. This will mean that some people's test suites will 
>> likely be broken when they upgrade to 1.6. They would have two options, 
>> both documented in the release notes: they can update their test suite 
>> to be discovery-compatible immediately, or they can just set TEST_RUNNER 
>> to point to the old runner and get back the old behavior, which they can 
>> keep using until Django 1.8 (or longer, if they package the old runner 
>> externally). 
>>
>> The alternative would be to keep the old test runner as the default in 
>> global_settings.py until it is removed in 1.8, and add the new runner as 
>> the TEST_RUNNER in the startproject-generated settings.py. This would 
>> provide a gentler transition for upgrading projects. On the other hand, 
>> we just recently got the startproject settings.py all cleaned-up, 
>> slimmed-down, and newbie-friendly, so it would make me sad to start 
>> polluting it again with stuff that new projects generally shouldn't care 
>> about, for solely legacy reasons. 
>>
>> Thoughts, questions, comments, code review and testing welcome! I'd like 
>> to merge this on Friday (it's a bear to keep updated with trunk), so let 
>> me know if you need longer. 
>>
>> Carl 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Yo-Yo Ma]

Wow, this is awesome! Thanks so much guys. Too long have I dreamt of the day 
when I could include tests for my "lib" and "util", etc. without having to 
couple them to one app or another. I'm so excited that this was added so 
quickly. 


Thanks
Yo-yo

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

I merged this patch tonight. Thanks to everyone who contributed! Now let's 
see how the CI servers feel about it...

Carl

On Wednesday, May 8, 2013 5:00:56 PM UTC-4, Carl Meyer wrote:
>
> Preston Timmons and I have been working the last several weeks to get 
> the test discovery branch (#17365) ready for merge, and I think we now 
> have a pull request ready for consideration: 
> https://github.com/django/django/pull/1050 
>
> Brief summary of the features, changes, and open questions: 
>
> * You can now place tests in any file matching the pattern 'test*.py', 
> anywhere in your codebase, rather than only in tests.py and models.py 
> modules of an INSTALLED_APP. The filename pattern is customizable via 
> the --pattern argument to "manage.py test". 
>
> * When you run "manage.py test" with no arguments, tests are discovered 
> and run in all subdirectories (recursively) of the current working 
> directory. (This means that django.contrib and other third-party app 
> tests are no longer run by default). 
>
> * When you pass arguments to "manage.py test", they are now full Python 
> dotted module paths. So if you have a "myproject.myapp" app, and its 
> tests.py contains "SomeTestCase", you would now run that single test 
> case via "manage.py myproject.myapp.tests.SomeTestCase" rather than 
> "manage.py myapp.SomeTestCase". This is longer, but allows more control 
> when an app's tests are split into multiple files, and allows for tests 
> to be placed anywhere you like. 
>
> * Doctests are no longer discovered by default; you will need to 
> explicitly integrate them with your test suite as recommended in the 
> Python docs: http://docs.python.org/2/library/doctest.html#unittest-api 
>
> * Tests in models.py and tests/__init__.py will no longer be discovered 
> (as those don't match the default filename pattern). 
>
> * The old test runner, and Django's extensions to the doctest module, 
> are deprecated and will be removed in Django 1.8; they could of course 
> be packaged separately if some people would like to continue using them. 
>
> Open question: how to handle the transition? 
>
> Jacob has suggested that back-compat breaks in test-running are not as 
> serious as in production code, and that we should just switch to the new 
> test runner by default in Django 1.6. This is what the pull request 
> currently does. This will mean that some people's test suites will 
> likely be broken when they upgrade to 1.6. They would have two options, 
> both documented in the release notes: they can update their test suite 
> to be discovery-compatible immediately, or they can just set TEST_RUNNER 
> to point to the old runner and get back the old behavior, which they can 
> keep using until Django 1.8 (or longer, if they package the old runner 
> externally). 
>
> The alternative would be to keep the old test runner as the default in 
> global_settings.py until it is removed in 1.8, and add the new runner as 
> the TEST_RUNNER in the startproject-generated settings.py. This would 
> provide a gentler transition for upgrading projects. On the other hand, 
> we just recently got the startproject settings.py all cleaned-up, 
> slimmed-down, and newbie-friendly, so it would make me sad to start 
> polluting it again with stuff that new projects generally shouldn't care 
> about, for solely legacy reasons. 
>
> Thoughts, questions, comments, code review and testing welcome! I'd like 
> to merge this on Friday (it's a bear to keep updated with trunk), so let 
> me know if you need longer. 
>
> Carl 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

On Thursday, May 9, 2013 8:15:02 PM UTC-4, Russell Keith-Magee wrote:

> On Thu, May 9, 2013 at 11:30 PM, Carl Meyer <ca...@oddbird.net
> > wrote:
>
>> Hi Russ,
>>
>> On 05/09/2013 04:51 AM, Russell Keith-Magee wrote:
>> > Great work Carl and Preston! (And everyone else who had a hand in the
>> > patch - I know this has been kicking around for a while now)
>>
>> Thanks for the reminder; I should have mentioned that Mahdi Yusuf did
>> the initial draft patch, based on Jannis' django-discover-runner, which
>> was based on code I showed on a slide in my "Django and Testing" talk at
>> PyCon 2012; it all comes full circle :-)
>>
>> > I agree with Jacob's tests aren't production sensitive, so making a fast
>> > transition to the new test discovery has less impact.
>> >
>> > However, we know from experience that it doesn't matter how much we flag
>> > this change in the release notes, they won't be read, and *someone* is
>> > going to get stung. In an ideal world, it would be good to be able to
>> > warn people who upgrade that their test suite may not be run as 
>> expected.
>> >
>> > We've had a proposal kicking around for a while to add a management
>> > command that does an "upgrade check". I'm wondering if this might be an
>> > opportune time to dig into this some more.
>> >
>> > To be clear, this isn't something I consider to be a merge blocker. I'd
>> > be happy to see 1.6 released with a fast transition to the new test
>> > runner. It would just be nice icing on the cake if we can get an upgrade
>> > validator included in the same release.
>>
>> It wouldn't be too hard to write something that tried test discovery
>> using the old runner and using the new runner and did a simple
>> comparison of test counts to warn you if they differ.
>>
>> I'm not personally likely to work on this "upgrade checks" framework,
>> but if it does happen I'd be happy to contribute this check to it.
>
>  
> I thought about that approach, but my immediate reaction was that it will 
> end up producing more false positives than helpful feedback. Even the 
> simple case -- myapp.TestCase -> myapp.tests.TestCase -- is going to 
> require some name munging to work out if the old test and the new test are 
> actually the same one. Since this is for a short-lived transitional tool, 
> the effort involved in getting it right exceeds the value gained by having 
> it, IMHO.
>

I wasn't anticipating trying to actually match up individual tests by 
name/path, just trying both runners and seeing whether the test counts 
match. On second thought, though, I realize that contrib / third-party apps 
will mean this check would alert "missing tests!" for pretty much everyone, 
so it would need to be smarter to be useful, and you're right that's likely 
not worth it.
 

> I'd prefer something much simpler:
>
>  * On syncdb/validate, check for a marker somewhere in user space. 
>
>  * If the marker isn't present, do a check for likely problems. In this 
> case, look for the value of TEST_RUNNER; if it's set to the new default 
> value, display a warning about the change in test discovery format. Add 
> similar checks for other setting changes, or warnings about features that 
> have been fully deprecated (e.g., the final transition of the {% url %} 
> behavior)
>
>  * Provide a management command to set marker. 
>
>  * On the next syncdb/validate, the marker is present, so the install is 
> verified as being "updated", and no warnings are displayed.
>
> This leaves the question of where to put the marker. Some initial ideas:
>
>  - a file on disk (e.g., an .updated file next to the settings file)
>  - a new setting (VERIFIED_VERSION = "1.5")
>  - a key-value table in a database used only for Django administrivia
>
> Essentially, this would give us a place to put the "NO REALLY, READ THIS" 
> warnings that we need in release notes, and make those messages slightly 
> targeted.
>

In theory this makes sense to me, but I think the question of where to put 
the "I already ran the upgrade check" marker is thorny. Database is ugly, 
since there's no requirement for a Django project to even use the ORM at 
all; and requiring projects using the ORM to have a magical extra 
not-related-to-a-voluntarily-installed-app table in their database sticks 
in my craw. Similar with adding a file to the filesystem; we'd essentially 
be requiring everyone to have this file in order to avoid the extra 
overhead (and possible noise) of the check on startup; forcing that type of 
cruft on everyone u

Re: test discovery

[Russell Keith-Magee]

On Thu, May 9, 2013 at 11:30 PM, Carl Meyer <c...@oddbird.net> wrote:

> Hi Russ,
>
> On 05/09/2013 04:51 AM, Russell Keith-Magee wrote:
> > Great work Carl and Preston! (And everyone else who had a hand in the
> > patch - I know this has been kicking around for a while now)
>
> Thanks for the reminder; I should have mentioned that Mahdi Yusuf did
> the initial draft patch, based on Jannis' django-discover-runner, which
> was based on code I showed on a slide in my "Django and Testing" talk at
> PyCon 2012; it all comes full circle :-)
>
> > I agree with Jacob's tests aren't production sensitive, so making a fast
> > transition to the new test discovery has less impact.
> >
> > However, we know from experience that it doesn't matter how much we flag
> > this change in the release notes, they won't be read, and *someone* is
> > going to get stung. In an ideal world, it would be good to be able to
> > warn people who upgrade that their test suite may not be run as expected.
> >
> > We've had a proposal kicking around for a while to add a management
> > command that does an "upgrade check". I'm wondering if this might be an
> > opportune time to dig into this some more.
> >
> > To be clear, this isn't something I consider to be a merge blocker. I'd
> > be happy to see 1.6 released with a fast transition to the new test
> > runner. It would just be nice icing on the cake if we can get an upgrade
> > validator included in the same release.
>
> It wouldn't be too hard to write something that tried test discovery
> using the old runner and using the new runner and did a simple
> comparison of test counts to warn you if they differ.
>
> I'm not personally likely to work on this "upgrade checks" framework,
> but if it does happen I'd be happy to contribute this check to it.


I thought about that approach, but my immediate reaction was that it will
end up producing more false positives than helpful feedback. Even the
simple case -- myapp.TestCase -> myapp.tests.TestCase -- is going to
require some name munging to work out if the old test and the new test are
actually the same one. Since this is for a short-lived transitional tool,
the effort involved in getting it right exceeds the value gained by having
it, IMHO.

I'd prefer something much simpler:

 * On syncdb/validate, check for a marker somewhere in user space.

 * If the marker isn't present, do a check for likely problems. In this
case, look for the value of TEST_RUNNER; if it's set to the new default
value, display a warning about the change in test discovery format. Add
similar checks for other setting changes, or warnings about features that
have been fully deprecated (e.g., the final transition of the {% url %}
behavior)

 * Provide a management command to set marker.

 * On the next syncdb/validate, the marker is present, so the install is
verified as being "updated", and no warnings are displayed.

This leaves the question of where to put the marker. Some initial ideas:

 - a file on disk (e.g., an .updated file next to the settings file)
 - a new setting (VERIFIED_VERSION = "1.5")
 - a key-value table in a database used only for Django administrivia

Essentially, this would give us a place to put the "NO REALLY, READ THIS"
warnings that we need in release notes, and make those messages slightly
targeted.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

On 05/09/2013 07:50 AM, Andrew Godwin wrote:
> Just want to say that I'm happy with a "fast transition".
> 
> Is there a possibility we can detect the case where the tests might be
> broken (how might they be?) and print a helpful error message?

Two kinds of potential breakage:

- The most common will be missing tests; doctests, or tests in files
that don't match the test*.py pattern, which are currently run because
they are in models.py or manually imported into a tests/__init__.py,
won't be discovered by the new runner. Some people may also consider it
a "breakage" that tests in their pip-installed external apps are no
longer run by default.

- Less common will be tests that are in the codebase for some reason
(vendored third-party code, for instance?) that weren't found or run by
the previous runner, but are found and run (and perhaps fail) with the
new runner.

The first kind of breakage I don't think we can reasonably discover at
runtime; it would mean trying both discovery methods on every single
test run and comparing test counts. We could do this in a special
"upgrade check" like Russ suggested, but not every time you run tests.

The second issue I'm not too concerned about. If the
previously-undiscovered tests pass, then there's not really a problem.
If they fail, it shouldn't be too hard to figure out what's going on.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Russ,

On 05/09/2013 04:51 AM, Russell Keith-Magee wrote:
> Great work Carl and Preston! (And everyone else who had a hand in the
> patch - I know this has been kicking around for a while now)

Thanks for the reminder; I should have mentioned that Mahdi Yusuf did
the initial draft patch, based on Jannis' django-discover-runner, which
was based on code I showed on a slide in my "Django and Testing" talk at
PyCon 2012; it all comes full circle :-)

> I agree with Jacob's tests aren't production sensitive, so making a fast
> transition to the new test discovery has less impact.
> 
> However, we know from experience that it doesn't matter how much we flag
> this change in the release notes, they won't be read, and *someone* is
> going to get stung. In an ideal world, it would be good to be able to
> warn people who upgrade that their test suite may not be run as expected.
> 
> We've had a proposal kicking around for a while to add a management
> command that does an "upgrade check". I'm wondering if this might be an
> opportune time to dig into this some more.
> 
> To be clear, this isn't something I consider to be a merge blocker. I'd
> be happy to see 1.6 released with a fast transition to the new test
> runner. It would just be nice icing on the cake if we can get an upgrade
> validator included in the same release.

It wouldn't be too hard to write something that tried test discovery
using the old runner and using the new runner and did a simple
comparison of test counts to warn you if they differ.

I'm not personally likely to work on this "upgrade checks" framework,
but if it does happen I'd be happy to contribute this check to it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Tom,

On 05/09/2013 03:27 AM, Tom Christie wrote:
> * Will it be possible to globally configure the default file/path pattern?
>   Jannis's django-discover-runner includes support for a
> `TEST_DISCOVER_PATTERN` - is there anything similar, or just the command
> line `--patterns` option?  The use case I'm looking for is a single
> `tests` directory containing all the test modules, and I'm wondering if
> that'd be easy to support?

One of the goals here was to avoid adding more settings, so there isn't
a way to configure the pattern globally at this point. That may be
something we want to revisit, though; if you do need a different
pattern, you'll need it every time you run your tests, which basically
means you're either wrapping up "manage.py test" in a shell script or
overriding the test command with your own tweaked version. If we can
avoid the need for that by adding a setting, it might be worth it.

Your use case is very easy to support if you don't mind naming your test
files inside the tests/ directory all test_something.py. If you don't
like that, then you might need to use something like --pattern "*.py".

> * Is this work broadly compatible with `django-discover-runner`?  I'd
> really like to make use of it in third party packages, but I'm wondering
> if I'd be able to do so in a way that lets me also
> provide compatibility with older versions of Django.

I think the capabilities of the command-line interface in this patch are
a superset of the capabilities of discover-runner. It's smarter about
figuring some pythonpath things out, and better about handling multiple
test labels, but anything that works with discover-runner should work
here too.

If necessary it wouldn't be at all hard to backport this code into
discover-runner so they are exactly the same; I haven't discussed this
with Jannis yet.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Andrew Godwin]

Just want to say that I'm happy with a "fast transition".

Is there a possibility we can detect the case where the tests might be
broken (how might they be?) and print a helpful error message?

Andrew


On Thu, May 9, 2013 at 11:51 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

>
> On Thu, May 9, 2013 at 5:00 AM, Carl Meyer <c...@oddbird.net> wrote:
>
>> Hi all,
>>
>> Preston Timmons and I have been working the last several weeks to get
>> the test discovery branch (#17365) ready for merge, and I think we now
>> have a pull request ready for consideration:
>> https://github.com/django/django/pull/1050
>>
>> Brief summary of the features, changes, and open questions:
>>
>> * You can now place tests in any file matching the pattern 'test*.py',
>> anywhere in your codebase, rather than only in tests.py and models.py
>> modules of an INSTALLED_APP. The filename pattern is customizable via
>> the --pattern argument to "manage.py test".
>>
>> * When you run "manage.py test" with no arguments, tests are discovered
>> and run in all subdirectories (recursively) of the current working
>> directory. (This means that django.contrib and other third-party app
>> tests are no longer run by default).
>>
>> * When you pass arguments to "manage.py test", they are now full Python
>> dotted module paths. So if you have a "myproject.myapp" app, and its
>> tests.py contains "SomeTestCase", you would now run that single test
>> case via "manage.py myproject.myapp.tests.SomeTestCase" rather than
>> "manage.py myapp.SomeTestCase". This is longer, but allows more control
>> when an app's tests are split into multiple files, and allows for tests
>> to be placed anywhere you like.
>>
>> * Doctests are no longer discovered by default; you will need to
>> explicitly integrate them with your test suite as recommended in the
>> Python docs: http://docs.python.org/2/library/doctest.html#unittest-api
>>
>> * Tests in models.py and tests/__init__.py will no longer be discovered
>> (as those don't match the default filename pattern).
>>
>> * The old test runner, and Django's extensions to the doctest module,
>> are deprecated and will be removed in Django 1.8; they could of course
>> be packaged separately if some people would like to continue using them.
>>
>>
> Great work Carl and Preston! (And everyone else who had a hand in the
> patch - I know this has been kicking around for a while now)
>
>
>> Open question: how to handle the transition?
>>
>> Jacob has suggested that back-compat breaks in test-running are not as
>> serious as in production code, and that we should just switch to the new
>> test runner by default in Django 1.6. This is what the pull request
>> currently does. This will mean that some people's test suites will
>> likely be broken when they upgrade to 1.6. They would have two options,
>> both documented in the release notes: they can update their test suite
>> to be discovery-compatible immediately, or they can just set TEST_RUNNER
>> to point to the old runner and get back the old behavior, which they can
>> keep using until Django 1.8 (or longer, if they package the old runner
>> externally).
>>
>> The alternative would be to keep the old test runner as the default in
>> global_settings.py until it is removed in 1.8, and add the new runner as
>> the TEST_RUNNER in the startproject-generated settings.py. This would
>> provide a gentler transition for upgrading projects. On the other hand,
>> we just recently got the startproject settings.py all cleaned-up,
>> slimmed-down, and newbie-friendly, so it would make me sad to start
>> polluting it again with stuff that new projects generally shouldn't care
>> about, for solely legacy reasons.
>>
>
> I agree with Jacob's tests aren't production sensitive, so making a fast
> transition to the new test discovery has less impact.
>
> However, we know from experience that it doesn't matter how much we flag
> this change in the release notes, they won't be read, and *someone* is
> going to get stung. In an ideal world, it would be good to be able to warn
> people who upgrade that their test suite may not be run as expected.
>
> We've had a proposal kicking around for a while to add a management
> command that does an "upgrade check". I'm wondering if this might be an
> opportune time to dig into this some more.
>
> To be clear, this isn't something I consider to be a merge blocker. I'd be
> happy to see 1.6 released with a fast transition to

Re: test discovery

[Russell Keith-Magee]

On Thu, May 9, 2013 at 5:00 AM, Carl Meyer <c...@oddbird.net> wrote:

> Hi all,
>
> Preston Timmons and I have been working the last several weeks to get
> the test discovery branch (#17365) ready for merge, and I think we now
> have a pull request ready for consideration:
> https://github.com/django/django/pull/1050
>
> Brief summary of the features, changes, and open questions:
>
> * You can now place tests in any file matching the pattern 'test*.py',
> anywhere in your codebase, rather than only in tests.py and models.py
> modules of an INSTALLED_APP. The filename pattern is customizable via
> the --pattern argument to "manage.py test".
>
> * When you run "manage.py test" with no arguments, tests are discovered
> and run in all subdirectories (recursively) of the current working
> directory. (This means that django.contrib and other third-party app
> tests are no longer run by default).
>
> * When you pass arguments to "manage.py test", they are now full Python
> dotted module paths. So if you have a "myproject.myapp" app, and its
> tests.py contains "SomeTestCase", you would now run that single test
> case via "manage.py myproject.myapp.tests.SomeTestCase" rather than
> "manage.py myapp.SomeTestCase". This is longer, but allows more control
> when an app's tests are split into multiple files, and allows for tests
> to be placed anywhere you like.
>
> * Doctests are no longer discovered by default; you will need to
> explicitly integrate them with your test suite as recommended in the
> Python docs: http://docs.python.org/2/library/doctest.html#unittest-api
>
> * Tests in models.py and tests/__init__.py will no longer be discovered
> (as those don't match the default filename pattern).
>
> * The old test runner, and Django's extensions to the doctest module,
> are deprecated and will be removed in Django 1.8; they could of course
> be packaged separately if some people would like to continue using them.
>
>
Great work Carl and Preston! (And everyone else who had a hand in the patch
- I know this has been kicking around for a while now)


> Open question: how to handle the transition?
>
> Jacob has suggested that back-compat breaks in test-running are not as
> serious as in production code, and that we should just switch to the new
> test runner by default in Django 1.6. This is what the pull request
> currently does. This will mean that some people's test suites will
> likely be broken when they upgrade to 1.6. They would have two options,
> both documented in the release notes: they can update their test suite
> to be discovery-compatible immediately, or they can just set TEST_RUNNER
> to point to the old runner and get back the old behavior, which they can
> keep using until Django 1.8 (or longer, if they package the old runner
> externally).
>
> The alternative would be to keep the old test runner as the default in
> global_settings.py until it is removed in 1.8, and add the new runner as
> the TEST_RUNNER in the startproject-generated settings.py. This would
> provide a gentler transition for upgrading projects. On the other hand,
> we just recently got the startproject settings.py all cleaned-up,
> slimmed-down, and newbie-friendly, so it would make me sad to start
> polluting it again with stuff that new projects generally shouldn't care
> about, for solely legacy reasons.
>

I agree with Jacob's tests aren't production sensitive, so making a fast
transition to the new test discovery has less impact.

However, we know from experience that it doesn't matter how much we flag
this change in the release notes, they won't be read, and *someone* is
going to get stung. In an ideal world, it would be good to be able to warn
people who upgrade that their test suite may not be run as expected.

We've had a proposal kicking around for a while to add a management command
that does an "upgrade check". I'm wondering if this might be an opportune
time to dig into this some more.

To be clear, this isn't something I consider to be a merge blocker. I'd be
happy to see 1.6 released with a fast transition to the new test runner. It
would just be nice icing on the cake if we can get an upgrade validator
included in the same release.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Tom Christie]

Hi Carl,

  Excellent, excellent stuff - many thanks to both yourself and Preston.

Couple of questions:

* Will it be possible to globally configure the default file/path pattern?
  Jannis's django-discover-runner includes support for a 
`TEST_DISCOVER_PATTERN` - is there anything similar, or just the command 
line `--patterns` option?  The use case I'm looking for is a single `tests` 
directory containing all the test modules, and I'm wondering if that'd be 
easy to support?
* Is this work broadly compatible with `django-discover-runner`?  I'd 
really like to make use of it in third party packages, but I'm wondering if 
I'd be able to do so in a way that lets me also provide compatibility with 
older versions of Django.

Thanks again,

  Tom

On Wednesday, 8 May 2013 23:04:29 UTC+1, Carl Meyer wrote:
>
> Hi Anssi, 
>
> On 05/08/2013 03:31 PM, Anssi K��ri�inen wrote: 
> > It would be really nice to be able to use same syntax for running a 
> > single Django's testcase at least for a while. I expect that there 
> > will be problems if inspecting how a given test case behaves in older 
> > versions compared to HEAD. Also, bisecting over the test case renaming 
> > boundary will be really ugly. I don't believe end users will have too 
> > much problems with this, but bug hunting and development in Django 
> > itself will be a bit more tedious to do. 
>
> I don't think this is enough of a problem to warrant more code to try to 
> work around it. 
>
> If you're bisecting, at worst this means having two commands in your 
> shell history instead of one and hopping up to the right one for each 
> bisect step. I think this is a minor irritation that will fade away soon 
> enough. 
>
> Most of our test apps are small/fast enough that you could just run the 
> whole app when bisecting instead of an individual TestCase, making this 
> a non-issue (since Django's test apps are top-level on sys.path, the 
> command to run a single app in Django's test suite is the same with the 
> new runner as with the old: "python runtests.py app_name"). 
>
> Carl 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Carl Meyer]

Hi Anssi,

On 05/08/2013 03:31 PM, Anssi Kääriäinen wrote:
> It would be really nice to be able to use same syntax for running a
> single Django's testcase at least for a while. I expect that there
> will be problems if inspecting how a given test case behaves in older
> versions compared to HEAD. Also, bisecting over the test case renaming
> boundary will be really ugly. I don't believe end users will have too
> much problems with this, but bug hunting and development in Django
> itself will be a bit more tedious to do.

I don't think this is enough of a problem to warrant more code to try to
work around it.

If you're bisecting, at worst this means having two commands in your
shell history instead of one and hopping up to the right one for each
bisect step. I think this is a minor irritation that will fade away soon
enough.

Most of our test apps are small/fast enough that you could just run the
whole app when bisecting instead of an individual TestCase, making this
a non-issue (since Django's test apps are top-level on sys.path, the
command to run a single app in Django's test suite is the same with the
new runner as with the old: "python runtests.py app_name").

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Jacob Kaplan-Moss]

On Wed, May 8, 2013 at 2:00 PM, Carl Meyer  wrote:
> Jacob has suggested that back-compat breaks in test-running are not as
> serious as in production code, and that we should just switch to the new
> test runner by default in Django 1.6.

I still think this. I don't see the need to maintain compatibility in
testing, especially when compatibility is just a TEST_RUNNER change
away.

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Anssi Kääriäinen]

It would be really nice to be able to use same syntax for running a
single Django's testcase at least for a while. I expect that there
will be problems if inspecting how a given test case behaves in older
versions compared to HEAD. Also, bisecting over the test case renaming
boundary will be really ugly. I don't believe end users will have too
much problems with this, but bug hunting and development in Django
itself will be a bit more tedious to do.

I don't have any bright ideas of how to do this nicely. Maybe just try
the new way, if no tests found, then try to convert appname.SomeClass
to projectname.appname.tests.SomeClass, and somehow do this only for
tests/runtests.py.

Otherwise the plan looks good to me. And the above issue isn't
anything severe, just something that would make developing Django a
bit easier in certain cases.

 - Anssi

On 9 touko, 00:00, Carl Meyer <c...@oddbird.net> wrote:
> Hi all,
>
> Preston Timmons and I have been working the last several weeks to get
> the test discovery branch (#17365) ready for merge, and I think we now
> have a pull request ready for 
> consideration:https://github.com/django/django/pull/1050
>
> Brief summary of the features, changes, and open questions:
>
> * You can now place tests in any file matching the pattern 'test*.py',
> anywhere in your codebase, rather than only in tests.py and models.py
> modules of an INSTALLED_APP. The filename pattern is customizable via
> the --pattern argument to "manage.py test".
>
> * When you run "manage.py test" with no arguments, tests are discovered
> and run in all subdirectories (recursively) of the current working
> directory. (This means that django.contrib and other third-party app
> tests are no longer run by default).
>
> * When you pass arguments to "manage.py test", they are now full Python
> dotted module paths. So if you have a "myproject.myapp" app, and its
> tests.py contains "SomeTestCase", you would now run that single test
> case via "manage.py myproject.myapp.tests.SomeTestCase" rather than
> "manage.py myapp.SomeTestCase". This is longer, but allows more control
> when an app's tests are split into multiple files, and allows for tests
> to be placed anywhere you like.
>
> * Doctests are no longer discovered by default; you will need to
> explicitly integrate them with your test suite as recommended in the
> Python docs:http://docs.python.org/2/library/doctest.html#unittest-api
>
> * Tests in models.py and tests/__init__.py will no longer be discovered
> (as those don't match the default filename pattern).
>
> * The old test runner, and Django's extensions to the doctest module,
> are deprecated and will be removed in Django 1.8; they could of course
> be packaged separately if some people would like to continue using them.
>
> Open question: how to handle the transition?
>
> Jacob has suggested that back-compat breaks in test-running are not as
> serious as in production code, and that we should just switch to the new
> test runner by default in Django 1.6. This is what the pull request
> currently does. This will mean that some people's test suites will
> likely be broken when they upgrade to 1.6. They would have two options,
> both documented in the release notes: they can update their test suite
> to be discovery-compatible immediately, or they can just set TEST_RUNNER
> to point to the old runner and get back the old behavior, which they can
> keep using until Django 1.8 (or longer, if they package the old runner
> externally).
>
> The alternative would be to keep the old test runner as the default in
> global_settings.py until it is removed in 1.8, and add the new runner as
> the TEST_RUNNER in the startproject-generated settings.py. This would
> provide a gentler transition for upgrading projects. On the other hand,
> we just recently got the startproject settings.py all cleaned-up,
> slimmed-down, and newbie-friendly, so it would make me sad to start
> polluting it again with stuff that new projects generally shouldn't care
> about, for solely legacy reasons.
>
> Thoughts, questions, comments, code review and testing welcome! I'd like
> to merge this on Friday (it's a bear to keep updated with trunk), so let
> me know if you need longer.
>
> Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Aymeric Augustin]

On 8 mai 2013, at 23:00, Carl Meyer  wrote:

> Jacob has suggested that back-compat breaks in test-running are not as
> serious as in production code, and that we should just switch to the new
> test runner by default in Django 1.6. This is what the pull request
> currently does.

That seems reasonable to me.

Thank you very much for working on this.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: test discovery

[Donald Stufft]

On May 8, 2013, at 5:00 PM, Carl Meyer  wrote:

> Jacob has suggested that back-compat breaks in test-running are not as
> serious as in production code, and that we should just switch to the new
> test runner by default in Django 1.6. This is what the pull request
> currently does. This will mean that some people's test suites will
> likely be broken when they upgrade to 1.6. They would have two options,
> both documented in the release notes: they can update their test suite
> to be discovery-compatible immediately, or they can just set TEST_RUNNER
> to point to the old runner and get back the old behavior, which they can
> keep using until Django 1.8 (or longer, if they package the old runner
> externally).


This sounds reasonable to me. Tests are not production code so I agree with 
Jacob.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail