#21197: 1.6 compatibility checks don't correctly validate TEST_RUNNER
-------------------------------------+-------------------------------------
Reporter: russellm | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version:
commands) | 1.6-alpha-1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by carljm):
Hi Russ - this is all making much more sense now. I think really my only
concern is that I think an "upgrade warning" is in practice a very
different thing from a "project validation check", and I'm not sure it
will serve us well to conflate the two.
In my mind, an upgrade warning is where we want to draw to your attention
something that changed in Django that may require attention in your
project, but where we really can't determine in an automated way whether a
specific configuration is "broken" or "valid" - only you can determine
that. An upgrade warning is essentially a slightly-automated version of
the backwards-incompatible section of the release notes, to assist the
many who don't read them. An upgrade warning should err on the side of
being loud rather than quiet if we can't be sure whether you need to do
anything, and it should point you to release notes that more fully
describe what changed and where you may want to look for possible
brokenness. Because it errs on the side of being loud, an upgrade warning
shouldn't be run as a part of regular project validation checks, unless
there's an easy way to say "Ok, I'm satisfied my project upgraded well and
runs fine on Django 1.6, I don't want to see any Django 1.6 upgrade
warnings anymore" (something like Shai's "expects Django version"
setting). An upgrade warning is never an "error", it is only ever an
informational note.
A project validation check, on the other hand, should be quiet unless we
are pretty sure something is broken and needs to be fixed in your project.
I am concerned about trying to convert the TEST_RUNNER check, which was
written as an upgrade warning (i.e. it errs on the side of being loud, and
was expected to only be run manually at upgrade time), into a project
validation check. I don't think that a 1.6 project without an explicit
TEST_RUNNER setting is "broken" and needs to be "fixed" by adding an
explicit TEST_RUNNER setting, and I don't want to add a project validation
check that thinks the lack of an explicit TEST_RUNNER setting is an error.
I don't see how we can possibly decide in an automated way whether a
project with a custom TEST_RUNNER is "valid" or not, and I don't think
knowing whether said custom TEST_RUNNER subclasses `DiscoverRunner` helps
us to make such a determination at all.
So basically I am quite skeptical of the direction you seem to be heading,
which is to try to build a project validation check that attempts to
correctly determine in all cases whether a project's TEST_RUNNER setting
is "valid" or "in error". I don't think it is possible to do this with any
confidence. I think what we need to be doing instead is figuring out how
to handle "upgrade warnings" where we simply can't make reliable automated
determinations and just want to nudge possibly-affected upgrading users
towards checking out important info in the release notes. I'm not sure
this is a use case that was addressed by the GSoC work. At the moment,
something like Shai's "expected Django version" setting is the best of bad
options for this that I've seen suggested.
--
Ticket URL: <https://code.djangoproject.com/ticket/21197#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.acd1b1193e850c5fa968290e686660b7%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.
