#8138: Switch django tests to use transactions
----------------------------------------+-----------------------------------
Reporter: mremolt | Owner: nobody
Status: new | Milestone: post-1.0
Component: Testing framework | Version: SVN
Resolution: | Keywords:
Stage: Accepted | Has_patch: 1
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
----------------------------------------+-----------------------------------
Comment (by kmtracey):
OK, with Ramiro's pointers as a guide, I've figured out what, exactly,
causes the time zone mismatch reported by modeltests/fixtures with the
current patch. (Attributing it to the rollback usage didn't quite answer
the question for me completely since I couldn't see why just the time zone
setting and not the actual data addition was being rolled back using the
new approach.) What's happening is the result of bracketing the doctest
run by the transaction stuff:
{{{
#!python
def run(self, test, compileflags=None, out=None, clear_globs=True):
"""
Wraps the parent run() and encloses it in a transaction.
"""
transaction.enter_transaction_management()
transaction.managed(True)
result = doctest.DocTestRunner.run(self, test, compileflags, out,
clear_globs)
transaction.rollback()
transaction.leave_transaction_management()
return result
}}}
The sequence of tests that is failing is first:
{{{
#!python
# Load fixture 1 again, using format discovery
>>> management.call_command('loaddata', 'fixture1', verbosity=0)
}}}
That loaddata uses a connection that has had its timezone set to whatever
is in settings. The loaddata works, the data is committed to the DB, and
that connection is closed by this code at the very end of loaddata.py:
{{{
#!python
# Close the DB connection. This is required as a workaround for an
# edge case in MySQL: if the same connection is used to
# create tables, load data, and query, the query can return
# incorrect results. See Django #7572, MySQL #37735.
if commit:
connection.close()
}}}
Next thing in the test is this:
{{{
#!python
>>> Article.objects.all()
[<Article: Time to reform copyright>, <Article: Poker has no place on
ESPN>, <Article: Python program becomes self aware>]
}}}
which opens a new connection, sets the timezone, and retrieves these
results from the DB. No commit yet, so the timezone setting is vulnerable
to rollback.
Next in the test is:
{{{
#!python
# Try to load fixture 2 using format discovery; this will fail
# because there are two fixture2's in the fixtures directory
>>> management.call_command('loaddata', 'fixture2', verbosity=0) #
doctest: +ELLIPSIS
Multiple fixtures named 'fixture2' in '...fixtures'. Aborting.
}}}
This loaddata is going to fail, and it is what causes the timezone setting
to be rolled back, since loaddata calls rollback in the case of failure.
The dumpdata is then called subsequently on the same connection, where the
timezone has reverted to whatever matches the actual OS timezone. If this
is different from what is in settings (or the default of America/Chicago
if settings is silent on the matter of timezone) then the times in the
dumpdata results will be off.
But still it wasn't clear why this rollback by loaddata was only a problem
with the patch code. The answer to that is this code near the beginning
of loaddata:
{{{
#!python
# Start transaction management. All fixtures are installed in a
# single transaction to ensure that all references are resolved.
if commit:
transaction.commit_unless_managed()
transaction.enter_transaction_management()
transaction.managed(True)
}}}
So on entry to loaddata, the uncommitted set timezone will be committed
before the failure-induced rollback if the doctest itself has not been
bracketed by the `enter_transaction_management`, etc.
Not sure what the right fix is. A brief experiment with sqlite seems to
indicate that the patch's changes to bracket `DocTestRunner.run` in a
transaction that is rolled back are not actually necessary for
successfully running the Django test suite. That is I can remove that
whole `run` override and all Django tests pass. (This is also consistent
with not seeing any problems caused by this newly added rollback not
working on MySQL/MyISAM.) So apparently anything done to the DB by
doctests don't interfere with subsequent tests in the Django testsuite.
But I'm a bit out of my depth here in knowing if that would be an OK
approach -- it seems inconsistent for one type of test to use rolled-back
transactions and the other to not.
Alternatively, always immediately committing the timezone setting seems
harmless, since it is only done when a new connection is opened, so it's
not like anything other than the timezone setting will be accidentally
committed, right? And that does solve the problem. But I feel a bit
uneasy with using that approach to 'fix' this problem because I really get
the impression this doctest isn't expecting to be run within an
externally-set-up transaction. It seems like this test is expecting to
control the transaction boundaries, and enclosing it in a transaction it
doesn't know about is interfering with that. I believe there was some
discussion initially about doctests somehow indicating that they wanted
control over transactions, but I don't see if that discussion came to a
conclusion as to how that could/should be done?
--
Ticket URL: <http://code.djangoproject.com/ticket/8138#comment:23>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en
-~----------~----~----~----~------~----~------~--~---
