#23372: Test suite initial database construction is too slow
--------------------------------------+------------------------------------
Reporter: manfre | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Testing framework | Version: 1.7-rc-3
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by shaib):
I've seen `loaddata` take eons on SQL Server, and even in the context of
migrations (with South). As far as we could figure out, the problem was
not transaction management per se, but the handling of constraints: For
`loaddata` to avoid complex and sometimes non-satisfiable ordering
requirements, it needs FK constraints to be deferred while it works. On
core databases, each for its own reasons, this is not an issue.
SQL Server does not support defering in the normal sense -- what you need
to do is turn the constraints off in the beginning of worlk and back on in
the end. I don't remember if this is an operation you can do within a
transaction, but for sure it is not tied to a transaction -- you can turn
the constraints off and then run several transactions before turning them
back on. When you do turn them back on, however, you can do it `WITH CHECK
OPTION` -- which loaddata does, or else it might put constraint-violating
data into the database.
SQL Server does not -- perhaps, given the semantics, cannot -- keep track
of data changed while constraints were off. When you turn them on `WITH
CHECK OPTION`, it checks every single record in the database -- or, at
least, takes long enough to do so.
We had a project with several dozen apps, and relatively simple sets of
migrations on a full database (for upgrades) would take minutes on Oracle
and hours on SQL Server. Then we told SQL Server users to use `migrate
--all --no-initial-data` (we didn't have any initial data anywhere), and
sanity returned.
I'm not sure what this implies with regards to a solution. Some of the
problem should go away if `loaddata` only defered constraints when initial
data actually existed. Perhaps there's a way to avoid the check in most
cases (say, most data loaded in the Django test suite can be given a free
ticket because it is verified by the other backends).
--
Ticket URL: <https://code.djangoproject.com/ticket/23372#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.d7a5554ef02a44a9397959195b76d488%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
