#33275: django.test.runner.DiscoverRunner.setup_databases() appears to create db
based on ordering of aliases from unordered set
-------------------------------------+-------------------------------------
Reporter: augb | Owner: nobody
Type: Bug | Status: new
Component: Testing | Version: 3.2
framework | Keywords: DiscoverRunner
Severity: Normal | setup_database test database
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
We use {{{settings.DATABASES}}} to change which database user is used to
connect to our database. For example, 'default' might use something along
the lines of {{{admin_user}}} for use in migrations, tests, etc.
{{{restricted_user}}} might be used for the actual Django web app.
Given a {{{settings.DATABASES}}} config along these lines:
{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'my_database_name',
'USER': 'admin_user',
# rest of actual config here
},
'restricted': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'my_database_name',
'USER': 'restricted_user',
# rest of actual config here
},
# other databases, as needed ...
}
}}}
The problem I appear to be running into is that the stock test runner
sometimes creates a test database using the "default" alias. At other
times, it appears to create the test database using the "restricted"
alias. (Actual alias names may be different, but illustrative.) For the
actual db tests, we have middleware in place to decide which connection
should be used for that test. (Sometimes we may want to use
{{{admin_user}}} in the test, sometimes {{{restricted_user}}}, but this is
not the current concern.)
Given our setup, the actual database to be created will be correctly
identified but with the wrong database user, in some cases.
({{{admin_user}}} has the ability to create a database, but
{{{restricted_user}}} does not.) If the 'restricted' alias is used the
tests will fail since {{{restricted_user}}} does not have permission to
create a database.
Sometimes it picks 'default' ...
{{{
(Pdb) kwargs
{'aliases': {'default', 'restricted'}}
(Pdb) n
Using existing test database for alias 'default'...
}}}
Sometimes it picks 'restricted' ...
{{{
(Pdb) kwargs
{'aliases': {'restricted', 'default'}}
(Pdb) n
Using existing test database for alias 'restricted'...
}}}
In practice this seems to be hit or miss due to the apparent use of a set
(which does not guarantee ordering; see:
https://docs.python.org/3.8/library/stdtypes.html#set) in an ordered
scenario:
Relevant code in Django:
{{{
for alias in aliases:
connection = connections[alias]
old_names.append((connection, db_name, first_alias is None))
# Actually create the database for the first connection
if first_alias is None:
first_alias = alias
}}}
from django/django/test/utils.py / setup_databases (see:
https://github.com/django/django/blob/1b3c0d3b54d4ff5f75af57d3130180b1d22468e9/django/test/utils.py#L171)
django/django/test/runner.py / DiscoverRunner appears to import
setup_databases from django.test.utils (see:
https://github.com/django/django/blob/1b3c0d3b54d4ff5f75af57d3130180b1d22468e9/django/test/runner.py#L642)
Simply overriding the passed in {{{aliases}}} kwarg with an ordered list,
does not appear to have any effect. This appears to be due to pulling the
alias from the connections instead of the aliases parameter in
{{{get_unique_databases_and_mirrors}}}:
{{{
def get_unique_databases_and_mirrors(aliases=None):
"""
Figure out which databases actually need to be created.
Deduplicate entries in DATABASES that correspond the same database or
are
configured as test mirrors.
Return two values:
- test_databases: ordered mapping of signatures to (name, list of
aliases)
where all aliases share the same underlying
database.
- mirrored_aliases: mapping of mirror aliases to original aliases.
"""
if aliases is None:
aliases = connections
mirrored_aliases = {}
test_databases = {}
dependencies = {}
default_sig =
connections[DEFAULT_DB_ALIAS].creation.test_db_signature()
for alias in connections:
# elided ...
}}}
(see:
https://github.com/django/django/blob/1b3c0d3b54d4ff5f75af57d3130180b1d22468e9/django/test/utils.py#L270)
Indeed line 270 referenced above appears to be the culprit.
For our use case, either always using the 'default' alias to create the
test database, truly honoring the order of aliases, or fixing the aliases
kwarg would solve our problem.
We are using 3.1; however, the code references above are essentially
identical and are from 3.2.
--
Ticket URL: <https://code.djangoproject.com/ticket/33275>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/047.450bc45b2c8f1f0d54debd425806b248%40djangoproject.com.
