#16088: Multi-database tests fail in ContentType
-------------------------------------+-------------------------------------
Reporter: def@… | Owner: nobody
Type: Bug | Status: reopened
Milestone: | Component: Database layer
Version: 1.2 | (models, ORM)
Resolution: | Severity: Normal
Triage Stage: | Keywords: ContentType
Unreviewed | Has patch: 1
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Old description:
> I have a multi-database setup:
>
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'wall2',
> 'HOST': 'localhost',
> 'PORT': '',
> },
>
> 'slave': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'wall2',
> 'HOST': 'localhost',
> 'PORT': '',
> 'TEST_MIRROR': 'default',
> },
>
> 'station': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'station',
> 'HOST': 'localhost',
> 'PORT': '',
> 'TEST_DEPENDENCIES': ['default',]
> },
>
> 'catalog': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'catalog',
> 'HOST': 'localhost',
> 'PORT': '',
> 'TEST_DEPENDENCIES': ['default', 'station']
> },
> }
>
> i have 2 routers, one handles the master/slave and one handles the 2
> "data" database station and catalog. they run fine on production, but
> when we try to run our unit tests i get the following stack trace:
>
> Traceback (most recent call last):
> File "./27.py", line 11, in <module>
> execute_manager(settings)
> File "/usr/lib/python2.7/site-
> packages/django/core/management/__init__.py", line 438, in
> execute_manager
> utility.execute()
> File "/usr/lib/python2.7/site-
> packages/django/core/management/__init__.py", line 379, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
> line 191, in run_from_argv
> self.execute(*args, **options.__dict__)
> File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
> line 220, in execute
> output = self.handle(*args, **options)
> File "/usr/lib/python2.7/site-
> packages/south/management/commands/test.py", line 8, in handle
> super(Command, self).handle(*args, **kwargs)
> File "/usr/lib/python2.7/site-
> packages/django/core/management/commands/test.py", line 37, in handle
> failures = test_runner.run_tests(test_labels)
> File "/usr/lib/python2.7/site-packages/django/test/simple.py", line
> 396, in run_tests
> old_config = self.setup_databases()
> File "/usr/lib/python2.7/site-packages/django/test/simple.py", line
> 334, in setup_databases
> test_db_name = connection.creation.create_test_db(self.verbosity,
> autoclobber=not self.interactive)
> File "/usr/lib/python2.7/site-packages/django/db/backends/creation.py",
> line 357, in create_test_db
> load_initial_data=False)
> File "/usr/lib/python2.7/site-
> packages/django/core/management/__init__.py", line 166, in call_command
> return klass.execute(*args, **defaults)
> File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
> line 220, in execute
> output = self.handle(*args, **options)
> File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
> line 351, in handle
> return self.handle_noargs(**options)
> File "/usr/lib/python2.7/site-
> packages/django/core/management/commands/syncdb.py", line 107, in
> handle_noargs
> emit_post_sync_signal(created_models, verbosity, interactive, db)
> File "/usr/lib/python2.7/site-packages/django/core/management/sql.py",
> line 182, in emit_post_sync_signal
> interactive=interactive, db=db)
> File "/usr/lib/python2.7/site-packages/django/dispatch/dispatcher.py",
> line 172, in send
> response = receiver(signal=self, sender=sender, **named)
> File "/usr/lib/python2.7/site-
> packages/django/contrib/contenttypes/management.py", line 25, in
> update_contenttypes
> ct.save()
> File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line
> 458, in save
> self.save_base(using=using, force_insert=force_insert,
> force_update=force_update)
> File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line
> 551, in save_base
> result = manager._insert(values, return_id=update_pk, using=using)
> File "/usr/lib/python2.7/site-packages/django/db/models/manager.py",
> line 195, in _insert
> return insert_query(self.model, values, **kwargs)
> File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line
> 1524, in insert_query
> return query.get_compiler(using=using).execute_sql(return_id)
> File "/usr/lib/python2.7/site-
> packages/django/db/models/sql/compiler.py", line 788, in execute_sql
> cursor = super(SQLInsertCompiler, self).execute_sql(None)
> File "/usr/lib/python2.7/site-
> packages/django/db/models/sql/compiler.py", line 732, in execute_sql
> cursor.execute(sql, params)
> File "/usr/lib/python2.7/site-
> packages/django/db/backends/mysql/base.py", line 86, in execute
> return self.cursor.execute(query, args)
> File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174,
> in execute
> self.errorhandler(self, exc, value)
> File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line
> 36, in defaulterrorhandler
> raise errorclass, errorvalue
> django.db.utils.IntegrityError: (1062, "Duplicate entry 'tools-station'
> for key 'app_label'")
>
> after some debugging i've found that the
> def update_contenttypes(app, created_models, verbosity=2, **kwargs):
> method looks up the tools-station content type in the station database,
> doesn't find it, and then tries to write it to the default database,
> where it already exists.
>
> by simply adding "using('default')" to the two ContentType.objects calls
> in the above method everything works fine. i'm not sure if all content
> types are meant to be in the 'default' database, but that seems to be the
> only way it can work at this point?
>
> i'll attach a simple patch,
New description:
I have a multi-database setup:
{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'wall2',
'HOST': 'localhost',
'PORT': '',
},
'slave': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'wall2',
'HOST': 'localhost',
'PORT': '',
'TEST_MIRROR': 'default',
},
'station': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'station',
'HOST': 'localhost',
'PORT': '',
'TEST_DEPENDENCIES': ['default',]
},
'catalog': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'catalog',
'HOST': 'localhost',
'PORT': '',
'TEST_DEPENDENCIES': ['default', 'station']
},
}
}}}
I have 2 routers, one handles the master/slave and one handles the 2
"data" database station and catalog. they run fine on production, but when
we try to run our unit tests i get the following stack trace:
{{{
Traceback (most recent call last):
File "./27.py", line 11, in <module>
execute_manager(settings)
File "/usr/lib/python2.7/site-
packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/lib/python2.7/site-
packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
line 220, in execute
output = self.handle(*args, **options)
File "/usr/lib/python2.7/site-
packages/south/management/commands/test.py", line 8, in handle
super(Command, self).handle(*args, **kwargs)
File "/usr/lib/python2.7/site-
packages/django/core/management/commands/test.py", line 37, in handle
failures = test_runner.run_tests(test_labels)
File "/usr/lib/python2.7/site-packages/django/test/simple.py", line 396,
in run_tests
old_config = self.setup_databases()
File "/usr/lib/python2.7/site-packages/django/test/simple.py", line 334,
in setup_databases
test_db_name = connection.creation.create_test_db(self.verbosity,
autoclobber=not self.interactive)
File "/usr/lib/python2.7/site-packages/django/db/backends/creation.py",
line 357, in create_test_db
load_initial_data=False)
File "/usr/lib/python2.7/site-
packages/django/core/management/__init__.py", line 166, in call_command
return klass.execute(*args, **defaults)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
line 220, in execute
output = self.handle(*args, **options)
File "/usr/lib/python2.7/site-packages/django/core/management/base.py",
line 351, in handle
return self.handle_noargs(**options)
File "/usr/lib/python2.7/site-
packages/django/core/management/commands/syncdb.py", line 107, in
handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/lib/python2.7/site-packages/django/core/management/sql.py",
line 182, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/lib/python2.7/site-packages/django/dispatch/dispatcher.py",
line 172, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/lib/python2.7/site-
packages/django/contrib/contenttypes/management.py", line 25, in
update_contenttypes
ct.save()
File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line
458, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line
551, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/lib/python2.7/site-packages/django/db/models/manager.py",
line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line
1524, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python2.7/site-
packages/django/db/models/sql/compiler.py", line 788, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/lib/python2.7/site-
packages/django/db/models/sql/compiler.py", line 732, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.7/site-
packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174,
in execute
self.errorhandler(self, exc, value)
File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line
36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.IntegrityError: (1062, "Duplicate entry 'tools-station'
for key 'app_label'")
}}}
after some debugging i've found that the
{{{
def update_contenttypes(app, created_models, verbosity=2, **kwargs):
}}}
method looks up the tools-station content type in the station database,
doesn't find it, and then tries to write it to the default database, where
it already exists.
by simply adding "using('default')" to the two ContentType.objects calls
in the above method everything works fine. i'm not sure if all content
types are meant to be in the 'default' database, but that seems to be the
only way it can work at this point?
i'll attach a simple patch,
--
Comment (by lrekucki):
Fixed formatting (please use preview!).
--
Ticket URL: <https://code.djangoproject.com/ticket/16088#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
