Bill
I'm about to disappear for a few days. Thank you very much for your
suggestions - I'll tackle them when I get back.
Cheers
Mike
On 18/09/2018 11:34 PM, Bill-Torcaso-Oxfam wrote:
I two comments, and as always Your Milage May Vary.
* I wonder if you have the right indexes on your Postgres database?
The previous people report much faster completion times for test
that use Postgres as the database. Perhaps your domain is just
hard (and you description makes it sound that way), but "Explain
Plan" might be your best friend here.
* Sqlite does not enforce length limit constraints on text fields.
You can declare a field to be CharField(max_length=200), and
Sqlite will happily put a longer string into the field. Postgres
will not.
This came up when I was importing data from an older generation of
our product. My custom import code ran successfully against
Sqlite and the exact same command failed against Postgres.
* (And yes, I said only two points. But this just occurs to me...)
Can you divide your test suite into distinct independent chunks,
and run them in parallel? Either against one common database or
using one database per test-chunk?
On Friday, September 14, 2018 at 11:32:49 AM UTC-4, Hezi Halpert wrote:
I would like to share with you an issue we encounter while moving
from sqlite to postgres with heavily use of Django testing.
We have Django app with ~700 tests. Most of them accessing
database. We recently migrated the database from sqlite to postgres.
Many of our tests were written in a way that compares actual pk’s
(hard-coded pks or just file/json comparisons) . Since Django
testing on sqlite (testcases.TestCase class) creates in-memory
database (which is being reseted every unit test by default), we
never had a problem with it.
However, Django TestCase on postgres create completely another
test db which preserves the pk sequences between different tests.
And since many of our tests were written in a way that compares
actual pk’s they all start fail - depends on the exact tests
execution order.
Even tests which expect some pk and are were not failed yet, can
potentially failed in the future - depends on adding/editing other
tests which may change the db sequence
We consider the following solutions:
1. Move to TransactionTestCase (instead of TestCase) and use
“reset_sequences = True” flag. Cons:
TransactionTestCase reduces performance dramatically (~4 times
longer in some of the tests)
2. Refactor all failed tests: remove all hard-coded references to
the pk. Cons: Require much Dev effort (we had more then 70
such tests)
3. Route the database in settings.py such it will use sqlite
instead of postgres when running tests. Cons: It will not
actually test the real scenarios - not an option
4. Combine reset_sequences flag with TestCase in our own version
to TestCase: OurTestCase class and make everything to inherit
from it. This is the option we finally decided of. See below.
fromdjango.test import TestCase, testcases
class OurTestCase(TestCase):
reset_sequences =True def _fixture_setup(self):
for db_namein self._databases_names(include_mirrors=False):
if self.reset_sequences:
self._reset_sequences(db_name)
if self.fixtures:
call_command('loaddata', *self.fixtures, **{'verbosity':0,'database':
db_name})
if not testcases.connections_support_transactions():
self.setUpTestData()
return super(TestCase,self)._fixture_setup()
self.atomics =self._enter_atomics()
Another problem of these kind of tests is the default ordering
assumption of Django which changes significantly between postgres
and sqlite when testing.
Therefore, models included in such tests must have a hint for
Django regarding the default ordering retrieval.
Our solution was to make all models inherit from
DexterModelDefaultOrder (below)
class DexterModelDefaultOrder(models.Model):
class Meta:
abstract =True ordering = ['id']
I hope it (will) help someone
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to django-users+unsubscr...@googlegroups.com
<mailto:django-users+unsubscr...@googlegroups.com>.
To post to this group, send email to django-users@googlegroups.com
<mailto:django-users@googlegroups.com>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/1e8ec552-d793-4161-ba10-070263586698%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/1e8ec552-d793-4161-ba10-070263586698%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/cd9ae433-a9b0-7f5c-e3ed-bb997b75ef7b%40dewhirst.com.au.
For more options, visit https://groups.google.com/d/optout.
