#28153: patch: it's now easier to populate data databases in test databases that
run code in parallel
-----------------------------------+--------------------------------------
Reporter: Marcos Diez | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------------------------
Description changed by Marcos Diez:
Old description:
> Although Django makes very easy for one to extend `
> django.test.runner.DiscoverRunner ` , it's ` setup_databases() ` does too
> much.
>
> Currently, it
>
> - creates all the test databases (for single thread unit tests)
> - duplicates all the test databases (in case of parallel unit tests)
>
> In case I am running not running tests in parallel, I can just populate
> the DB after running unit tests without any issues.
>
> But if I care about my time and want to run tests in parallel, I can
> either:
>
> a) populate my data after setup_databases() is executed, once for each
> thread of the parallel tests, which is slow
> b) get my hands dirty and reimplement ` setup_databases() `
>
> I propose (and I am sending the code to do so) a better solution. We just
> have to break `setup_databases()` in 3 functions:
>
> `DiscoverRunner.prepare_databases() `
> `DiscoverRunner.populate_databases() # noop by default `
> `DiscoverRunner.duplicate_databases_if_necessary()`
>
> The idea is quite simple: in order to be backward compatible, `
> setup_databases() `, will still exist but only call three functions above
> in that order.
>
> The first function will create all the test databases necessary for non
> parallel tests to run.
>
> ` populate_databases() `, which should be a no op, can be overwritten by
> the user who extends ` django.test.runner.DiscoverRunner ` so his/her
> data can be populated
>
> Afterwards, all the test DBs are copied as many times as necessary in
> case parallel tests are run via `
> DiscoverRunner.duplicate_databases_if_necessary() `
>
> I believe this change on Django will have no downside, will be backward
> compatible and help people who needs to populate real data on the DB for
> their tests.
New description:
Although Django makes very easy for one to extend `
django.test.runner.DiscoverRunner ` , it's ` setup_databases() ` does too
much.
Currently, it
- creates all the test databases (for single thread unit tests)
- duplicates all the test databases (in case of parallel unit tests)
In case I am running not running tests in parallel, I can just populate
the DB after running unit tests without any issues.
But if I care about my time and want to run tests in parallel, I can
either:
a) populate my data after setup_databases() is executed, once for each
thread of the parallel tests, which is slow
b) get my hands dirty and reimplement ` setup_databases() `
I propose (and I am sending the code to do so) a better solution. We just
have to break `setup_databases()` in 3 functions:
`DiscoverRunner.prepare_databases() `
`DiscoverRunner.populate_databases() # noop by default `
`DiscoverRunner.duplicate_databases_if_necessary()`
The idea is quite simple: in order to be backward compatible, `
setup_databases() `, will still exist but only call three functions above
in that order.
The first function will create all the test databases necessary for non
parallel tests to run.
` populate_databases() `, which should be a no op, can be overwritten by
the user who extends ` django.test.runner.DiscoverRunner ` so his/her
data can be populated
Afterwards, all the test DBs are copied as many times as necessary in case
parallel tests are run via `
DiscoverRunner.duplicate_databases_if_necessary() `
I believe this change on Django will have no downside, will be backward
compatible and help people who needs to populate real data on the DB for
their tests.
The PR with the changes for this to work is available here:
https://github.com/django/django/pull/8437
--
--
Ticket URL: <https://code.djangoproject.com/ticket/28153#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/068.5d8283840f2d21c9612d73acfcc03675%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
