Re: Speeding the code-test-debug cycle

[Russell Keith-Magee]

On Sat, Feb 6, 2010 at 1:25 AM, Malcolm Box  wrote:

>> From there, I have only been able to reduce test time down from hours
>> to minutes. Tests ought to run in seconds - so fast that you can run
>> them after every few edits - TDD.
>
> Totally agree, and what I'm looking to make Django do.  At the moment a full
> test run for my stuff takes about 5 minutes - not impossibly long, but long
> enough for a task switch to something else.

You won't get any disagreement out of me on this point. We have made
lots of speed improvements since the test framework was first
introduced. However, despite these changes, Django's tests still take
too long to run. They *should* be much faster.

>> One
>> of Django's particular sins is it drags all the data from your JSON
>> fixtures into fully resolved models.py objects before saving them.
>> This runs tons of Python code for no reason - you are not testing your
>> models' validations or save routines. The test runner should instead
>> push the JSON contents directly into the database as bulk import
>> statements.
>>
>
>  Hmm, that sounds like something that could be fixed.  Does anyone know if
> this is on the Django dev's radar and/or if someone's working on it?

I'm not aware of any specific test-speedup efforts at the moment.

In this general vein, I've had some vague thoughts about adding a
"Bulk create" method to the ORM for a while. In the "i've got an itch"
category, I've got a use case at work where I need to do this, and I
end up reverting to raw SQL to do it. This works, but given that it's
a pretty banal use case, it would be nice to have ORM support. More
broadly, all the m2m operations (which are currently multiple single
inserts/deletes) could benefit from this sort of API, as could fixture
loading.

However, the severity of my itch hasn't yet become enough to actually
make time on my schedule to write code. If you want to make this your
pet project, you certainly have my blessing.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Speeding the code-test-debug cycle

[Malcolm Box]

Hi,

Thanks for the suggestions.

2010/2/2 Phlip 

> Malcolm Box wrote:
>
> > Has anyone got any ideas on how to speed this up - for instance, is there
> a
> > way to say "use an existing test database"?
>
> Per my other advice to use fab and a fabfile with a test() target in
> it, you should set its (local()) target to something like
>
> OK, I need to have a look at fab to see what it can do for me.  I suspect
it won't solve the slow tests problem though - that seems to be a Django
issue.


> Do not use production fixtures for test fixtures, and if you must use
> the spew of dumpdata for fixtures you should manually trim them down
> to the minimum needed. Name them different from production fixtures.
> We have, naturally, a "countries.json", for all the regions we are
> allowed to sell to, and it takes forever to load. I pushed all our
> tests to use "sample_countries.json", instead, containing only
> countries that Sarah Palin could name on the spot. This made our tests
> much faster.
>
>
Nope, not using production data for fixtures.  Most of my fixtures contain a
handful of objects at most.


> And, finally, your test_settings.py file should use a sqlite3, in-
> memory database:
>
>
That would be wonderful, except my code doesn't run on sqlite3 as it relies
on some extra sql that sqlite doesn't support.


> From there, I have only been able to reduce test time down from hours
> to minutes. Tests ought to run in seconds - so fast that you can run
> them after every few edits - TDD.


Totally agree, and what I'm looking to make Django do.  At the moment a full
test run for my stuff takes about 5 minutes - not impossibly long, but long
enough for a task switch to something else.


> One
> of Django's particular sins is it drags all the data from your JSON
> fixtures into fully resolved models.py objects before saving them.
> This runs tons of Python code for no reason - you are not testing your
> models' validations or save routines. The test runner should instead
> push the JSON contents directly into the database as bulk import
> statements.
>
>
 Hmm, that sounds like something that could be fixed.  Does anyone know if
this is on the Django dev's radar and/or if someone's working on it?

Cheers,

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Speeding the code-test-debug cycle

[Phlip]

Malcolm Box wrote:

> Has anyone got any ideas on how to speed this up - for instance, is there a
> way to say "use an existing test database"?

Per my other advice to use fab and a fabfile with a test() target in
it, you should set its (local()) target to something like

   python manage.py test --settings=test_settings

You can also look up my _three_most_recently_changed_applications,
which looks a little bit like this:

  http://permalink.gmane.org/gmane.comp.python.fab.user/1093

to test only 3 apps. But I suspect that's not your problem.

Do not use production fixtures for test fixtures, and if you must use
the spew of dumpdata for fixtures you should manually trim them down
to the minimum needed. Name them different from production fixtures.
We have, naturally, a "countries.json", for all the regions we are
allowed to sell to, and it takes forever to load. I pushed all our
tests to use "sample_countries.json", instead, containing only
countries that Sarah Palin could name on the spot. This made our tests
much faster.

And, finally, your test_settings.py file should use a sqlite3, in-
memory database:

  from settings import *
  DATABASE_ENGINE = 'sqlite3'
  DATABASE_NAME = ":memory:"

>From there, I have only been able to reduce test time down from hours
to minutes. Tests ought to run in seconds - so fast that you can run
them after every few edits - TDD. The LAMP architectures are
spectacularly broke for this endeavor, for a variety of reasons. One
of Django's particular sins is it drags all the data from your JSON
fixtures into fully resolved models.py objects before saving them.
This runs tons of Python code for no reason - you are not testing your
models' validations or save routines. The test runner should instead
push the JSON contents directly into the database as bulk import
statements.

The final benefit of ":memory:" is SQLite3 is very fast when it does
not save anything to disk, so it's the fastest backend possible for
test.

--
  Phlip
  http://penbird.tumblr.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Speeding the code-test-debug cycle

[Malcolm Box]

Hi,

As my django application has grown I'm noticing I'm spending longer and
longer waiting for ./manage.py test to start up - ie create all the tables
etc.  Not helped of course by working on a laptop with a slow disk, as I
suspect many of us do these days.

This is slowing down my test-debug cycle quite noticably - e.g. when an
import fails due to an typo, the fix is extremely quick but then I have to
wait for ./manage.py test to crank through creating the databases again
before I can find the next bug.

Has anyone got any ideas on how to speed this up - for instance, is there a
way to say "use an existing test database"?

Cheers,

Malcolm

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.