OK, here's a brief summary of what I was thinking about. It dovetails
nicely with Simon's thinking, so I don't have to rewrite all my notes.
This is a brain dump and can probably be safely ignored if it sounds
like rubbish, for reasons I mention at the end.
On Thu, 2006-07-13 at 01:41 -0700, Simon Willison wrote:
>
> On 12 Jul 2006, at 02:28, Malcolm Tredinnick wrote:
>
> > I approached things from a different direction, though, more
> > concentrating on how to unit test templates and views (as in not
> > requiring a full end-to-end application setup in order to test
> > them). So
> > it's actually a orthogonal to what you are proposing here and fits in
> > nicely.
>
> I've been thinking about unit testing templates and views on-and-off
> for the past few months. Here's the bulk of my thinking.
>
> The best way of running tests against a Django application, to my
> mind, is to run a "fake" web server (i.e. one that doesn't actually
> bind to a port) around the application. This fake server acts as a
> test harness. Tests can then create HttpRequest objects (probably a
> subclass called something like TestHttpRequest), pass them to the
> Django application, get back an HttpResponse object and run
> assertions against that.
>
> There's just one problem with this approach: while running assertions
> against an HttpResponse object is adequate, the ideal situation would
> be to run assertions against the template context as well. If you
> just run assertions against the HttpResponse object you end up having
> to process the HTML in some way to work out if the test has passed.
Agreed.
I was working out how to divorce each of the three components (M/V/T)
from each other so that I could unit test each in isolation. Part of the
logic here is to provide a way for template designers to test invariants
of the template without having to worry about views. Similarly, view
refactoring should not require delving deep inside the templates to work
out what happens: that makes the test results too fragile in the face of
any template changes. If it takes too long to maintain the tests, people
don't do it, because it's just a chore, rather than pure benefit.
Views first: we can make a TestHttpRequest style object and some utility
functions to make simulating a request simple. We pass this plus canned
parameters to the view function we are testing. We can override the
template class (create a TestTemplate class) that does not actually
render to a string (although it might need to behave string-like). This
way we can test this object to verify that it was constructed from the
right source and given the right context. We can also validate that the
right properties were set on the HttpResponse object that is returned.
Testing whether or not the template would have generated the right
result is not part of "view" testing, that is "template" testing. In
view testing we are verifying that the right input returns the right
result.
Initially, I think it will be hard to have the views entirely divorced
from the model framework. To do that (separate them for testing
purposes) would require making it very easy to creat mock objects. This
isn't too painful, but capturing all the necessary data via all the
function calls a view can take can be tricky. I have a sketch of how to
capture this via the profiling hooks in Python (as a dev server runtime
option) for seeding tests, but it's not implemented at the moment, so
there are no doubt hidden problems (I don't think we can use signal
dispatches here as Simon wants to for the view/template interaction,
because we would need to add so many of them). So figure that we
probably, unfortunately, need a seeded database and model setup for this
to work initially.
It's still useful, though, since now when we are working on views, we
can check that we are still passing the necessary information into the
template layer and back to the client and we can do this in a way that
is independent of the particular way the template has been written.
For template testing, I was thinking about passing in an object that
supported the necessary attribute accesses (the beauty of everything
being accessed as "foo.bar.baz" is that faking it is reasonably
straightforward). The test takes the template source filename (or source
string), the canned input data and we can run assertions against the
result.
The hard bit, that I haven't worked out a solution for -- and it may not
exist nicely -- is how to handle custom template tags. We could test
them as opaque, mock objects that return a canned response for testing
purposes, but that only provides a way to test the templates they live
in. How do you test the tags themselves? Maybe template tags can be
tested similarly to views. Not sure about this part, it hasn't
crystallised in my brain yet.
> What you really want to be able to do is this (pseudocode):
>
> send_get('/polls/1/')
>
> assert_response_status(200)
> assert_response_contains('This is poll 1')
>
> assert_template_rendered('poll_detail.html')
> assert_template_context_match('poll.id', 1)
> assert_template_context_match('poll.title', 'poll 1')
>
> (These function names are deliberately horrible - I haven't thought
> about a nice API for this yet).
I can't help here. All my thinking has been along the lines of how to
duck imposing much API policy at all. Testing frameworks are such a
religious issue (and not without reason) that I wanted to supply the
tools so that people could easy run assertions against Django results,
without us imposing too much in the way of "you have to use this
framework".
So, view test results would contain a way to access everything in the
context, everything in the response and the template source (name).
Django can provide a framework for making it easy to test for invariants
there, but it also gives everybody a way to use these things from there
own preferred harness, be it nose or test.py or
crazy-test-thing-from-outer-space.py.
> The first line above fakes sending a GET request to /polls/1/. The
> next line checks that the response status code was 200, and the line
> after checks that the response body contained 'This is poll 1'. So
> far, all of these things can be done just using an HttpResponse object.
>
> The next three lines assert that the correct template was rendered,
> and check that the template context contained the expected data. This
> isn't possible with just an HttpResponse - the template and context
> have been rendered and forgotten by the time the HttpResponse is
> returned.
>
> TurboGears and Rails don't have this problem (in fact Rails is where
> the idea for running assertions against the template context comes
> from) because they couple their template systems to the view. We
> don't want to do that in Django.
>
> The solution I've been thinking about involves Django's event
> dispatch system. I think we should be firing an event when a template
> is rendered. The testing harness can then listen out for that event
> and use it to capture the template and context, meaning you can run
> assertions against them. This keeps our template system decoupled
> while letting us write tests against the template activity within the
> view functions.
I was going to poke about under the covers during test setup. For
example, a TestTemplate class would have all the functions of a Template
class, except that render() would just store the context for later
referral (and maybe return a stub string in case somebody really
expected a string back).
Your way is probably simpler, though, and a much better first step. If
somebody does not want to test the template contents in one particular
suite, they can just ignore those bits and focus on the results. Since
the signal you want to dispatch probably isn't that useful in production
(and signal dispatches do add overhead), we can poke in a "test" render
method as part of the test setup that adds the new dispatch call.
> I hope the above makes sense. I'll try to follow up later with a
> better idea of how I think the assertions API should look.
Now that I've written all that, I think I might be stepping back from
this a bit. Whilst eating lunch today, I sat down and looked at
everything I have going on professionally and in my Django sandboxes and
I think that since there are going to be a million people wanting to
work on the testing, I can spend my efforts elsewhere. There are a bunch
of medium-sized bits in Django I want to work on as well as getting the
open ticket count down and this part is already in good hands. It was
just fortuitous timing that Russell spoke up not three days after I had
spent an afternoon writing stuff down.
Regards,
Malcolm
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~----------~----~----~----~------~----~------~--~---
