Re: New to unit testing, need advice

[Joshua Russo]

Great ideas! I've started moving some of my logic that I had in
admin.py and views.py into model.py and things are already starting to
look easier in terms of testing.

My only real hurdle now is testing validation logic on the admin pages
and the form creation logic that is the main page to the application.

Any thoughts on form testing via the unit test. It seems like I just
want to use the test client, find a good group of post data sets, and
use the assertContains() and assertFormError().

Does that sound about right?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Javier Guerra]

On Thu, Jul 16, 2009 at 3:16 PM, Jumpfroggy wrote:
> You could put all these functions into the
> models.py file if you wanted, you'd just end up with huge models.py
> objects, and you might also run into cases where there is not an
> obvious place to put a bit of business logic.  Not every business
> logic function corresponds exactly to one model.

totally agree.

in my case, sometimes i just add a small utils.py with some of these;
but still haven't feel the need for a real business.py layer, since
until now for each 'user leve' concept there's usually a DB table,
therefore the obvious place for almost every method is right in that
model.

granted, these are mostly office automation intranet webapps, so the
user is already familiar with the main concepts that can be modeled in
the DB.  for more 'public' apps, the desired user-level concepts could
be alot more detached from the DB, and a thick business.py layer (with
lots of tests!) would be definitely the best answer.

-- 
Javier

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Jumpfroggy]

> besides the testing issues (which are certainly a heated debate!), i have to 
> say that my Django projects became far better organized and a lot more 
> flexible when i learned to put most of the code on the models, and not on the 
> views.

This is a pretty interesting topic.  Coming from a PHP background, I
started out with very messy code (each .php file contained it's own
program logic at the top).  From there I went to Javabeans (tomcat)
and got a crash course in MVC & OO web apps, almost to an extreme I
wouldn't want to duplicate.  Now that I'm in django, I have a nice
middle ground.

There are certain ideals (ie. programming paradigms) that might make
unit testing easier.  One is separating the different parts of MVC
(model, view, controller).  Controller in our case is mostly the
django code (if I'm not wrong).  The views are our views.py
functions.  The models are our models.py file.  But there's the
unspoken part of business logic.  By default, most django apps seem to
have very dumb models.py files (the classes are simply wrappers around
database rows & tables), and all the logic is in views.py.  However,
the business logic should really be pulled out of views.py, whether
that goes into heavy models.py objects (like Javier has), or a
separate business_logic.py file.

It helps to keep asking yourself, what is this line of code doing?

For example, in views.py:

def handle_add_item_form(request):
form = AddItemForm(request.POST)
if form.is_valid():
item = Item.objects.create(
name=form.cleaned_data['name'],
price=form.cleaned_data['price'],
status=form.cleaned_data['status'],
)
if item.status == 'out of stock':
# Order 100 more items for later
order_more_items(item, 100)
email_admin_about_oos(settings.ADMINS)

This handles a simple form to create an item.  However, there's
clearly business logic here... the whole "status == 'out of stock'"
sticks out like a sore thumb.  It'd be better like this:

in views.py:

def handle_add_item_form(request):
form = AddItemForm(request.POST)
if form.is_valid():
item = ItemManager.create(
name=form.cleaned_data['name'],
price=form.cleaned_data['price'],
status=form.cleaned_data['status'],
)

and in managers.py:

class ItemManager:
def create(name, price, status):
item = Item.objects.create(
name=form.cleaned_data['name'],
price=form.cleaned_data['price'],
status=form.cleaned_data['status'],
)
if item.status == 'out of stock':
# Order 100 more items for later
order_more_items(item, 100)
email_admin_about_oos(settings.ADMINS)

You have to type a bit more (to create all those managers, or service
objects, or whatever you call them), but the benefit is that business
logic is now in one place.  Views.py should only be the dumb glue that
connects the HTTP requests with the service objects, and returns the
results.

Like Daniel said, models.py are usually so simple/dumb that you don't
need to test them much.  Your views.py should be the same way.
They're simply glue, so the only things that will go wrong are simple
logic errors and typos.  All the complicated stuff is in the business
logic objects (the managers.py here), and that's what you spend a lot
of time unit testing.

In this example, you can use the ItemManager.create() function
anywhere you create items... adding new ones, copying existing ones,
importing items from a file, etc.  In the old example, you'd have to
duplicate the views.py code in each view that handles this situation
and test each one.  You could put all these functions into the
models.py file if you wanted, you'd just end up with huge models.py
objects, and you might also run into cases where there is not an
obvious place to put a bit of business logic.  Not every business
logic function corresponds exactly to one model.

Another great change... with all your logic separated like this, if
you have to change something very complicated (like the way the whole
system handles out of stock, ordering more inventory, permissions,
etc), you have one place to change & test.  It's great.

So do I use this?  No, all my code is in wrapped up in my views.py and
I have no unit testing.  I try to break out duplicated functionality
into their own functions, but that's more for convenience than true
separation.  I'd recommend starting with the "keep them separated"
mentality, since it gets harder and harder as a project goes on.

Sorry for the long rant, but it touched on something I've been
observing in my own code.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to 

Re: New to unit testing, need advice

[Joshua Russo]

On Jul 16, 1:35 pm, Javier Guerra  wrote:
> On Thu, Jul 16, 2009 at 9:27 AM, Joshua Russo wrote:
> > What are some examples of mutating operations (and other operations
> > for that matter) that you use in your models?
>
> the most obvious are:
>
> - 'calculated' fields.  the first example is the __unicode__() method,
> but lots others, like multiplying widht/height fields to get an 'area'
> field, (en/de)coding JSON data in one field to/from a python object,
> fetching a filesize from the filesystem,  things like that.
>
> - 'pre-made filters'. like alex's example
>
> - overriding the save() method, to add some consistency checks, update
> other tables (like maintaining denormalized fields), override one
> field with another, etc.
>
> - other kinds of objects that could present a nicer API than just a
> bunch of DB records and tables, for example a tag soup, even if it's
> an extra table, it's presented as extra fields and operations on the
> 'tagged' model.
>
> - anything that Java developers could call the 'business layer', or
> anything that a DB manager would like to put into an stored procedure
> (unless you're able to actually do that!)
>
> a worthwhile goal is to make view functions really really thin.  maybe
> just a few lines before calling the template or a generic view.
>
> if your model layer presents an API that is close to the concepts the
> user manages, but at the same time isn't tied to the specific web
> interface, you'll be able to easily rework the whole appearance of
> your website, quite possibly even change the whole user experience
> without rewriting the 'operational' code.
>
> --
> Javier

Great suggestions. Thanks for helping me clear that up.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Javier Guerra]

On Thu, Jul 16, 2009 at 9:27 AM, Joshua Russo wrote:
> What are some examples of mutating operations (and other operations
> for that matter) that you use in your models?

the most obvious are:

- 'calculated' fields.  the first example is the __unicode__() method,
but lots others, like multiplying widht/height fields to get an 'area'
field, (en/de)coding JSON data in one field to/from a python object,
fetching a filesize from the filesystem,  things like that.

- 'pre-made filters'. like alex's example

- overriding the save() method, to add some consistency checks, update
other tables (like maintaining denormalized fields), override one
field with another, etc.

- other kinds of objects that could present a nicer API than just a
bunch of DB records and tables, for example a tag soup, even if it's
an extra table, it's presented as extra fields and operations on the
'tagged' model.

- anything that Java developers could call the 'business layer', or
anything that a DB manager would like to put into an stored procedure
(unless you're able to actually do that!)

a worthwhile goal is to make view functions really really thin.  maybe
just a few lines before calling the template or a generic view.

if your model layer presents an API that is close to the concepts the
user manages, but at the same time isn't tied to the specific web
interface, you'll be able to easily rework the whole appearance of
your website, quite possibly even change the whole user experience
without rewriting the 'operational' code.

-- 
Javier

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Alex Robbins]

Whenever I do the same query in a couple of different views, I try to
pull it out into a manager method[1]. For instance, get all the
stories that are marked published from the last two days. I would make
this a method on the manager so I can reuse it other places. Then I
write a test for that manager method to make sure it was selecting the
right stuff. (Not a mutating operation, but it is model-related)

Hope that helps,
Alex

[1] http://docs.djangoproject.com/en/dev/topics/db/managers/#topics-db-managers

On Jul 16, 4:27 am, Joshua Russo  wrote:
> > besides the testing issues (which are certainly a heated debate!), i have 
> > to say that my Django projects became far better organized and a lot more 
> > flexible when i learned to put most of the code on the models, and not on 
> > the views.
>
> I find this really interesting because I wanted to put more code into
> the models but couldn't seem to find the right hooks to accomplish my
> specific tasks.
>
> What are some examples of mutating operations (and other operations
> for that matter) that you use in your models?
>
> Thanks for your suggestions
> Josh
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Joshua Russo]

> besides the testing issues (which are certainly a heated debate!), i have to 
> say that my Django projects became far better organized and a lot more 
> flexible when i learned to put most of the code on the models, and not on the 
> views.

I find this really interesting because I wanted to put more code into
the models but couldn't seem to find the right hooks to accomplish my
specific tasks.

What are some examples of mutating operations (and other operations
for that matter) that you use in your models?

Thanks for your suggestions
Josh
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Daniel Roseman]

On Jul 16, 1:20 am, Joshua Russo  wrote:
> I'm in the process of implementing testing (both doc tests and unit
> tests) though I'm having some conceptual difficulty. I'm not sure how
> far to take the testing. I'm curious what people concider an
> appropriate level of testing.
>
> This thought struck me most when I was going through the testing
> documentation, in the section regarding the testing of models. It
> seems to me that very little logic generally goes into the model
> classes. At least for me, I have far more logic in admin.py and
> views.py.
>
> What are peoples thoughts on testing models? What exactly are you
> testing for there? How do you gauge code that should be tested versus
> code that doesn't really need it?
>
> Thanks for your help
> Josh

The key thing is to know when to rely on Django's own tests, and when
to write your own. There's absolutely no point, for example, to write
tests that check that you can instantiate and save a model, and that
the values in the database match the values you pass in. That's
covered entirely by Django, and is very well-tested.

So what your unit tests need to cover is the bits where your code
interacts with and builds on the things that Django provides - ie the
business logic. This can be in custom model methods, or in views, or
library modules.

Personally, I always have trouble separating functional testing from
unit testing, so my so-called unit tests seem to always end up using
the test client to get a page and checking to see if it contains the
right data. That's certainly a *part* of unit testing, but you also
need to break things down to smaller units so you can test individual
bits of logic, and this is very difficult to do right.
--
DR.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Javier Guerra]

Joshua Russo wrote:
> This thought struck me most when I was going through the testing
> documentation, in the section regarding the testing of models. It
> seems to me that very little logic generally goes into the model
> classes. At least for me, I have far more logic in admin.py and
> views.py.

besides the testing issues (which are certainly a heated debate!), i have to 
say that my Django projects became far better organized and a lot more flexible 
when i learned to put most of the code on the models, and not on the views.

IOW, it has become a lot more productive for me not to think in terms of the 
webapp, but in terms of the data objects.  first invest a good portion of time 
(and several diagrams) getting the right structures to represent your data, add 
several methods to do any needed manipulation right in terms of the data, not 
in terms of the user's actions.  sometimes it's necessary to add manager 
objects to make model objects more 'high-level' and not as tied to the RDBMS 
tables used to store them.

with that structures in place, the view functions become very thin, just to get 
the needed objects for the templates, and interpret the user's actions to call 
any mutating operation on the relevant models.

when you do that, it becomes clear that a lot of the tests can be right in the 
models.py file, to exercise said functionality.  the tests.py is (mostly) 
reserved for testing the user interface, calling the URLs and verifying the 
response and effects.

hope that helps,

-- 
Javier

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Shawn Milochik]

Basically, you want to test anything that might break if another part  
of the code is changed that interacts with it,  might break if the  
application takes an alternative flow (maybe finally hits the 'else'  
of that if/else statement), or could possibly receive invalid input.

Unfortunately, that is theoretically everything.

I don't know what the correct answer is, and I suspect that the  
correct answer is (as is so often the case) "it depends." All  
applications are unique.

In any case, if you care enough to be doing it at all, and are  
concerned about doing it the right way, then you'll probably be fine.



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: New to unit testing, need advice

[Wayne Koorts]

> I'm in the process of implementing testing (both doc tests and unit
> tests) though I'm having some conceptual difficulty. I'm not sure how
> far to take the testing. I'm curious what people concider an
> appropriate level of testing.

Uh oh, be ready for a huge thread.  This is one of those unending debates.

Basically:

- More thorough test coverage == more confidence as you know there is
less chance that when you break something it will go unnoticed.
- Creating complete test coverage and keeping it up to date takes a
lot of time, and sometimes the decision is made to use that time for
other things.

As with anything, there's a balance.  It's very dependant on the
project and team involved.

Regards,
Wayne Koorts
http://www.wkoorts.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

New to unit testing, need advice

[Joshua Russo]

I'm in the process of implementing testing (both doc tests and unit
tests) though I'm having some conceptual difficulty. I'm not sure how
far to take the testing. I'm curious what people concider an
appropriate level of testing.

This thought struck me most when I was going through the testing
documentation, in the section regarding the testing of models. It
seems to me that very little logic generally goes into the model
classes. At least for me, I have far more logic in admin.py and
views.py.

What are peoples thoughts on testing models? What exactly are you
testing for there? How do you gauge code that should be tested versus
code that doesn't really need it?

Thanks for your help
Josh
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---