Re: Fixing a poorly written Django website (no unit tests)

[Kevin Ndung'u]

On Sunday, March 8, 2015 at 12:16:43 AM UTC+3, Some Developer wrote:
>
> On 06/03/15 16:23, Ilya Kazakevich wrote: 
> > You may start from highest level testing: 
> > 1) create "usage scenarios" for your website. Like "customer opens page 
> > 'foo', and should see 'bar'". You use such scenarios for manual testing, 
> > right? 
> > 2) code such scenarios against Django project. You may use BDD testing 
> > tools (like lettuce or behave), or do it in pure python. You may call 
> > Django views and templates directly (using django unittesting support or 
> > lettuce/harvest), or do it through the browser using Selenium. One 
> > scenario -- one method. Yes, there is a lot of boilerplate code, but you 
> > only need to write it once. BDD tools are used to simplify this process 
> > by writing scenarios on DSL called Gherkin, which is easier. 
> > 
> > This is some kind of "acceptance testing", the one that helps you to be 
> > sure website works like customer expects it to work. Every project 
> > should have one, I believe. 
> > 
> > After it, you may find that some units of your system are complex and 
> > error prone. What is unit? Unit is module, function or even package with 
> > _clear interface_. You then create unit tests against this module. Only 
> > units with complex logic need to be tested. No need to test simple view 
> > that just returns render(): this view is tested as part of high level 
> > testing. But if you have function "calc_user_amount" and complex 
> > business logic stands behind it, you may create unit test for it. 
> > 
> > There are 3 mistakes people do about testing: 
> > 1) Testing each function, even private function, even 1 line function. 
> > It takes a lot of time, and such tests are fragile. You throw'em away 
> > after simple refactoring. 
> > 2) Testing "in the middle of abstraction": I mean testing functions with 
> > out of clear interface, like private functions. If you need to read 
> > function code before writing test (pydoc is not enough), you should not 
> > test this function. Try a higher level of abstraction. 
> > 3) Skipping high level testing: even if all your code is covered with 
> > low-level unit-tests, you still need high level testing like the one 
> > with Selenium to make sure everything works correctly, and when you 
> > refactor your code you use such testing to make sure you did not break 
> > anything. 
> > 
> > So, you start with high level, and then cover _some_ units with unit 
> > test, if you believe they may contain error. 
> > 
>
> Awesome advice. Thanks! 
>
> I bought a book called "Test Driven Development using Python" and have 
> had a look through it a bit. It does seem a bit over the top (write no 
> code until you have a unit test for it in existence even if it is just a 
> one liner view function) but hopefully I'll be able to take some of the 
> advice and use it to fix up my code base. 
>
> Has anyone else read the book? Any opinions? 
>


I have read the book and it got me started in Django when I knew absolutely 
nothing about testing.
Though it might be a bit over the top, it manages to teach you the concepts 
so that you can later on apply on them on your own.
I highly recommend it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/24e0e29c-5e3d-4750-aebe-96d49fac247f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fixing a poorly written Django website (no unit tests)

[Some Developer]

On 06/03/15 16:23, Ilya Kazakevich wrote:

You may start from highest level testing:
1) create "usage scenarios" for your website. Like "customer opens page
'foo', and should see 'bar'". You use such scenarios for manual testing,
right?
2) code such scenarios against Django project. You may use BDD testing
tools (like lettuce or behave), or do it in pure python. You may call
Django views and templates directly (using django unittesting support or
lettuce/harvest), or do it through the browser using Selenium. One
scenario -- one method. Yes, there is a lot of boilerplate code, but you
only need to write it once. BDD tools are used to simplify this process
by writing scenarios on DSL called Gherkin, which is easier.

This is some kind of "acceptance testing", the one that helps you to be
sure website works like customer expects it to work. Every project
should have one, I believe.

After it, you may find that some units of your system are complex and
error prone. What is unit? Unit is module, function or even package with
_clear interface_. You then create unit tests against this module. Only
units with complex logic need to be tested. No need to test simple view
that just returns render(): this view is tested as part of high level
testing. But if you have function "calc_user_amount" and complex
business logic stands behind it, you may create unit test for it.

There are 3 mistakes people do about testing:
1) Testing each function, even private function, even 1 line function.
It takes a lot of time, and such tests are fragile. You throw'em away
after simple refactoring.
2) Testing "in the middle of abstraction": I mean testing functions with
out of clear interface, like private functions. If you need to read
function code before writing test (pydoc is not enough), you should not
test this function. Try a higher level of abstraction.
3) Skipping high level testing: even if all your code is covered with
low-level unit-tests, you still need high level testing like the one
with Selenium to make sure everything works correctly, and when you
refactor your code you use such testing to make sure you did not break
anything.

So, you start with high level, and then cover _some_ units with unit
test, if you believe they may contain error.



Awesome advice. Thanks!

I bought a book called "Test Driven Development using Python" and have 
had a look through it a bit. It does seem a bit over the top (write no 
code until you have a unit test for it in existence even if it is just a 
one liner view function) but hopefully I'll be able to take some of the 
advice and use it to fix up my code base.


Has anyone else read the book? Any opinions?

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54FB6A26.5050906%40googlemail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fixing a poorly written Django website (no unit tests)

[Carl Meyer]

On 03/06/2015 11:23 AM, Ilya Kazakevich wrote:
> You may start from highest level testing: 
> 1) create "usage scenarios" for your website. Like "customer opens page
> 'foo', and should see 'bar'". You use such scenarios for manual testing,
> right?
> 2) code such scenarios against Django project. You may use BDD testing
> tools (like lettuce or behave), or do it in pure python. You may call
> Django views and templates directly (using django unittesting support or
> lettuce/harvest), or do it through the browser using Selenium. One
> scenario -- one method. Yes, there is a lot of boilerplate code, but you
> only need to write it once. BDD tools are used to simplify this process
> by writing scenarios on DSL called Gherkin, which is easier. 
> 
> This is some kind of "acceptance testing", the one that helps you to be
> sure website works like customer expects it to work. Every project
> should have one, I believe.
> 
> After it, you may find that some units of your system are complex and
> error prone. What is unit? Unit is module, function or even package with
> _clear interface_. You then create unit tests against this module. Only
> units with complex logic need to be tested. No need to test simple view
> that just returns render(): this view is tested as part of high level
> testing. But if you have function "calc_user_amount" and complex
> business logic stands behind it, you may create unit test for it.
> 
> There are 3 mistakes people do about testing:
> 1) Testing each function, even private function, even 1 line function.
> It takes a lot of time, and such tests are fragile. You throw'em away
> after simple refactoring.
> 2) Testing "in the middle of abstraction": I mean testing functions with
> out of clear interface, like private functions. If you need to read
> function code before writing test (pydoc is not enough), you should not
> test this function. Try a higher level of abstraction.
> 3) Skipping high level testing: even if all your code is covered with
> low-level unit-tests, you still need high level testing like the one
> with Selenium to make sure everything works correctly, and when you
> refactor your code you use such testing to make sure you did not break
> anything.
> 
> So, you start with high level, and then cover _some_ units with unit
> test, if you believe they may contain error.

This is really excellent advice.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F9FA6E.8050001%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Fixing a poorly written Django website (no unit tests)

[Ilya Kazakevich]

You may start from highest level testing: 
1) create "usage scenarios" for your website. Like "customer opens page 
'foo', and should see 'bar'". You use such scenarios for manual testing, 
right?
2) code such scenarios against Django project. You may use BDD testing 
tools (like lettuce or behave), or do it in pure python. You may call 
Django views and templates directly (using django unittesting support or 
lettuce/harvest), or do it through the browser using Selenium. One scenario 
-- one method. Yes, there is a lot of boilerplate code, but you only need 
to write it once. BDD tools are used to simplify this process by writing 
scenarios on DSL called Gherkin, which is easier. 

This is some kind of "acceptance testing", the one that helps you to be 
sure website works like customer expects it to work. Every project should 
have one, I believe.

After it, you may find that some units of your system are complex and error 
prone. What is unit? Unit is module, function or even package with _clear 
interface_. You then create unit tests against this module. Only units with 
complex logic need to be tested. No need to test simple view that just 
returns render(): this view is tested as part of high level testing. But if 
you have function "calc_user_amount" and complex business logic stands 
behind it, you may create unit test for it.

There are 3 mistakes people do about testing:
1) Testing each function, even private function, even 1 line function. It 
takes a lot of time, and such tests are fragile. You throw'em away after 
simple refactoring.
2) Testing "in the middle of abstraction": I mean testing functions with 
out of clear interface, like private functions. If you need to read 
function code before writing test (pydoc is not enough), you should not 
test this function. Try a higher level of abstraction.
3) Skipping high level testing: even if all your code is covered with 
low-level unit-tests, you still need high level testing like the one with 
Selenium to make sure everything works correctly, and when you refactor 
your code you use such testing to make sure you did not break anything.

So, you start with high level, and then cover _some_ units with unit test, 
if you believe they may contain error.





On Wednesday, March 4, 2015 at 3:03:14 PM UTC+3, Some Developer wrote:
>
> Hi, 
>
> I've been working on a Django website for about 2 months on and off and 
> am nearing the end of development work where I can start thinking about 
> making it look pretty and the after that deploy to production. 
>
> I've been doing lots of manual testing and I'm sure that the website 
> works correctly but due to the need to get the website in production 
> ASAP and my lack of unit testing experience with Django (I'm still not 
> entirely sure what the point of unit testing a 2 or 3 line Django view 
> is when you can clearly see if it is correct or not) I've neglected 
> automated testing. 
>
> While I'm still going to go ahead and launch the site in production as 
> soon as it is deployed I want to go back and add in all the unit tests 
> that are missing. How would you tackle this problem? 
>
> Most of the code is pretty simple but there are ecommerce elements that 
> I have tested extensively by running my code through the Python 
> debugger. These must always work. 
>
> I'm a bit ashamed that it has got this far but I'm mainly a C developer 
> and unit testing isn't pushed quite so hard there (even though it should 
> be). 
>
> Any help appreciated. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/62a7cff3-bb01-42a2-9c7d-d28780033dd5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fixing a poorly written Django website (no unit tests)

[Mike Dewhirst]

On 4/03/2015 11:01 PM, Some Developer wrote:

Hi,

I've been working on a Django website for about 2 months on and off and
am nearing the end of development work where I can start thinking about
making it look pretty and the after that deploy to production.

I've been doing lots of manual testing and I'm sure that the website
works correctly but due to the need to get the website in production
ASAP and my lack of unit testing experience with Django (I'm still not
entirely sure what the point of unit testing a 2 or 3 line Django view
is when you can clearly see if it is correct or not) I've neglected
automated testing.

While I'm still going to go ahead and launch the site in production as
soon as it is deployed I want to go back and add in all the unit tests
that are missing. How would you tackle this problem?


Use Coverage. https://pypi.python.org/pypi/coverage/3.7.1

Best thing since sliced bread ...



Most of the code is pretty simple but there are ecommerce elements that
I have tested extensively by running my code through the Python
debugger. These must always work.

I'm a bit ashamed that it has got this far but I'm mainly a C developer
and unit testing isn't pushed quite so hard there (even though it should
be).

Any help appreciated.



--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F77F91.3080408%40dewhirst.com.au.
For more options, visit https://groups.google.com/d/optout.

Re: Fixing a poorly written Django website (no unit tests)

[Dan Gentry]

No need to test the Django provided logic, but I like to write a few tests 
for each view that check the permissions, urls, updates, etc.  More of a 
functional test than a unit test.  I find that when these tests fail it is 
usually something changed somewhere else in the app.  For example, a change 
to a model that alters validation of an update.

In the case of your ecommerce code, complete tests can make your process of 
testing much faster compared to manual debugging work, plus they will test 
all areas of the code the same way every time.  Less chance for omission or 
error.  Confidence in the code goes up.

On Wednesday, March 4, 2015 at 7:03:14 AM UTC-5, Some Developer wrote:
>
> Hi, 
>
> I've been working on a Django website for about 2 months on and off and 
> am nearing the end of development work where I can start thinking about 
> making it look pretty and the after that deploy to production. 
>
> I've been doing lots of manual testing and I'm sure that the website 
> works correctly but due to the need to get the website in production 
> ASAP and my lack of unit testing experience with Django (I'm still not 
> entirely sure what the point of unit testing a 2 or 3 line Django view 
> is when you can clearly see if it is correct or not) I've neglected 
> automated testing. 
>
> While I'm still going to go ahead and launch the site in production as 
> soon as it is deployed I want to go back and add in all the unit tests 
> that are missing. How would you tackle this problem? 
>
> Most of the code is pretty simple but there are ecommerce elements that 
> I have tested extensively by running my code through the Python 
> debugger. These must always work. 
>
> I'm a bit ashamed that it has got this far but I'm mainly a C developer 
> and unit testing isn't pushed quite so hard there (even though it should 
> be). 
>
> Any help appreciated. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/bcfa95ac-8116-427d-bad0-25dc535161b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fixing a poorly written Django website (no unit tests)

[Avraham Serour]

Manual testing takes time and are prone to errors because they are done by
humans, automatic testing means that when changing something in the code
you can just run the tests, you don't need to run the server, open the
browser, click, click and see if something seems out of order, then open
the js console to see if it throws any error, then add some console.log,
then remove them and so on...

It is actually hard to understand why you would need testing for something
small, but as with every project, you'll need to add features and the code
will grow slowly with time, before you realize you already have something
big

On Wed, Mar 4, 2015 at 2:01 PM, Some Developer 
wrote:

> Hi,
>
> I've been working on a Django website for about 2 months on and off and am
> nearing the end of development work where I can start thinking about making
> it look pretty and the after that deploy to production.
>
> I've been doing lots of manual testing and I'm sure that the website works
> correctly but due to the need to get the website in production ASAP and my
> lack of unit testing experience with Django (I'm still not entirely sure
> what the point of unit testing a 2 or 3 line Django view is when you can
> clearly see if it is correct or not) I've neglected automated testing.
>
> While I'm still going to go ahead and launch the site in production as
> soon as it is deployed I want to go back and add in all the unit tests that
> are missing. How would you tackle this problem?
>
> Most of the code is pretty simple but there are ecommerce elements that I
> have tested extensively by running my code through the Python debugger.
> These must always work.
>
> I'm a bit ashamed that it has got this far but I'm mainly a C developer
> and unit testing isn't pushed quite so hard there (even though it should
> be).
>
> Any help appreciated.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/54F6F431.2080800%40googlemail.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tL3vbmC0L_Cb-Nsj0AXR1y__Ba_sCLOAKx1TJD8s%3DH%3DtQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Fixing a poorly written Django website (no unit tests)

[Some Developer]

Hi,

I've been working on a Django website for about 2 months on and off and 
am nearing the end of development work where I can start thinking about 
making it look pretty and the after that deploy to production.


I've been doing lots of manual testing and I'm sure that the website 
works correctly but due to the need to get the website in production 
ASAP and my lack of unit testing experience with Django (I'm still not 
entirely sure what the point of unit testing a 2 or 3 line Django view 
is when you can clearly see if it is correct or not) I've neglected 
automated testing.


While I'm still going to go ahead and launch the site in production as 
soon as it is deployed I want to go back and add in all the unit tests 
that are missing. How would you tackle this problem?


Most of the code is pretty simple but there are ecommerce elements that 
I have tested extensively by running my code through the Python 
debugger. These must always work.


I'm a bit ashamed that it has got this far but I'm mainly a C developer 
and unit testing isn't pushed quite so hard there (even though it should 
be).


Any help appreciated.

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F6F431.2080800%40googlemail.com.
For more options, visit https://groups.google.com/d/optout.