Hello, everybody.
Jorge, thanks for starting this thread.
OK, well, I'd like to start from what I'd call the "current state" -- this is,
the most frequent problems I see when it comes to testing protected actions:
First of all, testing protected areas in Web sites powered by repoze.who +
repoze.what, independently of TG2, has been *badly* supported "officially",
which is mostly my fault. As a result, I've seen several testing approaches
which will prove to be a bad choice in the medium/long term, including:
1.- GETting /login_handler?login=userid&password=pass every time you want to
test that authorization is granted to the right subject. This is bad because:
* It's anti-DRY. It's annoying to write that line so often.
* It doesn't scale: It assumes that, out of the wide range of
authentication methods, your application will be tied to the
RedirectingFormPlugin. If this changes, you'll have to update your tests
accordingly.
2.- Forging the repoze.what credentials dict:
* It's not part of the API, it's an internal variable. Its disposition or
availability may change at any moment, and because it's internal, the built-in
predicate checkers will be updated internally, accordingly -- but your tests
will break.
* It bypasses identification, which will cause side effects the sooner or
later. For example, identity['user'] won't be defined.
3.- Independently of the way authentication was forged, when authorization is
denied and the user was anonymous, you check whether you got a 302 status code
and got redirected to the login form. That's bad:
* Again, it doesn't scale: It assumes that, out of the wide range of
challengers, your application will be tied to the RedirectingFormPlugin. If
this changes, you'll have to update your tests accordingly.
* There must be only two possible 4XX HTTP status codes, _never_ a 3XX: 401
(if the user was anonymous) or 403 (if the user was authenticated).
I'm not blaming those of you who have been using any of the methods above. I'm
blaming myself for not filling the gap before :)
(Jorge's proposals fall into first approach mentioned above, so from my point
of view they are just an alternate way to do the same thing and get the same
side effects).
So, I'll start supporting, with documentation and ready-to-use test utilities,
the long awaited recommended/official way to test protected areas:
Authentication must be ignored while testing the protected areas. Although at
first you may think that I am crazy, in fact it's *totally* useless to test
authentication while testing a protected area. At most, it only makes sense to
test authentication *once*, if and only if:
1.- You want to make sure repoze.who is properly integrated (integration
tests). In this situation, you just need a *separate* test case made up of
around 3 tests: One to test that the user is challenged when authorization is
denied with a 401 status code, another to test that voluntary logins work and
another to test that logout works. That's it. So, as your application evolves
and your authentication method changes, you'd just have to update three tests,
not the whole test suite.
2.- You wrote your own repoze.who plugin. In this situation, you need plain
old *unit* tests, not functional tests. Take a look into the official
repoze.who plugins, all of them have unit tests only: The are so isolated that
they only depend on the WSGI environ, nothing else, so functional tests (i.e.,
with your application "loaded") are not necessary.
I can tell you that testing authentication with official repoze.who plugins is
a waste of time, because they are all pretty well tested. At most you may want
to make sure that they are integrated correctly, with the ~3 tests I mentioned
above.
***** So, while testing protected areas, what you really must test is
identification and authorization. Authentication must be absolutely bypassed
-- Tested separately if you really have to. *****
SHOW ME SOME CODE!
If I've not already bored you with so much text, at this point you must be
eager to see some code. So here I go...
The "reasonable"/"official" way to test protected areas, using "/admin/" as an
example, which should had been available since day 1 is:
<=====
class TestAdmin(TestController):
def test_index(self):
resp = self.app.get('/admin/', extra_environ={'userid': 'manager'},
status=200)
assert "some text" in resp.body
def test_foo(self):
resp = self.app.get('/admin/foo', extra_environ={'userid': 'manager'},
status=200)
assert "some text" in resp.body
def test_authorization_denied_anonymous(self):
"""If the user is anonymous, the status must be 401"""
self.app.get('/admin/', status=401)
self.app.get('/admin/foo', status=401)
self.app.get('/admin/bar', status=401)
def test_authorization_denied_authenticated(self):
"""If the user is authenticated, but still authorization is denied,
the status must be 403"""
self.app.get('/admin/', extra_environ={'userid': 'editor'},
status=403)
self.app.get('/admin/foo', extra_environ={'userid': 'editor'},
status=403)
self.app.get('/admin/bar', extra_environ={'userid': 'editor'},
status=403)
=====>
instead of:
<=====
class TestAdmin(TestController):
def test_index(self):
self.app.get('/login_handler?login=manager&password=managepass')
resp = self.app.get('/admin/', status=200)
assert "some text" in resp.body
def test_foo(self):
self.app.get('/login_handler?login=manager&password=managepass')
resp = self.app.get('/admin/foo', status=200)
assert "some text" in resp.body
def test_authorization_denied_anonymous(self):
"""If the user is anonymous, she must be redirected to the login
form"""
resp = self.app.get('/admin/', status=302)
assert resp.follow().location.startswith('/login_handler')
resp = self.app.get('/admin/foo', status=302)
assert resp.follow().location.startswith('/login_handler')
resp = self.app.get('/admin/bar', status=302)
assert resp.follow().location.startswith('/login_handler')
def test_authorization_denied_authenticated(self):
"""If the user is authenticated, but still authorization is denied,
the status must be 403"""
self.app.get('/login_handler?login=editor&password=editpass')
self.app.get('/admin/', status=403)
self.app.get('/admin/foo', status=403)
self.app.get('/admin/bar', status=403)
=====>
I think the advantages are obvious:
1.- Less code, much less code.
2.- No need to deal with authentication. As a consequence, it's easy to
change the authentication methods whenever you want.
3.- Identification is not bypassed.
4.- No need to deal with the internal repoze.what credentials dict.
5.- Semantic: 401 and 403 are the HTTP status codes to deny authorization.
Then, you can use a test case like the one below to test authentication
itself, as it will behave in the real-world:
<=====
class TestAuthentication(unittest.TestCase):
def setUp(self):
config = loadapp('config:test.ini', relative_to=conf_dir)
config.global_conf['skip_authentication'] = False
self.app = webtest.TestApp(config)
def test_forced_login(self):
# Requesting a protected area:
resp = self.app.get('/admin/', status=302)
# We're redirected to the login form:
resp = resp.follow()
assert resp.location.startswith('/login_handler')
# Submitting the login form:
login_form = resp.forms['login']
login_form['login'] = u'manager'
login_form['password'] = 'managepass'
login_handler = login_form.submit()
# Checking that we're redirected to the initially requested page:
assert login_handler.location.startswith('/post_login')
original_page = login_handler.follow()
assert 'http://127.0.0.1:8080/admin/' == original_page.location
# Checking that she's not anonymous:
assert 'authtkt' in original_page.cookies
def test_voluntary_login(self):
# Requesting the login form:
resp = self.app.get('/login', status=302)
# Submitting the login form:
login_form = resp.forms['login']
login_form['login'] = u'manager'
login_form['password'] = 'managepass'
login_handler = login_form.submit()
# Checking that we're redirected to the home page:
assert login_handler.location.startswith('/post_login')
original_page = login_handler.follow()
assert 'http://127.0.0.1:8080/' == original_page.location
# Checking that she's not anonymous:
assert 'authtkt' in original_page.cookies
def test_logout(self):
# Logging in:
resp = self.app.get('/login_handler?login=editor&password=editpass')
# Checking that she's not anonymous:
assert 'authtkt' in resp.cookies
# Logging out:
resp = self.app.get('/logout_handler')
# Checking that she *is* anonymous:
assert 'authtkt' not in resp.cookies
=====>
So, as I've already said, as your application evolves and your authentication
method changes, you would just have to update the three tests above -- *not*
every single test that covers a protected area.
Finally, this is how I'll implement this:
1.- I have a repoze.who plugin which acts as identifier and challenger to
make the tests above possible.
2.- I'll make repoze.what-quickstart use it, replacing the default form
plugin, when we're asked to skip authentication.
Now, I can integrate this functionality nicely in TG2 if you accept it. The
implementation would be simple and backwards compatible:
1.- We'd have to add one line to test.ini (under [DEFAULT]), in the
quickstart template:
skip_authentication = True
2.- Next, pass the value of this variable, if defined, to repoze.what-
quickstart. Specifically, I'd have to pass its value as an argument to
tg.configuration:AppConfig.add_auth_middleware(), which will pass it to
repoze.what.plugins.quickstart:setup_sql_auth().
I hope I've made the problem clear. I'm absolutely convinced that this is the
way to go and I want to correct it in TG2 apps.
Cheers!
On Monday February 23, 2009 07:34:49 Jorge Vargas wrote:
> As agreed with gustavo we'll continue this on the mailing list.
>
> To the ones just joining we are discussing how to write webtests for
> auth enable pages.
>
> My suggestion is to add something like the following to the end of
> http://trac.turbogears.org/browser/projects/tg.devtools/trunk/devtools/temp
>lates/turbogears/%2Bpackage%2B/tests/__init__.py_tmpl
>
> class AuthTestController(TestController):
> def login(self,username,password):
> <some code>
> def logout(self):
> <some code>
>
> --------------------------------------
> So you will write tests like
>
> class TestAdminPermissions(AuthTestController):
> def setup():
> self.login('manager',managepass')
> def test_go_admin(self):
> self.app.get('/admin')
> <success manager can see /admin>
> .....
>
> class TestUserPermission(AuthTestController):
> def test_go_admin(self):
> self.app.get('/admin')
> <fail anon can't see /admin>
>
>
> This means one test class = one set of rules satisfied by one
> predicate (or set of predicates) the drawback is that you will have to
> set users for each type/group of predicate instead of testing directly
> for the predicate.
>
> A second option is to simply provide login and logout as two module
> level functions this will allow even more rich testing sets as you can
> call them from any of the nose fixtures. (package, module, class or
> even function)
>
> Another variable of this was 'contributed' by ChrisP and is posted
> here http://paste.turbogears.org/paste/34752 the problem with this one
> is that it
> a- by passes authentication
> b- uses "private" r.what api
--
Gustavo Narea <http://gustavonarea.net/>.
Get rid of unethical constraints! Get freedomware:
http://www.getgnulinux.org/
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"TurboGears Trunk" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/turbogears-trunk?hl=en
-~----------~----~----~----~------~----~------~--~---
