Hi, all. I've got a large Tomboy note going for how to write Windmill tests. It's made of stuff from my lightning talk at the Launchpad Thunderdome, interleaved with notes made to Tim and Ian recently. I paste it here to share knowledge. I'm also in the process of reworking this into the Windmill/JavaScript wiki pages.
So then, here lies my notes. Don't write a Windmill test unless you embrace the gospel below. ;) Goal: We are interested in testing that the XHR for JavaScript on a given page is hooked up to the server correctly, receives a response, and that response is generally sane and shown to the user. That is *all* we test. Windmill tests are not yet written to be web page smoke tests (i.e. a replacement for the stuff in stories that exercises the basics of each page). • Write the least amount of code you can to achieve this goal • Think about what the user is doing • Test what the user will see • Trust your other tests • Provide YUI test coverage to exercise the widget itself We are *not* testing: • JavaScript • Widgets • User interactions There are other tests to cover these elements. We only care about the XHR interactions between web server and the web page. Some tips to keep you focused on the goal: 1) Avoid waits.sleep Sleep can't be avoided completely. Some tests just won't pass without it. But you should never use sleep when you can wait on your element to appear. If you have to use sleep to get a test to pass, dig deeper and see if the js or the test can be rewritten. 2) Never use xpath expressions This is a big one, so some explanation... == xpath is fragile== This comes down to developers not understanding how to write xpath expressions and xpath expressions relying on html. Since HTML often changes before id or class names do, this can cause tests to fail and have to be rewritten, which causes people to trust Windmill tests less. Really, this is not so bad on one hand, but it is annoying and creates work and there are better ways to get a nodes than xpath anyway. jquery selectors rock for replacing all the stuff we've used xpath for in the past. == xpath is hard to write correctly == Worst case, developers write arcane xpath expressions that cause the test to pass, but they are not really testing what they think they are testing. This masks deeper problems with the test. I've never met someone who *really* knew xpath expressions, have you? ;) == xpath creates the wrong kind of test == It should also be avoided because if you're writing some xpath expression, you're probably already thinking about testing the wrong thing. Windmill tests should be XHR integration tests and they should be written from the user's point of view. id and class names help us identify the parts of the page the user needs to interact with, so all tests should be written off of these, rather than the html around these. In other words, xpath pushes the test writer to think about html layout and page arrangement, when we really don't care about those in these tests. We care about what the user is doing. Anything the user clicks or interacts with should have a unique id or can be identified with a combination of id and class names. So "avoid xpath" is a good general rule to producing better tests. It also produces better code because you have to think about the id and class names you're using more, which makes the page, the js, and the widgets themselves better. == xpath seems slower == I don't have benchmarks for this, so I'm hesitant to say it *is* slower. But test times have been faster for me, for the times I've migrated lines away from xpath. I've never done this in isolation, i.e. there were other test improvements, too, but my sense is that xpath takes longer to deal with than selecting on id, classname, or query selectors. 3) Always use a timeout on waits.forElement I can't prove it yet, but I think missing timeout values is what causes Windmill tests to hang. 4) Do not use a page reload to confirm data Your test can access the same objects as a unit test. Use that power. We want to confirm the XHR post saved the data correctly, so check the data. We don't need to confirm via the page. Just confirm one small bit on the page, so we know the data was presented back to the user correctly or that success was accurately reported. 5) Never use assertElemJS or assertProperty These are not JS or CSS tests remember! We only care about the server interactions. Push this stuff down to YUI tests. 6) Do not do error checking Use YUI test for making sure the widget does what it should when it errors. Hope that helps! Cheers, deryck _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
