Thank you for starting this conversation. On Tue, 2010-11-02 at 08:59 -0500, Deryck Hodge wrote: > I'd like to propose an experiment to see if this holds true. I would > propose that for 3 months (or until the Thunderdome) that we: > > 1.) disable page tests and windmill tests > 2.) leave windmill around for running yuitests and API integration > tests > 3.) anytime we touch UI in this time we must add unit test coverage > for browser classes (if the page test was acting as a unit test for > the view) > 4.) we track closely any regression that slips through and the impact > of this regression
I am +1 for the test. > A note about #2, #3, and #4 here: > > For #2, I think we should still run the JS unit tests automatically > and in my experience Windmill is not fragile for yui tests. Migrating > to jstestdriver, if we decide post-experiement to abandon Windmill > completely, should be a different issue, IMHO. Francis noted > yesterday that the API tests for the js LP.client have no other > coverage, so we should leave those around as well. Again, I don't > think these change much or are subject to the same fragility. I agree that the YUI test harness has never failed, but It will always be possible since there is exactly 1 page call though windmill for each registered suite. > My assumption about #3 is that we have tested some bits in page tests > that have no other test coverage. For example, view methods that > should be tested in browser unit tests but are covered by the page > test. I'm proposing we disable these tests but leave them around and > that we add unit test coverage where appropriate when touching UI. I > am not suggesting we have browser unit tests attempt to replicate > story tests. I think it would be could to run the page tests locally > when working on UI to see if something needs unit test coverage, too. > But this can't really be enforced by the experiment. I always delete any page test that contradicts a unittest. I have seen page tests that are real stories (a user acceptance test); they only fail when a feature is dramatically changed and often require a total re-write. I still favour assigning a several engineers to purge pagetests/stories of duplication. I cringe every time I see a page test check for form failures and permission denied. -- __Curtis C. Hovey_________ http://launchpad.net/
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
