Am Freitag, dem 21.09.2012 um 11:09 schrieb Arthur Schiwon: > Hey Jörn, > > thanks a lot, it's too long of course, but i read it anyway. > > You know i would really love to automatically test the LDAP backend. However, > it requires an available LDAP server, otherwise there is not much to test. Is > it possible to have one running on Jenkins? >
Hi Arthur, My plan in terms of executing the test suite is based on heavy usage of automated vm creation using vagrant (http://vagrantup.com/) This is necessary to test ownCloud with a great variety of server configurations: [sqlite, mysql, pg, (oracle?)] x [apache, nginx, lighttpd, (IIS?)] x [....] ldap - among other authentication backends will 'just' add another dimension to this configuration space. > Plus, it is not enough to check if configuring and login works, but also all > the other lovely things which are used for local users. Then however, some > features do not work (e.g. current encryption, because password changes are > unknown). Do you have a fluffy sweet recipe for this, too? :) > From a pure behavior driven testing approach I would expect that all scenarios should work in any configuration combination. Exceptions - as usual - prove the rule! ;-) Take care, Tom aka DeepDiver > Cheers > Arthur > > On Friday, September 21, 2012 12:34:42 AM Jörn Friedrich Dreyer wrote: > > TLDR: Describe a feature with scenarios in gherkin[1]! Do it! Do it now! > > > > Have you, too, asked yourself what Frank actually means when he says > > "Please give it a really good hard test."? I always imagine a dark > > cellar, chained to a desk, where I have to sit late into the night to > > click through every possible action ownCloud offers. What else would a > > good hard test be? > > > > But I don't want to ruin my evenings like this. I prefer sitting in my > > garden, sipping a cold beer (you are hereby invited to join me, should > > you visit Hannover). There must be a better solution! I mean, we are > > software developers ... we automate things! So, Tom (DeepDiver) and I > > started working on setting up an automated test suite that emulates a > > user sitting in front of a browser, doing all the "really good hard > > test"ing. Now we need to tell it what to do! > > > > As an example I just committed some Gherkin scenarios for the core files > > app[1]. The nice thing about them is: they are readable by your mom! > > Take for example, this excerpt: > > > > <gherkin> > > Feature: files > > In order to organize my files > > As a mom > > I want to have basic file management > > > > Background > > Given I am logged in > > # these are the files hosted on demo.owncloud.org > > And I am in the files app > > > > Scenario: create files and folders > > When I click on the new button > > And I click on the new <type> action > > And I enter <filename> > > Then I should see <filename> > > And <filename> should have mimetype <mimetype> > > And <filename> should have size <size> > > > > Examples: > > | type | mimetype | filename | size | > > | file | text/plain | simplefile.txt | 0 | > > | file | text/plain | äöü ß ÄÖÜ € @.txt | 0 | > > | file | text/x-c | test.cc | 0 | > > | file | text/x-php | test.php | 0 | > > | folder | httpd/unix-directory | Testfolder | 0 | > > | folder | httpd/unix-directory | Testfolder with spaces | 0 | > > > > </gherkin> > > > > Not that hard to understand, is it? > > > > Until now, this is basically a checklist, a tester could use to do > > Franks bidding. Now, as I said earlier, I tend to find this kind of > > tedious task ... very boring. Enter cucumber[2]: with a little glue > > code, these gherkin scenarios can be executed as unit tests! The clue > > code translates "I click on the new button" into code that cucumber uses > > to execute actions in a browser and check results. > > > > At this point I admit that I sometimes forget to execute the test suite > > before commiting. Enter our continuous integration server[3] (thx again > > tom for your awesome work there, too!): > > Jenkins will monitor the commits and execute the cucumber test suite > > after each checkin. He currently posts the result of the simpletest test > > suite into the IRC dev channel allowing you to blame me whenever I break > > a feature our users expect ;) > > > > In the end, this process will enable us to automate large parts of QA by > > testing each commit against every feature with combination of sqlite, > > mysql, Postgresql, Oracle, Firefox and Chrome/Chromium. We will always > > be able to say "Feature A works, which means that you can do scenario X, > > Y and Z". I cannot stress enough how awwwesome ... err ... sweeeeeeet > > this is! > > > > Sure,there is still a ton of work left to do, and as we all know the > > first steps are always the hardest. Fortunately, even your mom, dad, > > sister, brother or significant other can help us with the first step - > > writing textual scenarios for features: > > > > 1. Go to http://demo.owncloud.org > > 2. Pick an app from the navigation bar > > 3. Keep the gherkin syntax page open for reference > > https://github.com/cucumber/cucumber/wiki/Gherkin > > 4. Learn from feature examples > > https://github.com/aslakhellesoy/cucumber-rails-test/tree/master/features > > (all the *.feature files) > > 5. Start with a general Feature description: > > > > Feature: [the app name is enough for now] > > In order to [goal] > > [stakeholder] > > wants [behavior] > > > > 6. Flesh out some scenarios with Given-When-Then: > > https://github.com/cucumber/cucumber/wiki/Given-When-Then > > > > Scenario: browsing the gallery > > Given I am logged in > > And I have uploaded the demo files > > When I go to the gallery > > Then I want to see a thumbnail of <image> > > > > Examples: > > | image | > > | Demo Image - ccc.jpg | > > | Demo Image - Laser Towards Milky Ways Centre.jpg | > > | Demo Image - Northern Lights.jpg | > > > > 7. Send the .feature as a response to this message or directly to me. > > > > I'll take care of aggregating the first round of scenarios into > > features. We'll then create a public repository on github and start > > writing the glue code (Step Definitions[4]). But thats another topic for > > the future. > > > > Now, go ahead, write a few feature scenarios, and ask your neighbor to > > do it, too! It'll save you lots of hours of silly clicking in the night! > > And it will bring world peace! > > > > so long > > > > Jörn > > > > [1] > > https://gitorious.org/owncloud-acceptance-test-suite/owncloud-acceptance-tes > > t-suite [2] http://cukes.info/ > > [3] http://ci.tmit.eu/ > > [4] https://github.com/cucumber/cucumber/wiki/Step-Definitions > _______________________________________________ > Owncloud mailing list > [email protected] > https://mail.kde.org/mailman/listinfo/owncloud _______________________________________________ Owncloud mailing list [email protected] https://mail.kde.org/mailman/listinfo/owncloud
