On Fri, May 29, 2009 at 8:31 AM, Ashley Moran <[email protected]> wrote: > > > On 28 May 2009, at 15:46, John Jones wrote: > >> After Ashley's recent excellent intro to cucumber and BDD I have >> finally got around to getting it running. BTW, a belated BIG thank you >> to Ashley for a great presentation and a well run session. > > Haha, that made me laugh. I thought it was a model of how not to run > a presentation :) But thanks for the kind words...
Don't be too modest. Standing up in front of a bunch of your opinionated peers, and typing, thinking and speaking all while fielding questions and consuming alcohol has just got to be admired! > > >> The four scenarios I found I was interested in evaluating cucmber in >> are: rails (obviously), stand alone running against an embedded web >> server, a regular ruby app, and linking it to a set of existing >> external perl tests and programs. These evaluations are all done and >> now running nicely in their relative applications. >> >> For some time I have been wondering how to bring all the perl tests, >> in a perl app, together within a sensible top down test strategy. It >> looks like cucumber provides the glue. Currently I run these perl >> tests in batches using perl's test::more harness. Now with the wonder >> of cucumber I have integrate the ruby/perl world enabling me to build >> on the automation already in place around the perl app with BDD. >> Before going too much further with this approach does anyone know if >> this is already being done? If so can you point me in the right >> direction? > > Are each of these applications independent, or do they integrate to > form a larger system? The short answer is they integrate to form a large system. It is quite a mixture: external services (that change from time to time so I run tests against them), perl modules doing DSP/FFT as well as running four expert systems (in three threads) processing real time data, an application build suite (for continuous integration of the main application), a web interface running on localhost fronts the application, a ruby REST service in our intranet fronts a ruby application supporting another interface protocol to third parties and used by the application plus an Internet side web site coordinates the running of potentially many thousands of applications (at the moment we are in small numbers). > > Can you describe more about these Perl tests? Are they code level > tests? If so, you're stuck with them in Perl. If they are black-box > tests, you could move them to Cucumber. Some are code level and others are more like integration tests. There are approaching five years worth of tests and these have evolved as testing has evolved in this time -- along with my own interpretation to suite my agenda. Fortunately they are quite granular and very encapsulated so I can build on them with simple step handlers/helpers. My plan is to do black box testing with webrat steps. This provides the interface for doing testing via the embedded web server running on localhost. Also, this black box approach can be used to assess changes to the rules in each of its four expert systems. How to test these four knowledge bases has been puzzling me for some time too, because the sum of the tested parts certainly does not guarantee its performance. For example, one of the applications significant features is given the same input real time data streams, each separate running application will produce similar end results but make different decisions along the way. In fact they do this slightly too well at the moment (for testing purposes). A little less lopsidity in their actual decision making was anticipated. > > What I suspect you want is a `rake predeploy` (or something) task that > runs all the Cucumber features and all the Perl tests together, and > fails unless both subcommands are successful. At the moment I see cucumber providing this layer. What are your thoughts on this? Are there goals the cucumber development path has that might make this impractical? The big attraction (for me) to taking this approach lies in using the feature and scenario descriptions to capture and present application knowledge/expertise related to the feature-scenario and test steps each scenario fires. Capturing this application expertise is a big issue in this project. If I just run the perl test harness(es) from a rake task, I know they all pass or fail, which is great, but to another engineer coming along later it is going to be more opaque than seeing immediately which step failed. For example, in the tests that run against one of the external services if the date format changes in the data feed the feedback is instant, no digging down and running the test again manually to find out what has happened. Plus, ongoing development and maintenance is all driven agilely from one cucumber layer - even if we are going to add additional functionality, test::more tests in perl, ruby or python. I felt really backed into a corner when I looked at all the testing needs this application demands and while there was a way to tackle each part, each solution was piece meal and lacked a top down coherence and feel good factor. Cucumber is helping me to put some order and positive 'feel good' into this testing process and so far it appears to work well. I can think top down from the business needs and build down into the piece meal testing layer as needed by adding suitable step helpers. Up to now just three step helpers seem to provide all I need to interface into scripts and I can see the need for just two more, though these are more nice to have than vital. John > > Ashley > > -- > http://www.patchspace.co.uk/ > http://www.linkedin.com/in/ashleymoran > http://aviewfromafar.net/ > http://twitter.com/ashleymoran > > > > > > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "NWRUG" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nwrug-members?hl=en -~----------~----~----~----~------~----~------~--~---
