I have started working on such project, first effects are here: https://github.com/mspanc/vspec
m. 2016-02-02 3:25 GMT+01:00 Chris Daley <chebiza...@gmail.com>: > Hi Evan, > Thanks for the offer, I've sent you a pull request on Github. I've got some > new Vala posts in the can that aren't related to Valadate that should be up > later this week as well. > Cheers > Chris D > > 2016-02-01 16:12 GMT-08:00 Evan Nemerson <e...@nemerson.com>: > > > Since you're actively blogging about Vala, how would you feel about > > being added to Planet Vala (planet.vala-project.org)? Nobody on is > > very active right now, so it would be good to get some new blood… > > > > > > On Mon, 2016-02-01 at 14:37 -0800, Chris Daley wrote: > > > Hi Steven, > > > > > > It's still very much active, I've just had to take some time out to > > > do some > > > work on my night job (managing a Rock band). I'm drafting a blog post > > > this > > > afternoon with an update on progress and some new ways to contribute, > > > so > > > stay tuned. > > > > > > Cheers > > > Chris D > > > > > > > > > > > > 2016-02-01 14:22 GMT-08:00 Steven Oliver <oliver.ste...@gmail.com>: > > > > > > > Hey, > > > > I just wanted to drop in and see if any progress had been made on > > > > this? > > > > > > > > I for one am very excited about the possibility. > > > > > > > > Thank you, > > > > Steven N. Oliver > > > > > > > > > > > > > > > > On Fri, Jan 8, 2016 at 2:55 PM -0800, "Chris Daley" <chebizarro@gma > > > > il.com> > > > > wrote: > > > > > > > > Hi Al, > > > > > > > > > > Thanks for the input, this is very much appreciated and > > > > > incredibly useful! > > > > > I had in fact just written a post on my blog asking for exactly > > > > > this sort > > > > > of feedback - you can read it here -http://chrisdaley.biz/test-dr > > > > > iven-development-in-vala-pt-1.html > > > > > > > > > > I've got some time to work on this over the next few weeks and > > > > > hope to have > > > > > a usable release out for testing not long after, so now's the > > > > > time for > > > > > anyone else interested in a testing framework to hit me up with > > > > > your ideas! > > > > > > > > > > Have a good weekend, > > > > > Cheers > > > > > Chris D > > > > > > > > > > > > > > > 2016-01-08 14:29 GMT-08:00 Al Thomas : > > > > > > > > > > > > From: Chris Daley > > > > > > > > > > > > > > > > > > Sent: Tuesday, 5 January 2016, 1:16 > > > > > > > Subject: Re: [Vala] Testing framework > > > > > > > > > > > > > > I've done some thinking about this over the last couple of > > > > > > > months and > > > > > > used > > > > > > > the holiday period to finish off a few things, namely a port > > > > > > > of Gherkin > > > > > > to > > > > > > > Vala. You can grab it from here if you want to check it out: > > > > > > > https://github.com/chebizarro/gherkin-vala > > > > > > > > > > > > > > I'm going to be sketching out what I think is a reasonable > > > > > > > roadmap over the next week or so with a view to an alpha > > > > > > > release around > > > > > > the > > > > > > > end of February. If anyone is interested in contributing, or > > > > > > > has any > > > > > > > specific ideas about what sort of features they would find > > > > > > > the most > > > > > > useful, > > > > > > > > > > > > > please get in touch > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > Great work on porting Gherkin 3 to Vala. > > > > > > > > > > > > I wanted to put forward a few ideas that I have been slowly > > > > > > researching over the past year or so in the hope they are > > > > > > > > > > > > insightful to any developments of testing tools for Vala. > > > > > > > > > > > > Gherkin > > > > > > ------- > > > > > > Gherkin is a language for structuring human language in a > > > > > > way that allows business analysts, developers and testers to > > > > > > define > > > > > > features of an application. > > > > > > > > > > > > Some computer languages have tools available to developers to > > > > > > convert > > > > > > > > > > > > Gherkin to an outline of a computer program. The following > > > > > > meaningless > > > > > > > > > > > > example shows the structure. This uses PHP's Behat to convert > > > > > > Gherkin > > > > > > > > > > > > to PHP: > > > > > > > > > > > > Feature: test > > > > > > > > > > > > Background: given this is a test # features/test.feature:3 > > > > > > > > > > > > Scenario: testing of app # features/test.feature:5 > > > > > > When i run something > > > > > > Then it passes > > > > > > > > > > > > 1 scenario (1 undefined) > > > > > > 2 steps (2 undefined) > > > > > > 0m0.17s (9.41Mb) > > > > > > > > > > > > --- FeatureContext has missing steps. Define them with these > > > > > > snippets: > > > > > > > > > > > > /** > > > > > > * @When i run something > > > > > > */ > > > > > > public function iRunSomething() > > > > > > { > > > > > > throw new PendingException(); > > > > > > } > > > > > > > > > > > > /** > > > > > > * @Then it passes > > > > > > */ > > > > > > public function itPasses() > > > > > > { > > > > > > throw new PendingException(); > > > > > > } > > > > > > > > > > > > It is the feature context that forms the basis of generating > > > > > > automated acceptance tests from the features specified in > > > > > > > > > > > > Gherkin. The developer then fills in the gaps with code that > > > > > > drives the tests. In PHP web development this is a tool like > > > > > > Mink that can drive various headless web browsers. > > > > > > > > > > > > Automatic code generation for Vala > > > > > > ---------------------------------- > > > > > > I can think of two approaches to generation code. I have > > > > > > tried neither. > > > > > > > > > > > > The first is to use libvala to generate a Vala AST then output > > > > > > the AST. Potentially libvala could be modified to generate a > > > > > > > > > > > > Genie version of the AST. This appeals to me. > > > > > > > > > > > > The other approach would be similar to Valadoc: > > > > > > > > > > > > https://git.gnome.org/browse/valadoc/tree/src/libvaladoc/api/fo > > > > > > rmalparameter.vala#n131 > > > > > > > > > > > > Code generation could also be useful for anyone wanting to > > > > > > > > > > > > develop a tool similar to RSpec. So a common approach using > > > > > > libvala > > > > > > may be helpful. I think Anjuta CTags also uses libvala for > > > > > > > > > > > > autosuggestion of function names etc. See: > > > > > > > > > > > > https://github.com/GNOME/anjuta/blob/master/plugins/symbol-db/a > > > > > > njuta-tags/ctags-visitor.vala > > > > > > > > > > > > Acceptance Testing Drivers > > > > > > -------------------------- > > > > > > This is probably the hardest part given the wide range of > > > > > > interfaces available. > > > > > > > > > > > > Gherkin is from Cucumber written in Ruby with web application > > > > > > development in mind. So I think most tools there use web > > > > > > > > > > > > interfaces. In the Vala world this could be done with libsoup > > > > > > for a text analysis of the web interface, but also embedding > > > > > > Webkit or Gecko which also allows Javascript to be tested. > > > > > > > > > > > > Vala is often used for desktop GUI development. So Linux > > > > > > Desktop Testing Project ( http://ldtp.freedesktop.org/wiki/ ) > > > > > > using Assistive Technology Service Provider Interface ( > > > > > > > > > > > > > > > > > > https://en.wikipedia.org/wiki/Assistive_Technology_Service_Prov > > > > > > ider_Interface > > > > > > ) may be relevant. > > > > > > > > > > > > Of course software is also developed for technical users. So > > > > > > there are potentially command line interfaces, D-Bus > > > > > > interfaces, shared > > > > > > library interfaces and so on to cater for. > > > > > > > > > > > > For command line interfaces I'm starting to think GLib's > > > > > > trap_subprocess may be useful: > > > > > > > > > > > > http://valadoc.org/#!api=glib-2.0/GLib.Test.trap_subprocess > > > > > > I'm trying to write functional tests for Genie, but some > > > > > > > > > > > > features need to stop the compilation process. e.g. attempting > > > > > > to override a protected method in a class. The test should > > > > > > > > > > > > trap the error from valac and make sure it matches the > > > > > > > > > > > > expected error. > > > > > > > > > > > > > > > > > > Test Output > > > > > > ----------- > > > > > > This is probably moving away from specification by example > > > > > > with Gherkin and moving towards unit testing. Specifically > > > > > > it's about my experience with GLib testing framework reports. > > > > > > > > > > > > Each test can output the results as TAP (Test Anything > > > > > > Protocol) > > > > > > by using the --tap switch. I would recommend TAP because it > > > > > > is processed more easily by a lot of tools.e.g. Jenkins build > > > > > > server. > > > > > > > > > > > > > > > > > > In the past I have done unit tests using a script such as: > > > > > > > > > > > > #/bin/sh > > > > > > > > > > > > tests="arrays_multiline > > > > > > variables_declarations > > > > > > variables_type_inference" > > > > > > > > > > > > # Build test binaries > > > > > > for test in $tests > > > > > > do > > > > > > valac $test.gs > > > > > > done > > > > > > > > > > > > # Run test binaries > > > > > > gtester --keep-going -o results.xml $tests > > > > > > > > > > > > # Fix missing info section from results - see > > > > > > https://bugzilla.gnome.org/show_bug.cgi?id=668035 > > > > > > > > > > > > > > > > > info="\n\nUnknown\nUnknown\n\n" > > > > > > sed -i 's||'$info'|g' results.xml > > > > > > > > > > > > # Generate HTML report > > > > > > gtester-report results.xml > report.html > > > > > > > > > > > > > > > > > > I have not figured out a convenient way to get the output as > > > > > > TAP, > > > > > > > > > > > > although a script for automake is available: > > > > > > https://git.gnome.org/browse/glib/tree/tap-driver.sh > > > > > > > > > > > > > > > > > Unit Test of Binaries > > > > > > --------------------- > > > > > > Vala and Genie produce binaries of course. > > > > > > > > > > > > So far I have produced a test binary that is a Position > > > > > > Independent Executable ( > > > > > > > > > > > > https://securityblog.redhat.com/2012/11/28/position-independent > > > > > > -executables-pie/ > > > > > > ) that has all the symbols in the dynamic table. This > > > > > > allows each unit test to treat the test binary as a > > > > > > > > > > > > shared object to it can test individual functions, but also > > > > > > be run as an executable for functional testing. > > > > > > > > > > > > I haven't figured out how to do this for a production binary. > > > > > > This would probably a static build so the production binary has > > > > > > the unit tests aggregated after it in memory and the linker > > > > > > should use the symbol table and not the dynamic symbol table. > > > > > > > > > > > > For a test binary you may also want to include the --debug > > > > > > flag in Valac to include Vala filenames. I think this was tried > > > > > > > > > > > > for code coverage reports with Valum - see > > > > > > > > > > > > https://coveralls.io/builds/4147345 for example. Not sure of th > > > > > > e results. > > > > > > > > > > > > > > > > > > Vala Bugs to be Aware Off > > > > > > ------------------------- > > > > > > May be of interest: > > > > > > https://bugzilla.gnome.org/show_bug.cgi?id=704072 > > > > > > https://bugzilla.gnome.org/show_bug.cgi?id=597999 > > > > > > https://bugzilla.gnome.org/show_bug.cgi?id=739725 > > > > > > > > > > > > Hope all this helps frame any work you may do on Vala > > > > > > testing. > > > > > > > > > > > > All the best with it, > > > > > > > > > > > > Al > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Chris Daley > > > > > Pacific Northwest > > > > > > > > > > e: chebiza...@gmail.com > > > > > w: http://chrisdaley.biz > > > > > m: +1601 980 1249 > > > > > > > > > > s: chebizarro > > > > > tw: chebizarro > > > > > tz: PDT > > > > > _______________________________________________ > > > > > vala-list mailing listvala-list@gnome.orghttps://mail.gnome.org/m > > > > > ailman/listinfo/vala-list > > > > > > > > > > > > > > > > > > > > > > -- > Chris Daley > Pacific Northwest > > e: chebiza...@gmail.com > w: http://chrisdaley.biz > m: +1601 980 1249 > s: chebizarro > tw: chebizarro > tz: PDT > _______________________________________________ > vala-list mailing list > vala-list@gnome.org > https://mail.gnome.org/mailman/listinfo/vala-list > _______________________________________________ vala-list mailing list vala-list@gnome.org https://mail.gnome.org/mailman/listinfo/vala-list
