Re: [Elementary-dev-community] Testing Round 2
Hi,
I have just subscribed to this list and saw this thread.
satisfactory (see this:
http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html);
however, Vala's development environment is too complicated for me (a Vala
I use similar setup and am using it heavily and it works perfectly. I
reused some libgee-s AbstracTestCase and using cmake to build and run
tests coexist /w the normal app and/or shared libs in the similar
directory structure.
The output likes this:
/ValueHolderTest/null_value_test: OK
/ValueHolderTest/string_value_test: OK
/ValueHolderTest/enum_value_test: OK
...
/AspectTriggerTest/test_trigger_discard: OK
/BindingsTest/bind_aspect_adapter_test: OK
The main test program:
static int main (string[] args) {
GLib.Test.init (ref args);
TestSuite.get_root ().add_suite (new ValueHolderTest ().get_suite ());
TestSuite.get_root ().add_suite (new BindingsTest ().get_suite ());
return GLib.Test.run ();
}
And a simple test suite /w some unit tests:
public class ModelPresenterTest : AbstractTestCase {
...
public ModelPresenterTest () {
base (ModelPresenterTest);
add_test (change_model_under_presenter_test,
change_model_under_presenter_test);
add_test (base_test, base_test);
}
public override void set_up () {
value_holder = new ValueHolder ();
}
public override void tear_down () {
value_holder = null;
}
public void change_model_under_presenter_test () {
}
My vision for this experiment is that someone can set up Noise (or some
other project) for testing, and then I (and/or other volunteers) can start
writing a few tests for it and sort of coach the developers in TDD for the
purpose of getting the developers to a self-sufficient state before
potentially moving onto another project and doing the same.
I think it's a very good idea and I recommend the following post to
all of those who have never heard about nor used TDD:
http://blogs.agilefaqs.com/2011/11/01/importance-of-unit-testing-and-test-driven-development-tdd/
Also the libgee (Vala) guys are using tests heavily either.
--
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : elementary-dev-community@lists.launchpad.net
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing Round 2
Hey, I do not really have the time to read the whole conversation, but a few points: An unfinished article on the subject (lack of time, etc...): http://xapantu.wordpress.com/2013/04/07/writing-tests-using-ldtp-for-elementary/ I have already implented a few tests in the past (unfortunately, due to lack of time there is nearly nothing), for instance for scratch: http://bazaar.launchpad.net/~elementary-apps/scratch/scratch/view/head:/scratchcore/core-tests/main.vala http://bazaar.launchpad.net/%7Eelementary-apps/scratch/scratch/view/head:/scratchcore/core-tests/main.vala (and more generally all the scratchcore/ folder) I think there were a few things for marlin/pantheon files too, but I cannot remember wether it has been erased by some crazy merges or not. You can use accessibility features to do efficient testing, see mago, ldtp, at-spi: https://live.gnome.org/DesktopTesting http://ldtp.freedesktop.org/ (it looks like it is down at this moment but this is the most interesting one). I recommend you to not sart with Noise, it has a complex UI (code side), and it does some weird things when you try to do automation with it (I tested, at-spi was not strong enough to work with Noise if I recall correctly). Marlin/pantheon-files seems a better target for an experiment (the widgets are simplier). Inline replies below: On 06/04/2013 19:56, Craig wrote: Alright guys, I think it would be a good start to select an application and try to start writing tests with it. It seems like GLib's testing environment should be satisfactory (see this: http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html); however, Vala's development environment is too complicated for me (a Vala noob) to get a project set up for tests. See scratch build systems and our custom cmake modules, it should be sufficient to do everything you want, and quite easily I think ;) (for someone who knows CMake reasonably). If someone would be willing to volunteer to get GLib's testing set up for Noise (or something else if they so desire) including a quick example test (so those of us with testing--but not Vala--experience can have a starting point), I would be willing to start writing tests and working with the project's primary developer(s) to get them started toward Test Driven Development (other developers with TDD experience also expressed interest). Unfortunately, starting unit testing means a lot of code reorganization, so this is actually the point that someone needs to work on. Writing tests once the API is clear is the easiest part -.- [...] My vision for this experiment is that someone can set up Noise (or some other project) for testing, and then I (and/or other volunteers) can start writing a few tests for it and sort of coach the developers in TDD for the purpose of getting the developers to a self-sufficient state before potentially moving onto another project and doing the same. Thoughts? In my opinion, writing a few blog posts would be also an interesting thing, there are very few good articles about GUI testing for Vala/GTK/CMake. Lucas -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
[Elementary-dev-community] Testing
Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Individual comment from a non-dev who has just observed devs: it would seem we have no formal testing policy but that we gravely need one. Code is developed and requested to be merged by dev A, then dev B looks the code itself over. Hopefully at this point dev B would then build it and test the crap out of it to make sure everything still works perfectly, but I feel that doesn't get done thoroughly as often as it should and consequently regressions slip through. Automated and standardized testing would be boss. Regards, Cassidy James On Apr 4, 2013 7:52 AM, Craig webe...@gmail.com wrote: Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Yes, testing is sorely needed. I started down the path of finding the best solution for Vala Unit Testing, but couldn't find any great options. Currently, all projects that pursued the xUnit (https://en.wikipedia.org/wiki/XUnit) model for Vala have fallen out of development and only their shells remain. I reached out to the Yorba devs, who appear to hold a repository for the old 'valadate' project but they are in the same position we are in. I believe they have done some testing with the built-in GObject unit test suite ( https://live.gnome.org/Vala/TestSample), but it has been found to be severely lacking by anyone who knows anything about unit testing. As with most OSS developers, I started to scratch that itch and realized that it is a little bit beyond my skill set. The code I started to write remains unusable, rotting away somewhere on one of my hdd platters in a folder called 'vunit'. If anyone is interested in starting a Vala Unit Test project under the umbrella of the elementary community, I'm sure we could get quite a bit of traction from the Vala community at large and I would love to help out. On Thu, Apr 4, 2013 at 7:59 AM, Sergey Shnatsel Davidoff ser...@elementaryos.org wrote: This is what I've developed for testing: https://lists.launchpad.net/elementary-dev-community/msg01948.html I've also created Glimpse - a system for sandboxed testing, but it lacks a D-bus proxy which is needed for adding/removing apps via GUI. See http://shnatsel.blogspot.com/2011/07/weve-just-revolutionized-alpha-testing.htmland http://shnatsel.blogspot.com/2012/05/state-of-glimpse.html I don't think we have any unit testing in place yet. Midori probably has but that's it. 2013/4/4 Craig webe...@gmail.com Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. For now what we just do is we test the interface, the new features and some old features too to see if they remain unaffected. On Thu, Apr 4, 2013 at 1:51 PM, Craig webe...@gmail.com wrote: Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Our m̶o̶s̶t̶ ̶e̶f̶f̶e̶c̶t̶i̶v̶e̶ only regression testing is in Files. It goes Cody, I think I fixed crash X and I add it to my list of crash report email titles I look for to reoccur. On Thu, Apr 4, 2013 at 8:25 AM, David Gomes da...@elementaryos.org wrote: Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. For now what we just do is we test the interface, the new features and some old features too to see if they remain unaffected. On Thu, Apr 4, 2013 at 1:51 PM, Craig webe...@gmail.com wrote: Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Cody Garver -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
We also had a regression on Granite the other day. It goes I fixed this bug and I tested that bug but then I didn't test crazy-weird behaviour and it turned out it broke some things. Of course, action has been taken to fix it, but it can't be merged yet. (sorry for sending this email to Cody twice, I meant to Reply To All. On Thu, Apr 4, 2013 at 2:31 PM, Cody Garver c...@elementaryos.org wrote: Our m̶o̶s̶t̶ ̶e̶f̶f̶e̶c̶t̶i̶v̶e̶ only regression testing is in Files. It goes Cody, I think I fixed crash X and I add it to my list of crash report email titles I look for to reoccur. On Thu, Apr 4, 2013 at 8:25 AM, David Gomes da...@elementaryos.orgwrote: Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. For now what we just do is we test the interface, the new features and some old features too to see if they remain unaffected. On Thu, Apr 4, 2013 at 1:51 PM, Craig webe...@gmail.com wrote: Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Cody Garver -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
David, I understand where you're coming from and I have worried about that in the past. There are two approaches for Unit Testing. The strong approach is to figure out what you want the app to do (i.e. design the framework), describe that app's functions and libraries using tests, then write your code around it. If at some point the requirements change, you can modify the Unit Tests. It's boring, but provides a very stable foundation for apps. The second approach (and what we would have to do for all existing projects) is to write the tests around the app. This approach is like trying to build a bucket around the water, but it can be done and any development going forward is more likely to not break the core functionality of the app. Would it be feasible to create a Unit Test team on launchpad with the sole purpose of specializing in adding testing to projects and writing the tests required to kill regression bugs before they kill us? Also, Adam (at Yorba) and I came up with the conclusion that rolling a new Unit Test framework might be the best option. Here's a quote from Adam at Yorba: You might also want to consider rolling your own simple unit testing framework in Vala without even using Glib.Test - sometimes I think that might be the easiest path forward here. So, this could be a really big project, but I see a lot of merit to it. On Thu, Apr 4, 2013 at 8:25 AM, David Gomes da...@elementaryos.org wrote: Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. For now what we just do is we test the interface, the new features and some old features too to see if they remain unaffected. On Thu, Apr 4, 2013 at 1:51 PM, Craig webe...@gmail.com wrote: Hello everyone, I'm curious what you devs do for testing? I'm not particularly familiar with Vala, but I'm learning a lot about testing at work and I'm trying to develop myself to that end in my free time. I'm sending this email because I'd like to get a pulse on what you Elementary devs think about testing and what you actually do to test your code. Also, please feel encouraged to talk about what you've done in the past, what has/hasn't worked for you, and generally what your philosophy is about testing (or whether you have no philosophy). Individual comments and comments on behalf of elementary as a whole are both welcome. Sound off! Thanks, Craig -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
2013/4/4 Craig webe...@gmail.com Sergey, that looks very interesting, although I'm a little confused--what are the differences between your first link and Glimpse? My first link is my scripts for easily testing code proposed for merging. I created them to offload the burden of testing proposed code from developers to adventurous users. However, nobody understand what are those scripts and why they're needed unless I explain it personally on IRC .I guess I suck at emailing people, but I always knew that... -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
I strongly recommend anyone interested in automated testing to read through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one responsible for most of unit testing in Ubuntu (he's also the author of Apport which we already use). His IRC nick is pitti and the session logs can be found at http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Here is another practical post I found interesting regarding setting up Unit Tests in Vala with Cmake: http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html I apologize for spamming the mailing list. On Thu, Apr 4, 2013 at 8:56 AM, Sergey Shnatsel Davidoff ser...@elementaryos.org wrote: I strongly recommend anyone interested in automated testing to read through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one responsible for most of unit testing in Ubuntu (he's also the author of Apport which we already use). His IRC nick is pitti and the session logs can be found at http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
We do unit and behavorial testing at where i work, and i think it's great. Writing tests before you develop the actual feature really makes you think why your feature should do what and when it should do so. Maybe we could build vala tests for granite. That would mean we would have to write tests for existing features too, but when that is done, we can implement features with first creating a test (a spec) for it, and then actually implementing it according to the spec. We could then integrate some kind of continuous integration using something like Travis (if it does vala), or if launchpad offers that functionality, we can use that. If it works for granite, we could slowly and steadily move to testing in all elementary projects. I think this would in the long term greatly improve stability of the system and individual apps, will encourage good and maintainable code and architecture of apps, and it would make elementary developing more professional. Just my 2 cents... Jaap Op 4 apr. 2013 16:11 schreef Dane Henson d...@elementaryos.org het volgende: Here is another practical post I found interesting regarding setting up Unit Tests in Vala with Cmake: http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html I apologize for spamming the mailing list. On Thu, Apr 4, 2013 at 8:56 AM, Sergey Shnatsel Davidoff ser...@elementaryos.org wrote: I strongly recommend anyone interested in automated testing to read through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one responsible for most of unit testing in Ubuntu (he's also the author of Apport which we already use). His IRC nick is pitti and the session logs can be found at http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
*If anyone is interested in starting a Vala Unit Test project under the
umbrella of the elementary community, I'm sure we could get quite a bit of
traction from the Vala community at large and I would love to help out. -
Dane Henson*
*
*
Not only that, but I imagine it would garner quite a lot of professional
developer attention for elementary, since professional devs seem
dramatically more exposed to (and interested in) formal testing than
hobbyist developers.
*I apologize for spamming the mailing list. - Dane Henson*
Please don't apologize--from the sounds of it, there is quite a bit of
consensus that this is a subject that needs to be discussed. And I for one
have found each of your messages profitable.
*Unit testing is boring to write so if we just said Everybody. Write unit
tests. All Projects. Now. it would really take on. On companies and when
developers are paid to work, they can write and put tests everywhere, but
it's harder for us. - David Gomes*
*
*
David, I used to think that as well; however, once you learn to write tests
correctly (I'm starting that process myself) it becomes more exciting. As
Dane mentions later in the thread, when you build the bucket around the
water (write tests for existing code), it is certainly drudgery; however,
when you write tests *before* you implement the functionality, you
1. wind up with better code (because code must be written in neat
modules to provide your tests access to the inputs/outputs they need)
2. get the excitement of writing code and watching your tests pass/fail
(and as you learn to write better tests, you get detailed information about
exactly what caused the failure)
3. can change your code fearlessly, knowing that if anything breaks, you
will be notified immediately
The whole process becomes at least a little more exciting, especially if
you've experienced a huge untested code base and the fear that comes with
having to make a change or implement a feature lest the whole thing come
tumbling down on you.
*While if we (developers) use the first approach, which is called TDD is
much better. We write the test first so we define how our app should behave
and how our code is structured already (so all the thinking of the code
structure you do it in the tests) which is really great. - Goncalo*
*
*
TDD = Test Driven Development (for the uninitiated). I elaborated on this
process to some extent in my previous paragraph.
We can also do BDD , if there's no framework for this we can probably
create something, for more infos you can check cucumber for Ruby. *Bdd
let's you write the tests in PLAIN english, so who does the mockups could
write the tests as well. that would be great. - Goncalo*
*
*
BDD = Behavior Driven Development. It's either the same thing or very
closely related to ATDD (Acceptance Test Driven Development). In either
case, the general idea is that you specify what the entire system
*must do* (system
requirements/acceptance criteria) and then you write tests to verify each
requirement. This sort of test might look like:
WHEN the bold button is pressed
AND the string 'abcd' is typed,
EXPECT the text view to contain the string 'abcd' in bold
Then, for each line, you write a function that implements either the setup
(the WHEN/AND portion) or the evaluation (the EXPECT portion) and then the
framework puts the pieces together and executes the test. The code for the
test (in pseudocode) might look like this:
func pressBoldButton () {
theApp.boldButton.setPressed (true)
}
func type(string input) {
theApp.InsertAtCursor (input)
}
func verifyContents () {
start := 0
end := 3
contents := theApp.GetFormattedSubStr (start, end)
contentsAreBold := contents.isBold()
letters := contents.getPlainTextString()
Testing.Expect (contentsAreBold);
Testing.Expect (letters == abcd)
}
On Thu, Apr 4, 2013 at 9:11 AM, Dane Henson d...@elementaryos.org wrote:
Here is another practical post I found interesting regarding setting up
Unit Tests in Vala with Cmake:
http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html
I apologize for spamming the mailing list.
On Thu, Apr 4, 2013 at 8:56 AM, Sergey Shnatsel Davidoff
ser...@elementaryos.org wrote:
I strongly recommend anyone interested in automated testing to read
through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
responsible for most of unit testing in Ubuntu (he's also the author of
Apport which we already use). His IRC nick is pitti and the session logs
can be found at
http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
--
Sergey Shnatsel Davidoff
OS architect @ elementary
--
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : elementary-dev-community@lists.launchpad.net
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help : https://help.launchpad.net/ListHelp
--
Mailing list: https://launchpad.net/~elementary-dev-community
Re: [Elementary-dev-community] Testing
Craig, thanks to make everything more clear.
Unit test is not just having something to compare stuff but we need a great
mocking system. Does anyone have links for that as well?
On Thu, Apr 4, 2013 at 4:00 PM, Craig webe...@gmail.com wrote:
*If anyone is interested in starting a Vala Unit Test project under the
umbrella of the elementary community, I'm sure we could get quite a bit of
traction from the Vala community at large and I would love to help out. -
Dane Henson*
*
*
Not only that, but I imagine it would garner quite a lot of professional
developer attention for elementary, since professional devs seem
dramatically more exposed to (and interested in) formal testing than
hobbyist developers.
*I apologize for spamming the mailing list. - Dane Henson*
Please don't apologize--from the sounds of it, there is quite a bit of
consensus that this is a subject that needs to be discussed. And I for one
have found each of your messages profitable.
*Unit testing is boring to write so if we just said Everybody. Write
unit tests. All Projects. Now. it would really take on. On companies and
when developers are paid to work, they can write and put tests everywhere,
but it's harder for us. - David Gomes*
*
*
David, I used to think that as well; however, once you learn to write
tests correctly (I'm starting that process myself) it becomes more
exciting. As Dane mentions later in the thread, when you build the bucket
around the water (write tests for existing code), it is certainly drudgery;
however, when you write tests *before* you implement the functionality,
you
1. wind up with better code (because code must be written in neat
modules to provide your tests access to the inputs/outputs they need)
2. get the excitement of writing code and watching your tests
pass/fail (and as you learn to write better tests, you get detailed
information about exactly what caused the failure)
3. can change your code fearlessly, knowing that if anything breaks,
you will be notified immediately
The whole process becomes at least a little more exciting, especially if
you've experienced a huge untested code base and the fear that comes with
having to make a change or implement a feature lest the whole thing come
tumbling down on you.
*While if we (developers) use the first approach, which is called TDD is
much better. We write the test first so we define how our app should behave
and how our code is structured already (so all the thinking of the code
structure you do it in the tests) which is really great. - Goncalo*
*
*
TDD = Test Driven Development (for the uninitiated). I elaborated on this
process to some extent in my previous paragraph.
We can also do BDD , if there's no framework for this we can probably
create something, for more infos you can check cucumber for Ruby. *Bdd
let's you write the tests in PLAIN english, so who does the mockups could
write the tests as well. that would be great. - Goncalo*
*
*
BDD = Behavior Driven Development. It's either the same thing or very
closely related to ATDD (Acceptance Test Driven Development). In either
case, the general idea is that you specify what the entire system *must do
* (system requirements/acceptance criteria) and then you write tests to
verify each requirement. This sort of test might look like:
WHEN the bold button is pressed
AND the string 'abcd' is typed,
EXPECT the text view to contain the string 'abcd' in bold
Then, for each line, you write a function that implements either the setup
(the WHEN/AND portion) or the evaluation (the EXPECT portion) and then the
framework puts the pieces together and executes the test. The code for the
test (in pseudocode) might look like this:
func pressBoldButton () {
theApp.boldButton.setPressed (true)
}
func type(string input) {
theApp.InsertAtCursor (input)
}
func verifyContents () {
start := 0
end := 3
contents := theApp.GetFormattedSubStr (start, end)
contentsAreBold := contents.isBold()
letters := contents.getPlainTextString()
Testing.Expect (contentsAreBold);
Testing.Expect (letters == abcd)
}
On Thu, Apr 4, 2013 at 9:11 AM, Dane Henson d...@elementaryos.org wrote:
Here is another practical post I found interesting regarding setting up
Unit Tests in Vala with Cmake:
http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html
I apologize for spamming the mailing list.
On Thu, Apr 4, 2013 at 8:56 AM, Sergey Shnatsel Davidoff
ser...@elementaryos.org wrote:
I strongly recommend anyone interested in automated testing to read
through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
responsible for most of unit testing in Ubuntu (he's also the author of
Apport which we already use). His IRC nick is pitti and the session logs
can be found at
http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
--
Sergey Shnatsel Davidoff
OS architect @
Re: [Elementary-dev-community] Testing
If we can get a number of experienced test practitioners and Vala developers to commit to it, I wouldn't mind contributing to a test framework development. Would anyone else be interested? On Thu, Apr 4, 2013 at 10:13 AM, Craig webe...@gmail.com wrote: *Would it be feasible to create a Unit Test team on launchpad with the sole purpose of specializing in adding testing to projects and writing the tests required to kill regression bugs before they kill us? - Dane* * * If we do this, I would expect it to be short term only. Developers should be responsible for testing their own code and Test Engineers should be primarily responsible for giving the devs the tools/training they need to test their own work. I propose instead that we only implement tests when we do bug fixes. If a bug crops up, we analyze what caused it and then we write a test to prevent any such bug from appearing (not just that specific bug, but any bug in that class of bugs). For example, I recently worked on a system that takes a config file, parses its key/value pairs into a map, and then exposes its values in the form of methods called string GetValueOfKey1(), string GetValueOfKey2(), etc. These functions simply contained return map.GetValue(key1);. This worked fine as long as the configuration file was setup correctly; however, as soon as someone made a mistake in the config file (accidentally renamed key1 to Key1 or some such) the application crashed because GetValueOfKey1() couldn't find key1 in the map structure. This error *ultimately resulted from* unhandled input, I created a test to test for *all kinds* of bad input and then implemented the input handling. Applying tests where they're useful prevents us from testing stable code. And then moving forward, we can write tests for all new functionality. On Thu, Apr 4, 2013 at 10:00 AM, Craig webe...@gmail.com wrote: *If anyone is interested in starting a Vala Unit Test project under the umbrella of the elementary community, I'm sure we could get quite a bit of traction from the Vala community at large and I would love to help out. - Dane Henson* * * Not only that, but I imagine it would garner quite a lot of professional developer attention for elementary, since professional devs seem dramatically more exposed to (and interested in) formal testing than hobbyist developers. *I apologize for spamming the mailing list. - Dane Henson* Please don't apologize--from the sounds of it, there is quite a bit of consensus that this is a subject that needs to be discussed. And I for one have found each of your messages profitable. *Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. - David Gomes* * * David, I used to think that as well; however, once you learn to write tests correctly (I'm starting that process myself) it becomes more exciting. As Dane mentions later in the thread, when you build the bucket around the water (write tests for existing code), it is certainly drudgery; however, when you write tests *before* you implement the functionality, you 1. wind up with better code (because code must be written in neat modules to provide your tests access to the inputs/outputs they need) 2. get the excitement of writing code and watching your tests pass/fail (and as you learn to write better tests, you get detailed information about exactly what caused the failure) 3. can change your code fearlessly, knowing that if anything breaks, you will be notified immediately The whole process becomes at least a little more exciting, especially if you've experienced a huge untested code base and the fear that comes with having to make a change or implement a feature lest the whole thing come tumbling down on you. *While if we (developers) use the first approach, which is called TDD is much better. We write the test first so we define how our app should behave and how our code is structured already (so all the thinking of the code structure you do it in the tests) which is really great. - Goncalo* * * TDD = Test Driven Development (for the uninitiated). I elaborated on this process to some extent in my previous paragraph. We can also do BDD , if there's no framework for this we can probably create something, for more infos you can check cucumber for Ruby. *Bdd let's you write the tests in PLAIN english, so who does the mockups could write the tests as well. that would be great. - Goncalo* * * BDD = Behavior Driven Development. It's either the same thing or very closely related to ATDD (Acceptance Test Driven Development). In either case, the general idea is that you specify what the entire system *must do* (system requirements/acceptance criteria) and then you write tests to verify each requirement. This
Re: [Elementary-dev-community] Testing
I would be interested but I'm not the best vala developer for sure. On Thu, Apr 4, 2013 at 4:40 PM, Craig webe...@gmail.com wrote: If we can get a number of experienced test practitioners and Vala developers to commit to it, I wouldn't mind contributing to a test framework development. Would anyone else be interested? On Thu, Apr 4, 2013 at 10:13 AM, Craig webe...@gmail.com wrote: *Would it be feasible to create a Unit Test team on launchpad with the sole purpose of specializing in adding testing to projects and writing the tests required to kill regression bugs before they kill us? - Dane* * * If we do this, I would expect it to be short term only. Developers should be responsible for testing their own code and Test Engineers should be primarily responsible for giving the devs the tools/training they need to test their own work. I propose instead that we only implement tests when we do bug fixes. If a bug crops up, we analyze what caused it and then we write a test to prevent any such bug from appearing (not just that specific bug, but any bug in that class of bugs). For example, I recently worked on a system that takes a config file, parses its key/value pairs into a map, and then exposes its values in the form of methods called string GetValueOfKey1(), string GetValueOfKey2(), etc. These functions simply contained return map.GetValue(key1);. This worked fine as long as the configuration file was setup correctly; however, as soon as someone made a mistake in the config file (accidentally renamed key1 to Key1 or some such) the application crashed because GetValueOfKey1() couldn't find key1 in the map structure. This error *ultimately resulted from* unhandled input, I created a test to test for *all kinds* of bad input and then implemented the input handling. Applying tests where they're useful prevents us from testing stable code. And then moving forward, we can write tests for all new functionality. On Thu, Apr 4, 2013 at 10:00 AM, Craig webe...@gmail.com wrote: *If anyone is interested in starting a Vala Unit Test project under the umbrella of the elementary community, I'm sure we could get quite a bit of traction from the Vala community at large and I would love to help out. - Dane Henson* * * Not only that, but I imagine it would garner quite a lot of professional developer attention for elementary, since professional devs seem dramatically more exposed to (and interested in) formal testing than hobbyist developers. *I apologize for spamming the mailing list. - Dane Henson* Please don't apologize--from the sounds of it, there is quite a bit of consensus that this is a subject that needs to be discussed. And I for one have found each of your messages profitable. *Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. - David Gomes* * * David, I used to think that as well; however, once you learn to write tests correctly (I'm starting that process myself) it becomes more exciting. As Dane mentions later in the thread, when you build the bucket around the water (write tests for existing code), it is certainly drudgery; however, when you write tests *before* you implement the functionality, you 1. wind up with better code (because code must be written in neat modules to provide your tests access to the inputs/outputs they need) 2. get the excitement of writing code and watching your tests pass/fail (and as you learn to write better tests, you get detailed information about exactly what caused the failure) 3. can change your code fearlessly, knowing that if anything breaks, you will be notified immediately The whole process becomes at least a little more exciting, especially if you've experienced a huge untested code base and the fear that comes with having to make a change or implement a feature lest the whole thing come tumbling down on you. *While if we (developers) use the first approach, which is called TDD is much better. We write the test first so we define how our app should behave and how our code is structured already (so all the thinking of the code structure you do it in the tests) which is really great. - Goncalo * * * TDD = Test Driven Development (for the uninitiated). I elaborated on this process to some extent in my previous paragraph. We can also do BDD , if there's no framework for this we can probably create something, for more infos you can check cucumber for Ruby. *Bdd let's you write the tests in PLAIN english, so who does the mockups could write the tests as well. that would be great. - Goncalo* * * BDD = Behavior Driven Development. It's either the same thing or very closely related to ATDD (Acceptance Test Driven Development). In either case, the general idea is that you
Re: [Elementary-dev-community] Testing
I'm not so sure we need a solution bound to Vala specifically, because: 1. We have automated UI testing covered by Ubuntu's regression testing framework 2. We have D-bus testing coverted by Ubuntu's regression testing framework 3. Vala translates to C so we can use a C unit testing system for functions For details on what Ubuntu do see Martin Pitt's UDW session, http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Sergey, how do you write code in Vala and write tests in C? it becomes too difficult for a developer, don't you think? On Thu, Apr 4, 2013 at 5:18 PM, Sergey Shnatsel Davidoff ser...@elementaryos.org wrote: I'm not so sure we need a solution bound to Vala specifically, because: 1. We have automated UI testing covered by Ubuntu's regression testing framework 2. We have D-bus testing coverted by Ubuntu's regression testing framework 3. Vala translates to C so we can use a C unit testing system for functions For details on what Ubuntu do see Martin Pitt's UDW session, http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
+1 Sergey, that page is massive. Could you send us the interesting parts? On Thu, Apr 4, 2013 at 11:59 AM, Goncalo Margalho g...@margalho.info wrote: Sergey, how do you write code in Vala and write tests in C? it becomes too difficult for a developer, don't you think? On Thu, Apr 4, 2013 at 5:18 PM, Sergey Shnatsel Davidoff ser...@elementaryos.org wrote: I'm not so sure we need a solution bound to Vala specifically, because: 1. We have automated UI testing covered by Ubuntu's regression testing framework 2. We have D-bus testing coverted by Ubuntu's regression testing framework 3. Vala translates to C so we can use a C unit testing system for functions For details on what Ubuntu do see Martin Pitt's UDW session, http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html -- Sergey Shnatsel Davidoff OS architect @ elementary -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
Re: [Elementary-dev-community] Testing
Nor am I. But I think the first step would be to explore what can be done with the existing tools (assuming there are any that are still supported). I find that I rarely need much more than a simple facility with the ability to create test cases and do setup/teardown so if we can find anything that provides even that, I think we can go a long way. On Thu, Apr 4, 2013 at 10:45 AM, Goncalo Margalho g...@margalho.info wrote: I would be interested but I'm not the best vala developer for sure. On Thu, Apr 4, 2013 at 4:40 PM, Craig webe...@gmail.com wrote: If we can get a number of experienced test practitioners and Vala developers to commit to it, I wouldn't mind contributing to a test framework development. Would anyone else be interested? On Thu, Apr 4, 2013 at 10:13 AM, Craig webe...@gmail.com wrote: *Would it be feasible to create a Unit Test team on launchpad with the sole purpose of specializing in adding testing to projects and writing the tests required to kill regression bugs before they kill us? - Dane* * * If we do this, I would expect it to be short term only. Developers should be responsible for testing their own code and Test Engineers should be primarily responsible for giving the devs the tools/training they need to test their own work. I propose instead that we only implement tests when we do bug fixes. If a bug crops up, we analyze what caused it and then we write a test to prevent any such bug from appearing (not just that specific bug, but any bug in that class of bugs). For example, I recently worked on a system that takes a config file, parses its key/value pairs into a map, and then exposes its values in the form of methods called string GetValueOfKey1(), string GetValueOfKey2(), etc. These functions simply contained return map.GetValue(key1);. This worked fine as long as the configuration file was setup correctly; however, as soon as someone made a mistake in the config file (accidentally renamed key1 to Key1 or some such) the application crashed because GetValueOfKey1() couldn't find key1 in the map structure. This error *ultimately resulted from* unhandled input, I created a test to test for *all kinds* of bad input and then implemented the input handling. Applying tests where they're useful prevents us from testing stable code. And then moving forward, we can write tests for all new functionality. On Thu, Apr 4, 2013 at 10:00 AM, Craig webe...@gmail.com wrote: *If anyone is interested in starting a Vala Unit Test project under the umbrella of the elementary community, I'm sure we could get quite a bit of traction from the Vala community at large and I would love to help out. - Dane Henson* * * Not only that, but I imagine it would garner quite a lot of professional developer attention for elementary, since professional devs seem dramatically more exposed to (and interested in) formal testing than hobbyist developers. *I apologize for spamming the mailing list. - Dane Henson* Please don't apologize--from the sounds of it, there is quite a bit of consensus that this is a subject that needs to be discussed. And I for one have found each of your messages profitable. *Unit testing is boring to write so if we just said Everybody. Write unit tests. All Projects. Now. it would really take on. On companies and when developers are paid to work, they can write and put tests everywhere, but it's harder for us. - David Gomes* * * David, I used to think that as well; however, once you learn to write tests correctly (I'm starting that process myself) it becomes more exciting. As Dane mentions later in the thread, when you build the bucket around the water (write tests for existing code), it is certainly drudgery; however, when you write tests *before* you implement the functionality, you 1. wind up with better code (because code must be written in neat modules to provide your tests access to the inputs/outputs they need) 2. get the excitement of writing code and watching your tests pass/fail (and as you learn to write better tests, you get detailed information about exactly what caused the failure) 3. can change your code fearlessly, knowing that if anything breaks, you will be notified immediately The whole process becomes at least a little more exciting, especially if you've experienced a huge untested code base and the fear that comes with having to make a change or implement a feature lest the whole thing come tumbling down on you. *While if we (developers) use the first approach, which is called TDD is much better. We write the test first so we define how our app should behave and how our code is structured already (so all the thinking of the code structure you do it in the tests) which is really great. - Goncalo * * * TDD = Test Driven Development (for the uninitiated). I elaborated on this process to some extent in my previous paragraph. We can also do BDD , if there's no
[Elementary-dev-community] Testing new ISOs
Hello everyone, As many of you probably already know, we are now building ISOs again. This is great news, of course, but we need some testers for a couple of things on the new builds. 1. NVIDIA fixes 2. UEFI boot First of all, you can find the new ISOs herehttps://sourceforge.net/projects/elementaryos/files/unstable/and UEFI is now (supposedly) working well. I'm not exactly sure of what happened with NVIDIA but I think we're shipping some different (more recent) drivers that fix many problems. If you have some time for flashing the latest iso and booting it, please reply to this post so that we can get some very needed feedback. Best regards, David Munchor Gomes -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
[Elementary-dev-community] Testing Dynamic Notebook
Hi everyone, so finally I got a pure Gtk implementation of a dynamic notebook pretty much ready. It would be great if you'd try to integrate it in your apps to see if it works properly. If it does, it will go to granite. You can find it at * lp:~tombeckmann/granite/granite-dynamic-notebook-gkt-based*. In case you find any issues or you miss features, please list them at the merge request: https://code.launchpad.net/~tombeckmann/granite/granite-dynamic-notebook-gkt-based/+merge/104050 So, thanks for trying, hopefully we will have an awesome notebook in granite very soon! Tom -- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
