Re: [Elementary-dev-community] Testing Round 2

2013-04-20 Thread Pál Dorogi 
Hi,

I have just subscribed to this list and saw this thread.

> satisfactory (see this:
> http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html);
> however, Vala's development environment is too complicated for me (a Vala

I use similar setup and am using it heavily and it works perfectly. I
reused some libgee-s AbstracTestCase and using cmake to build and run
tests coexist /w the normal app and/or shared libs in the similar
directory structure.
The output likes this:
/ValueHolderTest/null_value_test: OK
/ValueHolderTest/string_value_test: OK
/ValueHolderTest/enum_value_test: OK
...
/AspectTriggerTest/test_trigger_discard: OK
/BindingsTest/bind_aspect_adapter_test: OK

The main test program:

static int main (string[] args) {

GLib.Test.init (ref args);

TestSuite.get_root ().add_suite (new ValueHolderTest ().get_suite ());
 
 TestSuite.get_root ().add_suite (new BindingsTest ().get_suite ());

 return GLib.Test.run ();
}

And a simple test suite /w some unit tests:

public class ModelPresenterTest : AbstractTestCase {
...
public ModelPresenterTest () {
base ("ModelPresenterTest");

add_test ("change_model_under_presenter_test",
change_model_under_presenter_test);
add_test ("base_test", base_test);

}

public override void set_up () {
 value_holder = new ValueHolder ();
}

public override void tear_down () {
value_holder = null;
}

public void change_model_under_presenter_test () {

}

> My "vision" for this experiment is that someone can set up Noise (or some
> other project) for testing, and then I (and/or other volunteers) can start
> writing a few tests for it and sort of coach the developers in TDD for the
> purpose of getting the developers to a self-sufficient state before
> potentially moving onto another project and doing the same.

I think it's a very good idea and I recommend the following post to
all of those who have never heard about nor used TDD:
http://blogs.agilefaqs.com/2011/11/01/importance-of-unit-testing-and-test-driven-development-tdd/

Also the libgee (Vala) guys are using tests heavily either.

-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing Round 2

2013-04-07 Thread xapantu 

Hey,

I do not really have the time to read the whole conversation, but a few 
points:


An unfinished article on the subject (lack of time, etc...): 
http://xapantu.wordpress.com/2013/04/07/writing-tests-using-ldtp-for-elementary/



I have already implented a few tests in the past (unfortunately, due to 
lack of time there is nearly nothing), for instance for scratch:
http://bazaar.launchpad.net/~elementary-apps/scratch/scratch/view/head:/scratchcore/core-tests/main.vala 
 
(and more generally all the scratchcore/ folder)
I think there were a few things for marlin/pantheon files too, but I 
cannot remember wether it has been erased by some crazy merges or not.


You can use accessibility features to do efficient testing, see mago, 
ldtp, at-spi:

https://live.gnome.org/DesktopTesting
http://ldtp.freedesktop.org/ (it looks like it is down at this moment 
but this is the most interesting one).


I recommend you to not sart with Noise, it has a complex UI (code side), 
and it does some weird things when you try to do automation with it (I 
tested, at-spi was not strong enough to work with Noise if I recall 
correctly). Marlin/pantheon-files seems a better target for an 
experiment (the widgets are simplier).


Inline replies below:

On 06/04/2013 19:56, Craig wrote:

Alright guys,

I think it would be a good start to select an application and try to 
start writing tests with it. It seems like GLib's testing environment 
should be satisfactory (see this: 
http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html); 
however, Vala's development environment is too complicated for me (a 
Vala noob) to get a project set up for tests.
See scratch build systems and our custom cmake modules, it should be 
sufficient to do everything you want, and quite easily I think ;) (for 
someone who knows CMake reasonably).
If someone would be willing to volunteer to get GLib's testing set up 
for Noise (or something else if they so desire) including a quick 
example test (so those of us with testing--but not Vala--experience 
can have a starting point), I would be willing to start writing tests 
and working with the project's primary developer(s) to get them 
started toward Test Driven Development (other developers with TDD 
experience also expressed interest).
Unfortunately, starting unit testing means a lot of code reorganization, 
so this is actually the point that someone needs to work on. Writing 
tests once the API is clear is the easiest part -.-


[...]

My "vision" for this experiment is that someone can set up Noise (or 
some other project) for testing, and then I (and/or other volunteers) 
can start writing a few tests for it and sort of coach the developers 
in TDD for the purpose of getting the developers to a self-sufficient 
state before potentially moving onto another project and doing the same.


Thoughts?
In my opinion, writing a few blog posts would be also an interesting 
thing, there are very few good articles about GUI testing for 
Vala/GTK/CMake.


Lucas

--
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
Nor am I. But I think the first step would be to explore what can be done
with the existing tools (assuming there are any that are still supported).
I find that I rarely need much more than a simple facility with the ability
to create test cases and do setup/teardown so if we can find anything that
provides even that, I think we can go a long way.


On Thu, Apr 4, 2013 at 10:45 AM, Goncalo Margalho  wrote:

> I would be interested but I'm not the best vala developer for sure.
>
>
> On Thu, Apr 4, 2013 at 4:40 PM, Craig  wrote:
>
>> If we can get a number of experienced test practitioners and Vala
>> developers to commit to it, I wouldn't mind contributing to a test
>> framework development. Would anyone else be interested?
>>
>>
>> On Thu, Apr 4, 2013 at 10:13 AM, Craig  wrote:
>>
>>> *Would it be feasible to create a Unit Test team on launchpad with the
>>> sole purpose of specializing in adding testing to projects and writing the
>>> tests required to kill regression bugs before they kill us? - Dane*
>>> *
>>> *
>>> If we do this, I would expect it to be short term only. Developers
>>> should be responsible for testing their own code and "Test Engineers"
>>> should be primarily responsible for giving the devs the tools/training they
>>> need to test their own work. I propose instead that we only implement tests
>>> when we do bug fixes. If a bug crops up, we analyze what caused it and then
>>> we write a test to prevent any such bug from appearing (not just that
>>> specific bug, but any bug in that class of bugs).
>>>
>>> For example, I recently worked on a system that takes a config file,
>>> parses its key/value pairs into a map, and then exposes its values in the
>>> form of methods called "string GetValueOfKey1()", "string
>>> GetValueOfKey2()", etc. These functions simply contained "return
>>> map.GetValue("key1");". This worked fine as long as the configuration file
>>> was setup correctly; however, as soon as someone made a mistake in the
>>> config file (accidentally renamed "key1" to "Key1" or some such) the
>>> application crashed because GetValueOfKey1() couldn't find "key1" in the
>>> map structure. This error *ultimately resulted from* unhandled input, I
>>> created a test to test for *all kinds* of bad input and then
>>> implemented the input handling.
>>>
>>> Applying tests where they're useful prevents us from testing stable
>>> code. And then moving forward, we can write tests for all new functionality.
>>>
>>>
>>> On Thu, Apr 4, 2013 at 10:00 AM, Craig  wrote:
>>>
 *If anyone is interested in starting a Vala Unit Test project under
 the umbrella of the elementary community, I'm sure we could get quite a bit
 of traction from the Vala community at large and I would love to help out.
 - Dane Henson*
 *
 *
 Not only that, but I imagine it would garner quite a lot of
 professional developer attention for elementary, since professional devs
 seem dramatically more exposed to (and interested in) formal testing than
 hobbyist developers.

 *I apologize for spamming the mailing list. - Dane Henson*
 Please don't apologize--from the sounds of it, there is quite a bit of
 consensus that this is a subject that needs to be discussed. And I for one
 have found each of your messages profitable.

 *Unit testing is boring to write so if we just said "Everybody. Write
 unit tests. All Projects. Now." it would really take on. On companies and
 when developers are paid to work, they can write and put tests everywhere,
 but it's harder for us. - David Gomes*
 *
 *
 David, I used to think that as well; however, once you learn to write
 tests correctly (I'm starting that process myself) it becomes more
 exciting. As Dane mentions later in the thread, when you build the bucket
 around the water (write tests for existing code), it is certainly drudgery;
 however, when you write tests *before* you implement the
 functionality, you

1. wind up with better code (because code must be written in neat
modules to provide your tests access to the inputs/outputs they need)
2. get the excitement of writing code and watching your tests
pass/fail (and as you learn to write better tests, you get detailed
information about exactly what caused the failure)
3. can change your code fearlessly, knowing that if anything
breaks, you will be notified immediately

 The whole process becomes at least a little more exciting, especially
 if you've experienced a huge untested code base and the fear that comes
 with having to make a change or implement a feature lest the whole thing
 come tumbling down on you.

 *While if we (developers) use the first approach, which is called TDD
 is much better. We write the test first so we define how our app should
 behave and how our code is structured already (so all the thinking of the

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
+1

Sergey, that page is massive. Could you send us the interesting parts?

On Thu, Apr 4, 2013 at 11:59 AM, Goncalo Margalho  wrote:

> Sergey, how do you write code in Vala and write tests in C? it becomes too
> difficult for a developer, don't you think?
>
>
> On Thu, Apr 4, 2013 at 5:18 PM, Sergey "Shnatsel" Davidoff <
> [email protected]> wrote:
>
>> I'm not so sure we need a solution bound to Vala specifically, because:
>>
>>1. We have automated UI testing covered by Ubuntu's regression
>>testing framework
>>2. We have D-bus testing coverted by Ubuntu's regression testing
>>framework
>>3. Vala translates to C so we can use a C unit testing system for
>>functions
>>
>> For details on what Ubuntu do see Martin Pitt's UDW session,
>> http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
>>
>> --
>> Sergey "Shnatsel" Davidoff
>> OS architect @ elementary
>>
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Goncalo Margalho 
Sergey, how do you write code in Vala and write tests in C? it becomes too
difficult for a developer, don't you think?


On Thu, Apr 4, 2013 at 5:18 PM, Sergey "Shnatsel" Davidoff <
[email protected]> wrote:

> I'm not so sure we need a solution bound to Vala specifically, because:
>
>1. We have automated UI testing covered by Ubuntu's regression testing
>framework
>2. We have D-bus testing coverted by Ubuntu's regression testing
>framework
>3. Vala translates to C so we can use a C unit testing system for
>functions
>
> For details on what Ubuntu do see Martin Pitt's UDW session,
> http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
>
> --
> Sergey "Shnatsel" Davidoff
> OS architect @ elementary
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Sergey "Shnatsel" Davidoff 
I'm not so sure we need a solution bound to Vala specifically, because:

   1. We have automated UI testing covered by Ubuntu's regression testing
   framework
   2. We have D-bus testing coverted by Ubuntu's regression testing
   framework
   3. Vala translates to C so we can use a C unit testing system for
   functions

For details on what Ubuntu do see Martin Pitt's UDW session,
http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html

-- 
Sergey "Shnatsel" Davidoff
OS architect @ elementary
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Goncalo Margalho 
I would be interested but I'm not the best vala developer for sure.


On Thu, Apr 4, 2013 at 4:40 PM, Craig  wrote:

> If we can get a number of experienced test practitioners and Vala
> developers to commit to it, I wouldn't mind contributing to a test
> framework development. Would anyone else be interested?
>
>
> On Thu, Apr 4, 2013 at 10:13 AM, Craig  wrote:
>
>> *Would it be feasible to create a Unit Test team on launchpad with the
>> sole purpose of specializing in adding testing to projects and writing the
>> tests required to kill regression bugs before they kill us? - Dane*
>> *
>> *
>> If we do this, I would expect it to be short term only. Developers should
>> be responsible for testing their own code and "Test Engineers" should be
>> primarily responsible for giving the devs the tools/training they need to
>> test their own work. I propose instead that we only implement tests when we
>> do bug fixes. If a bug crops up, we analyze what caused it and then we
>> write a test to prevent any such bug from appearing (not just that specific
>> bug, but any bug in that class of bugs).
>>
>> For example, I recently worked on a system that takes a config file,
>> parses its key/value pairs into a map, and then exposes its values in the
>> form of methods called "string GetValueOfKey1()", "string
>> GetValueOfKey2()", etc. These functions simply contained "return
>> map.GetValue("key1");". This worked fine as long as the configuration file
>> was setup correctly; however, as soon as someone made a mistake in the
>> config file (accidentally renamed "key1" to "Key1" or some such) the
>> application crashed because GetValueOfKey1() couldn't find "key1" in the
>> map structure. This error *ultimately resulted from* unhandled input, I
>> created a test to test for *all kinds* of bad input and then implemented
>> the input handling.
>>
>> Applying tests where they're useful prevents us from testing stable code.
>> And then moving forward, we can write tests for all new functionality.
>>
>>
>> On Thu, Apr 4, 2013 at 10:00 AM, Craig  wrote:
>>
>>> *If anyone is interested in starting a Vala Unit Test project under the
>>> umbrella of the elementary community, I'm sure we could get quite a bit of
>>> traction from the Vala community at large and I would love to help out. -
>>> Dane Henson*
>>> *
>>> *
>>> Not only that, but I imagine it would garner quite a lot of professional
>>> developer attention for elementary, since professional devs seem
>>> dramatically more exposed to (and interested in) formal testing than
>>> hobbyist developers.
>>>
>>> *I apologize for spamming the mailing list. - Dane Henson*
>>> Please don't apologize--from the sounds of it, there is quite a bit of
>>> consensus that this is a subject that needs to be discussed. And I for one
>>> have found each of your messages profitable.
>>>
>>> *Unit testing is boring to write so if we just said "Everybody. Write
>>> unit tests. All Projects. Now." it would really take on. On companies and
>>> when developers are paid to work, they can write and put tests everywhere,
>>> but it's harder for us. - David Gomes*
>>> *
>>> *
>>> David, I used to think that as well; however, once you learn to write
>>> tests correctly (I'm starting that process myself) it becomes more
>>> exciting. As Dane mentions later in the thread, when you build the bucket
>>> around the water (write tests for existing code), it is certainly drudgery;
>>> however, when you write tests *before* you implement the functionality,
>>> you
>>>
>>>1. wind up with better code (because code must be written in neat
>>>modules to provide your tests access to the inputs/outputs they need)
>>>2. get the excitement of writing code and watching your tests
>>>pass/fail (and as you learn to write better tests, you get detailed
>>>information about exactly what caused the failure)
>>>3. can change your code fearlessly, knowing that if anything breaks,
>>>you will be notified immediately
>>>
>>> The whole process becomes at least a little more exciting, especially if
>>> you've experienced a huge untested code base and the fear that comes with
>>> having to make a change or implement a feature lest the whole thing come
>>> tumbling down on you.
>>>
>>> *While if we (developers) use the first approach, which is called TDD
>>> is much better. We write the test first so we define how our app should
>>> behave and how our code is structured already (so all the thinking of the
>>> code structure you do it in the tests)  which is really great. - Goncalo
>>> *
>>> *
>>> *
>>> TDD = Test Driven Development (for the uninitiated). I elaborated on
>>> this process to some extent in my previous paragraph.
>>>
>>> We can also do BDD , if there's no framework for this we can probably
>>> create something, for more infos you can check cucumber for Ruby. *Bdd
>>> let's you write the tests in PLAIN english, so who does the mockups could
>>> write the tests as well. that would be great.

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
If we can get a number of experienced test practitioners and Vala
developers to commit to it, I wouldn't mind contributing to a test
framework development. Would anyone else be interested?


On Thu, Apr 4, 2013 at 10:13 AM, Craig  wrote:

> *Would it be feasible to create a Unit Test team on launchpad with the
> sole purpose of specializing in adding testing to projects and writing the
> tests required to kill regression bugs before they kill us? - Dane*
> *
> *
> If we do this, I would expect it to be short term only. Developers should
> be responsible for testing their own code and "Test Engineers" should be
> primarily responsible for giving the devs the tools/training they need to
> test their own work. I propose instead that we only implement tests when we
> do bug fixes. If a bug crops up, we analyze what caused it and then we
> write a test to prevent any such bug from appearing (not just that specific
> bug, but any bug in that class of bugs).
>
> For example, I recently worked on a system that takes a config file,
> parses its key/value pairs into a map, and then exposes its values in the
> form of methods called "string GetValueOfKey1()", "string
> GetValueOfKey2()", etc. These functions simply contained "return
> map.GetValue("key1");". This worked fine as long as the configuration file
> was setup correctly; however, as soon as someone made a mistake in the
> config file (accidentally renamed "key1" to "Key1" or some such) the
> application crashed because GetValueOfKey1() couldn't find "key1" in the
> map structure. This error *ultimately resulted from* unhandled input, I
> created a test to test for *all kinds* of bad input and then implemented
> the input handling.
>
> Applying tests where they're useful prevents us from testing stable code.
> And then moving forward, we can write tests for all new functionality.
>
>
> On Thu, Apr 4, 2013 at 10:00 AM, Craig  wrote:
>
>> *If anyone is interested in starting a Vala Unit Test project under the
>> umbrella of the elementary community, I'm sure we could get quite a bit of
>> traction from the Vala community at large and I would love to help out. -
>> Dane Henson*
>> *
>> *
>> Not only that, but I imagine it would garner quite a lot of professional
>> developer attention for elementary, since professional devs seem
>> dramatically more exposed to (and interested in) formal testing than
>> hobbyist developers.
>>
>> *I apologize for spamming the mailing list. - Dane Henson*
>> Please don't apologize--from the sounds of it, there is quite a bit of
>> consensus that this is a subject that needs to be discussed. And I for one
>> have found each of your messages profitable.
>>
>> *Unit testing is boring to write so if we just said "Everybody. Write
>> unit tests. All Projects. Now." it would really take on. On companies and
>> when developers are paid to work, they can write and put tests everywhere,
>> but it's harder for us. - David Gomes*
>> *
>> *
>> David, I used to think that as well; however, once you learn to write
>> tests correctly (I'm starting that process myself) it becomes more
>> exciting. As Dane mentions later in the thread, when you build the bucket
>> around the water (write tests for existing code), it is certainly drudgery;
>> however, when you write tests *before* you implement the functionality,
>> you
>>
>>1. wind up with better code (because code must be written in neat
>>modules to provide your tests access to the inputs/outputs they need)
>>2. get the excitement of writing code and watching your tests
>>pass/fail (and as you learn to write better tests, you get detailed
>>information about exactly what caused the failure)
>>3. can change your code fearlessly, knowing that if anything breaks,
>>you will be notified immediately
>>
>> The whole process becomes at least a little more exciting, especially if
>> you've experienced a huge untested code base and the fear that comes with
>> having to make a change or implement a feature lest the whole thing come
>> tumbling down on you.
>>
>> *While if we (developers) use the first approach, which is called TDD is
>> much better. We write the test first so we define how our app should behave
>> and how our code is structured already (so all the thinking of the code
>> structure you do it in the tests)  which is really great. - Goncalo*
>> *
>> *
>> TDD = Test Driven Development (for the uninitiated). I elaborated on this
>> process to some extent in my previous paragraph.
>>
>> We can also do BDD , if there's no framework for this we can probably
>> create something, for more infos you can check cucumber for Ruby. *Bdd
>> let's you write the tests in PLAIN english, so who does the mockups could
>> write the tests as well. that would be great. - Goncalo*
>> *
>> *
>> BDD = Behavior Driven Development. It's either the same thing or very
>> closely related to ATDD (Acceptance Test Driven Development). In either
>> case, the general idea is that you specify wha

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
*Would it be feasible to create a Unit Test team on launchpad with the sole
purpose of specializing in adding testing to projects and writing the tests
required to kill regression bugs before they kill us? - Dane*
*
*
If we do this, I would expect it to be short term only. Developers should
be responsible for testing their own code and "Test Engineers" should be
primarily responsible for giving the devs the tools/training they need to
test their own work. I propose instead that we only implement tests when we
do bug fixes. If a bug crops up, we analyze what caused it and then we
write a test to prevent any such bug from appearing (not just that specific
bug, but any bug in that class of bugs).

For example, I recently worked on a system that takes a config file, parses
its key/value pairs into a map, and then exposes its values in the form of
methods called "string GetValueOfKey1()", "string GetValueOfKey2()", etc.
These functions simply contained "return map.GetValue("key1");". This
worked fine as long as the configuration file was setup correctly; however,
as soon as someone made a mistake in the config file (accidentally renamed
"key1" to "Key1" or some such) the application crashed because
GetValueOfKey1() couldn't find "key1" in the map structure. This error
*ultimately
resulted from* unhandled input, I created a test to test for *all kinds* of
bad input and then implemented the input handling.

Applying tests where they're useful prevents us from testing stable code.
And then moving forward, we can write tests for all new functionality.


On Thu, Apr 4, 2013 at 10:00 AM, Craig  wrote:

> *If anyone is interested in starting a Vala Unit Test project under the
> umbrella of the elementary community, I'm sure we could get quite a bit of
> traction from the Vala community at large and I would love to help out. -
> Dane Henson*
> *
> *
> Not only that, but I imagine it would garner quite a lot of professional
> developer attention for elementary, since professional devs seem
> dramatically more exposed to (and interested in) formal testing than
> hobbyist developers.
>
> *I apologize for spamming the mailing list. - Dane Henson*
> Please don't apologize--from the sounds of it, there is quite a bit of
> consensus that this is a subject that needs to be discussed. And I for one
> have found each of your messages profitable.
>
> *Unit testing is boring to write so if we just said "Everybody. Write
> unit tests. All Projects. Now." it would really take on. On companies and
> when developers are paid to work, they can write and put tests everywhere,
> but it's harder for us. - David Gomes*
> *
> *
> David, I used to think that as well; however, once you learn to write
> tests correctly (I'm starting that process myself) it becomes more
> exciting. As Dane mentions later in the thread, when you build the bucket
> around the water (write tests for existing code), it is certainly drudgery;
> however, when you write tests *before* you implement the functionality,
> you
>
>1. wind up with better code (because code must be written in neat
>modules to provide your tests access to the inputs/outputs they need)
>2. get the excitement of writing code and watching your tests
>pass/fail (and as you learn to write better tests, you get detailed
>information about exactly what caused the failure)
>3. can change your code fearlessly, knowing that if anything breaks,
>you will be notified immediately
>
> The whole process becomes at least a little more exciting, especially if
> you've experienced a huge untested code base and the fear that comes with
> having to make a change or implement a feature lest the whole thing come
> tumbling down on you.
>
> *While if we (developers) use the first approach, which is called TDD is
> much better. We write the test first so we define how our app should behave
> and how our code is structured already (so all the thinking of the code
> structure you do it in the tests)  which is really great. - Goncalo*
> *
> *
> TDD = Test Driven Development (for the uninitiated). I elaborated on this
> process to some extent in my previous paragraph.
>
> We can also do BDD , if there's no framework for this we can probably
> create something, for more infos you can check cucumber for Ruby. *Bdd
> let's you write the tests in PLAIN english, so who does the mockups could
> write the tests as well. that would be great. - Goncalo*
> *
> *
> BDD = Behavior Driven Development. It's either the same thing or very
> closely related to ATDD (Acceptance Test Driven Development). In either
> case, the general idea is that you specify what the entire system *must do
> * (system requirements/acceptance criteria) and then you write tests to
> verify each requirement. This sort of test might look like:
>
> "WHEN the bold button is pressed
>  AND the string 'abcd' is typed,
>  EXPECT the text view to contain the string 'abcd' in bold"
>
> Then, for each line, you write a function that implem

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Goncalo Margalho 
Craig, thanks to make everything more clear.
Unit test is not just having something to compare stuff but we need a great
mocking system. Does anyone have links for that as well?


On Thu, Apr 4, 2013 at 4:00 PM, Craig  wrote:

> *If anyone is interested in starting a Vala Unit Test project under the
> umbrella of the elementary community, I'm sure we could get quite a bit of
> traction from the Vala community at large and I would love to help out. -
> Dane Henson*
> *
> *
> Not only that, but I imagine it would garner quite a lot of professional
> developer attention for elementary, since professional devs seem
> dramatically more exposed to (and interested in) formal testing than
> hobbyist developers.
>
> *I apologize for spamming the mailing list. - Dane Henson*
> Please don't apologize--from the sounds of it, there is quite a bit of
> consensus that this is a subject that needs to be discussed. And I for one
> have found each of your messages profitable.
>
> *Unit testing is boring to write so if we just said "Everybody. Write
> unit tests. All Projects. Now." it would really take on. On companies and
> when developers are paid to work, they can write and put tests everywhere,
> but it's harder for us. - David Gomes*
> *
> *
> David, I used to think that as well; however, once you learn to write
> tests correctly (I'm starting that process myself) it becomes more
> exciting. As Dane mentions later in the thread, when you build the bucket
> around the water (write tests for existing code), it is certainly drudgery;
> however, when you write tests *before* you implement the functionality,
> you
>
>1. wind up with better code (because code must be written in neat
>modules to provide your tests access to the inputs/outputs they need)
>2. get the excitement of writing code and watching your tests
>pass/fail (and as you learn to write better tests, you get detailed
>information about exactly what caused the failure)
>3. can change your code fearlessly, knowing that if anything breaks,
>you will be notified immediately
>
> The whole process becomes at least a little more exciting, especially if
> you've experienced a huge untested code base and the fear that comes with
> having to make a change or implement a feature lest the whole thing come
> tumbling down on you.
>
> *While if we (developers) use the first approach, which is called TDD is
> much better. We write the test first so we define how our app should behave
> and how our code is structured already (so all the thinking of the code
> structure you do it in the tests)  which is really great. - Goncalo*
> *
> *
> TDD = Test Driven Development (for the uninitiated). I elaborated on this
> process to some extent in my previous paragraph.
>
> We can also do BDD , if there's no framework for this we can probably
> create something, for more infos you can check cucumber for Ruby. *Bdd
> let's you write the tests in PLAIN english, so who does the mockups could
> write the tests as well. that would be great. - Goncalo*
> *
> *
> BDD = Behavior Driven Development. It's either the same thing or very
> closely related to ATDD (Acceptance Test Driven Development). In either
> case, the general idea is that you specify what the entire system *must do
> * (system requirements/acceptance criteria) and then you write tests to
> verify each requirement. This sort of test might look like:
>
> "WHEN the bold button is pressed
>  AND the string 'abcd' is typed,
>  EXPECT the text view to contain the string 'abcd' in bold"
>
> Then, for each line, you write a function that implements either the setup
> (the WHEN/AND portion) or the evaluation (the EXPECT portion) and then the
> framework puts the pieces together and executes the test. The code for the
> test (in pseudocode) might look like this:
> func pressBoldButton () {
>theApp.boldButton.setPressed (true)
> }
> func type(string input) {
>theApp.InsertAtCursor (input)
> }
> func verifyContents () {
>start := 0
>end := 3
>contents := theApp.GetFormattedSubStr (start, end)
>contentsAreBold := contents.isBold()
>letters := contents.getPlainTextString()
>
>Testing.Expect (contentsAreBold);
>Testing.Expect (letters == "abcd")
> }
>
> On Thu, Apr 4, 2013 at 9:11 AM, Dane Henson  wrote:
>
>> Here is another practical post I found interesting regarding setting up
>> Unit Tests in Vala with Cmake:
>>
>>
>> http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html
>>
>> I apologize for spamming the mailing list.
>>
>>
>> On Thu, Apr 4, 2013 at 8:56 AM, Sergey "Shnatsel" Davidoff <
>> [email protected]> wrote:
>>
>>> I strongly recommend anyone interested in automated testing to read
>>> through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
>>> responsible for most of unit testing in Ubuntu (he's also the author of
>>> Apport which we already use). His IRC nick is "pitti" and the session logs
>>> can be found at
>>> http:/

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
*If anyone is interested in starting a Vala Unit Test project under the
umbrella of the elementary community, I'm sure we could get quite a bit of
traction from the Vala community at large and I would love to help out. -
Dane Henson*
*
*
Not only that, but I imagine it would garner quite a lot of professional
developer attention for elementary, since professional devs seem
dramatically more exposed to (and interested in) formal testing than
hobbyist developers.

*I apologize for spamming the mailing list. - Dane Henson*
Please don't apologize--from the sounds of it, there is quite a bit of
consensus that this is a subject that needs to be discussed. And I for one
have found each of your messages profitable.

*Unit testing is boring to write so if we just said "Everybody. Write unit
tests. All Projects. Now." it would really take on. On companies and when
developers are paid to work, they can write and put tests everywhere, but
it's harder for us. - David Gomes*
*
*
David, I used to think that as well; however, once you learn to write tests
correctly (I'm starting that process myself) it becomes more exciting. As
Dane mentions later in the thread, when you build the bucket around the
water (write tests for existing code), it is certainly drudgery; however,
when you write tests *before* you implement the functionality, you

   1. wind up with better code (because code must be written in neat
   modules to provide your tests access to the inputs/outputs they need)
   2. get the excitement of writing code and watching your tests pass/fail
   (and as you learn to write better tests, you get detailed information about
   exactly what caused the failure)
   3. can change your code fearlessly, knowing that if anything breaks, you
   will be notified immediately

The whole process becomes at least a little more exciting, especially if
you've experienced a huge untested code base and the fear that comes with
having to make a change or implement a feature lest the whole thing come
tumbling down on you.

*While if we (developers) use the first approach, which is called TDD is
much better. We write the test first so we define how our app should behave
and how our code is structured already (so all the thinking of the code
structure you do it in the tests)  which is really great. - Goncalo*
*
*
TDD = Test Driven Development (for the uninitiated). I elaborated on this
process to some extent in my previous paragraph.

We can also do BDD , if there's no framework for this we can probably
create something, for more infos you can check cucumber for Ruby. *Bdd
let's you write the tests in PLAIN english, so who does the mockups could
write the tests as well. that would be great. - Goncalo*
*
*
BDD = Behavior Driven Development. It's either the same thing or very
closely related to ATDD (Acceptance Test Driven Development). In either
case, the general idea is that you specify what the entire system
*must do* (system
requirements/acceptance criteria) and then you write tests to verify each
requirement. This sort of test might look like:

"WHEN the bold button is pressed
 AND the string 'abcd' is typed,
 EXPECT the text view to contain the string 'abcd' in bold"

Then, for each line, you write a function that implements either the setup
(the WHEN/AND portion) or the evaluation (the EXPECT portion) and then the
framework puts the pieces together and executes the test. The code for the
test (in pseudocode) might look like this:
func pressBoldButton () {
   theApp.boldButton.setPressed (true)
}
func type(string input) {
   theApp.InsertAtCursor (input)
}
func verifyContents () {
   start := 0
   end := 3
   contents := theApp.GetFormattedSubStr (start, end)
   contentsAreBold := contents.isBold()
   letters := contents.getPlainTextString()

   Testing.Expect (contentsAreBold);
   Testing.Expect (letters == "abcd")
}

On Thu, Apr 4, 2013 at 9:11 AM, Dane Henson  wrote:

> Here is another practical post I found interesting regarding setting up
> Unit Tests in Vala with Cmake:
>
>
> http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html
>
> I apologize for spamming the mailing list.
>
>
> On Thu, Apr 4, 2013 at 8:56 AM, Sergey "Shnatsel" Davidoff <
> [email protected]> wrote:
>
>> I strongly recommend anyone interested in automated testing to read
>> through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
>> responsible for most of unit testing in Ubuntu (he's also the author of
>> Apport which we already use). His IRC nick is "pitti" and the session logs
>> can be found at
>> http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
>>
>>
>> --
>> Sergey "Shnatsel" Davidoff
>> OS architect @ elementary
>>
>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launc

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Jaap Broekhuizen 
We do unit and behavorial testing at where i work, and i think it's great.
Writing tests before you develop the actual feature really makes you think
why your feature should do what and when it should do so.

Maybe we could build vala tests for granite. That would mean we would have
to write tests for existing features too, but when that is done, we can
implement features with first creating a test (a spec) for it, and then
actually implementing it according to the spec. We could then integrate
some kind of continuous integration using something like Travis (if it does
vala), or if launchpad offers that functionality, we can use that.

If it works for granite, we could slowly and steadily move to testing in
all elementary projects.

I think this would in the long term greatly improve stability of the system
and individual apps, will encourage good and maintainable code and
architecture of apps, and it would make elementary developing more
professional.

Just my 2 cents...

Jaap
Op 4 apr. 2013 16:11 schreef "Dane Henson"  het
volgende:

> Here is another practical post I found interesting regarding setting up
> Unit Tests in Vala with Cmake:
>
>
> http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html
>
> I apologize for spamming the mailing list.
>
>
> On Thu, Apr 4, 2013 at 8:56 AM, Sergey "Shnatsel" Davidoff <
> [email protected]> wrote:
>
>> I strongly recommend anyone interested in automated testing to read
>> through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
>> responsible for most of unit testing in Ubuntu (he's also the author of
>> Apport which we already use). His IRC nick is "pitti" and the session logs
>> can be found at
>> http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
>>
>>
>> --
>> Sergey "Shnatsel" Davidoff
>> OS architect @ elementary
>>
>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Dane Henson 
Here is another practical post I found interesting regarding setting up
Unit Tests in Vala with Cmake:

http://blog.remysaissy.com/2012/11/setting-up-unit-tests-in-vala-project.html

I apologize for spamming the mailing list.


On Thu, Apr 4, 2013 at 8:56 AM, Sergey "Shnatsel" Davidoff <
[email protected]> wrote:

> I strongly recommend anyone interested in automated testing to read
> through Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
> responsible for most of unit testing in Ubuntu (he's also the author of
> Apport which we already use). His IRC nick is "pitti" and the session logs
> can be found at
> http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html
>
>
> --
> Sergey "Shnatsel" Davidoff
> OS architect @ elementary
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Sergey "Shnatsel" Davidoff 
I strongly recommend anyone interested in automated testing to read through
Martin Pitt's Ubuntu Dev Week session on the topic. He's the one
responsible for most of unit testing in Ubuntu (he's also the author of
Apport which we already use). His IRC nick is "pitti" and the session logs
can be found at
http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html

-- 
Sergey "Shnatsel" Davidoff
OS architect @ elementary
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Goncalo Margalho 
I love this topic.
David Gomes, you said that writing tests is boring, probably you are right
but that's because you tried to write them in the "second approach"
explained earlier by Dane Henson.

The problem with that is that the code that you wrote before MAYBE is not
testable, which makes your life horrible and boring.

While if we (developers) use the first approach, which is called TDD is
much better. We write the test first so we define how our app should behave
and how our code is structured already (so all the thinking of the code
structure you do it in the tests)  which is really great.

We can also do BDD , if there's no framework for this we can probably
create something, for more infos you can check cucumber for Ruby.
Bdd let's you write the tests in PLAIN english, so who does the mockups
could write the tests as well. that would be great.

I used to hate testing but Since I started doing a bit of TDD my code looks
much better.

I will try the tools suggested above and see if I can use something.

Goncalo


On Thu, Apr 4, 2013 at 2:38 PM, Dane Henson  wrote:

> David,
> I understand where you're coming from and I have worried about that in the
> past.  There are two approaches for Unit Testing.
>
> The strong approach is to figure out what you want the app to do (i.e.
> design the framework), describe that app's functions and libraries using
> tests, then write your code around it.  If at some point the requirements
> change, you can modify the Unit Tests.  It's boring, but provides a very
> stable foundation for apps.
>
> The second approach (and what we would have to do for all existing
> projects) is to write the tests around the app.  This approach is like
> trying to build a bucket around the water, but it can be done and any
> development going forward is more likely to not break the core
> functionality of the app.
>
> Would it be feasible to create a Unit Test team on launchpad with the sole
> purpose of specializing in adding testing to projects and writing the tests
> required to kill regression bugs before they kill us?
>
> Also, Adam (at Yorba) and I came up with the conclusion that rolling a new
> Unit Test framework might be the best option.  Here's a quote from Adam at
> Yorba: "You might also want to consider rolling your own simple unit
> testing framework in Vala without even using Glib.Test - sometimes I think
> that might be the easiest path forward here."
>
> So, this could be a really big project, but I see a lot of merit to it.
>
>
> On Thu, Apr 4, 2013 at 8:25 AM, David Gomes wrote:
>
>> Unit testing is boring to write so if we just said "Everybody. Write unit
>> tests. All Projects. Now." it would really take on. On companies and when
>> developers are paid to work, they can write and put tests everywhere, but
>> it's harder for us.
>>
>> For now what we just do is we test the interface, the new features and
>> some old features too to see if they remain unaffected.
>>
>>
>> On Thu, Apr 4, 2013 at 1:51 PM, Craig  wrote:
>>
>>> Hello everyone,
>>>
>>> I'm curious what you devs do for testing? I'm not particularly familiar
>>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>>> develop myself to that end in my free time. I'm sending this email because
>>> I'd like to get a pulse on what you Elementary devs think about testing and
>>> what you actually do to test your code. Also, please feel encouraged to
>>> talk about what you've done in the past, what has/hasn't worked for you,
>>> and generally what your philosophy is about testing (or whether you have no
>>> philosophy). Individual comments and comments on behalf of elementary as a
>>> whole are both welcome.
>>>
>>> Sound off!
>>>
>>> Thanks,
>>> Craig
>>>
>>> --
>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>> Post to : [email protected]
>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Sergey "Shnatsel" Davidoff 
2013/4/4 Craig 

> Sergey, that looks very interesting, although I'm a little confused--what
> are the differences between your first link and Glimpse?
>

My first link is my scripts for easily testing code proposed for merging. I
created them to offload the burden of testing proposed code from developers
to adventurous users. However, nobody understand what are those scripts and
why they're needed unless I explain it personally on IRC .I guess I suck at
emailing people, but I always knew that...

-- 
Sergey "Shnatsel" Davidoff
OS architect @ elementary
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Dane Henson 
David,
I understand where you're coming from and I have worried about that in the
past.  There are two approaches for Unit Testing.

The strong approach is to figure out what you want the app to do (i.e.
design the framework), describe that app's functions and libraries using
tests, then write your code around it.  If at some point the requirements
change, you can modify the Unit Tests.  It's boring, but provides a very
stable foundation for apps.

The second approach (and what we would have to do for all existing
projects) is to write the tests around the app.  This approach is like
trying to build a bucket around the water, but it can be done and any
development going forward is more likely to not break the core
functionality of the app.

Would it be feasible to create a Unit Test team on launchpad with the sole
purpose of specializing in adding testing to projects and writing the tests
required to kill regression bugs before they kill us?

Also, Adam (at Yorba) and I came up with the conclusion that rolling a new
Unit Test framework might be the best option.  Here's a quote from Adam at
Yorba: "You might also want to consider rolling your own simple unit
testing framework in Vala without even using Glib.Test - sometimes I think
that might be the easiest path forward here."

So, this could be a really big project, but I see a lot of merit to it.


On Thu, Apr 4, 2013 at 8:25 AM, David Gomes  wrote:

> Unit testing is boring to write so if we just said "Everybody. Write unit
> tests. All Projects. Now." it would really take on. On companies and when
> developers are paid to work, they can write and put tests everywhere, but
> it's harder for us.
>
> For now what we just do is we test the interface, the new features and
> some old features too to see if they remain unaffected.
>
>
> On Thu, Apr 4, 2013 at 1:51 PM, Craig  wrote:
>
>> Hello everyone,
>>
>> I'm curious what you devs do for testing? I'm not particularly familiar
>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>> develop myself to that end in my free time. I'm sending this email because
>> I'd like to get a pulse on what you Elementary devs think about testing and
>> what you actually do to test your code. Also, please feel encouraged to
>> talk about what you've done in the past, what has/hasn't worked for you,
>> and generally what your philosophy is about testing (or whether you have no
>> philosophy). Individual comments and comments on behalf of elementary as a
>> whole are both welcome.
>>
>> Sound off!
>>
>> Thanks,
>> Craig
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread David Gomes 
We also had a regression on Granite the other day. It goes "I fixed this
bug" and I tested that bug but then I didn't test crazy-weird behaviour and
it turned out it broke some things. Of course, action has been taken to fix
it, but it can't be merged yet.

(sorry for sending this email to Cody twice, I meant to Reply To All.


On Thu, Apr 4, 2013 at 2:31 PM, Cody Garver  wrote:

> Our  m̶o̶s̶t̶ ̶e̶f̶f̶e̶c̶t̶i̶v̶e̶ only regression testing is in Files. It
> goes "Cody, I think I fixed crash X" and I add it to my list of crash
> report email titles I look for to reoccur.
>
>
> On Thu, Apr 4, 2013 at 8:25 AM, David Gomes wrote:
>
>> Unit testing is boring to write so if we just said "Everybody. Write unit
>> tests. All Projects. Now." it would really take on. On companies and when
>> developers are paid to work, they can write and put tests everywhere, but
>> it's harder for us.
>>
>> For now what we just do is we test the interface, the new features and
>> some old features too to see if they remain unaffected.
>>
>>
>> On Thu, Apr 4, 2013 at 1:51 PM, Craig  wrote:
>>
>>> Hello everyone,
>>>
>>> I'm curious what you devs do for testing? I'm not particularly familiar
>>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>>> develop myself to that end in my free time. I'm sending this email because
>>> I'd like to get a pulse on what you Elementary devs think about testing and
>>> what you actually do to test your code. Also, please feel encouraged to
>>> talk about what you've done in the past, what has/hasn't worked for you,
>>> and generally what your philosophy is about testing (or whether you have no
>>> philosophy). Individual comments and comments on behalf of elementary as a
>>> whole are both welcome.
>>>
>>> Sound off!
>>>
>>> Thanks,
>>> Craig
>>>
>>> --
>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>> Post to : [email protected]
>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Cody Garver
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Cody Garver 
Our  m̶o̶s̶t̶ ̶e̶f̶f̶e̶c̶t̶i̶v̶e̶ only regression testing is in Files. It
goes "Cody, I think I fixed crash X" and I add it to my list of crash
report email titles I look for to reoccur.


On Thu, Apr 4, 2013 at 8:25 AM, David Gomes  wrote:

> Unit testing is boring to write so if we just said "Everybody. Write unit
> tests. All Projects. Now." it would really take on. On companies and when
> developers are paid to work, they can write and put tests everywhere, but
> it's harder for us.
>
> For now what we just do is we test the interface, the new features and
> some old features too to see if they remain unaffected.
>
>
> On Thu, Apr 4, 2013 at 1:51 PM, Craig  wrote:
>
>> Hello everyone,
>>
>> I'm curious what you devs do for testing? I'm not particularly familiar
>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>> develop myself to that end in my free time. I'm sending this email because
>> I'd like to get a pulse on what you Elementary devs think about testing and
>> what you actually do to test your code. Also, please feel encouraged to
>> talk about what you've done in the past, what has/hasn't worked for you,
>> and generally what your philosophy is about testing (or whether you have no
>> philosophy). Individual comments and comments on behalf of elementary as a
>> whole are both welcome.
>>
>> Sound off!
>>
>> Thanks,
>> Craig
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Cody Garver
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread David Gomes 
Unit testing is boring to write so if we just said "Everybody. Write unit
tests. All Projects. Now." it would really take on. On companies and when
developers are paid to work, they can write and put tests everywhere, but
it's harder for us.

For now what we just do is we test the interface, the new features and some
old features too to see if they remain unaffected.


On Thu, Apr 4, 2013 at 1:51 PM, Craig  wrote:

> Hello everyone,
>
> I'm curious what you devs do for testing? I'm not particularly familiar
> with Vala, but I'm learning a lot about testing at work and I'm trying to
> develop myself to that end in my free time. I'm sending this email because
> I'd like to get a pulse on what you Elementary devs think about testing and
> what you actually do to test your code. Also, please feel encouraged to
> talk about what you've done in the past, what has/hasn't worked for you,
> and generally what your philosophy is about testing (or whether you have no
> philosophy). Individual comments and comments on behalf of elementary as a
> whole are both welcome.
>
> Sound off!
>
> Thanks,
> Craig
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Dane Henson 
Craig, I have seen that and it looks promising.  That post also mentions
Yorba and valadate, but the Yorba devs have told me they have no intention
of continuing development on it.


On Thu, Apr 4, 2013 at 8:15 AM, Dane Henson  wrote:

> Yes, testing is sorely needed.
>
> I started down the path of finding the best solution for Vala Unit
> Testing, but couldn't find any great options.  Currently, all projects that
> pursued the xUnit (https://en.wikipedia.org/wiki/XUnit) model for Vala
> have fallen out of development and only their shells remain.  I reached out
> to the Yorba devs, who appear to hold a repository for the old 'valadate'
> project but they are in the same position we are in.  I believe they have
> done some testing with the built-in GObject unit test suite (
> https://live.gnome.org/Vala/TestSample), but it has been found to be
> severely lacking by anyone who knows anything about unit testing.
>
> As with most OSS developers, I started to scratch that itch and realized
> that it is a little bit beyond my skill set.  The code I started to write
> remains unusable, rotting away somewhere on one of my hdd platters in a
> folder called 'vunit'.  If anyone is interested in starting a Vala Unit
> Test project under the umbrella of the elementary community, I'm sure we
> could get quite a bit of traction from the Vala community at large and I
> would love to help out.
>
>
> On Thu, Apr 4, 2013 at 7:59 AM, Sergey "Shnatsel" Davidoff <
> [email protected]> wrote:
>
>> This is what I've developed for testing:
>> https://lists.launchpad.net/elementary-dev-community/msg01948.html
>>
>> I've also created Glimpse - a system for sandboxed testing, but it lacks
>> a D-bus proxy which is needed for adding/removing apps via GUI. See
>> http://shnatsel.blogspot.com/2011/07/weve-just-revolutionized-alpha-testing.htmland
>> http://shnatsel.blogspot.com/2012/05/state-of-glimpse.html
>>
>> I don't think we have any unit testing in place yet. Midori probably has
>> but that's it.
>>
>>
>> 2013/4/4 Craig 
>>
>>> Hello everyone,
>>>
>>> I'm curious what you devs do for testing? I'm not particularly familiar
>>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>>> develop myself to that end in my free time. I'm sending this email because
>>> I'd like to get a pulse on what you Elementary devs think about testing and
>>> what you actually do to test your code. Also, please feel encouraged to
>>> talk about what you've done in the past, what has/hasn't worked for you,
>>> and generally what your philosophy is about testing (or whether you have no
>>> philosophy). Individual comments and comments on behalf of elementary as a
>>> whole are both welcome.
>>>
>>> Sound off!
>>>
>>> Thanks,
>>> Craig
>>>
>>> --
>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>> Post to : [email protected]
>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Sergey "Shnatsel" Davidoff
>> OS architect @ elementary
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Dane Henson 
Yes, testing is sorely needed.

I started down the path of finding the best solution for Vala Unit Testing,
but couldn't find any great options.  Currently, all projects that pursued
the xUnit (https://en.wikipedia.org/wiki/XUnit) model for Vala have fallen
out of development and only their shells remain.  I reached out to the
Yorba devs, who appear to hold a repository for the old 'valadate' project
but they are in the same position we are in.  I believe they have done some
testing with the built-in GObject unit test suite (
https://live.gnome.org/Vala/TestSample), but it has been found to be
severely lacking by anyone who knows anything about unit testing.

As with most OSS developers, I started to scratch that itch and realized
that it is a little bit beyond my skill set.  The code I started to write
remains unusable, rotting away somewhere on one of my hdd platters in a
folder called 'vunit'.  If anyone is interested in starting a Vala Unit
Test project under the umbrella of the elementary community, I'm sure we
could get quite a bit of traction from the Vala community at large and I
would love to help out.


On Thu, Apr 4, 2013 at 7:59 AM, Sergey "Shnatsel" Davidoff <
[email protected]> wrote:

> This is what I've developed for testing:
> https://lists.launchpad.net/elementary-dev-community/msg01948.html
>
> I've also created Glimpse - a system for sandboxed testing, but it lacks a
> D-bus proxy which is needed for adding/removing apps via GUI. See
> http://shnatsel.blogspot.com/2011/07/weve-just-revolutionized-alpha-testing.htmland
> http://shnatsel.blogspot.com/2012/05/state-of-glimpse.html
>
> I don't think we have any unit testing in place yet. Midori probably has
> but that's it.
>
>
> 2013/4/4 Craig 
>
>> Hello everyone,
>>
>> I'm curious what you devs do for testing? I'm not particularly familiar
>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>> develop myself to that end in my free time. I'm sending this email because
>> I'd like to get a pulse on what you Elementary devs think about testing and
>> what you actually do to test your code. Also, please feel encouraged to
>> talk about what you've done in the past, what has/hasn't worked for you,
>> and generally what your philosophy is about testing (or whether you have no
>> philosophy). Individual comments and comments on behalf of elementary as a
>> whole are both welcome.
>>
>> Sound off!
>>
>> Thanks,
>> Craig
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Sergey "Shnatsel" Davidoff
> OS architect @ elementary
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Craig 
I just came across this: http://esite.ch/2012/06/26/writing-tests-for-vala/.
It sounds very, very relevant. Comments?

Also, at work, we have a build server that kicks off a new build (and tests
immediately thereafter) roughly every time someone pushes changes to the
repo (sorry if that's not bzr terminology), which means we have a single
base environment to perform our tests. It works fairly well except that we
have 200+ people working on a single (C++--to make matters worse) project
and colossal communication is required to keep everything out of a FUBAR
state. However, an Elementary project probably doesn't have ~200 people
working on it at once, so you might be able to copy/tweak our methodology
without inheriting some of our headaches. I can elaborate more if desired.
Thoughts?

Sergey, that looks very interesting, although I'm a little confused--what
are the differences between your first link and Glimpse?


On Thu, Apr 4, 2013 at 7:59 AM, Sergey "Shnatsel" Davidoff <
[email protected]> wrote:

> This is what I've developed for testing:
> https://lists.launchpad.net/elementary-dev-community/msg01948.html
>
> I've also created Glimpse - a system for sandboxed testing, but it lacks a
> D-bus proxy which is needed for adding/removing apps via GUI. See
> http://shnatsel.blogspot.com/2011/07/weve-just-revolutionized-alpha-testing.htmland
> http://shnatsel.blogspot.com/2012/05/state-of-glimpse.html
>
> I don't think we have any unit testing in place yet. Midori probably has
> but that's it.
>
>
> 2013/4/4 Craig 
>
>> Hello everyone,
>>
>> I'm curious what you devs do for testing? I'm not particularly familiar
>> with Vala, but I'm learning a lot about testing at work and I'm trying to
>> develop myself to that end in my free time. I'm sending this email because
>> I'd like to get a pulse on what you Elementary devs think about testing and
>> what you actually do to test your code. Also, please feel encouraged to
>> talk about what you've done in the past, what has/hasn't worked for you,
>> and generally what your philosophy is about testing (or whether you have no
>> philosophy). Individual comments and comments on behalf of elementary as a
>> whole are both welcome.
>>
>> Sound off!
>>
>> Thanks,
>> Craig
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to : [email protected]
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Sergey "Shnatsel" Davidoff
> OS architect @ elementary
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Sergey "Shnatsel" Davidoff 
This is what I've developed for testing:
https://lists.launchpad.net/elementary-dev-community/msg01948.html

I've also created Glimpse - a system for sandboxed testing, but it lacks a
D-bus proxy which is needed for adding/removing apps via GUI. See
http://shnatsel.blogspot.com/2011/07/weve-just-revolutionized-alpha-testing.htmland
http://shnatsel.blogspot.com/2012/05/state-of-glimpse.html

I don't think we have any unit testing in place yet. Midori probably has
but that's it.


2013/4/4 Craig 

> Hello everyone,
>
> I'm curious what you devs do for testing? I'm not particularly familiar
> with Vala, but I'm learning a lot about testing at work and I'm trying to
> develop myself to that end in my free time. I'm sending this email because
> I'd like to get a pulse on what you Elementary devs think about testing and
> what you actually do to test your code. Also, please feel encouraged to
> talk about what you've done in the past, what has/hasn't worked for you,
> and generally what your philosophy is about testing (or whether you have no
> philosophy). Individual comments and comments on behalf of elementary as a
> whole are both welcome.
>
> Sound off!
>
> Thanks,
> Craig
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Sergey "Shnatsel" Davidoff
OS architect @ elementary
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp

Re: [Elementary-dev-community] Testing

2013-04-04 Thread Cassidy James 
Individual comment from a non-dev who has just observed devs: it would seem
we have no formal testing policy but that we gravely need one. Code is
developed and requested to be merged by dev A, then dev B looks the code
itself over. Hopefully at this point dev B would then build it and test the
crap out of it to make sure everything still works perfectly, but I feel
that doesn't get done thoroughly as often as it should and consequently
regressions slip through.

Automated and standardized testing would be boss.

Regards,
Cassidy James
On Apr 4, 2013 7:52 AM, "Craig"  wrote:

> Hello everyone,
>
> I'm curious what you devs do for testing? I'm not particularly familiar
> with Vala, but I'm learning a lot about testing at work and I'm trying to
> develop myself to that end in my free time. I'm sending this email because
> I'd like to get a pulse on what you Elementary devs think about testing and
> what you actually do to test your code. Also, please feel encouraged to
> talk about what you've done in the past, what has/hasn't worked for you,
> and generally what your philosophy is about testing (or whether you have no
> philosophy). Individual comments and comments on behalf of elementary as a
> whole are both welcome.
>
> Sound off!
>
> Thanks,
> Craig
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>
-- 
Mailing list: https://launchpad.net/~elementary-dev-community
Post to : [email protected]
Unsubscribe : https://launchpad.net/~elementary-dev-community
More help   : https://help.launchpad.net/ListHelp