Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Paulo van Breugel]

On 7 October 2016 3:14:14 pm "Blumentrath, Stefan" 
<stefan.blumentr...@nina.no> wrote:



Hi again,

Interesting discussion!

With my user and amateur addon-developer perspective I would conclude that:
- we all agree that tests/unittests are very useful even if not "the silver 
bullet"
- things one does in "unittests" might also partly be done in the code of 
the module (e.g. check for a specific state of input or environment 
variables ...), so a requirement for unittest might have to go hand in hand 
with a requirement for "maturity" of the module?
- "unittests" can also differ a lot in complexity so it is probably not too 
easy to say which level of unit tests should be required for a new core 
module, and the presence of a test alone does not necessarily mean that a 
module is "well tested"...
- it might be a good starting point to "require" only "simple" unittests 
for new core modules in that sense that a unit test makes sure that at 
least examples from the manual work without throwing an error? That would 
at the same time test code and manual/documentation... There was the idea 
to use examples from the manual to auto-generate tests, was`nt it?
- with more tests on "higher level" modules, also "lower level functions" 
(e.g. library) might be covered. This could on the one hand help to trace 
(side) effect of changes on lower level functions, but on the other hand 
might require to also update all output-tests which were written based on 
that bogus library function, would not it?
- like some others here I never wrote a test for my addons, but I would be 
willing to learn!


Same here

Here the simple examples Vaclav provided were very helpful, as for me as a 
rookie the more involved examples and the full documentation of the 
unittests are hard to understand.


Again, same here

Maybe you can add a very minimalistic example to the test documentation 
that only checks (e.g. based on NC data set) that e.g. "r.example -g 
input=mymap" works without throwing an error?


+1



And finally, Martins request regarding an RFC for "promoting modules to 
core" should not drown in the discussion about unittests (even if the 
latter probably is a precondition to the former)...


Cheers
Stefan

P.S.: For automatized tests of addons there might be a dependency issue, as 
a test-server would have to be equipped with all possible dependencies in 
order to be able to run the addons?




-Original Message-
From: grass-dev [mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of 
Moritz Lennert

Sent: 7. oktober 2016 08:32
To: Vaclav Petras <wenzesl...@gmail.com>; Anna Petrášová 
<kratocha...@gmail.com>
Cc: GRASS developers list <grass-dev@lists.osgeo.org>; Markus Metz 
<markus.metz.gisw...@gmail.com>
Subject: Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to 
move to core


On 06/10/16 23:34, Vaclav Petras wrote:

On Tue, Oct 4, 2016 at 4:55 PM, Anna Petrášová <kratocha...@gmail.com
<mailto:kratocha...@gmail.com>> wrote:

[...]

c) test correctness of results.
It just depends how you write them, and yes, for some modules c) is
more difficult to implement than for others.


On Thu, Oct 6, 2016 at 5:00 PM, Anna Petrášová <kratocha...@gmail.com
<mailto:kratocha...@gmail.com>> wrote:

There is correct result in the sense "as author intended". So you can
use pen and paper and solve very small example. That doesn't mean it
will cover all options, but better than nothing. [...]


For r.forestfrag, I wrote a test which was based on an example in the
original paper which was computing value of one cell in 3x3 window. It
is a really trivial example which tests 1 number and two other numbers
which are intermediate results. However, by writing it, I actually
discovered that although the results for a large area look good, this
small example, which I was able compute by hand, is failing. After
investigation, it turned out that the error is actually in r.mapcalc.
Computing the result outside of GRASS was crucial in this case. It was
an index and I was able to compute it by hand (and check it with the
example in the paper). For some other modules it could be more
difficult and I don't want to compute it without GRASS for more than
one cell even in this case, but it was certainly possible to write a
test of correctness for this module. Note that the test doesn't show
if the (forest fragmentation) index makes sense or if it is useful,
but it shows that the module gives the result it promised.

This is the original test:

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.fore
stfrag/testsuite/r_forestfrag_trivial.py


Nice example, thanks !



This is the r.mapcalc bug:

https://trac.osgeo.org/grass/ticket/3067

And this is test which specifically shows the bug (would show if it
wou

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Blumentrath, Stefan]

Hi again,

Interesting discussion!

With my user and amateur addon-developer perspective I would conclude that:
- we all agree that tests/unittests are very useful even if not "the silver 
bullet"
- things one does in "unittests" might also partly be done in the code of the 
module (e.g. check for a specific state of input or environment variables ...), 
so a requirement for unittest might have to go hand in hand with a requirement 
for "maturity" of the module?
- "unittests" can also differ a lot in complexity so it is probably not too 
easy to say which level of unit tests should be required for a new core module, 
and the presence of a test alone does not necessarily mean that a module is 
"well tested"...
- it might be a good starting point to "require" only "simple" unittests for 
new core modules in that sense that a unit test makes sure that at least 
examples from the manual work without throwing an error? That would at the same 
time test code and manual/documentation... There was the idea to use examples 
from the manual to auto-generate tests, was`nt it?
- with more tests on "higher level" modules, also "lower level functions" (e.g. 
library) might be covered. This could on the one hand help to trace (side) 
effect of changes on lower level functions, but on the other hand might require 
to also update all output-tests which were written based on that bogus library 
function, would not it?
- like some others here I never wrote a test for my addons, but I would be 
willing to learn! Here the simple examples Vaclav provided were very helpful, 
as for me as a rookie the more involved examples and the full documentation of 
the unittests are hard to understand. Maybe you can add a very minimalistic 
example to the test documentation that only checks (e.g. based on NC data set) 
that e.g. "r.example -g input=mymap" works without throwing an error?

And finally, Martins request regarding an RFC for "promoting modules to core" 
should not drown in the discussion about unittests (even if the latter probably 
is a precondition to the former)...

Cheers
Stefan

P.S.: For automatized tests of addons there might be a dependency issue, as a 
test-server would have to be equipped with all possible dependencies in order 
to be able to run the addons?



-Original Message-
From: grass-dev [mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of Moritz 
Lennert
Sent: 7. oktober 2016 08:32
To: Vaclav Petras <wenzesl...@gmail.com>; Anna Petrášová <kratocha...@gmail.com>
Cc: GRASS developers list <grass-dev@lists.osgeo.org>; Markus Metz 
<markus.metz.gisw...@gmail.com>
Subject: Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move 
to core

On 06/10/16 23:34, Vaclav Petras wrote:
> On Tue, Oct 4, 2016 at 4:55 PM, Anna Petrášová <kratocha...@gmail.com 
> <mailto:kratocha...@gmail.com>> wrote:
>
> [...]
>
> c) test correctness of results.
> It just depends how you write them, and yes, for some modules c) is
> more difficult to implement than for others.
>
>
> On Thu, Oct 6, 2016 at 5:00 PM, Anna Petrášová <kratocha...@gmail.com 
> <mailto:kratocha...@gmail.com>> wrote:
>
> There is correct result in the sense "as author intended". So you can
> use pen and paper and solve very small example. That doesn't mean it
> will cover all options, but better than nothing. [...]
>
>
> For r.forestfrag, I wrote a test which was based on an example in the 
> original paper which was computing value of one cell in 3x3 window. It 
> is a really trivial example which tests 1 number and two other numbers 
> which are intermediate results. However, by writing it, I actually 
> discovered that although the results for a large area look good, this 
> small example, which I was able compute by hand, is failing. After 
> investigation, it turned out that the error is actually in r.mapcalc.
> Computing the result outside of GRASS was crucial in this case. It was 
> an index and I was able to compute it by hand (and check it with the 
> example in the paper). For some other modules it could be more 
> difficult and I don't want to compute it without GRASS for more than 
> one cell even in this case, but it was certainly possible to write a 
> test of correctness for this module. Note that the test doesn't show 
> if the (forest fragmentation) index makes sense or if it is useful, 
> but it shows that the module gives the result it promised.
>
> This is the original test:
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.fore
> stfrag/testsuite/r_forestfrag_trivial.py

Nice example, thanks !

>
> This is the r.mapcalc bug:
>
> https://trac.osgeo.org/grass/ticket/3067
>
> And th

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Blumentrath, Stefan]

Agreed with Paulo.
My impression is that new users rely on the menu to explore the software and 
its modules...


From: Paulo van Breugel [mailto:p.vanbreu...@gmail.com]
Sent: 7. oktober 2016 13:35
To: Moritz Lennert <mlenn...@club.worldonline.be>; Blumentrath, Stefan 
<stefan.blumentr...@nina.no>; Vaclav Petras <wenzesl...@gmail.com>; Anna 
Petrášová <kratocha...@gmail.com>
Cc: GRASS developers list <grass-dev@lists.osgeo.org>; Markus Metz 
<markus.metz.gisw...@gmail.com>
Subject: Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move 
to core


On 7 October 2016 1:26:09 pm Moritz Lennert 
<mlenn...@club.worldonline.be<mailto:mlenn...@club.worldonline.be>> wrote:

> On 07/10/16 11:35, Blumentrath, Stefan wrote:
>> Ahhh, yes, there was that branch of the discussion too...
>>
>> g.extension is great and I do not see any limit with that, esp. not
>> for single user environments.
>>
>> However, on our server I download, compile and install centrally all
>> addons from source using a script. Thus, the main perceivable
>> difference between core modules and addons would be visibility in the
>> GUI...
>>
>> For me personally, that is no issue. But people who start with GRASS
>> likely often explore GRASS "graphically", thus a toolbox-approach
>> would be very nice I think. Having an "Addons" entry in the menu (if
>> there is space for that) where all installed addons register, would
>> be a demonstration of existing addional functionality...
>
> [...]
>
> On 05/10/16 23:10, Anna Petrášová wrote:
>  > Hmm, this is already implemented (several years I would say). Look
>  > into the tab Modules in layer manager. At least we now know, that
>  > nobody here is using it...
>
> Yes, I have to admit that I rarely use the other tabs in the GUI, to my
> own disadvantage most probably...
>
> But maybe we can think more radically. Just as several OS have moved
> away from the classical menu paradigm to search-based access to
> functionalities (e.g. gnome-shell, etc), why not completely get rid of
> the menus and only leave the modules tab as entry point ?

That works for some but surely not all users.

>
> Again, just brainstorming...
>
> Moritz
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org<mailto:grass-dev@lists.osgeo.org>
> http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Paulo van Breugel]

On 7 October 2016 1:26:09 pm Moritz Lennert  
wrote:



On 07/10/16 11:35, Blumentrath, Stefan wrote:

Ahhh, yes, there was that branch of the discussion too...

g.extension is great and I do not see any limit with that, esp. not
for single user environments.

However, on our server I download, compile and install centrally all
addons from source using a script. Thus, the main perceivable
difference between core modules and addons would be visibility in the
GUI...

For me personally, that is no issue. But people who start with GRASS
likely often explore GRASS "graphically", thus a toolbox-approach
would be very nice I think. Having an "Addons" entry in the menu (if
there is space for that) where all installed addons register, would
be a demonstration of existing addional functionality...


[...]

On 05/10/16 23:10, Anna Petrášová wrote:
 > Hmm, this is already implemented (several years I would say). Look
 > into the tab Modules in layer manager. At least we now know, that
 > nobody here is using it...

Yes, I have to admit that I rarely use the other tabs in the GUI, to my
own disadvantage most probably...

But maybe we can think more radically. Just as several OS have moved
away from the classical menu paradigm to search-based access to
functionalities (e.g. gnome-shell, etc), why not completely get rid of
the menus and only leave the modules tab as entry point ?


That works for some but surely not all users.



Again, just brainstorming...

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 07/10/16 11:35, Blumentrath, Stefan wrote:

Ahhh, yes, there was that branch of the discussion too...

g.extension is great and I do not see any limit with that, esp. not
for single user environments.

However, on our server I download, compile and install centrally all
addons from source using a script. Thus, the main perceivable
difference between core modules and addons would be visibility in the
GUI...

For me personally, that is no issue. But people who start with GRASS
likely often explore GRASS "graphically", thus a toolbox-approach
would be very nice I think. Having an "Addons" entry in the menu (if
there is space for that) where all installed addons register, would
be a demonstration of existing addional functionality...


[...]

On 05/10/16 23:10, Anna Petrášová wrote:
> Hmm, this is already implemented (several years I would say). Look
> into the tab Modules in layer manager. At least we now know, that
> nobody here is using it...

Yes, I have to admit that I rarely use the other tabs in the GUI, to my 
own disadvantage most probably...


But maybe we can think more radically. Just as several OS have moved 
away from the classical menu paradigm to search-based access to 
functionalities (e.g. gnome-shell, etc), why not completely get rid of 
the menus and only leave the modules tab as entry point ?


Again, just brainstorming...

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Blumentrath, Stefan]

Ahhh, yes, there was that branch of the discussion too...

g.extension is great and I do not see any limit with that, esp. not for single 
user environments.

However, on our server I download, compile and install centrally all addons 
from source using a script.
Thus, the main perceivable difference between core modules and addons would be 
visibility in the GUI...

For me personally, that is no issue. But people who start with GRASS likely 
often explore GRASS "graphically", thus a toolbox-approach would be very nice I 
think. Having an "Addons" entry in the menu (if there is space for that) where 
all installed addons register, would be a demonstration of existing addional 
functionality...
However, as you can imagine manually editing xml files for the numerous addons 
is no option for me. If it would be possible to implement your suggestion to 
add modules automatically to the GUI at install by means of Makefile (instead 
of g.extension) that would be splendid. But maybe I could change my script to 
use g.extension -s...

Advantages I see for moving addons to core:
- GRASS offers more "popular" functionality out of the box (new users probably 
do not check addons immediately when starting with GRASS)
- usually functionality is maintained to new major version (even if probably 
some exceptions exist, e.g. orthorectification GUI which has not been ported to 
G7 yet)
- moving an addon to core can be a kind of "award" or "reward" for an addon 
developer, thus it might be an incentive to e.g. add tests, documentation ...
- it might also help to motivate / involve more devs to contribute to core 
development?

Cheers
Stefan


-Original Message-
From: Moritz Lennert [mailto:mlenn...@club.worldonline.be] 
Sent: 7. oktober 2016 11:00
To: Blumentrath, Stefan <stefan.blumentr...@nina.no>; Vaclav Petras 
<wenzesl...@gmail.com>; Anna Petrášová <kratocha...@gmail.com>
Cc: GRASS developers list <grass-dev@lists.osgeo.org>; Markus Metz 
<markus.metz.gisw...@gmail.com>
Subject: RE: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move 
to core



Le 7 octobre 2016 10:41:50 GMT+02:00, "Blumentrath, Stefan" 
<stefan.blumentr...@nina.no> a écrit :
>Hi again,
>
>Interesting discussion!
>
>With my user and amateur addon-developer perspective I would conclude
>that:
[...]
>And finally, Martins request regarding an RFC for "promoting modules to 
>core" should not drown in the discussion about unittests (even if the 
>latter probably is a precondition to the former)...

And what is your take on the actual need to move modules to core ? Why is it an 
advantage ? And as a corollary to that: what limits/issues do you see with 
g.extension ?

Moritz



___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Markus Neteler]

[... two emails with lots of pointer to tests ...]

Please someone may create a (trac) wiki page since this will get lost
in the emails in no time.

thanks
Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

Le 7 octobre 2016 10:41:50 GMT+02:00, "Blumentrath, Stefan" 
 a écrit :
>Hi again,
>
>Interesting discussion!
>
>With my user and amateur addon-developer perspective I would conclude
>that:
[...]
>And finally, Martins request regarding an RFC for "promoting modules to
>core" should not drown in the discussion about unittests (even if the
>latter probably is a precondition to the former)...

And what is your take on the actual need to move modules to core ? Why is it an 
advantage ? And as a corollary to that: what limits/issues do you see with 
g.extension ?

Moritz



___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 06/10/16 23:34, Vaclav Petras wrote:

On Tue, Oct 4, 2016 at 4:55 PM, Anna Petrášová > wrote:

[...]

c) test correctness of results.
It just depends how you write them, and yes, for some modules c) is
more difficult to implement than for others.


On Thu, Oct 6, 2016 at 5:00 PM, Anna Petrášová > wrote:

There is correct result in the sense "as author intended". So you can
use pen and paper and solve very small example. That doesn't mean it
will cover all options, but better than nothing. [...]


For r.forestfrag, I wrote a test which was based on an example in the
original paper which was computing value of one cell in 3x3 window. It
is a really trivial example which tests 1 number and two other numbers
which are intermediate results. However, by writing it, I actually
discovered that although the results for a large area look good, this
small example, which I was able compute by hand, is failing. After
investigation, it turned out that the error is actually in r.mapcalc.
Computing the result outside of GRASS was crucial in this case. It was
an index and I was able to compute it by hand (and check it with the
example in the paper). For some other modules it could be more difficult
and I don't want to compute it without GRASS for more than one cell even
in this case, but it was certainly possible to write a test of
correctness for this module. Note that the test doesn't show if the
(forest fragmentation) index makes sense or if it is useful, but it
shows that the module gives the result it promised.

This is the original test:

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.forestfrag/testsuite/r_forestfrag_trivial.py


Nice example, thanks !



This is the r.mapcalc bug:

https://trac.osgeo.org/grass/ticket/3067

And this is test which specifically shows the bug (would show if it
would be reintroduced):

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.mapcalc/testsuite/test_row_above_below_bug.py


Here we have an example of a test with 70 lines of code which was 
comitted almost at the same time as the issue was fixed (ok 3 days 
before), with a one-line fix. Now, it's good to safeguard against this 
bug reappearing, but I would argue that this example shows that we often 
only think of relevant tests after a bug appears, and I'm not sure that 
it is very efficient to write a test at that point, given the low 
probability of that bug reappearing ;-)


But hey, bear with me, I'm learning...

Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Vaclav Petras]

Hi Nikos,

On Wed, Oct 5, 2016 at 7:39 PM, Nikos Alexandris 
wrote:

> could you name some good examples (not to the
> manual/s), some real modules with integrated tests?  I am much in need
> for good code.
>

I'm not sure if they are good, but here are some examples based on Anna's
test categories.

On Tue, Oct 4, 2016 at 4:55 PM, Anna Petrášová 
wrote:

> [...]
> gunittests can serve to a) test inputs/outputs b) catch changes in
> results (whether correct or incorrect) c) test correctness of results.
> It just depends how you write them, and yes, for some modules c) is
> more difficult to implement than for others.
>

a) inputs/outputs

This one tests if the outputs are there (module runs, accepts flags, gives
outputs) and if the output values are in expected ranges ("basic
correctness") and the output are expected data types. It also tests some
stats like sum and variance of the result, but that's more for the category
b).

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.forestfrag/testsuite/r_forestfrag_ncspm.py

b) result changes

This uses a miniature dataset and results obtained by the module itself. It
tests all values of the result and expects them to match the reference.
This will catch a regression in a module code (or in dependencies).

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.forestfrag/testsuite/r_forestfrag_xy.py

It assumes that the model worked at the point these were obtained. If that
was not the case, the data must be changed like in this commit:

https://trac.osgeo.org/grass/changeset/68722

c) correctness

Here is a test on a miniature example which was possible to compute by hand
and was part of the original publication.

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.forestfrag/testsuite/r_forestfrag_trivial.py

Here is a test which uses reference data computed using another software:

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.slope.aspect/testsuite/test_r_slope_aspect.py?rev=68557#L44
https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.slope.aspect/testsuite/data

Using gdaldem to test r.slope.aspect it is actually not ideal, because
gdaldem uses old r.slope.aspect code. So in this case, it is more a test
against very old version of r.slope.aspect.

This is definitively a creative part because you need to get the results
without using the code you are developing to get the result. For example,
if you have prototype implementation in Python or r.mapcalc and then your
write C code to do the same more effectively with more options, you can use
the prototype to cover part of your outputs. This of course does not give a
guarantee that you did not make the same mistake twice, but it is almost
guaranteed that you will make two separate sets of mistakes and this will
expose them.

It of course never hurts to test also the "basic correctness", e.g. that
the values are all positive or that they are from 0 to 255 and are integers.

d) small parts and dependencies

There is one more category I would add to this list. These are the tests of
building blocks (units) you are using in your module. It can be functions
your wrote but also functions or modules you are using. When writing a C
module you can write a separate module which uses the functions from your
module one by one to generate some testable outputs. These output can be
then tested using the gunittest procedure.

https://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.flow/test_main.c
https://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.flow/testsuite/r3flow_test.py

Further, you can write a ctypes-based (fully) Python test for the library
functions your are using:

https://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/testsuite/gis_lib_tokenize.py

For Python modules, you can do the same it is just much easier and Soeren
already provided examples for it.

Hope this helps,
Vaclav
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Vaclav Petras]

On Tue, Oct 4, 2016 at 4:55 PM, Anna Petrášová 
wrote:

> [...]
>
c) test correctness of results.
> It just depends how you write them, and yes, for some modules c) is
> more difficult to implement than for others.
>

On Thu, Oct 6, 2016 at 5:00 PM, Anna Petrášová 
wrote:

> There is correct result in the sense "as author intended". So you can
> use pen and paper and solve very small example. That doesn't mean it
> will cover all options, but better than nothing. [...]
>

For r.forestfrag, I wrote a test which was based on an example in the
original paper which was computing value of one cell in 3x3 window. It is a
really trivial example which tests 1 number and two other numbers which are
intermediate results. However, by writing it, I actually discovered that
although the results for a large area look good, this small example, which
I was able compute by hand, is failing. After investigation, it turned out
that the error is actually in r.mapcalc. Computing the result outside of
GRASS was crucial in this case. It was an index and I was able to compute
it by hand (and check it with the example in the paper). For some other
modules it could be more difficult and I don't want to compute it without
GRASS for more than one cell even in this case, but it was certainly
possible to write a test of correctness for this module. Note that the test
doesn't show if the (forest fragmentation) index makes sense or if it is
useful, but it shows that the module gives the result it promised.

This is the original test:

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.forestfrag/testsuite/r_forestfrag_trivial.py

This is the r.mapcalc bug:

https://trac.osgeo.org/grass/ticket/3067

And this is test which specifically shows the bug (would show if it would
be reintroduced):

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.mapcalc/testsuite/test_row_above_below_bug.py
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Anna Petrášová]

On Thu, Oct 6, 2016 at 4:08 PM, Markus Metz
 wrote:
> On Thu, Oct 6, 2016 at 12:09 PM, Moritz Lennert
>  wrote:
>> On 06/10/16 01:39, Nikos Alexandris wrote:
>>>
>>> The question, the way I understand things, is not if and at what extent
>>> we need tests. GRASS-GIS needs them.
>>> However, no matter the effort, tests will almost never be complete.-
>>> That's what I was told by computer scientists, and what I can confirm
>>> with my minimal experiences by doing, for at least two years now, in a
>>> dozen of mini-projects.
>>
>>
>> This is true, in my experience, but Markus' remarks (AFAIU) go beyond the
>> question of completeness. One big issue we have with some modules is how to
>> define the 'correct' result. Unless we use a tautological reasoning to say
>> that the current state of the module gives the correct results (thus
>> reducing the signification of the tests to regression tests), there are a
>> series of modules for which no third-party validation data exists.
>>
>> For example: what is a "correct" segmentation by i.segment ? Or what is the
>> "correct" output of some of r.watershed's results ?
>
> Very good examples because there is no "correct" output of i.segment
> or r.watershed because there is no agreement about what is correct.
> Different implementations of image segmentation or 2D hydrological
> modelling make different assumptions. Most of these assumptions are
> supported by peer-reviewed literature. That means there is no
> generally accepted "correct" result, and all we can do is to ensure
> that the output of GRASS modules (obviously we assume that the output
> matches the expectations of the author of the module) should not
> change.
>

There is correct result in the sense "as author intended". So you can
use pen and paper and solve very small example. That doesn't mean it
will cover all options, but better than nothing. This is obviously
hard to do if you are not the original author of the algorithm, and
that's the reason we should insist on tests for new modules.

Anna


> Markus M
>
>>
>> A second issue is that some bugs only appear with large amounts of data (see
>> #3084 for example), but I don't think that it is feasible to test all our
>> modules on multi-GB datasets. And sometimes it is not the size of the data
>> as such, but very specific combinations of data.
>>
>> This second issue obviously should not stop us from writing tests for less
>> sizeable data. The first issue is a bit more difficult to solve.
>>
>>> Yes, testing costs a lot of time. But, if comparing it with the time
>>> spent afterwards in debugging bad code, therein lies the wish to have
>>> spent the "less" time right in the beginning.
>>
>>
>> I agree...to a point. However, don't forget that tests are code just like
>> any other code and thus increasing the lines of code by writing all these
>> tests also increases the lines of code that need to be maintained. So, a big
>> yes for testing, but let's be careful and efficient about it and not pretend
>> we can solve everything with testing. And let's be especially wary of the
>> danger cited often in the literature about testing: while thinking about
>> tests is important, don't forget to think about the actual code you write
>> ;-)
>>
>> Moritz
>>
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Markus Metz]

On Thu, Oct 6, 2016 at 12:09 PM, Moritz Lennert
 wrote:
> On 06/10/16 01:39, Nikos Alexandris wrote:
>>
>> The question, the way I understand things, is not if and at what extent
>> we need tests. GRASS-GIS needs them.
>> However, no matter the effort, tests will almost never be complete.-
>> That's what I was told by computer scientists, and what I can confirm
>> with my minimal experiences by doing, for at least two years now, in a
>> dozen of mini-projects.
>
>
> This is true, in my experience, but Markus' remarks (AFAIU) go beyond the
> question of completeness. One big issue we have with some modules is how to
> define the 'correct' result. Unless we use a tautological reasoning to say
> that the current state of the module gives the correct results (thus
> reducing the signification of the tests to regression tests), there are a
> series of modules for which no third-party validation data exists.
>
> For example: what is a "correct" segmentation by i.segment ? Or what is the
> "correct" output of some of r.watershed's results ?

Very good examples because there is no "correct" output of i.segment
or r.watershed because there is no agreement about what is correct.
Different implementations of image segmentation or 2D hydrological
modelling make different assumptions. Most of these assumptions are
supported by peer-reviewed literature. That means there is no
generally accepted "correct" result, and all we can do is to ensure
that the output of GRASS modules (obviously we assume that the output
matches the expectations of the author of the module) should not
change.

Markus M

>
> A second issue is that some bugs only appear with large amounts of data (see
> #3084 for example), but I don't think that it is feasible to test all our
> modules on multi-GB datasets. And sometimes it is not the size of the data
> as such, but very specific combinations of data.
>
> This second issue obviously should not stop us from writing tests for less
> sizeable data. The first issue is a bit more difficult to solve.
>
>> Yes, testing costs a lot of time. But, if comparing it with the time
>> spent afterwards in debugging bad code, therein lies the wish to have
>> spent the "less" time right in the beginning.
>
>
> I agree...to a point. However, don't forget that tests are code just like
> any other code and thus increasing the lines of code by writing all these
> tests also increases the lines of code that need to be maintained. So, a big
> yes for testing, but let's be careful and efficient about it and not pretend
> we can solve everything with testing. And let's be especially wary of the
> danger cited often in the literature about testing: while thinking about
> tests is important, don't forget to think about the actual code you write
> ;-)
>
> Moritz
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Sören Gebbert]

Hi Nikos,

2016-10-06 1:39 GMT+02:00 Nikos Alexandris :

> Apologies for removing all but Soeren's last post.
>
> * Sören Gebbert:
>
>
> I would like to put the GRASS test framework into perspective, since i
>> think that its capabilities are not well known.
>> The gunittest framework is not about unit tests. It was designed to test
>> all aspects of the GRASS development. This framework allows you to:
>>
>> * Implement unit tests for the Python libraries, their mthods and classes
>> * Implement and run doctests in the source code of the Python libraries
>> * Run integration tests for all modules, checking correct output for
>> almost
>> all datatypes in GRASS (raster, vector, 3D raster, space-time datasets,
>> categories, color definitions, stdout, ...). Module tests are IMHO
>> integration tests, since module make use of different library methods and
>> classes and combine them.
>> * Run C-library tests as unit and integration tests. C-library unit and
>> integration tests can either be implemented in C or via ctypes in Python
>> * Run tests on library level, module level or for all libraries and
>> modules
>> in the whole GRASS source tree using a single command, ...
>> * Perform regression tests in dedicated test locations, autimatically
>> triggered by a cronjob or a commit
>> * The framework allows you to run all library unit tests, before module
>> integration tests are performed
>> * It creates temporary mapsest to run without problems in production
>> locations
>> * It logs all tests in detail  and generates easy to inspect HTML output
>> at
>> runtime, so you can check the progress of the tests and its gradually
>> available results
>> * It allows on the fly mapset creation and deletion
>> * It supports temporary region environments
>> * It support user defined test data for input generation and output
>> validation
>>
>> These capabilities allow a wide range of tests to be created, covering
>> most
>> aspects of the GRASS development, with the exception of the GUI.
>> So please no excuses that the gunittest framework is not capable of
>> implementing a test that is required to assure module correctness.
>>
>
> Soeren, could you name some good examples (not to the
> manual/s), some real modules with integrated tests?  I am much in need
> for good code.
>

Some examples:
***

C-Code
--

Unittest for the linear equation solver in gmath library:

https://trac.osgeo.org/grass/browser/grass/trunk/lib/gmath/test/test_solvers.c

Integration test of the groundwater flow module, that make use of many
gmath functions. The result of the simulation is tested against reference
values that are based on literature, see the header of the file:

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.gwflow/testsuite/validation_7x7_grid.py

Python-Code
-

The PyGRASS unittests are a great example and inspiration for testing:

https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/raster/testsuite

https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/pygrass/vector/testsuite

Integration tests:

The v.stream.order uses the PyGRASS vector library and implements two
integration tests:

https://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.stream.order/testsuite/test_stream_order.py

The temporal module t.rast.univar uses integration tests to validate the
combination of the temporal framework, PyGRASS and the functionality of
r.univar:

https://trac.osgeo.org/grass/browser/grass/trunk/temporal/t.rast.univar/testsuite/test_univar.py

The module t.rast.aggreagte combines the temporal framework, PyGRASS and
r.series. Its functionality is tested in several integration tests:

https://trac.osgeo.org/grass/browser/grass/trunk/temporal/t.rast.aggregate#testsuite

Best regards
Soeren


> By the way, working code does not necessarily mean good code.  And vice
> versa, good code does not mean that it works the way it should.
> Yet, broken but "good" code (cleanly documented, tested as much as
> possible)
> can be easily fixed and probably easier optimised.  Working, though
> poor/uncommented/untested code, is frequently, if not always, a
> big-time-consuming mission.
>
> The question, the way I understand things, is not if and at what extent
> we need tests. GRASS-GIS needs them.
> However, no matter the effort, tests will almost never be complete.-
> That's what I was told by computer scientists, and what I can confirm
> with my minimal experiences by doing, for at least two years now, in a
> dozen of mini-projects.
>
> Rather, the question is how can we bring, most, of the theory of testing
> in to
> reality, put it in real action, beautify GRASS' code, whether core or
> addons.  My understanding is that "our" common understanding is to focus
> in raising the overall quality level, make it easy for some-one to read
> the code, to understand the code almost as if reading simple text, to
> enable
> some-one to improve or adapt 

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 06/10/16 01:39, Nikos Alexandris wrote:

The question, the way I understand things, is not if and at what extent
we need tests. GRASS-GIS needs them.
However, no matter the effort, tests will almost never be complete.-
That's what I was told by computer scientists, and what I can confirm
with my minimal experiences by doing, for at least two years now, in a
dozen of mini-projects.


This is true, in my experience, but Markus' remarks (AFAIU) go beyond 
the question of completeness. One big issue we have with some modules is 
how to define the 'correct' result. Unless we use a tautological 
reasoning to say that the current state of the module gives the correct 
results (thus reducing the signification of the tests to regression 
tests), there are a series of modules for which no third-party 
validation data exists.


For example: what is a "correct" segmentation by i.segment ? Or what is 
the "correct" output of some of r.watershed's results ?


A second issue is that some bugs only appear with large amounts of data 
(see #3084 for example), but I don't think that it is feasible to test 
all our modules on multi-GB datasets. And sometimes it is not the size 
of the data as such, but very specific combinations of data.


This second issue obviously should not stop us from writing tests for 
less sizeable data. The first issue is a bit more difficult to solve.



Yes, testing costs a lot of time. But, if comparing it with the time
spent afterwards in debugging bad code, therein lies the wish to have
spent the "less" time right in the beginning.


I agree...to a point. However, don't forget that tests are code just 
like any other code and thus increasing the lines of code by writing all 
these tests also increases the lines of code that need to be maintained. 
So, a big yes for testing, but let's be careful and efficient about it 
and not pretend we can solve everything with testing. And let's be 
especially wary of the danger cited often in the literature about 
testing: while thinking about tests is important, don't forget to think 
about the actual code you write ;-)


Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Nikos Alexandris]

Apologies for removing all but Soeren's last post.

* Sören Gebbert:


I would like to put the GRASS test framework into perspective, since i
think that its capabilities are not well known.
The gunittest framework is not about unit tests. It was designed to test
all aspects of the GRASS development. This framework allows you to:

* Implement unit tests for the Python libraries, their mthods and classes
* Implement and run doctests in the source code of the Python libraries
* Run integration tests for all modules, checking correct output for almost
all datatypes in GRASS (raster, vector, 3D raster, space-time datasets,
categories, color definitions, stdout, ...). Module tests are IMHO
integration tests, since module make use of different library methods and
classes and combine them.
* Run C-library tests as unit and integration tests. C-library unit and
integration tests can either be implemented in C or via ctypes in Python
* Run tests on library level, module level or for all libraries and modules
in the whole GRASS source tree using a single command, ...
* Perform regression tests in dedicated test locations, autimatically
triggered by a cronjob or a commit
* The framework allows you to run all library unit tests, before module
integration tests are performed
* It creates temporary mapsest to run without problems in production
locations
* It logs all tests in detail  and generates easy to inspect HTML output at
runtime, so you can check the progress of the tests and its gradually
available results
* It allows on the fly mapset creation and deletion
* It supports temporary region environments
* It support user defined test data for input generation and output
validation

These capabilities allow a wide range of tests to be created, covering most
aspects of the GRASS development, with the exception of the GUI.
So please no excuses that the gunittest framework is not capable of
implementing a test that is required to assure module correctness.


Soeren, could you name some good examples (not to the
manual/s), some real modules with integrated tests?  I am much in need
for good code.

By the way, working code does not necessarily mean good code.  And vice
versa, good code does not mean that it works the way it should.
Yet, broken but "good" code (cleanly documented, tested as much as possible)
can be easily fixed and probably easier optimised.  Working, though
poor/uncommented/untested code, is frequently, if not always, a
big-time-consuming mission.

The question, the way I understand things, is not if and at what extent
we need tests. GRASS-GIS needs them.
However, no matter the effort, tests will almost never be complete.-
That's what I was told by computer scientists, and what I can confirm
with my minimal experiences by doing, for at least two years now, in a
dozen of mini-projects.

Rather, the question is how can we bring, most, of the theory of testing in to
reality, put it in real action, beautify GRASS' code, whether core or
addons.  My understanding is that "our" common understanding is to focus
in raising the overall quality level, make it easy for some-one to read
the code, to understand the code almost as if reading simple text, to enable
some-one to improve or adapt a piece of code relatively fast.

Step-by-step.  Nothing is going to change overnight.
Not to forget, the community is a mixture of members with a high
diversity degree when looking at coding skills.  Everyone should be
invited. And everyone is invited. Experienced developers, are asked
and, with all the respect, (you) carry the responsibility to shed light for
the newcomers in coding for GRASS-GIS.

Allow me please to reiterate what I have repeated indirectly
or directly, some times:  please, dear reader/coder/scripter, 
when you write code, no matter what you write, comment your code,

use longer variable names, don't hesitate to repeat explaining, document
here and there. Test.  Use assertions in your code, even for simple,
single functions. Write code that "invites" the reader.

Yes, testing costs a lot of time.  But, if comparing it with the time
spent afterwards in debugging bad code, therein lies the wish to have spent the 
"less"
time right in the beginning.

Testing will boldly save big, valuable time.-
It'll beautify GRASS-GIS and grow the community.
It'll work for us, not against us.

Nikos
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Paulo van Breugel]

On October 5, 2016 23:10:27 Anna Petrášová  wrote:


On Wed, Oct 5, 2016 at 10:46 AM, Moritz Lennert
 wrote:

On 05/10/16 16:29, Paulo van Breugel wrote:



On 05-10-16 15:20, Moritz Lennert wrote:


Maybe we should ask the users of whether this distinction between
modules and core and extensions is really relevant for them, or
whether most are perfectly happy to just install extensions.



Since you are asking :-) , as an user, my main interest is in good
documentation and reproducible examples (which I can than also use to
see if outputs make sense to me as an user). In that respect there is no
inherent difference between core modules and extensions. What is
different is that many (most?) of the core functions are accessible
through the menu. I personally don't find that very important,
especially with the modules tab giving fairly easy access to extensions,
but I can imagine that for new / other users, especially those more
inclined to menu-driven applications, this may make a difference.



Thank you, Paulo, for this feedback.

This means that finding a means to more easily integrate addon modules into
menus might be a task to keep in mind.

Maybe the startup script could check the .grass7/addons directory and
automatically create an 'addons' menu using [1] ? Maybe the mecanism can be
changed in a way to just allow to create toolboxes and that these are
automatically integrated into the main menu ?

Just brainstorming, here...

Moritz


[1] https://grass.osgeo.org/grass70/manuals/wxGUI.toolboxes.html


Hmm, this is already implemented (several years I would say). Look
into the tab Modules in layer manager. At least we now know, that
nobody here is using it...


I do use it. As I mentioned above, I find it a convenient way to access / 
find extensions. My point was that others might prefer to have access to 
these functions through the menu bar. And initially I was one of them, for 
some reason it took me time to get used using this modules tab, and to 
realize / internalize that the modules tab is in fact more powerful having 
the search function.


So I guess it is a matter of preference, which in turn will often depend on 
what a person is used to. I have seen a similar discussion concerning 
moving functions to the processing toolbox, but keeping a link to these 
functions in the menu.



Before adding additional menu Addons, I suggest reading
https://trac.osgeo.org/grass/ticket/1742


But yes, there are other issues to consider, like space on the menu bar,, true.




Anna



___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev



___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Anna Petrášová]

On Wed, Oct 5, 2016 at 10:46 AM, Moritz Lennert
 wrote:
> On 05/10/16 16:29, Paulo van Breugel wrote:
>>
>>
>> On 05-10-16 15:20, Moritz Lennert wrote:
>>>
>>> Maybe we should ask the users of whether this distinction between
>>> modules and core and extensions is really relevant for them, or
>>> whether most are perfectly happy to just install extensions.
>>
>>
>> Since you are asking :-) , as an user, my main interest is in good
>> documentation and reproducible examples (which I can than also use to
>> see if outputs make sense to me as an user). In that respect there is no
>> inherent difference between core modules and extensions. What is
>> different is that many (most?) of the core functions are accessible
>> through the menu. I personally don't find that very important,
>> especially with the modules tab giving fairly easy access to extensions,
>> but I can imagine that for new / other users, especially those more
>> inclined to menu-driven applications, this may make a difference.
>>
>
> Thank you, Paulo, for this feedback.
>
> This means that finding a means to more easily integrate addon modules into
> menus might be a task to keep in mind.
>
> Maybe the startup script could check the .grass7/addons directory and
> automatically create an 'addons' menu using [1] ? Maybe the mecanism can be
> changed in a way to just allow to create toolboxes and that these are
> automatically integrated into the main menu ?
>
> Just brainstorming, here...
>
> Moritz
>
>
> [1] https://grass.osgeo.org/grass70/manuals/wxGUI.toolboxes.html

Hmm, this is already implemented (several years I would say). Look
into the tab Modules in layer manager. At least we now know, that
nobody here is using it...
Before adding additional menu Addons, I suggest reading
https://trac.osgeo.org/grass/ticket/1742


Anna

>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Sören Gebbert]

Hi,

2016-10-05 15:20 GMT+02:00 Moritz Lennert :

> On 05/10/16 14:24, Sören Gebbert wrote:
>
>> Hi,
>>
>> 2016-10-05 10:20 GMT+02:00 Moritz Lennert > >:
>>
>> [sent this from the wrong address, so it didn't get through to the
>> list]
>>
>>
>>  Message d'origine 
>> Envoyé : 5 octobre 2016 00:41:20 GMT+02:00
>>
>>
>>
>> Le 4 octobre 2016 22:55:35 GMT+02:00, "Anna Petrášová"
>> > a écrit :
>> >On Tue, Oct 4, 2016 at 4:22 PM, Markus Metz
>> >> > wrote:
>> >> On Tue, Oct 4, 2016 at 5:42 PM, Sören Gebbert
>> >> > > wrote:
>> >>> Hi,
>> 
>> 
>>  >
>>  > You are very welcome to write the missing tests for core
>> modules.
>>  >
>>  > However, i don't understand the argument that because many core
>> >modules
>>  > have
>>  > no tests, therefore new modules don't need them. If developers
>> of
>> >addon
>>  > module are serious about the attempt to make their modules
>> usable
>> >and
>>  > maintainable for others, then they have to implement tests. Its
>> >and
>>  > integral
>>  > part of the development process and GRASS has a beautiful test
>>  > environment
>>  > hat makes writing tests easy. Tests and documentation are part
>> of
>> >coding
>>  > and
>>  > not something special. I don't think this is a hard
>> requirement.
>>  >
>>  > There is a nice statement that is not far from the truth:
>> >Untested code
>>  > is
>>  > broken code.
>> 
>>  these gunittests only test if a module output stays the same.
>> This
>> >>>
>> >>>
>> >>> This is simply wrong, please read the gunittest documentation.
>> >>
>> >> but then why does
>> >>>
>> >>> The gunittest for the v.stream.order addon is an example how its
>> >done:
>> >>>
>> >https://trac.osgeo.org/grass/browser/grass-addons/grass7/ve
>> ctor/v.stream.order/testsuite/test_stream_order.py
>> > ctor/v.stream.order/testsuite/test_stream_order.py>
>> >>
>> >> assume certain order numbers for features 4 and 7? What if these
>> >order
>> >> numbers are wrong?
>> >>
>> >> Recently I fixed bugs in r.stream.order, related to stream length
>> >> calculations which are in turn used to determine stream orders. The
>> >> gunittest did not pick up 1) the bugs, 2) the bug fixes.
>> >>
>> >>>
>> >>> You can write gunittests that will test every flag, every option,
>> >their
>> >>> combination and any output of a module. I have implemented plenty
>> of
>> >tests,
>> >>> that check for correct error handling. Writing tests is effort,
>> but
>> >you have
>> >>> to do it anyway. Why not implementing a gunittest for every
>> feature
>> >while
>> >>> developing the module?
>> 
>> 
>>  My guess for the r.stream.* modules is at least 40 man hours of
>>  testing to make sure they work correctly. That includes
>> evaluation
>> >of
>>  float usage, handling of NULL data, comparison of results with
>> and
>>  without the -m flag. Testing should be done with both high-res
>> >(LIDAR)
>>  and low-res (e.g. SRTM) DEMs.
>> >>>
>> >>>
>> >>> Tests can be performed on artificial data that tests all aspects
>> of
>> >the
>> >>> algorithm. Tests that show the correctness of the algorithm for
>> >specific
>> >>> small cases should be preferred. However, large data should not be
>> >an
>> >>> obstacle to write a test.
>> >>
>> >> I agree, for tests during development, not for gunittests.
>> >>
>> >> From the examples I read, gunittests expect a specific output. If
>> the
>> >> expected output (obtained with an assumed correct version of the
>> >> module) is wrong, the gunittest is bogus. gunittests are ok to make
>> >> sure the output does not change, but not ok to make sure the output
>> >is
>> >> correct. Two random examples are r.stream.order and r.univar.
>> >
>> >
>> >I am not sure why are we discussing this, it's pretty obvious that
>> >gunittests can serve to a) test inputs/outputs b) catch changes in
>> >results (whether correct or incorrect) c) test correctness of
>> results.
>> >It just depends how you write them, and yes, for some modules c) is
>> >more difficult to implement than for others.
>>
>>
>> Well, I agree with Markus that unittests are not a panacea and that
>> we should not fall into 

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 05/10/16 17:14, Paulo van Breugel wrote:



On 05-10-16 16:50, Sören Gebbert wrote:



2016-10-05 16:46 GMT+02:00 Moritz Lennert
>:

On 05/10/16 16:29, Paulo van Breugel wrote:


On 05-10-16 15 :20, Moritz Lennert wrote:

Maybe we should ask the users of whether this distinction
between
modules and core and extensions is really relevant for
them, or
whether most are perfectly happy to just install extensions.


Since you are asking :-) , as an user, my main interest is in good
documentation and reproducible examples (which I can than also
use to
see if outputs make sense to me as an user). In that respect
there is no
inherent difference between core modules and extensions. What is
different is that many (most?) of the core functions are
accessible
through the menu. I personally don't find that very important,
especially with the modules tab giving fairly easy access to
extensions,
but I can imagine that for new / other users, especially those
more
inclined to menu-driven applications, this may make a difference.


Thank you, Paulo, for this feedback.

This means that finding a means to more easily integrate addon
modules into menus might be a task to keep in mind.

Maybe the startup script could check the .grass7/addons directory
and automatically create an 'addons' menu using [1] ? Maybe the
mecanism can be changed in a way to just allow to create toolboxes
and that these are automatically integrated into the main menu ?

Just brainstorming, here...


How about g.extension will create a menu entry based on the modules
keywords when it installs the module? And it will remove the entry
when the module gets removed?


These sound like interesting options. I don't have a very good idea what
would be the best approach, but I would caution against full freedom for
the module to install itself in any location in the menu. I am having in
mind how some QGIS addons create menu items, including in the top-level
menu, resulting in what for me is a more chaotic bloated environment. In
that sense, perhaps there could even be an option at installation to
prevent the module from creating a menu item.


The current toolboxes mecanism allows for a specific "user modules" menu 
which automatically contains anything which is in the 
.grass7/toolboxes/toolboxes.xml file. So if we just put all addons 
there, this might avoid the chaos you are talking about.


And as g.extension already does the work of reading the module 
description and keyword, it shouldn't be too complicated to integrate 
each new module into a toolbox named by the first keyword (raster, 
vector, imagery, etc) and to use its description as label.


Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Paulo van Breugel]

On 05-10-16 16:50, Sören Gebbert wrote:



2016-10-05 16:46 GMT+02:00 Moritz Lennert 
>:


On 05/10/16 16:29, Paulo van Breugel wrote:


On 05-10-16 15 :20, Moritz Lennert wrote:

Maybe we should ask the users of whether this distinction
between
modules and core and extensions is really relevant for
them, or
whether most are perfectly happy to just install extensions.


Since you are asking :-) , as an user, my main interest is in good
documentation and reproducible examples (which I can than also
use to
see if outputs make sense to me as an user). In that respect
there is no
inherent difference between core modules and extensions. What is
different is that many (most?) of the core functions are
accessible
through the menu. I personally don't find that very important,
especially with the modules tab giving fairly easy access to
extensions,
but I can imagine that for new / other users, especially those
more
inclined to menu-driven applications, this may make a difference.


Thank you, Paulo, for this feedback.

This means that finding a means to more easily integrate addon
modules into menus might be a task to keep in mind.

Maybe the startup script could check the .grass7/addons directory
and automatically create an 'addons' menu using [1] ? Maybe the
mecanism can be changed in a way to just allow to create toolboxes
and that these are automatically integrated into the main menu ?

Just brainstorming, here...


How about g.extension will create a menu entry based on the modules 
keywords when it installs the module? And it will remove the entry 
when the module gets removed?


These sound like interesting options. I don't have a very good idea what 
would be the best approach, but I would caution against full freedom for 
the module to install itself in any location in the menu. I am having in 
mind how some QGIS addons create menu items, including in the top-level 
menu, resulting in what for me is a more chaotic bloated environment. In 
that sense, perhaps there could even be an option at installation to 
prevent the module from creating a menu item.





Best regards
Soeren


Moritz


[1] https://grass.osgeo.org/grass70/manuals/wxGUI.toolboxes.html


___
grass-dev mailing list
grass-dev@lists.osgeo.org 
http://lists.osgeo.org/mailman/listinfo/grass-dev





___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 05/10/16 16:29, Paulo van Breugel wrote:


On 05-10-16 15:20, Moritz Lennert wrote:

Maybe we should ask the users of whether this distinction between
modules and core and extensions is really relevant for them, or
whether most are perfectly happy to just install extensions.


Since you are asking :-) , as an user, my main interest is in good
documentation and reproducible examples (which I can than also use to
see if outputs make sense to me as an user). In that respect there is no
inherent difference between core modules and extensions. What is
different is that many (most?) of the core functions are accessible
through the menu. I personally don't find that very important,
especially with the modules tab giving fairly easy access to extensions,
but I can imagine that for new / other users, especially those more
inclined to menu-driven applications, this may make a difference.



Thank you, Paulo, for this feedback.

This means that finding a means to more easily integrate addon modules 
into menus might be a task to keep in mind.


Maybe the startup script could check the .grass7/addons directory and 
automatically create an 'addons' menu using [1] ? Maybe the mecanism can 
be changed in a way to just allow to create toolboxes and that these are 
automatically integrated into the main menu ?


Just brainstorming, here...

Moritz


[1] https://grass.osgeo.org/grass70/manuals/wxGUI.toolboxes.html
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Sören Gebbert]

2016-10-05 16:46 GMT+02:00 Moritz Lennert :

> On 05/10/16 16:29, Paulo van Breugel wrote:
>
>>
>> On 05-10-16 15:20, Moritz Lennert wrote:
>>
>>> Maybe we should ask the users of whether this distinction between
>>> modules and core and extensions is really relevant for them, or
>>> whether most are perfectly happy to just install extensions.
>>>
>>
>> Since you are asking :-) , as an user, my main interest is in good
>> documentation and reproducible examples (which I can than also use to
>> see if outputs make sense to me as an user). In that respect there is no
>> inherent difference between core modules and extensions. What is
>> different is that many (most?) of the core functions are accessible
>> through the menu. I personally don't find that very important,
>> especially with the modules tab giving fairly easy access to extensions,
>> but I can imagine that for new / other users, especially those more
>> inclined to menu-driven applications, this may make a difference.
>>
>>
> Thank you, Paulo, for this feedback.
>
> This means that finding a means to more easily integrate addon modules
> into menus might be a task to keep in mind.
>
> Maybe the startup script could check the .grass7/addons directory and
> automatically create an 'addons' menu using [1] ? Maybe the mecanism can be
> changed in a way to just allow to create toolboxes and that these are
> automatically integrated into the main menu ?
>
> Just brainstorming, here...
>

How about g.extension will create a menu entry based on the modules
keywords when it installs the module? And it will remove the entry when the
module gets removed?

Best regards
Soeren


>
> Moritz
>
>
> [1] https://grass.osgeo.org/grass70/manuals/wxGUI.toolboxes.html
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Paulo van Breugel]

On 05-10-16 15:20, Moritz Lennert wrote:

On 05/10/16 14:24, Sören Gebbert wrote:

Hi,

2016-10-05 10:20 GMT+02:00 Moritz Lennert >:

[sent this from the wrong address, so it didn't get through to 
the list]



 Message d'origine 
Envoyé : 5 octobre 2016 00:41:20 GMT+02:00



Le 4 octobre 2016 22:55:35 GMT+02:00, "Anna Petrášová"
> a écrit :
>On Tue, Oct 4, 2016 at 4:22 PM, Markus Metz
>> wrote:
>> On Tue, Oct 4, 2016 at 5:42 PM, Sören Gebbert
>> > wrote:
>>> Hi,


 >
 > You are very welcome to write the missing tests for core 
modules.

 >
 > However, i don't understand the argument that because many 
core

>modules
 > have
 > no tests, therefore new modules don't need them. If 
developers of

>addon
 > module are serious about the attempt to make their modules 
usable

>and
 > maintainable for others, then they have to implement 
tests. Its

>and
 > integral
 > part of the development process and GRASS has a beautiful 
test

 > environment
 > hat makes writing tests easy. Tests and documentation are 
part of

>coding
 > and
 > not something special. I don't think this is a hard 
requirement.

 >
 > There is a nice statement that is not far from the truth:
>Untested code
 > is
 > broken code.

 these gunittests only test if a module output stays the 
same. This

>>>
>>>
>>> This is simply wrong, please read the gunittest documentation.
>>
>> but then why does
>>>
>>> The gunittest for the v.stream.order addon is an example how its
>done:
>>>
>https://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.stream.order/testsuite/test_stream_order.py

>>
>> assume certain order numbers for features 4 and 7? What if these
>order
>> numbers are wrong?
>>
>> Recently I fixed bugs in r.stream.order, related to stream length
>> calculations which are in turn used to determine stream 
orders. The

>> gunittest did not pick up 1) the bugs, 2) the bug fixes.
>>
>>>
>>> You can write gunittests that will test every flag, every 
option,

>their
>>> combination and any output of a module. I have implemented 
plenty of

>tests,
>>> that check for correct error handling. Writing tests is 
effort, but

>you have
>>> to do it anyway. Why not implementing a gunittest for every 
feature

>while
>>> developing the module?


 My guess for the r.stream.* modules is at least 40 man hours of
 testing to make sure they work correctly. That includes 
evaluation

>of
 float usage, handling of NULL data, comparison of results 
with and

 without the -m flag. Testing should be done with both high-res
>(LIDAR)
 and low-res (e.g. SRTM) DEMs.
>>>
>>>
>>> Tests can be performed on artificial data that tests all 
aspects of

>the
>>> algorithm. Tests that show the correctness of the algorithm for
>specific
>>> small cases should be preferred. However, large data should 
not be

>an
>>> obstacle to write a test.
>>
>> I agree, for tests during development, not for gunittests.
>>
>> From the examples I read, gunittests expect a specific output. 
If the

>> expected output (obtained with an assumed correct version of the
>> module) is wrong, the gunittest is bogus. gunittests are ok to 
make
>> sure the output does not change, but not ok to make sure the 
output

>is
>> correct. Two random examples are r.stream.order and r.univar.
>
>
>I am not sure why are we discussing this, it's pretty obvious that
>gunittests can serve to a) test inputs/outputs b) catch changes in
>results (whether correct or incorrect) c) test correctness of 
results.

>It just depends how you write them, and yes, for some modules c) is
>more difficult to implement than for others.


Well, I agree with Markus that unittests are not a panacea and that
we should not fall into the trap of thinking that these tests will
guarantee that the results of our modules are correct.


Then i live in a parallel universe. Simple question: How do you test
your software? How do you assure the correct functionality of your
software? Why is it impossible to implement your approach of testing in
a dedicated gunittest? How do you assure software quality, if you don't
provide tools so 

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

On 05/10/16 14:24, Sören Gebbert wrote:

Hi,

2016-10-05 10:20 GMT+02:00 Moritz Lennert >:

[sent this from the wrong address, so it didn't get through to the list]


 Message d'origine 
Envoyé : 5 octobre 2016 00:41:20 GMT+02:00



Le 4 octobre 2016 22:55:35 GMT+02:00, "Anna Petrášová"
> a écrit :
>On Tue, Oct 4, 2016 at 4:22 PM, Markus Metz
>> wrote:
>> On Tue, Oct 4, 2016 at 5:42 PM, Sören Gebbert
>> > wrote:
>>> Hi,


 >
 > You are very welcome to write the missing tests for core modules.
 >
 > However, i don't understand the argument that because many core
>modules
 > have
 > no tests, therefore new modules don't need them. If developers of
>addon
 > module are serious about the attempt to make their modules usable
>and
 > maintainable for others, then they have to implement tests. Its
>and
 > integral
 > part of the development process and GRASS has a beautiful test
 > environment
 > hat makes writing tests easy. Tests and documentation are part of
>coding
 > and
 > not something special. I don't think this is a hard requirement.
 >
 > There is a nice statement that is not far from the truth:
>Untested code
 > is
 > broken code.

 these gunittests only test if a module output stays the same. This
>>>
>>>
>>> This is simply wrong, please read the gunittest documentation.
>>
>> but then why does
>>>
>>> The gunittest for the v.stream.order addon is an example how its
>done:
>>>

>https://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.stream.order/testsuite/test_stream_order.py


>>
>> assume certain order numbers for features 4 and 7? What if these
>order
>> numbers are wrong?
>>
>> Recently I fixed bugs in r.stream.order, related to stream length
>> calculations which are in turn used to determine stream orders. The
>> gunittest did not pick up 1) the bugs, 2) the bug fixes.
>>
>>>
>>> You can write gunittests that will test every flag, every option,
>their
>>> combination and any output of a module. I have implemented plenty of
>tests,
>>> that check for correct error handling. Writing tests is effort, but
>you have
>>> to do it anyway. Why not implementing a gunittest for every feature
>while
>>> developing the module?


 My guess for the r.stream.* modules is at least 40 man hours of
 testing to make sure they work correctly. That includes evaluation
>of
 float usage, handling of NULL data, comparison of results with and
 without the -m flag. Testing should be done with both high-res
>(LIDAR)
 and low-res (e.g. SRTM) DEMs.
>>>
>>>
>>> Tests can be performed on artificial data that tests all aspects of
>the
>>> algorithm. Tests that show the correctness of the algorithm for
>specific
>>> small cases should be preferred. However, large data should not be
>an
>>> obstacle to write a test.
>>
>> I agree, for tests during development, not for gunittests.
>>
>> From the examples I read, gunittests expect a specific output. If the
>> expected output (obtained with an assumed correct version of the
>> module) is wrong, the gunittest is bogus. gunittests are ok to make
>> sure the output does not change, but not ok to make sure the output
>is
>> correct. Two random examples are r.stream.order and r.univar.
>
>
>I am not sure why are we discussing this, it's pretty obvious that
>gunittests can serve to a) test inputs/outputs b) catch changes in
>results (whether correct or incorrect) c) test correctness of results.
>It just depends how you write them, and yes, for some modules c) is
>more difficult to implement than for others.


Well, I agree with Markus that unittests are not a panacea and that
we should not fall into the trap of thinking that these tests will
guarantee that the results of our modules are correct.


Then i live in a parallel universe. Simple question: How do you test
your software? How do you assure the correct functionality of your
software? Why is it impossible to implement your approach of testing in
a dedicated gunittest? How do you assure software quality, if you don't
provide tools so that other developers are able to test your software
for correctness? 

Re: [GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Sören Gebbert]

Hi,

2016-10-05 10:20 GMT+02:00 Moritz Lennert :

> [sent this from the wrong address, so it didn't get through to the list]
>
>
>  Message d'origine 
> Envoyé : 5 octobre 2016 00:41:20 GMT+02:00
>
>
>
> Le 4 octobre 2016 22:55:35 GMT+02:00, "Anna Petrášová" <
> kratocha...@gmail.com> a écrit :
> >On Tue, Oct 4, 2016 at 4:22 PM, Markus Metz
> > wrote:
> >> On Tue, Oct 4, 2016 at 5:42 PM, Sören Gebbert
> >>  wrote:
> >>> Hi,
> 
> 
>  >
>  > You are very welcome to write the missing tests for core modules.
>  >
>  > However, i don't understand the argument that because many core
> >modules
>  > have
>  > no tests, therefore new modules don't need them. If developers of
> >addon
>  > module are serious about the attempt to make their modules usable
> >and
>  > maintainable for others, then they have to implement tests. Its
> >and
>  > integral
>  > part of the development process and GRASS has a beautiful test
>  > environment
>  > hat makes writing tests easy. Tests and documentation are part of
> >coding
>  > and
>  > not something special. I don't think this is a hard requirement.
>  >
>  > There is a nice statement that is not far from the truth:
> >Untested code
>  > is
>  > broken code.
> 
>  these gunittests only test if a module output stays the same. This
> >>>
> >>>
> >>> This is simply wrong, please read the gunittest documentation.
> >>
> >> but then why does
> >>>
> >>> The gunittest for the v.stream.order addon is an example how its
> >done:
> >>>
> >https://trac.osgeo.org/grass/browser/grass-addons/grass7/
> vector/v.stream.order/testsuite/test_stream_order.py
> >>
> >> assume certain order numbers for features 4 and 7? What if these
> >order
> >> numbers are wrong?
> >>
> >> Recently I fixed bugs in r.stream.order, related to stream length
> >> calculations which are in turn used to determine stream orders. The
> >> gunittest did not pick up 1) the bugs, 2) the bug fixes.
> >>
> >>>
> >>> You can write gunittests that will test every flag, every option,
> >their
> >>> combination and any output of a module. I have implemented plenty of
> >tests,
> >>> that check for correct error handling. Writing tests is effort, but
> >you have
> >>> to do it anyway. Why not implementing a gunittest for every feature
> >while
> >>> developing the module?
> 
> 
>  My guess for the r.stream.* modules is at least 40 man hours of
>  testing to make sure they work correctly. That includes evaluation
> >of
>  float usage, handling of NULL data, comparison of results with and
>  without the -m flag. Testing should be done with both high-res
> >(LIDAR)
>  and low-res (e.g. SRTM) DEMs.
> >>>
> >>>
> >>> Tests can be performed on artificial data that tests all aspects of
> >the
> >>> algorithm. Tests that show the correctness of the algorithm for
> >specific
> >>> small cases should be preferred. However, large data should not be
> >an
> >>> obstacle to write a test.
> >>
> >> I agree, for tests during development, not for gunittests.
> >>
> >> From the examples I read, gunittests expect a specific output. If the
> >> expected output (obtained with an assumed correct version of the
> >> module) is wrong, the gunittest is bogus. gunittests are ok to make
> >> sure the output does not change, but not ok to make sure the output
> >is
> >> correct. Two random examples are r.stream.order and r.univar.
> >
> >
> >I am not sure why are we discussing this, it's pretty obvious that
> >gunittests can serve to a) test inputs/outputs b) catch changes in
> >results (whether correct or incorrect) c) test correctness of results.
> >It just depends how you write them, and yes, for some modules c) is
> >more difficult to implement than for others.
>
>
> Well, I agree with Markus that unittests are not a panacea and that we
> should not fall into the trap of thinking that these tests will guarantee
> that the results of our modules are correct.
>

Then i live in a parallel universe. Simple question: How do you test your
software? How do you assure the correct functionality of your software? Why
is it impossible to implement your approach of testing in a dedicated
gunittest? How do you assure software quality, if you don't provide tools
so that other developers are able to test your software for correctness?
Regression tests are not possible then, because the effect of changes in
the core libraries can not be easily detected in modules without tests.

Can you explain to me why the developers of the sophisticated software
system VTK [1] implement unit and integration tests for all software
components to assure the correct functionality of the framework? They
didn't saw the trap? They are delusional to think that tests assure
software quality?

Why is test driven development [2] an integral part of agile software

[GRASS-dev] Fwd: Re: Upcoming 7.2.0: review which addons to move to core

[Moritz Lennert]

[sent this from the wrong address, so it didn't get through to the list]


 Message d'origine 
Envoyé : 5 octobre 2016 00:41:20 GMT+02:00



Le 4 octobre 2016 22:55:35 GMT+02:00, "Anna Petrášová"  
a écrit :
>On Tue, Oct 4, 2016 at 4:22 PM, Markus Metz
> wrote:
>> On Tue, Oct 4, 2016 at 5:42 PM, Sören Gebbert
>>  wrote:
>>> Hi,


 >
 > You are very welcome to write the missing tests for core modules.
 >
 > However, i don't understand the argument that because many core
>modules
 > have
 > no tests, therefore new modules don't need them. If developers of
>addon
 > module are serious about the attempt to make their modules usable
>and
 > maintainable for others, then they have to implement tests. Its
>and
 > integral
 > part of the development process and GRASS has a beautiful test
 > environment
 > hat makes writing tests easy. Tests and documentation are part of
>coding
 > and
 > not something special. I don't think this is a hard requirement.
 >
 > There is a nice statement that is not far from the truth:
>Untested code
 > is
 > broken code.

 these gunittests only test if a module output stays the same. This
>>>
>>>
>>> This is simply wrong, please read the gunittest documentation.
>>
>> but then why does
>>>
>>> The gunittest for the v.stream.order addon is an example how its
>done:
>>>
>https://trac.osgeo.org/grass/browser/grass-addons/grass7/vector/v.stream.order/testsuite/test_stream_order.py
>>
>> assume certain order numbers for features 4 and 7? What if these
>order
>> numbers are wrong?
>>
>> Recently I fixed bugs in r.stream.order, related to stream length
>> calculations which are in turn used to determine stream orders. The
>> gunittest did not pick up 1) the bugs, 2) the bug fixes.
>>
>>>
>>> You can write gunittests that will test every flag, every option,
>their
>>> combination and any output of a module. I have implemented plenty of
>tests,
>>> that check for correct error handling. Writing tests is effort, but
>you have
>>> to do it anyway. Why not implementing a gunittest for every feature
>while
>>> developing the module?


 My guess for the r.stream.* modules is at least 40 man hours of
 testing to make sure they work correctly. That includes evaluation
>of
 float usage, handling of NULL data, comparison of results with and
 without the -m flag. Testing should be done with both high-res
>(LIDAR)
 and low-res (e.g. SRTM) DEMs.
>>>
>>>
>>> Tests can be performed on artificial data that tests all aspects of
>the
>>> algorithm. Tests that show the correctness of the algorithm for
>specific
>>> small cases should be preferred. However, large data should not be
>an
>>> obstacle to write a test.
>>
>> I agree, for tests during development, not for gunittests.
>>
>> From the examples I read, gunittests expect a specific output. If the
>> expected output (obtained with an assumed correct version of the
>> module) is wrong, the gunittest is bogus. gunittests are ok to make
>> sure the output does not change, but not ok to make sure the output
>is
>> correct. Two random examples are r.stream.order and r.univar.
>
>
>I am not sure why are we discussing this, it's pretty obvious that
>gunittests can serve to a) test inputs/outputs b) catch changes in
>results (whether correct or incorrect) c) test correctness of results.
>It just depends how you write them, and yes, for some modules c) is
>more difficult to implement than for others.


Well, I agree with Markus that unittests are not a panacea and that we should 
not fall into the trap of thinking that these tests will guarantee that the 
results of our modules are correct.

However, I do agree that these tests are useful in detecting if any changes to 
the code change the output, thus raising a flag that the developer has to at 
least take into account.

I'll try to write some tests for the OBIA tools when I find the time, although 
I do agree with Markus that it wouldn't be useful to try to write tests that 
would cover each and every possible corner case...

In the meantime, g.extension is wonderful tool 

Moritz

>
>Anna
>
>>
>> Markus M
>>
>>>
>>> Best regards
>>> Soeren
>>>

 my2c

 Markus M

 >
 > Best
 > Sören
 >
 >>
 >> One thing we could think about is activating the toolbox idea a
>bit
 >> more
 >> and creating a specific OBIA toolbox in addons.
 >>
 >>> Identified candidates could be added to core once they fulfill
>the
 >>> requirements above. Would that happen only in minor releases or
>would
 >>> that also be possible in point releases?
 >>
 >>
 >> Adding modules to core is not an API change, so I don't see why
>they
 >> can't
 >> be added at any time. But then again, having a series of new
>modules
 >> can be
 >> sufficient