Hi Robert and devel,

Thank you for pointers, this is useful. The examples below illustrate how to test multiple items together. But I think the complexity of the 52(and counting) options (37 binary, the rest are multiples/multichoice) means, I think it'll be better to handwrite tests. I'll need to reuse existing test frameworks to attempt maximise coverage.


C

On 31/01/18 09:01, Robert Merkel wrote:
Sorry for the tardy response - I went away for the long weekend and was
busy yesterday!

As Phil says, you can test both leaf functions and controller code with
unit tests. With controller code, you may need to write mocks for some or
all of the called code.  I don't know whether there are prewritten mocking
libraries for guile, but given the extreme flexibility of scheme it
shouldn't be difficult to write code to replace one function call with
another where necessary.

As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
there is no particular reason from a testing perspective to throw in an
intermediate function.

As far as which options to test, if testing all the combinations
exhaustively results in too many tests (and it sounds like it does) try
pairwise combinatoric testing, which works as follows:

Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
can either be "on" or "off" it has two values).

For any pair of options o_x and o_y, for all the possible combinations of
option values there should be at least one test that has that combination.

To give a very simple case, for three options {a, b, c}  each of which can
take two values {off, on}, you could have the following tests:

      Option   a     b     c
Test
1                 off  off   off
2                 off   on  off
3                 on   off   on
4                 on    on  off
5                 off   off    on

For the pair (a,b) tests 1 through 4 give all the possible combinations,
for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
combinations.

5 is the minimum number of tests that can meet the pairwise combinatoric
criterion, as compared to 8 if you tried all the possible combinations.
However, it becomes an even bigger win if you're trying lots of options:
you only need 9 tests for 8 options with two values, whereas you would need
256 tests to try every possible option combination!

To generate optimal pairwise test sets, you can use "jenny" (this is a
really useful but unmaintained tool, I really should take over the
maintenance):

http://burtleburtle.net/bob/math/jenny.html

Hope this helps, and feel free to ask more questions.  I will try to
respond more promptly!


On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam <christopher....@gmail.com>
wrote:

Dear Devel



To rgmerk: Welcome back, and it was a nice to meet irl!



While simplifying transaction.scm and thinking of unit testing, I now have
a conundrum worthy of an expert view.



The reports require 2 main functions – the options generator and the
renderer; the options generator generates a options.scm controller object,
and the renderer takes options and outputs html.



I understand unit testing to handle testing of ‘leaf’ functions e.g.
(split->date), rather than the controller code (e.g. renderer takes options
and outputs html) – but to me this is rather silly because split->date only
tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
actual functionality.



With regards to unit testing I can see several issues



    1. The refactored report has inlined most single-use functions into
    lambda expressions – I figured that directly stating (xaccTransGetDate
    (xaccSplitGetParent split)) is much more descriptive to a programmer than
    to create a testable leaf function (split->date split). I can see the
    benefits of both – leave as lambda expressions which will can be
    understandable by anyone who is familiar with the API, or break them out
    into 100s of single use functions which can be tested, but introduces a
    whole layer of cognitive load to anyone hacking code – (what does
    split->date actually do? Where is its definition). Also, breaking the
    lambda functions into testable functions means the implementation is frozen
    and the next hacker will have lesser scope to rework/optimise the report.



    1. The refactored report is now flexible enough to accommodate derived
    reports with a different multicolumn data function – eg
    income-gst-statement.scm has been reworked into a transaction.scm
    derivative which passes its own calculated-cells to report on GST sales and
    purchases. This is not yet committed.



    1. I think the most useful testing approach for a complex
    transaction.scm will be to test functions of various combinations of
    options values, and test the resulting html for satisfactory output. There
    are now dozens of bools and multichoices that can be triggered, each
    effecting html in various ways. How best to test?



    1. My view would be the unit test would check that:
       1. the TR actually exists
       2. it can display empty-report
       3. it can understand passing of custom-calculated-cells
       4. each of the options can be toggled, and the resulting html
       displays/hides cells/detail as expected
       5. and sorting options generate sorted rows



Comments welcome, I had no formal training ☹


_______________________________________________
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Reply via email to