Hi all To aid in development and refactoring of the numerous reports, I've developed a report-options stress test.
*Premise:* Most reports are currently defined in scheme. The reporting engine expects, at a minimum, 2 functions to be defined - an options generator, and a renderer. The options generator will create a blank (gnc:report-options) object, modify it by adding various options eg (gnc:make-string-option), (gnc:make-simple-boolean-option), and return the options object to the engine. This forms the basis of the report options UI. The renderer will accept a report ID, and the options object. The renderer will read the options used, query the database, and present a gnc:html-document object to the engine. I'm not sure what happens afterwards; the engine will render the html-document in Webkit. *Refactoring* Most of the report infrastructure was designed about 15-20 years ago... this was using scheme/lisp conventions - bearing in mind LISP was designed in the 1970s, and modern scripting engines such as python etc were nascent - nonetheless scheme was chosen, for reasons that are apparent to schemers only -- pythonistas will argue otherwise, but I do now that scheme is a more flexible language to design financial reports; scheme has also evolved in the past 15 years to become a powerful compilable language... Guile is no different and will continue to evolve in a positive direction. Unfortunately the state of Gnucash scheme code as of 2018 is stuck in the past; for various reasons they have been kept on life support, limping forward via every bugzilla report, and very difficult to understand and create new types of reports. It would be good to refactor the numerous scheme functions but, as some users have found out, refactoring the cryptic code with good intentions do create some unusual report crashes. *Proposal* I have designed a mechanism whereby most standard reports in Gnucash have been queried, and their options (string/boolean/etc) dumped, and a test-stress-options is run continuously... This will repeatedly render the options with various combinations of options, simply testing whether the report will actually run. It does not verify the contents of the report. It's designed to catch errors whereby e.g. finding periodic average on a regular report is working well, but a refactored report whereby calculating the average will fail on an empty book causing (total amount / num columns) will trigger a division by zero. The test-stress-options will come in 2 guises: The first will perform a simple-stress-test, whereby all options are set to successive values... if a report has 3 booleans (t/f) and 1 multichoice with 4 values (a/b/c/d), it will run them with: (t,t,t,a), (f,f,f,b), (t,t,t,c), (f,f,f,d) - this is fairly simple. The second test will perform a combinatorial stress test, whereby the number of options is sent to a local build of jenny from http://burtleburtle.net/bob/math/jenny.html (thanks rgmerk); guile will subprocess "jenny 2 2 2 4" which will generate an efficient list of options (1a 2b 3a 4a), (1b 2b 3a 4b) etc, which is then interpreted by guile to run the report repeatedly, hopefully catching more errors. Either test is first run on an empty book, then on a book with sample transactions. On my laptop, a Intel i5, the simple-stress-test compiles in 1s and runs most standard reports in 5 seconds and the combinatorial-stress-test compiles in 8s and runs them in 20s. See its output at https://gist.github.com/christopherlam/c04d43398ecac486d662c4e74eb293ca Some reports e.g. multicolumn, hello-world, and eguile-based reports refuse to be run, and are excluded. IMHO this strategy, although complicated, is a safe and effective way to ensure that refactoring internal functions do not lead to unexpected report crashes. I'd think the simple stress test would be sufficient for daily use, and combinatorics stress test prior to release. Code currently lives on at https://github.com/Gnucash/gnucash/pull/378 Comments welcome! _______________________________________________ gnucash-devel mailing list [email protected] https://lists.gnucash.org/mailman/listinfo/gnucash-devel
