Dear AmForthers,
Monday evening doing a little AmForth related stuff, how is that?
Full disclosure: I am using emacs/org-mode. So I created a habit
for Monday. And boy those red colored bars saying "not done"
become annoying quickly. So better do something :)
Contemplating the Test Cases Thing
> 2020-11-02 I wrote:
> 3. Add testing scripts --- I have a set of scripts for that, but I
> have not run this stuff yet. However, in my opinion adding a
> working test suite is far more important at the moment, than
> anything else.
>
> This includes preparing some hardware with 4 relevant target boards
> in order to simplify the process.
Turns out that the set of scripts isn't all that great. So:
What would I like to achieve?
There will be a ./test/ directory, which holds the test cases and
the files to programm my available target hw for the tests.
: cd test
: make targets # shall produce firmware for target boards
: make install $target # flash the particular target board
: make test $target # calls the runner on a given list of test cases
The result should be something like this:
: make test $target
: . loading tester on $target, ok.
: . 0/12 test/common/test-A.frt, ok
: . 1/19 test/common/test-B.frt, 1 error
: 1 error in 21 tests
The verbatim output goes to runner.log or something such.
*configuration*
- which target board do I want to use?
- which programmer and its paramters do I have?
- which details for the serial (or other?) connection?
*terminology*
- unit-under-test :: this is the function (asm of forth word) to
be tested.
: : unit-under-test ( stack -- effect )
: definition
: ;
These live in the directories arm, avr8, common, msp430,
risc-v, shared ... these trees should remain unchanged imho.
- test case :: this is a statement to load unit-under-test and
its possible dependencies, plus a collection of "t{ ... }t"
statements to exercise unit-under-test:
: #include unit-under-test.fs
: t{ stack function changed stack }t
Other forms of test procedures should be possible as long as
they produce output, which can be understood by the runner.
There are dragons lurking here when it comes to #include or
#require.
- runner :: this is a script which can talk to a target board
(via serial), feed a testcase file, collect the resulting
output and extract the results
- tester :: this is an implementation of the Hayes tester. It
defines 't{' '}t' and a few more words.
- firmware :: for every target board to be used, we require
a well equipped AmForth firmware. I expect this firmware to be
somewhat special and separate from the examples in the ./appl/
subdirectory.
There are possible complications known as setup/teardown. Markers
could possibly help. On the other hand, I have not yet exhausted
the flash or RAM of a atmega-644a.
*possible tree layout*
: amforth/trunk
: ...
: +-- test/
: +-- Makefile \ top of magic
: +-- M.programmer.mk \ make vars to specify my programmer
: +-- M.connection.mk \ make vars to specify serial connection
: +-- bin/ \ any tools
: | +-- runner.pl
: |
: +-- tester/ \ place for tester words? optional?
: |
: +-- firmware/
: | +-- avr8-at644p/
: | | +-- Makefile
: | | +-- dict_appl_core.inc
: | | +-- dict_appl.inc \ includes dict_local.inc
: | | +-- dict_local.inc \ all additional words to load in this
context
: | | +-- testfirmware.asm \ top level definition for target
firmware
: | | +-- words/
: | | +-- applturnkey.asm \ project specific
: | |
: | +-- $target-$controller/
: | | +-- Makefile
: | | +-- ...
: | ...
: |
: +-- cases/
: | +-- common/ \ testcases for all target boards
: | | +-- stack-test.frt
: | | ...
: | |
: | +-- avr8/ \ testcases for specific target boards
: | | +-- ...
: | ...
: |
: +-- logs/ \ place for logfiles?
Assuming that Mr.Someone has installed the relevant tools,
editing the M.*.mk files should be enough to
- produce firmware for at least one target
- install the firmware on said target
- load and evaluate the available tests
All of this in a few lines of shell commands.
Does that sound like a plausible plan?
Next step: create this structure for one target and a few test
cases and see, how we fare.
Cheers,
Erich
--
May the Forth be with you ...
_______________________________________________
Amforth-devel mailing list for http://amforth.sf.net/
Amforth-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amforth-devel
