Hi John, I fully agree, testing a subset of AmForth in some sort of emulation is absolutely not a substitute for tests on actual hardware. What got me excited is the possibility of running the emulated test subset on some of the (free) CI platforms out there. I'm confident I could get github run the suite fully automatically on every commit I push. It could serve as an early warning system that my changes are breaking something I didn't expect. Manual tests will of course always be part of the development process.
Best regards, Martin On Sat, Jan 10, 2026 at 8:39 PM John Sarabacha <[email protected]> wrote: > Hi, > My approach to testing amForth is somewhat different. The MounStudio makes > it easier to deploy/debug the software on the chip level from Windows 11 > instead of using QEMU. > Testing the core software for amForth whether ARM32 or RISCV is going to be > similar, the way I am handling this is the riscv/words (arm/words) > directory contains the assembler code for the words 73 files 1 file for > each base word (same for arm). I am checking each one, some are visually > easily verifiable (e.g contain 1 or 2 assembler instructions). Then I will > check the runtime results of each of these 73 words. If those results are > correct (above) the (DOCOLLEN) words in shared/words directory should also > work since they use those base words. Same would be true for the > appl/.../words. > This is just a start, a runtime combination of words could still fail. That > is why an automated means of testing can help. > At some point I will be using CLIPS (expert system shell) to help this > testing process, CLIPS based rules will generate sequences of XT byte codes > and examine the results of execution to analyse and help diagnose problems. > > Regards, > John S > > > > > > > On Sat, Jan 10, 2026 at 3:18 PM Martin Kobetic <[email protected]> wrote: > > > Hello again, > > > > I keep running into basic things being broken on the ARM architecture; > > yesterday I found out that even division doesn't work correctly. It won't > > be hard to fix, but it made me yearn for the ability to run the standard > > Forth test suite against the ARM build. > > > > I noticed that the test/ directory is attempting something similar and > > arguably more ambitious by running a test suite against actual boards, > > which is certainly very useful, but I'm after something much more basic. > > Any 32-bit arm based target should do to test the core words. > > > > So I made another attempt at getting the linux-arm build running. I still > > haven't figured out how to make it run under bare QEMU (I'm lost in the > > various details that are required to emulate a Raspbery PI, I will get > > there eventually), but I did get it running in an emulated 32-bit ARM > > container. The Makefile on my branch shows the details how > > > > > https://github.com/mkobetic/amforth/blob/unor4/appl/linux-arm/Makefile#L36-L57 > > > > With that I was able to attempt the core test suite run. Running `make > > test` currently ends on this sour note: > > --- > > ... > > > TESTING BOOLEANS: INVERT AND OR XOR > > ok > > > > > ok > > > T{ 0 0 AND -> 0 }T > > ok > > > T{ 0 1 AND -> 0 }T > > ok > > > T{ 1 0 AND -> 0 }T > > ok > > > T{ 1 1 AND -> 1 }T > > qemu: uncaught target signal 11 (Segmentation fault) - core dumped > > Segmentation fault > > --- > > This is to be expected because variables are still broken on ARM and they > > are critical for the test framework to function. Frankly I'm surprised it > > managed to get that far. > > > > But anyway, what gets me excited is how easy it is to run the tests and > > collect results with the linux-arm build. With a bit of programming I can > > process the results and output pass/fail stats and make the job pass or > > fail based on that. From there it shouldn't be hard to get a fully > > automated CI run on any platform that supports 32-bit ARM (or at least > some > > sort of emulation). It looks like Github Actions should accommodate that > > reasonably easily. I'm not sure how hard it would be to get the same for > > RISC-V, but I'd be surprised if there weren't any (free) options for that > > somewhere. A Raspbery PI style linux-riscv build could probably be very > > similar to the linux-arm one. > > > > Anyway, please let me know if I'm rediscovering the wheel here. Any hints > > or pointers to previous efforts in this direction would be very welcome. > > > > Cheers, > > > > Martin > > > > _______________________________________________ > > Amforth-devel mailing list for http://amforth.sf.net/ > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/amforth-devel > > > > _______________________________________________ > Amforth-devel mailing list for http://amforth.sf.net/ > [email protected] > https://lists.sourceforge.net/lists/listinfo/amforth-devel > _______________________________________________ Amforth-devel mailing list for http://amforth.sf.net/ [email protected] https://lists.sourceforge.net/lists/listinfo/amforth-devel
