Hi, Once upon a time, testing flashrom was simple: Read, erase, write, verify, done.
Over the past few months, we've introduced some very important features to Flashrom. Unfortunately, testing is no longer quite so simple. Partial writes require that we test various patterns (and we do with a newly introduced torture script), and we do that to some extend with the torture test that we recently checked in. Embedded controllers introduce a whole other world of pain. Long story short, a better testing strategy is needed. To get the ball rolling, I have a work-in-progress patch for the Chromium.org branch of Flashrom right now to introduce a unit testing framework and would appreciate thoughts / comments: http://codereview.chromium.org/5136001 . You can download the unified diff from there (as to avoid duplication here). It's written in shell script (tested on dash and bash) since I felt that's the lowest common denominator and the lowest barrier-to-entry for folks wanting to add/modify the tests. Feel free to post comments in this thread, since the code review will be closed once the code is checked in. The idea is simple: One master script does generic setup, backs up the firmware image, runs unit tests, and unconditionally restores the firmware image at the end. The user sets a few (hopefully no more than two or three) environment variables and runs the master script, specifying desired unit tests as arguments on the command line. Each unit test should be independent, and should have its own documentation in case it needs special environment variables or something. Due to the variety of tests which may be performed, it's up to the user to decide which tests to run on their hardware. If you're curious to try the patch, here is a simple invocation that will test partial writes for an x86 BIOS chip: ./do_tests.sh partial_writes_x86_bios.sh I've attached some sample console output for a test run that does partial writes for x86 BIOS chip and some write protection commands. In this example, I had to pass in some env variables and specify multiple tests as arguments. The invocation: FLASHROM="../flashrom" FLASHROM_PARAM="-p internal:bus=spi" sh do_tests.sh partial_writes_x86_bios.sh chip_size.sh wp-toggle.sh wp-range.sh If I were to test some EC stuff, I'd need to add additional parameters since my EC test script requires a special layout file and an secondary EC firmware image: LAYOUT_FILE=/tmp/ec_layout.txt ALT_EC_IMAGE=/tmp/alt_ec_image.bin FLASHROM="../flashrom" FLASHROM_PARAM="-p internal:bus=lpc" ./do_tests.sh partial_writes_ec.sh -- David Hendricks (dhendrix) Systems Software Engineer, Google Inc.
localhost test # FLASHROM="../flashrom" FLASHROM_PARAM="-p internal:bus=spi" sh do_tests.sh partial_writes_x86_bios.sh chip_size.sh wp-toggle.sh wp-range.sh testing flashrom binary: ../flashrom Running test in /tmp/flashrom_test.gSEAqbYQ5U Reading BIOS image flashrom v0.9.2 [email protected]/flashrom : c6c9f82 : Nov 03 2010 22:07:01 UTC on Linux 2.6.32.23+drm33.10 (i686), built with libpci 3.1.4, GCC 4.4.3, little endian sh: dmidecode: not found Original image saved as backup.bin Test cases: partial_writes_x86_bios.sh chip_size.sh wp-toggle.sh wp-range.sh Running test: "partial_writes_x86_bios.sh" ffh pattern written in ff_4k.bin 00h pattern written in 00_4k.bin Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS flashrom v0.9.2 [email protected]/flashrom : c6c9f82 : Nov 03 2010 22:07:01 UTC on Linux 2.6.32.23+drm33.10 (i686), built with libpci 3.1.4, GCC 4.4.3, little endian sh: dmidecode: not found Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS dmidecode execution unsucessfull - continuing without DMI info Reading flash... SUCCESS Running test: "chip_size.sh" Running test: "wp-toggle.sh" SUCCESS. SUCCESS. Running test: "wp-range.sh" w25_set_range: new status: 0x24 w25_set_range: new status: 0x00 Result: PASSED restoring original image using system's flashrom flashrom v0.9.2 [email protected]/flashrom : c6c9f82 : Nov 03 2010 22:07:01 UTC on Linux 2.6.32.23+drm33.10 (i686), built with libpci 3.1.4, GCC 4.4.3, little endian sh: dmidecode: not found dmidecode execution unsucessfull - continuing without DMI info Writing flash chip... Erasing flash before programming... done. Programming flash... done. Verifying flash... VERIFIED. SUCCESS test files remain in /tmp/flashrom_test.gSEAqbYQ5U
aligned region 0 test: passed aligned region 1 test: passed aligned region 2 test: passed aligned region 3 test: passed aligned region 4 test: passed aligned region 5 test: passed aligned region 6 test: passed aligned region 7 test: passed aligned region 8 test: passed aligned region 9 test: passed aligned region 10 test: passed aligned region 11 test: passed aligned region 12 test: passed aligned region 13 test: passed aligned region 14 test: passed aligned region 15 test: passed aligned region 0 test: passed aligned region 1 test: passed aligned region 2 test: passed aligned region 3 test: passed aligned region 4 test: passed aligned region 5 test: passed aligned region 6 test: passed aligned region 7 test: passed aligned region 8 test: passed aligned region 9 test: passed aligned region 10 test: passed aligned region 11 test: passed aligned region 12 test: passed aligned region 13 test: passed aligned region 14 test: passed aligned region 15 test: passed
old start: 0x00000000, old length: 0x00000000 attempting to set write protect range: start=0x000000 0x010000 ./wp-range.sh: passed
old write protect status: 0 ./wp-toggle.sh: passed
_______________________________________________ flashrom mailing list [email protected] http://www.flashrom.org/mailman/listinfo/flashrom
