This is a pretty big cleanup, still in progress, but I think these parts are ready to merge.
I think we should change the test "protocol" to be a subprocess rather sourcing the .test file, as you alluded to. But that can be done in a separate step on top of this. There's also some cruft like '. testing.sh' at the top of most tests, but it doesn't seem to exist. As long as I'm going in the right direction I can fix all this. Summary after patch 1 but before patch 2: Tested 66 toybox commands: 90 test failures Commands with test failures: bzcat chattr date find groupadd groupdel hostname ls modinfo mount mv pgrep pkill renice tar touch xzcat zcat (skipped commands not listed) After patch 2: Tested 61 toybox commands: 36 test failures Commands with test failures: date find ls modinfo mv pgrep pkill renice tar zcat Commands skipped: chattr chgrp chown groupadd groupdel hostname ifconfig losetup mount useradd make: *** [tests] Error 1 ----- [PATCH 1/2] Refactor the test harness to triage tests, and fix some bugs. genconfig.sh: - Fix a name clash bug where 'make test' would run all tests and then try to build the 'test' binary. 'make test_bin' is special cased for now. - Make the test targets .PHONY, since we're not creating files called test_sed, etc. - Use here docs to write Makefile fragments, for readability and to be consistent with the rest of the script. Refactor into functions. - Add comments. make.sh: - Add the output binary name as the first param. This fixes an issue where 'make; make test_sed' would leave you without a 'toybox' binary because single.sh created another toybox binary and renamed it to sed. runtest.sh: - Changed the 'testing' function to return 0. This fixes a bug where the return value of the script depends on the return code of the last command -- NOT whether it passed or failed. For example, 'make test_test' would fail with code 1, even though all tests passed (because 'test' exits 1 validly). On the other hand, tests with failures would still return 0 if the last command didn't fail. - Use local variables instead of globals since we are sharing globals with the tests themselves. - Add a common function to skip tests if not running as root, and keep track of skipped commands. test.sh: - Refactor it into separate functions/actions and add usage. - Fix a bug where tests would succeed under 'make test_$COMMAND' but fail under 'make test'. The problem is if you are running the xzcat command, 'tar' could be either the host binary or the toybox binary, depending on what tools are in the modified test $PATH. To fix this, the test harness finds the host binary and sets $HOST_BIN_TAR, etc. before invoking the test. - Keep track of the number of failures per *command*, and exit 1 if any failed - Use a distinct test directory for each command, instead of repeatedly setting up and tearing down the same one. - When running all tests, print out a summary at the end. single.sh: - Call scripts/make.sh directly instead of 'make'. - Add comments. --- Makefile | 2 +- scripts/genconfig.sh | 103 ++++++++++++++++++++++++++--------- scripts/make.sh | 16 ++++-- scripts/runtest.sh | 40 +++++++++----- scripts/single.sh | 28 +++++++--- scripts/test.sh | 149 +++++++++++++++++++++++++++++++++++++++++---------- 6 files changed, 258 insertions(+), 80 deletions(-) [PATCH 2/2] Various test fixes and cleanup. bzcat: Fix a test that was succeeding for the wrong reason. The test was looking for a 1 exit code in the case of bad data, but there was a typo in the input path, so it failed 1 for the wrong reason. bzcat, xzcat, zcat: Use $HOST_BIN_TAR to fix the issue where they fail under 'make test' but not 'make test_bzcat'. hostname: Use $HOST_BIN_HOSTNAME, and skip one test if not root. chgrp, groupadd, groupdel, ifconfig, losetup, mount, useradd: Skip if not root. pgrep, pkill: style cleanup and speedup of the tests (change sleep 1 to sleep 0.1). No fixes yet. renice: Fix some quoting problems (revealing further errors, not fixed) tar: Style cleanup in preparation to fix failures under 'make test' but not 'make test_tar' (due to host/toybox discrepancy) touch: Use $HOST_BIN_DATE because toybox doesn't support %N nanoseconds --- tests/bzcat.test | 41 ++++++++++--------- tests/chattr.test | 4 +- tests/chgrp.test | 7 +--- tests/chown.test | 7 +--- tests/groupadd.test | 2 + tests/groupdel.test | 2 + tests/hostname.test | 16 +++++--- tests/ifconfig.test | 7 +--- tests/losetup.test | 7 +--- tests/mount.test | 6 ++- tests/pgrep.test | 51 ++++++++++++------------ tests/pkill.test | 54 ++++++++++++------------- tests/renice.test | 8 ++-- tests/tar.test | 111 ++++++++++++++++++++++++++++++++-------------------- tests/touch.test | 6 +-- tests/useradd.test | 7 +--- tests/xzcat.test | 36 +++++++++-------- tests/zcat.test | 39 +++++++++--------- 18 files changed, 216 insertions(+), 195 deletions(-)
From 071d2224309f527737db323876102d593d3dc543 Mon Sep 17 00:00:00 2001 From: Andy Chu <[email protected]> Date: Fri, 18 Mar 2016 18:15:15 -0700 Subject: [PATCH 2/2] Various test fixes and cleanup. bzcat: Fix a test that was succeeding for the wrong reason. The test was looking for a 1 exit code in the case of bad data, but there was a typo in the input path, so it failed 1 for the wrong reason. bzcat, xzcat, zcat: Use $HOST_BIN_TAR to fix the issue where they fail under 'make test' but not 'make test_bzcat'. hostname: Use $HOST_BIN_HOSTNAME, and skip one test if not root. chgrp, groupadd, groupdel, ifconfig, losetup, mount, useradd: Skip if not root. pgrep, pkill: style cleanup and speedup of the tests (change sleep 1 to sleep 0.1). No fixes yet. renice: Fix some quoting problems (revealing further errors, not fixed) tar: Style cleanup in preparation to fix failures under 'make test' but not 'make test_tar' (due to host/toybox discrepancy) touch: Use $HOST_BIN_DATE because toybox doesn't support %N nanoseconds --- tests/bzcat.test | 41 ++++++++++--------- tests/chattr.test | 4 +- tests/chgrp.test | 7 +--- tests/chown.test | 7 +--- tests/groupadd.test | 2 + tests/groupdel.test | 2 + tests/hostname.test | 16 +++++--- tests/ifconfig.test | 7 +--- tests/losetup.test | 7 +--- tests/mount.test | 6 ++- tests/pgrep.test | 51 ++++++++++++------------ tests/pkill.test | 54 ++++++++++++------------- tests/renice.test | 8 ++-- tests/tar.test | 111 ++++++++++++++++++++++++++++++++-------------------- tests/touch.test | 6 +-- tests/useradd.test | 7 +--- tests/xzcat.test | 36 +++++++++-------- tests/zcat.test | 39 +++++++++--------- 18 files changed, 216 insertions(+), 195 deletions(-) diff --git a/tests/bzcat.test b/tests/bzcat.test index ef1b07f..d28d8a2 100755 --- a/tests/bzcat.test +++ b/tests/bzcat.test @@ -6,25 +6,30 @@ [ -f testing.sh ] && . testing.sh #testing "name" "command" "result" "infile" "stdin" + testing "overflow" \ - 'bzcat "$TOPDIR/files/bzcat/overflow.bz2" >/dev/null 2>/dev/null; - [ $? -eq 1 ] && echo good' "good\n" "" "" + 'bzcat "$TOPDIR/tests/files/bzcat/overflow.bz2" 2>&1; echo $?' \ + "bzcat: bad data\n1\n" "" "" echo "hello" > file -tar -cjf file.tar.bz2 file -# Get system bzcat -bzcatExe=`which bzcat` -$bzcatExe file.tar.bz2 > bzcatOut -testing "- decompresses a single file" "bzcat file.tar.bz2 > Tempfile && echo "yes"; diff Tempfile bzcatOut && echo "yes"; rm -rf file* bzcatOut Tempfile" "yes\nyes\n" "" "" +$HOST_BIN_TAR -cjf file.tar.bz2 file -#testing "name" "command" "result" "infile" "stdin" -echo "hello" > file1 -echo "hi" > file2 -echo "Hi, Good morning !! I am a bzcat tester" > file3 -tar -cjf file1.tar.bz2 file1 -tar -cjf file2.tar.bz2 file2 -tar -cjf file3.tar.bz2 file3 -# Get system bzcat -bzcatExe=`which bzcat` -$bzcatExe file1.tar.bz2 file2.tar.bz2 file3.tar.bz2 > bzcatOut -testing "- decompresses multiple files" "bzcat file1.tar.bz2 file2.tar.bz2 file3.tar.bz2 > Tempfile && echo "yes" ; diff Tempfile bzcatOut && echo "yes"; rm -rf file* bzcatOut Tempfile " "yes\nyes\n" "" "" +$HOST_BIN_BZCAT file.tar.bz2 > expected_out + +testing "decompresses a single file" \ + 'bzcat file.tar.bz2 > Tempfile && echo yes; diff Tempfile expected_out && echo yes' \ + "yes\nyes\n" "" "" + +echo "one" > file1 +echo "two" > file2 +echo "three" > file3 + +$HOST_BIN_TAR -cjf file1.tar.bz2 file1 +$HOST_BIN_TAR -cjf file2.tar.bz2 file2 +$HOST_BIN_TAR -cjf file3.tar.bz2 file3 + +$HOST_BIN_BZCAT file1.tar.bz2 file2.tar.bz2 file3.tar.bz2 > expected_out + +testing "decompresses multiple files" \ + 'bzcat file1.tar.bz2 file2.tar.bz2 file3.tar.bz2 > Tempfile && echo yes; diff Tempfile expected_out && echo yes' \ + "yes\nyes\n" "" "" diff --git a/tests/chattr.test b/tests/chattr.test index bbf8b09..d06cff7 100755 --- a/tests/chattr.test +++ b/tests/chattr.test @@ -2,9 +2,9 @@ [ -f testing.sh ] && . testing.sh -#testing "name" "command" "result" "infile" "stdin" +skip_if_not_root -# chattr - Testcases +#testing "name" "command" "result" "infile" "stdin" mkdir testattr IN="cd testattr" diff --git a/tests/chgrp.test b/tests/chgrp.test index 16cd776..0a98349 100755 --- a/tests/chgrp.test +++ b/tests/chgrp.test @@ -2,12 +2,7 @@ [ -f testing.sh ] && . testing.sh -if [ "$(id -u)" -ne 0 ] -then - echo "$SHOWSKIP: chgrp (not root)" - continue 2>/dev/null - exit -fi +skip_if_not_root # We chgrp between "root" and the last group in /etc/group. diff --git a/tests/chown.test b/tests/chown.test index 95ad727..df31d67 100755 --- a/tests/chown.test +++ b/tests/chown.test @@ -2,12 +2,7 @@ [ -f testing.sh ] && . testing.sh -if [ "$(id -u)" -ne 0 ] -then - echo "$SHOWSKIP: chown (not root)" - continue 2>/dev/null - exit -fi +skip_if_not_root # We chown between user "root" and the last user in /etc/passwd, # and group "root" and the last group in /etc/group. diff --git a/tests/groupadd.test b/tests/groupadd.test index 091dccf..a1ff56a 100755 --- a/tests/groupadd.test +++ b/tests/groupadd.test @@ -4,6 +4,8 @@ [ -f testing.sh ] && . testing.sh +skip_if_not_root + # 70 characters long string; hereafter, we will use it as per our need. _s70="abcdefghijklmnopqrstuvwxyz123456789abcdefghijklmnopqrstuvwxyz123456789" diff --git a/tests/groupdel.test b/tests/groupdel.test index ea9ee9a..67ebdda 100755 --- a/tests/groupdel.test +++ b/tests/groupdel.test @@ -5,6 +5,8 @@ [ -f testing.sh ] && . testing.sh +skip_if_not_root + # Redirecting all output to /dev/null for grep and delgroup arg="&>/dev/null" diff --git a/tests/hostname.test b/tests/hostname.test index 5107ce2..b6a0610 100755 --- a/tests/hostname.test +++ b/tests/hostname.test @@ -7,12 +7,16 @@ #testing "name" "command" "result" "infile" "stdin" -# Get system hostname -hostnameExe=`which hostname` -hostnameOut=`$hostnameExe` +# Use host command to get the hostname +expected_out=$($HOST_BIN_HOSTNAME) # New hostname -NewHostname="NewHostName.system" +new_hostname="NewHostName.system" -testing "get" "hostname" "$hostnameOut\n" "" "" -testing "set, Get and then Reset" "hostname $NewHostname; hostname; hostname $hostnameOut; hostname" "$NewHostname\n$hostnameOut\n" "" "" +testing "get" "hostname" "$expected_out\n" "" "" + +skip_if_not_root + +testing "set, get and reset" \ + "hostname $new_hostname; hostname; hostname $expected_out; hostname" \ + "$new_hostname\n$expected_out\n" "" "" diff --git a/tests/ifconfig.test b/tests/ifconfig.test index 34fcde0..6583305 100755 --- a/tests/ifconfig.test +++ b/tests/ifconfig.test @@ -21,12 +21,7 @@ [ -f testing.sh ] && . testing.sh -if [ "$(id -u)" -ne 0 ] -then - echo "$SHOWSKIP: ifconfig (not root)" - continue 2>/dev/null - exit -fi +skip_if_not_root #testing "name" "command" "result" "infile" "stdin" diff --git a/tests/losetup.test b/tests/losetup.test index 651c693..930dd4a 100755 --- a/tests/losetup.test +++ b/tests/losetup.test @@ -2,12 +2,7 @@ [ -f testing.sh ] && . testing.sh -if [ "$(id -u)" -ne 0 ] -then - echo "$SHOWSKIP: losetup (not root)" - continue 2>/dev/null - exit -fi +skip_if_not_root #testing "name" "command" "result" "infile" "stdin" diff --git a/tests/mount.test b/tests/mount.test index 6590b94..9e376ca 100755 --- a/tests/mount.test +++ b/tests/mount.test @@ -5,6 +5,8 @@ [ -f testing.sh ] && . testing.sh +skip_if_not_root + #testing "name" "command" "result" "infile" "stdin" root_fs=`df | grep "\/$" | awk '{print $1}'` @@ -56,11 +58,11 @@ reCreateTmpFs mkdir -p testDir1/testDir2 testDir echo "abcdefghijklmnopqrstuvwxyz" > testDir1/testDir2/testFile testing "-o bind dir1 dir2" \ - 'mount -o bind testDir1 testDir >/dev/null 2>&1 && \ + 'mount -o bind testDir1 testDir && \ cat testDir/testDir2/testFile && sleep 1 && umount testDir' \ "abcdefghijklmnopqrstuvwxyz\n" "" "" testing "-o rbind dir1 dir2" \ - 'mount -o rbind testDir1 testDir >/dev/null 2>&1 && \ + 'mount -o rbind testDir1 testDir && \ cat testDir/testDir2/testFile && sleep 1 && umount testDir' \ "abcdefghijklmnopqrstuvwxyz\n" "" "" testing "-o loop $tmp_b_fs /mnt" \ diff --git a/tests/pgrep.test b/tests/pgrep.test index 753a55a..62cec34 100755 --- a/tests/pgrep.test +++ b/tests/pgrep.test @@ -1,33 +1,32 @@ -#!/bin/bash - # Copyright 2013 Divya Kothari <[email protected]> # Copyright 2013 Robin Mittal <[email protected]> -#cleaning 'yes' processes -killall yes >/dev/null 2>&1 +# Clean up other 'yes' processes. TODO: Don't mess with the host system. +killall yes + +# Starting processes to test pgrep command +yes >/dev/null & + +pid=$! +echo "PID $pid started" -[ -f testing.sh ] && . testing.sh +sleep 0.1 + +parent_pid=$(awk '{ print $4 }' /proc/$pid/stat) +echo "Parent of process $pid is $parent_pid" #testing "name" "command" "result" "infile" "stdin" -# Starting processes to test pgrep command -yes >/dev/null & -proc=$! -#echo "# Process created with id: $proc" -sleep .1 -session_id=0 -proc_parent=`cat /proc/${proc}/stat | awk '{ print $4 }'` -#echo "# Parent Process id of $proc is $proc_parent" - -# Testcases for pgrep command -testing "pattern" "pgrep yes" "$proc\n" "" "" -testing "wildCardPattern" "pgrep ^y.*s$" "$proc\n" "" "" -testing "-l pattern" "pgrep -l yes" "$proc yes\n" "" "" -testing "-f pattern" "pgrep -f yes" "$proc\n" "" "" -testing "-n pattern" "pgrep -n yes" "$proc\n" "" "" -testing "-o pattern" "pgrep -o yes" "$proc\n" "" "" -testing "-s" "pgrep -s $session_id yes" "$proc\n" "" "" -testing "-P" "pgrep -P $proc_parent yes" "$proc\n" "" "" - -#Clean-up -killall yes >/dev/null 2>&1 +testing "pattern" "pgrep yes" "$pid\n" "" "" +testing "wildCardPattern" "pgrep ^y.*s$" "$pid\n" "" "" +testing "-l pattern" "pgrep -l yes" "$pid yes\n" "" "" +testing "-f pattern" "pgrep -f yes" "$pid\n" "" "" +testing "-n pattern" "pgrep -n yes" "$pid\n" "" "" +testing "-o pattern" "pgrep -o yes" "$pid\n" "" "" +testing "-P" "pgrep -P $parent_pid yes" "$pid\n" "" "" + +# NOTE: Fails with pgrep: bad -s '0' +testing "-s" "pgrep -s 0 yes" "$pid\n" "" "" + +# Cleanup +kill $pid diff --git a/tests/pkill.test b/tests/pkill.test index 7a20ae5..359c30c 100755 --- a/tests/pkill.test +++ b/tests/pkill.test @@ -1,10 +1,8 @@ -#!/bin/bash - # Copyright 2013 Divya Kothari <[email protected]> # Copyright 2013 Robin Mittal <[email protected]> -#cleaning 'yes' processes -killall yes >/dev/null 2>&1 +# Clean up other 'yes' processes. TODO: Don't mess with the host system. +killall yes [ -f testing.sh ] && . testing.sh @@ -13,42 +11,42 @@ killall yes >/dev/null 2>&1 # Testcases for pkill command yes >/dev/null & -sleep 1 -testing "pattern" "pkill yes && sleep 1 && (pgrep yes || echo 'yes')" \ +sleep 0.1 +testing "pattern" "pkill yes && sleep 0.1 && (pgrep yes || echo 'yes')" \ "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & yes print1 >/dev/null & yes print2 >/dev/null & -sleep 1 -testing "pattern (multiple)" "pkill yes && sleep 1 && +sleep 0.1 +testing "pattern (multiple)" "pkill yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & -sleep 1 -testing "-f pattern (one)" "pkill -f yes && sleep 1 && +sleep 0.1 +testing "-f pattern (one)" "pkill -f yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes print1 >/dev/null & -sleep 1 -testing "-f pattern args" "pkill -f \"yes print1\" && sleep 1 && +sleep 0.1 +testing "-f pattern args" "pkill -f \"yes print1\" && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & yes print1 >/dev/null & yes print2 >/dev/null & -sleep 1 -testing "-f pattern (multiple)" "pkill -f yes && sleep 1 && +sleep 0.1 +testing "-f pattern (multiple)" "pkill -f yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & -sleep 1 -testing "-s 0 -f pattern (regexp)" "pkill -s 0 -f ye* && sleep 1 && +sleep 0.1 +testing "-s 0 -f pattern (regexp)" "pkill -s 0 -f ye* && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 @@ -56,8 +54,8 @@ yes >/dev/null & proc1=$! yes >/dev/null & proc2=$! -sleep 1 -testing "-n pattern" "pkill -n yes && sleep 1 && pgrep yes" \ +sleep 0.1 +testing "-n pattern" "pkill -n yes && sleep 0.1 && pgrep yes" \ "$proc1\n" "" "" killall yes >/dev/null 2>&1 @@ -65,36 +63,36 @@ yes >/dev/null & proc1=$! yes >/dev/null & proc2=$! -sleep 1 -testing "-o pattern" "pkill -o yes && sleep 1 && pgrep yes" \ +sleep 0.1 +testing "-o pattern" "pkill -o yes && sleep 0.1 && pgrep yes" \ "$proc2\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & -sleep 1 -testing "-s (blank) pattern" "pkill -s '' yes && sleep 1 && +sleep 0.1 +testing "-s (blank) pattern" "pkill -s '' yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & -sleep 1 -testing "-s 0 pattern" "pkill -s 0 yes && sleep 1 && +sleep 0.1 +testing "-s 0 pattern" "pkill -s 0 yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & proc=$! proc_p=`cat /proc/${proc}/stat | awk '{ print $4 }'` -sleep 1 -testing "-P parent_prodId pattern" "pkill -P $proc_p yes && sleep 1 && +sleep 0.1 +testing "-P parent_prodId pattern" "pkill -P $proc_p yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 yes >/dev/null & proc=$! proc_parent=`cat /proc/${proc}/stat | awk '{ print $4 }'` -sleep 1 -testing "-9 pattern" "pkill -9 yes && sleep 1 && +sleep 0.1 +testing "-9 pattern" "pkill -9 yes && sleep 0.1 && (pgrep yes || echo 'yes')" "yes\n" "" "" killall yes >/dev/null 2>&1 diff --git a/tests/renice.test b/tests/renice.test index e350c27..fc773a6 100755 --- a/tests/renice.test +++ b/tests/renice.test @@ -72,20 +72,20 @@ do old_nice_val=`fun_nice_val $proc1` new_nice_val=`renice -n $n_val $proc1 >/dev/null 2>&1 && fun_nice_val $proc1` echo -n "[$old_nice_val/$new_nice_val]," - if [ $old_nice_val -eq 39 -a $old_nice_val -eq $new_nice_val ] + if [ "$old_nice_val" -eq 39 -a "$old_nice_val" -eq "$new_nice_val" ] then echo -n " [Equals 39,doing -1] " n_val="-1" - elif [ $old_nice_val -eq 0 -a $old_nice_val -eq $new_nice_val ] + elif [ "$old_nice_val" -eq 0 -a "$old_nice_val" -eq "$new_nice_val" ] then echo -n " [Equals 0,doing +1] " n_val="1" - elif [ $new_nice_val -gt 39 -o $new_nice_val -lt 0 ] + elif [ "$new_nice_val" -gt 39 -o "$new_nice_val" -lt 0 ] then echo " [Test Fail] " echo "FAIL: renice with step 1 ($proc1) (boundary value)" cnt="-1" - elif [ $new_nice_val -eq $n_val1 -a $new_nice_val -eq $(($old_nice_val+1)) ] + elif [ "$new_nice_val" -eq "$n_val1" -a "$new_nice_val" -eq $(($old_nice_val+1)) ] then cnt=$(($cnt + 1)) if [ $cnt -eq $loop_cnt ] diff --git a/tests/tar.test b/tests/tar.test index 80ba555..1203b66 100755 --- a/tests/tar.test +++ b/tests/tar.test @@ -7,84 +7,111 @@ #testing "name" "command" "result" "infile" "stdin" -#Creating dir -mkdir dir/dir1 -p +mkdir -p dir/dir1 echo "This is testdata" > dir/dir1/file -testing "tgz - compession, extraction and data validation" "tar -czf dir.tgz dir/ && [ -e dir.tgz ] && echo 'yes'; rm -rf dir; tar -xf dir.tgz && [ -f dir/dir1/file ] && cat dir/dir1/file; rm -rf dir.tgz" "yes\nThis is testdata\n" "" "" +testing "tgz - compession, extraction and data validation" \ + "tar -czf dir.tgz dir/ && [ -e dir.tgz ] && echo 'yes'; rm -rf dir; tar -xf dir.tgz && [ -f dir/dir1/file ] && cat dir/dir1/file; rm -rf dir.tgz" \ + "yes\nThis is testdata\n" "" "" -#Creating dir -mkdir dir/dir1 -p +mkdir -p dir/dir1 echo "This is testdata" > dir/dir1/file -testing "tar.gz - compession, extraction and data validation" "tar -czf dir.tar.gz dir/ && [ -e dir.tar.gz ] && echo 'yes'; rm -rf dir; tar -xf dir.tar.gz && [ -f dir/dir1/file ] && cat dir/dir1/file; rm -rf dir.tar.gz" "yes\nThis is testdata\n" "" "" +testing "tar.gz - compession, extraction and data validation" \ + "tar -czf dir.tar.gz dir/ && [ -e dir.tar.gz ] && echo 'yes'; rm -rf dir; tar -xf dir.tar.gz && [ -f dir/dir1/file ] && cat dir/dir1/file; rm -rf dir.tar.gz" \ + "yes\nThis is testdata\n" "" "" -#Creating dir -mkdir dir/dir1 -p +mkdir -p dir/dir1 echo "This is testdata" > dir/dir1/file -testing "verbose compression" "tar -cvzf dir.tgz dir/; rm -rf dir.tgz" "dir/\ndir/dir1/\ndir/dir1/file\n" "" "" +testing "verbose compression" \ + "tar -cvzf dir.tgz dir/; rm -rf dir.tgz" \ + "dir/\ndir/dir1/\ndir/dir1/file\n" "" "" rm -rf dir/ -#creating test file dd if=/dev/zero of=testFile ibs=4096 obs=4096 count=1000 2>/dev/null -testing "- compession and extraction of a file" "tar -czf testFile.tgz testFile && [ -e testFile.tgz ] && echo 'yes'; rm -rf testFile; tar -xf testFile.tgz && [ -f testFile ] && echo 'yes'; rm -rf testFile.tgz" "yes\nyes\n" "" "" +testing "- compession and extraction of a file" \ + "tar -czf testFile.tgz testFile && [ -e testFile.tgz ] && echo 'yes'; rm -rf testFile; tar -xf testFile.tgz && [ -f testFile ] && echo 'yes'; rm -rf testFile.tgz" \ + "yes\nyes\n" "" "" -#creating empty test file touch testFile -testing "- compession and extraction of a empty file" "tar -czf testFile.tgz testFile && [ -e testFile.tgz ] && echo 'yes'; rm -rf testFile; tar -xf testFile.tgz && [ -f testFile ] && echo 'yes'; rm -rf testFile.tgz" "yes\nyes\n" "" "" +testing "- compession and extraction of a empty file" \ + "tar -czf testFile.tgz testFile && [ -e testFile.tgz ] && echo 'yes'; rm -rf testFile; tar -xf testFile.tgz && [ -f testFile ] && echo 'yes'; rm -rf testFile.tgz" \ + "yes\nyes\n" "" "" -#Creating dir -mkdir dir/dir1 -p +mkdir -p dir/dir1 touch dir/dir1/file1 dir/dir1/file2 dir/dir1/file3 dir/dir1/file4 -testing "-t option" "tar -czf dir.tar.gz dir/; rm -rf dir; tar -tf dir.tar.gz | sort; rm -rf dir.tar.gz" "dir/\ndir/dir1/\ndir/dir1/file1\ndir/dir1/file2\ndir/dir1/file3\ndir/dir1/file4\n" "" "" +testing "-t option" \ + "tar -czf dir.tar.gz dir/; rm -rf dir; tar -tf dir.tar.gz | sort; rm -rf dir.tar.gz" \ + "dir/\ndir/dir1/\ndir/dir1/file1\ndir/dir1/file2\ndir/dir1/file3\ndir/dir1/file4\n" "" "" rm -rf dir/ -#Creating nested directory mkdir -p dir/dir1 dir/dir2 dir/dir3 dir/dir4 -echo "This is testdata" > dir/dir1/file; echo "Dont exclude me" > dir/dir3/file1 ; -echo "Exclude me" > dir/dir3/file2 ; echo "YO" > dir/dir4/file1 ; echo "Hello" >dir/dir4/file2; echo "Dont" > dir/dir2/file1 +echo "This is testdata" > dir/dir1/file +echo "Dont exclude me" > dir/dir3/file1 +echo "Exclude me" > dir/dir3/file2 +echo "YO" > dir/dir4/file1 +echo "Hello" >dir/dir4/file2 +echo "Dont" > dir/dir2/file1 echo -ne "dir/dir4\ndir/dir3/file2\n" > exclude_file -testing "create with files excluded : -X" "tar -czf dir.tgz dir/ -X exclude_file ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir3/file1\n" "" "" + +testing "create with files excluded : -X" \ + "tar -czf dir.tgz dir/ -X exclude_file ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " \ + "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir3/file1\n" "" "" rm -rf exclude_file -#Creating nested directory -mkdir dir/dir1 -p ; mkdir dir/dir2 ; mkdir dir/dir3 ; mkdir dir/dir4 +mkdir -p dir/dir1 dir/dir2 dir/dir3 dir/dir4 echo "This is testdata" > dir/dir1/file -echo "Dont exclude me" > dir/dir3/file1 ; echo "Exclude me" > dir/dir3/file2 ; echo "YO" > dir/dir4/file1 ; echo "Hello" >dir/dir4/file2; echo "Dont" > dir/dir2/file1 -testing "with pattern --exclude" "tar --exclude=dir/dir3/* -czf dir.tgz dir/ ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir4/\ndir/dir4/file1\ndir/dir4/file2\n" "" "" - -#Creating directory to be compressed -mkdir dir/dir1 -p +echo "Dont exclude me" > dir/dir3/file1 +echo "Exclude me" > dir/dir3/file2 +echo "YO" > dir/dir4/file1 +echo "Hello" >dir/dir4/file2 +echo "Dont" > dir/dir2/file1 +testing "with pattern --exclude" "tar --exclude=dir/dir3/* -czf dir.tgz dir/ ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " \ + "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir4/\ndir/dir4/file1\ndir/dir4/file2\n" "" "" + +mkdir -p dir/dir1 echo "This is testdata" > dir/dir1/file mkdir temp -testing "extract with -C Dir" "tar -czf dir.tgz dir/ ;rm -rf dir ;tar -xf dir.tgz -C temp/ ; [ -e temp/dir ] && echo 'yes' ; rm -rf dir dir.tgz" "yes\n" "" "" +testing "extract with -C Dir" "tar -czf dir.tgz dir/ ;rm -rf dir ;tar -xf dir.tgz -C temp/ ; [ -e temp/dir ] && echo 'yes' ; rm -rf dir dir.tgz" \ + "yes\n" "" "" rm -rf temp -#Creating nested directory -mkdir dir/dir1 -p ; mkdir dir/dir2 ; mkdir dir/dir3 ; mkdir dir/dir4 ; mkdir temp_dir -echo "dir1/file" > dir/dir1/file ; echo "temp_dir/file" > temp_dir/file -echo "dir3/file1" > dir/dir3/file1 ; echo "dir3/file2" > dir/dir3/file2 ; echo "YO" > dir/dir4/file1 ; echo "Hello" >dir/dir4/file2; echo "dir2/file1" > dir/dir2/file1 +mkdir -p dir/dir1 dir/dir2 dir/dir3 dir/dir4 temp_dir +echo "dir1/file" > dir/dir1/file +echo "temp_dir/file" > temp_dir/file +echo "dir3/file1" > dir/dir3/file1 +echo "dir3/file2" > dir/dir3/file2 +echo "YO" > dir/dir4/file1 +echo "Hello" > dir/dir4/file2 +echo "dir2/file1" > dir/dir2/file1 echo "temp_dir/file" > exclude_file -testing "create with extra files/directory included : -T" "tar -czf dir.tgz dir/ -T exclude_file ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir3/file1\ndir/dir3/file2\ndir/dir4/\ndir/dir4/file1\ndir/dir4/file2\ntemp_dir/file\n" "" "" +testing "create with extra files/directory included : -T" \ + "tar -czf dir.tgz dir/ -T exclude_file ; rm -rf dir ; tar -tf dir.tgz | sort; rm -rf dir.tgz " \ + "dir/\ndir/dir1/\ndir/dir1/file\ndir/dir2/\ndir/dir2/file1\ndir/dir3/\ndir/dir3/file1\ndir/dir3/file2\ndir/dir4/\ndir/dir4/file1\ndir/dir4/file2\ntemp_dir/file\n" "" "" rm -rf exclude_file rm -rf temp_dir -#Creating dir mkdir dir/dir1 -p -echo "Inside dir/dir1" > dir/dir1/file ; echo "Hello Inside dir" > dir/file -testing "extract to STDOUT : -O" " tar -czf dir.tgz dir/ ; rm -rf dir ; tar -xf dir.tgz -O ; [ -e 'Inside dir/dir1/\nHello Inside dir\n' ] && echo 'yes'; rm -rf dir.tgz " "" "" "" +echo "Inside dir/dir1" > dir/dir1/file +echo "Hello Inside dir" > dir/file +testing "extract to STDOUT : -O" \ + "tar -czf dir.tgz dir/ ; rm -rf dir ; tar -xf dir.tgz -O ; [ -e 'Inside dir/dir1/\nHello Inside dir\n' ] && echo 'yes'; rm -rf dir.tgz " \ + "" "" "" -#Creating short filename f="filename_with_100_chars_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" echo "This is testdata" > $f -testing "shortname filename" "tar -cf testFile.tar $f && [ -e testFile.tar ] && echo 'yes'; rm -f $f; tar -xf testFile.tar && [ -f $f ] && cat $f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -f $f" "yes\nThis is testdata\n" "" "" +testing "short filename" \ + "tar -cf testFile.tar $f && [ -e testFile.tar ] && echo 'yes'; rm -f $f; tar -xf testFile.tar && [ -f $f ] && cat $f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -f $f" \ + "yes\nThis is testdata\n" "" "" -#Creating long filename f="filename_with_101_chars_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" echo "This is testdata" > $f -testing "longname filename" "tar -cf testFile.tar $f && [ -e testFile.tar ] && echo 'yes'; rm -f $f; tar -xf testFile.tar && [ -f $f ] && cat $f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -f $f" "yes\nThis is testdata\nLongLink\n" "" "" +testing "long filename" \ + "tar -cf testFile.tar $f && [ -e testFile.tar ] && echo 'yes'; rm -f $f; tar -xf testFile.tar && [ -f $f ] && cat $f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -f $f" \ + "yes\nThis is testdata\nLongLink\n" "" "" -#Creating long pathname d="dirname_with_50_chars_xxxxxxxxxxxxxxxxxxxxxxxxxxxx" f="filename_with_50_chars_xxxxxxxxxxxxxxxxxxxxxxxxxxx" mkdir $d echo "This is testdata" > $d/$f -testing "longname pathname" "tar -cf testFile.tar $d/$f && [ -e testFile.tar ] && echo 'yes'; rm -rf $d; tar -xf testFile.tar && [ -f $d/$f ] && cat $d/$f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -rf $d" "yes\nThis is testdata\nLongLink\n" "" "" +testing "long pathname" \ + "tar -cf testFile.tar $d/$f && [ -e testFile.tar ] && echo 'yes'; rm -rf $d; tar -xf testFile.tar && [ -f $d/$f ] && cat $d/$f && strings testFile.tar | grep -o LongLink; rm -f testFile.tar; rm -rf $d" \ + "yes\nThis is testdata\nLongLink\n" "" "" diff --git a/tests/touch.test b/tests/touch.test index d386156..98b3a03 100755 --- a/tests/touch.test +++ b/tests/touch.test @@ -15,16 +15,16 @@ testing "-d" \ "touch -d 2009-02-13T23:31:30Z walrus && date -r walrus +%s" \ "1234567890\n" "" "" +# NOTE: Using host date because toybox date doesn't support %N. testing "-d nanoseconds" \ - "touch -d 2009-02-13T23:31:30.123456789Z walrus && date -r walrus +%s.%N" \ + "touch -d 2009-02-13T23:31:30.123456789Z walrus && $HOST_BIN_DATE -r walrus +%s.%N" \ "1234567890.123456789\n" "" "" testing "-r" \ - "touch -r walrus walrus2 && date -r walrus2 +%s.%N" \ + "touch -r walrus walrus2 && $HOST_BIN_DATE -r walrus2 +%s.%N" \ "1234567890.123456789\n" "" "" #testing "-a" #testing "-m" #testing "-am" #testing "-t" -rm walrus walrus2 diff --git a/tests/useradd.test b/tests/useradd.test index d0b68ee..3e89084 100755 --- a/tests/useradd.test +++ b/tests/useradd.test @@ -5,12 +5,7 @@ [ -f testing.sh ] && . testing.sh -if [ "$(id -u)" -ne 0 ] -then - echo "$SHOWSKIP: useradd (not root)" - continue 2>/dev/null - exit -fi +skip_if_not_root # Redirecting all output to /dev/null for grep, adduser and deluser arg="&>/dev/null" diff --git a/tests/xzcat.test b/tests/xzcat.test index 4aaf35f..966164a 100755 --- a/tests/xzcat.test +++ b/tests/xzcat.test @@ -6,21 +6,25 @@ [ -f testing.sh ] && . testing.sh #testing "name" "command" "result" "infile" "stdin" + echo "hello" > file -tar -cJf file.xz file -# Get system xzcat -xzcatExe=`which xzcat` -$xzcatExe file.xz > xzcatOut -testing "- decompresses a single file" "xzcat file.xz > Tempfile && echo "yes"; diff Tempfile xzcatOut && echo "yes"; rm -rf file* xzcatOut Tempfile" "yes\nyes\n" "" "" +$HOST_BIN_TAR -cJf file.xz file -#testing "name" "command" "result" "infile" "stdin" -echo "hello" > file1 -echo "hi" > file2 -echo "Hi, Good morning !! I am a xzcat tester" > file3 -tar -cJf file1.xz file1 -tar -cJf file2.xz file2 -tar -cJf file3.xz file3 -# Get system xzcat -xzcatExe=`which xzcat` -$xzcatExe file1.xz file2.xz file3.xz > xzcatOut -testing "- decompresses multiple files" "xzcat file1.xz file2.xz file3.xz > Tempfile && echo "yes" ; diff Tempfile xzcatOut && echo "yes"; rm -rf file* xzcatOut Tempfile " "yes\nyes\n" "" "" +$HOST_BIN_XZCAT file.xz > expected_out +testing "decompresses a single file" \ + 'xzcat file.xz > Tempfile && echo yes; diff Tempfile expected_out && echo yes; rm -rf file* expected_out Tempfile' \ + "yes\nyes\n" "" "" + +echo "one" > file1 +echo "two" > file2 +echo "three" > file3 + +$HOST_BIN_TAR -cJf file1.xz file1 +$HOST_BIN_TAR -cJf file2.xz file2 +$HOST_BIN_TAR -cJf file3.xz file3 + +$HOST_BIN_XZCAT file1.xz file2.xz file3.xz > expected_out + +testing "decompresses multiple files" \ + 'xzcat file1.xz file2.xz file3.xz > Tempfile && echo yes; diff Tempfile expected_out && echo yes' \ + "yes\nyes\n" "" "" diff --git a/tests/zcat.test b/tests/zcat.test index ccd472c..3399057 100755 --- a/tests/zcat.test +++ b/tests/zcat.test @@ -5,22 +5,25 @@ [ -f testing.sh ] && . testing.sh -#testing "name" "command" "result" "infile" "stdin" +# testing "name" "command" "result" "infile" "stdin" + echo "hello" > file -tar -czf file.gz file -# Get system zcat -zcatExe=`which zcat` -$zcatExe file.gz > zcatOut -testing "- decompresses a single file" "zcat file.gz > Tempfile && echo "yes"; diff Tempfile zcatOut && echo "yes"; rm -rf file* zcatOut Tempfile" "yes\nyes\n" "" "" - -#testing "name" "command" "result" "infile" "stdin" -echo "hello" > file1 -echo "hi" > file2 -echo "Hi, Good morning !! I am a bzcat tester" > file3 -tar -czf file1.gz file1 -tar -czf file2.gz file2 -tar -czf file3.gz file3 -# Get system zcat -zcatExe=`which zcat` -$zcatExe file1.gz file2.gz file3.gz > zcatOut -testing "- decompresses multiple files" "zcat file1.gz file2.gz file3.gz > Tempfile && echo "yes" ; diff Tempfile zcatOut && echo "yes"; rm -rf file* zcatOut Tempfile " "yes\nyes\n" "" "" +$HOST_BIN_TAR -czf file.gz file + +$HOST_BIN_ZCAT file.gz > expected_out +testing "decompresses a single file" \ + 'zcat file.gz > Tempfile && echo yes; diff Tempfile expected_out && echo yes' \ + "yes\nyes\n" "" "" + +echo "one" > file1 +echo "two" > file2 +echo "three" > file3 + +$HOST_BIN_TAR -czf file1.gz file1 +$HOST_BIN_TAR -czf file2.gz file2 +$HOST_BIN_TAR -czf file3.gz file3 + +$HOST_BIN_ZCAT file1.gz file2.gz file3.gz > expected_out +testing "decompresses multiple files" \ + 'zcat file1.gz file2.gz file3.gz > Tempfile && echo yes; diff Tempfile expected_out && echo yes' \ + "yes\nyes\n" "" "" -- 1.9.1
From ef20cd5ab57b3f303aa87118bd68249e190c4b61 Mon Sep 17 00:00:00 2001 From: Andy Chu <[email protected]> Date: Fri, 18 Mar 2016 18:02:17 -0700 Subject: [PATCH 1/2] Refactor the test harness to triage tests, and fix some bugs. genconfig.sh: - Fix a name clash bug where 'make test' would run all tests and then try to build the 'test' binary. 'make test_bin' is special cased for now. - Make the test targets .PHONY, since we're not creating files called test_sed, etc. - Use here docs to write Makefile fragments, for readability and to be consistent with the rest of the script. Refactor into functions. - Add comments. make.sh: - Add the output binary name as the first param. This fixes an issue where 'make; make test_sed' would leave you without a 'toybox' binary because single.sh created another toybox binary and renamed it to sed. runtest.sh: - Changed the 'testing' function to return 0. This fixes a bug where the return value of the script depends on the return code of the last command -- NOT whether it passed or failed. For example, 'make test_test' would fail with code 1, even though all tests passed (because 'test' exits 1 validly). On the other hand, tests with failures would still return 0 if the last command didn't fail. - Use local variables instead of globals since we are sharing globals with the tests themselves. - Add a common function to skip tests if not running as root, and keep track of skipped commands. test.sh: - Refactor it into separate functions/actions and add usage. - Fix a bug where tests would succeed under 'make test_$COMMAND' but fail under 'make test'. The problem is if you are running the xzcat command, 'tar' could be either the host binary or the toybox binary, depending on what tools are in the modified test $PATH. To fix this, the test harness finds the host binary and sets $HOST_BIN_TAR, etc. before invoking the test. - Keep track of the number of failures per *command*, and exit 1 if any failed - Use a distinct test directory for each command, instead of repeatedly setting up and tearing down the same one. - When running all tests, print out a summary at the end. single.sh: - Call scripts/make.sh directly instead of 'make'. - Add comments. --- Makefile | 2 +- scripts/genconfig.sh | 103 ++++++++++++++++++++++++++--------- scripts/make.sh | 16 ++++-- scripts/runtest.sh | 40 +++++++++----- scripts/single.sh | 28 +++++++--- scripts/test.sh | 149 +++++++++++++++++++++++++++++++++++++++++---------- 6 files changed, 258 insertions(+), 80 deletions(-) diff --git a/Makefile b/Makefile index 40e95b1..2f92a12 100644 --- a/Makefile +++ b/Makefile @@ -61,7 +61,7 @@ distclean: clean test: tests tests: - scripts/test.sh + scripts/test.sh all help:: @echo ' toybox - Build toybox.' diff --git a/scripts/genconfig.sh b/scripts/genconfig.sh index 5e7d1df..6a15d09 100755 --- a/scripts/genconfig.sh +++ b/scripts/genconfig.sh @@ -1,5 +1,15 @@ #!/bin/bash - +# +# Inspects the system and generates various config files. +# +# Usage: +# scripts/genconfig.sh +# +# Outputs: +# - generated/Config.in, generated/Config.probed - kconfig input +# - generated/cflags - CFLAGS used by scripts/make.sh +# - .singlemake - targets included by Makefile +# # This has to be a separate file from scripts/make.sh so it can be called # before menuconfig. (It's called again from scripts/make.sh just to be sure.) @@ -124,9 +134,6 @@ genconfig() done } -probeconfig > generated/Config.probed || rm generated/Config.probed -genconfig > generated/Config.in || rm generated/Config.in - # Find names of commands that can be built standalone in these C files toys() { @@ -134,24 +141,70 @@ toys() sed -rn 's/([^:]*):.*(OLD|NEW)TOY\( *([a-zA-Z][^,]*) *,.*/\1:\3/p' } -WORKING= -PENDING= -toys toys/*/*.c | ( -while IFS=":" read FILE NAME -do - [ "$NAME" == help ] && continue - [ "$NAME" == install ] && continue - echo -e "$NAME: $FILE *.[ch] lib/*.[ch]\n\tscripts/single.sh $NAME\n" - echo -e "test_$NAME:\n\tscripts/test.sh $NAME\n" - [ "${FILE/pending//}" != "$FILE" ] && - PENDING="$PENDING $NAME" || - WORKING="$WORKING $NAME" -done > .singlemake && -echo -e "clean::\n\trm -f $WORKING $PENDING" >> .singlemake && -echo -e "list:\n\t@echo $(echo $WORKING $PENDING | tr ' ' '\n' | sort | xargs)"\ - >> .singlemake && -echo -e "list_working:\n\t@echo $(echo $WORKING | tr ' ' '\n' | sort | xargs)" \ - >> .singlemake && -echo -e "list_pending:\n\t@echo $(echo $PENDING | tr ' ' '\n' | sort | xargs)" \ - >> .singlemake -) +sort_words() +{ + tr ' ' '\n' | sort | xargs +} + +# Print Makefile targets to stdout. +print_singlemake() +{ + local working= + local pending= + local test_targets= + while IFS=":" read cmd_src cmd + do + [ "$cmd" == help ] && continue + [ "$cmd" == install ] && continue + + local test_name=test_$cmd + local build_name=$cmd + # 'make test' is already taken for running all tests, so the 'test' binary + # can be built with 'make test_bin'. + [ "$cmd" == test ] && build_name=test_bin + + # Print a build target and test target for each command. + cat <<EOF +$build_name: $cmd_src *.[ch] lib/*.[ch] + scripts/single.sh $cmd + +$test_name: + scripts/test.sh single $cmd + +EOF + + [ "${cmd_src/pending//}" != "$cmd_src" ] && + pending="$pending $cmd" || + working="$working $cmd" + test_targets="$test_targets $test_name" + done + + # Print more targets. + cat <<EOF +# test_bin builds the 'test' file, not a file named test_bin. And all the rest +# of the test targest are phony too. +.PHONY: test_bin $test_targets + +clean:: + rm -f $working $pending + +list: + @echo $(echo $working $pending | sort_words) + +list_working: + @echo $(echo "$working" | sort_words) + +list_pending: + @echo $(echo "$pending" | sort_words) +EOF +} + +main() +{ + probeconfig > generated/Config.probed || rm generated/Config.probed + genconfig > generated/Config.in || rm generated/Config.in + + toys toys/*/*.c | print_singlemake > .singlemake +} + +main "$@" diff --git a/scripts/make.sh b/scripts/make.sh index 8be4c3b..ab2719c 100755 --- a/scripts/make.sh +++ b/scripts/make.sh @@ -1,12 +1,18 @@ #!/bin/bash - -# Grab default values for $CFLAGS and such. +# +# Build toybox. Uses configured values for $CFLAGS and such. +# +# Usage: +# scripts/make.sh export LANG=c export LC_ALL=C set -o pipefail source ./configure +# output ${OUTNAME}_unstripped and and $OUTNAME +OUTNAME=${1:-toybox} + [ -z "$KCONFIG_CONFIG" ] && KCONFIG_CONFIG=".config" # Since each cc invocation is short, launch half again as many processes @@ -111,7 +117,7 @@ fi # LINK needs optlibs.dat, above -LINK="$(echo $LDOPTIMIZE $LDFLAGS -o toybox_unstripped -Wl,--as-needed $(cat generated/optlibs.dat))" +LINK="$(echo $LDOPTIMIZE $LDFLAGS -o ${OUTNAME}_unstripped -Wl,--as-needed $(cat generated/optlibs.dat))" genbuildsh > generated/build.sh && chmod +x generated/build.sh || exit 1 echo "Make generated/config.h from $KCONFIG_CONFIG." @@ -293,9 +299,9 @@ done [ $DONE -ne 0 ] && exit 1 do_loudly $BUILD $LFILES $LINK || exit 1 -if [ ! -z "$NOSTRIP" ] || ! do_loudly ${CROSS_COMPILE}strip toybox_unstripped -o toybox +if [ ! -z "$NOSTRIP" ] || ! do_loudly ${CROSS_COMPILE}strip ${OUTNAME}_unstripped -o $OUTNAME then - echo "strip failed, using unstripped" && cp toybox_unstripped toybox || + echo "strip failed, using unstripped" && cp ${OUTNAME}_unstripped $OUTNAME || exit 1 fi diff --git a/scripts/runtest.sh b/scripts/runtest.sh index 0c933c2..3d0fc8c 100644 --- a/scripts/runtest.sh +++ b/scripts/runtest.sh @@ -30,8 +30,9 @@ # else contains a colon-separated list of features (in which case the function # clears SKIP if the flag was found, or sets it to 1 if the flag was not found). -export FAILCOUNT=0 -export SKIP= +FAILCOUNT=0 +SKIPPED_COMMANDS= +SKIP= # Helper functions @@ -64,12 +65,12 @@ optional() testing() { - NAME="$CMDNAME $1" - [ -z "$1" ] && NAME=$2 + local test_desc="$CMDNAME $1" + [ -z "$1" ] && test_desc=$2 if [ $# -ne 5 ] then - echo "Test $NAME has the wrong number of arguments ($# $*)" >&2 + echo "Test $test_desc has the wrong number of arguments ($# $*)" >&2 exit fi @@ -77,24 +78,24 @@ testing() if [ -n "$SKIP" ] || ( [ -n "$SKIP_HOST" ] && [ -n "$TEST_HOST" ]) then - [ ! -z "$VERBOSE" ] && echo "$SHOWSKIP: $NAME" + [ ! -z "$VERBOSE" ] && echo "$SHOWSKIP: $test_desc" return 0 fi echo -ne "$3" > expected echo -ne "$4" > input echo -ne "$5" | eval "$2" > actual - RETVAL=$? + local status=$? # Catch segfaults - [ $RETVAL -gt 128 ] && [ $RETVAL -lt 255 ] && - echo "exited with signal (or returned $RETVAL)" >> actual + [ $status -gt 128 ] && [ $status -lt 255 ] && + echo "exited with signal (or returned $status)" >> actual cmp expected actual > /dev/null 2>&1 if [ $? -ne 0 ] then - FAILCOUNT=$[$FAILCOUNT+1] - echo "$SHOWFAIL: $NAME" + FAILCOUNT=$(($FAILCOUNT+1)) + echo "$SHOWFAIL: $test_desc" if [ -n "$VERBOSE" ] then [ ! -z "$4" ] && echo "echo -ne \"$4\" > input" @@ -103,13 +104,15 @@ testing() [ "$VERBOSE" == fail ] && exit 1 fi else - echo "$SHOWPASS: $NAME" + echo "$SHOWPASS: $test_desc" fi rm -f input expected actual [ -n "$DEBUG" ] && set +x - return $RETVAL + # Always succeed -- we don't want the exit code to depend on just the last + # "testing" statement. + return 0 } # Recursively grab an executable and all the libraries needed to run it. @@ -172,3 +175,14 @@ dochroot() rmdir tmpdir4chroot } +# Tests can call this if they require root. NOTE: This code relies on being in +# the 'for' loop in scripts/test.sh. +skip_if_not_root() +{ + if [ "$(id -u)" -ne 0 ] + then + echo "$SHOWSKIP: $CMDNAME (not root)" + SKIPPED_COMMANDS="$SKIPPED_COMMANDS $CMDNAME" + continue + fi +} diff --git a/scripts/single.sh b/scripts/single.sh index e3c9ebb..a29ff46 100755 --- a/scripts/single.sh +++ b/scripts/single.sh @@ -1,10 +1,19 @@ #!/bin/bash - -# Build a standalone toybox command - -if [ -z "$1" ] +# +# Build standalone toybox commands. +# +# Usage: +# scripts/single.sh COMMAND... +# +# The output is put in the repo root, or $PREFIX. +# +# Example: +# # Put grep and sed binaries in this dir +# $ PREFIX=generated/test/bin scripts/single.sh grep sed + +if [ $# -eq 0 ] then - echo "usage: single.sh command..." >&2 + echo "Usage: single.sh COMMAND..." >&2 exit 1 fi @@ -15,6 +24,12 @@ then exit 1 fi +# For each command: +# 1) write a .singleconfig file (I think this would be better as +# generated/singleconfig/sed) +# 2) make allnoconfig, except turn stuff related to the command on +# 3) make toybox, and then move it to the command name. + export KCONFIG_CONFIG=.singleconfig for i in "$@" do @@ -38,6 +53,5 @@ do echo "# CONFIG_TOYBOX is not set" >> "$KCONFIG_CONFIG" && grep "CONFIG_TOYBOX_" .config >> "$KCONFIG_CONFIG" && - make && - mv -f toybox $PREFIX$i || exit 1 + scripts/make.sh $PREFIX$i || exit 1 done diff --git a/scripts/test.sh b/scripts/test.sh index 0c3d7aa..00e01e6 100755 --- a/scripts/test.sh +++ b/scripts/test.sh @@ -1,49 +1,140 @@ #!/bin/bash +# +# Run toybox tests. + +usage() +{ + cat <<EOF +Usage: + scripts/test.sh all + scripts/test.sh single COMMAND... + +Examples: + $ scripts/test.sh all # run tests for all commands + $ scripts/test.sh single grep sed # run tests for these two commands + + # Test 'grep' on the system, not toybox + $ TEST_HOST=1 scripts/test.sh commands grep +EOF +} [ -z "$TOPDIR" ] && TOPDIR="$(pwd)" trap 'kill $(jobs -p) 2>/dev/null; exit 1' INT -rm -rf generated/testdir -mkdir -p generated/testdir/testdir +cd_test_dir() +{ + local cmd=$1 + local test_dir=$TEST_ROOT/test_$cmd + rm -rf $test_dir + mkdir -p $test_dir + cd $test_dir +} + +setup_test_env() +{ + # Before changing path, set variables for host executables. + HOST_BIN_TAR=$(which tar) + HOST_BIN_BZCAT=$(which bzcat) + HOST_BIN_XZCAT=$(which xzcat) + HOST_BIN_ZCAT=$(which zcat) + + HOST_BIN_DATE=$(which date) + HOST_BIN_HOSTNAME=$(which hostname) + + PATH="$BIN_DIR:$PATH" # Make sure the tests can use toybox tools + export LC_COLLATE=C + + # Library functions used by .test scripts, e.g. 'testing'. + . "$TOPDIR/scripts/runtest.sh" -if [ -z "$TEST_HOST" ] -then - if [ $# -ne 0 ] + if [ -f "$TOPDIR/generated/config.h" ] then - PREFIX=generated/testdir/ scripts/single.sh "$@" || exit 1 - else - make install_flat PREFIX=generated/testdir || exit 1 + export OPTIONFLAGS=:$(echo $(sed -nr 's/^#define CFG_(.*) 1/\1/p' "$TOPDIR/generated/config.h") | sed 's/ /:/g') fi -fi +} -cd generated/testdir -PATH="$PWD:$PATH" -cd testdir -export LC_COLLATE=C +# Run tests for specific commands. +single() +{ + # Build individual binaries, e.g. generated/testdir/expr + [ -z "$TEST_HOST" ] && PREFIX=$BIN_DIR/ scripts/single.sh "$@" || exit 1 -. "$TOPDIR"/scripts/runtest.sh -[ -f "$TOPDIR/generated/config.h" ] && export OPTIONFLAGS=:$(echo $(sed -nr 's/^#define CFG_(.*) 1/\1/p' "$TOPDIR/generated/config.h") | sed 's/ /:/g') + setup_test_env -if [ $# -ne 0 ] -then - for i in "$@" + for cmd in "$@" do - CMDNAME="${i##*/}" - CMDNAME="${CMDNAME%.test}" - . "$TOPDIR"/tests/$i.test + CMDNAME=$cmd # .test file uses this + cd_test_dir $cmd + # Run test. NOTE: it may 'continue' + . "$TOPDIR"/tests/$cmd.test done -else - for i in "$TOPDIR"/tests/*.test + + [ $FAILCOUNT -eq 0 ] || echo "toybox $cmd: $FAILCOUNT total failures" + [ $FAILCOUNT -eq 0 ] # exit success if there were 0 failures +} + +# Run tests for all commands. +all() +{ + # Build a toybox binary and create symlinks to it. + [ -z "$TEST_HOST" ] && make install_flat PREFIX=$BIN_DIR/ || exit 1 + + setup_test_env + + local failed_commands='' + local num_commands=0 + + for test_file in "$TOPDIR"/tests/*.test do - CMDNAME="${i##*/}" + # Strip off the front and back of the test filename to get the command + CMDNAME="${test_file##*/}" CMDNAME="${CMDNAME%.test}" - if [ -h ../$CMDNAME ] || [ ! -z "$TEST_HOST" ] + + if [ -h $BIN_DIR/$CMDNAME ] || [ -n "$TEST_HOST" ] then - cd .. && rm -rf testdir && mkdir testdir && cd testdir || exit 1 - . $i + local old_count=$FAILCOUNT + cd_test_dir $CMDNAME + + # Run test. NOTE: it may 'continue' + . $test_file + + if [ $FAILCOUNT -ne $old_count ] + then + echo "$CMDNAME: some tests failed ($FAILCOUNT failures so far)" + failed_commands="$failed_commands $CMDNAME" + fi + num_commands=$(($num_commands+1)) else - echo "$CMDNAME disabled" + echo "$CMDNAME not built" fi done -fi + + echo + echo -n "Tested $num_commands toybox commands: " + if [ $FAILCOUNT -eq 0 ] + then + echo "ALL PASSED" + else + echo "$FAILCOUNT test failures" + echo "Commands with test failures: $failed_commands" + fi + echo "Commands skipped: $SKIPPED_COMMANDS" + + [ $FAILCOUNT -eq 0 ] # exit success if there were 0 failures +} + +readonly TEST_ROOT=$TOPDIR/generated/test +readonly BIN_DIR=$TEST_ROOT/bin + +rm -rf $TEST_ROOT # clear out data from old runs +mkdir -p $BIN_DIR + +case $1 in + single|all) + "$@" + ;; + *) + usage + ;; +esac -- 1.9.1
_______________________________________________ Toybox mailing list [email protected] http://lists.landley.net/listinfo.cgi/toybox-landley.net
