Having played with this a little, I've concluded that the test runner should look at OUTPUT and not .../results when deciding if a test should be run; and probably only delete OUTPUT when UPDATE=1
It's much clearer. Andrew On 25 July 2015 at 03:35, D. Hugh Redelmeier <[email protected]> wrote: > As mentioned earlier, my test run from last night hung in one test. > I had initiated with > make check UPDATE=1 > > I killed the hung test run (it took a couple of ^C strokes to do it). I > then restarted it via > make check > > Somehow all the testing/pluto/*/OUTPUT directories that had been > painstakingly filled in the first run disappeared. I'm not used to this, > but it is the way things work now. They were probably moved to > ~/results/bluebird.mimosa.com/2015-07-24-bluebird.mimosa.com-v3.14rc3-66-g28f5374-master/*/OUTPUT/ > > The oldest testing/pluto/*/OUTPUT director after the second run is > drwxrwxr-x. 2 build build 4096 Jul 24 12:45 testing/pluto/ikev2-ddns-02/OUTPUT > This directory does not appear under results. What's that about? > > And it doesn't appear in the expected sequence as per TESTLIST > The next oldest is > drwxrwxr-x. 2 build build 4096 Jul 24 12:49 > testing/pluto/newoe-07-ike-replace-initiator/OUTPUT > Why is that? > > In order to salvage the run, I have changed my copy of > pluto-testlist-scan.sh to be willing to look in ~/results for the OUTPUT > directories. > But it fails. Why? > > Let's look at > > testing/pluto/netkey-passthrough-03/OUTPUT/RESULT: > > {"hostname": "west", "time": "2015-07-24 17:08", "runtime": 51.0, "msg": > "end", "testname": "netkey-passthrough-03", "epoch": 1437772082} > {"hostname": "east", "time": "2015-07-24 17:08", "runtime": 51.0, "msg": > "end", "testname": "netkey-passthrough-03", "epoch": 1437772082} > {"time": "2015-07-24 17:08", "runtime": 56.67, "testname": > "netkey-passthrough-03", "epoch": 1437772087, "expect": "good", "node": > "bluebird.mimosa.com", "result": "passed"} > > and > ~/results/bluebird.mimosa.com/2015-07-24-bluebird.mimosa.com-v3.14rc3-66-g28f5374-master/netkey-passthrough-03/OUTPUT/RESULT: > > {"testname": "netkey-passthrough-03", "epoch": 1434833680, "hostname": > "west", "time": "2015-06-20 16:54", "msg": "end", "runtime": 47.64} > {"testname": "netkey-passthrough-03", "epoch": 1434833680, "hostname": > "east", "time": "2015-06-20 16:54", "msg": "end", "runtime": 47.66} > {"testname": "netkey-passthrough-03", "epoch": 1434833683, "result": null, > "node": "bluebird.mimosa.com", "time": "2015-06-20 16:54", "expect": "good", > "runtime": 50.94} > > They ought to be the same since they describe the same test run (I think). > They are quite different. Why the heck is that? > > In fact, the one in ~/results seems to be from a different date! > $ ls -ltr > ~/results/bluebird.mimosa.com/2015-07-24-bluebird.mimosa.com-v3.14rc3-66-g28f5374-master/netkey-passthrough-03/OUTPUT/ > total 644 > -rw-rw-r--. 1 build build 0 Jun 20 16:53 swan12.pcap > -rw-rw-r--. 1 build build 20210 Jun 20 16:54 west.console.verbose.txt > -rw-rw-r--. 1 build build 16617 Jun 20 16:54 east.console.verbose.txt > -rw-rw-r--. 1 build build 448 Jun 20 16:54 RESULT > -rwxrwxrwx. 1 build qemu 288592 Jul 24 17:08 east.pluto.log > -rwxrwxrwx. 1 build qemu 308696 Jul 24 17:08 west.pluto.log > -rw-rw-r--. 1 build build 3789 Jul 24 17:08 east.console.txt > -rw-rw-r--. 1 build build 0 Jul 24 17:08 east.console.diff > -rw-rw-r--. 1 build build 6684 Jul 24 17:08 west.console.txt > -rw-rw-r--. 1 build build 0 Jul 24 17:08 west.console.diff > > Why the heck are things from June 20 here? After all, the directory > name includes the date July 24. > > [build@bluebird libreswan]$ ls -ltr > testing/pluto/netkey-passthrough-03/OUTPUT/ > total 644 > -rw-rw-r--. 1 build build 20209 Jul 24 17:08 west.console.verbose.txt > -rw-rw-r--. 1 build build 16616 Jul 24 17:08 east.console.verbose.txt > -rwxrwxrwx. 1 build qemu 288592 Jul 24 17:08 east.pluto.log > -rwxrwxrwx. 1 build qemu 308696 Jul 24 17:08 west.pluto.log > -rw-rw-r--. 1 build build 6684 Jul 24 17:08 west.console.txt > -rw-rw-r--. 1 build build 0 Jul 24 17:08 west.console.diff > -rw-rw-r--. 1 build build 3789 Jul 24 17:08 east.console.txt > -rw-rw-r--. 1 build build 0 Jul 24 17:08 east.console.diff > -rw-rw-r--. 1 build build 450 Jul 24 17:08 RESULT > > The are NOT in the original directory. > > This looks very very wrong. > > Can anyone explain? > > I was going to start another test run, but I think that I need to > leave the scene of the crime undisturbed. > _______________________________________________ > Swan-dev mailing list > [email protected] > https://lists.libreswan.org/mailman/listinfo/swan-dev _______________________________________________ Swan-dev mailing list [email protected] https://lists.libreswan.org/mailman/listinfo/swan-dev
