In running regression tests, I'm seeing a lot of config.ini differences; it looks like the latest stats updates have just updated stats.txt, and not the other output files (config.ini, simout, and simerr). Even though a test won't fail if the only change is in config.ini, simout, or simerr, it's very useful to keep those updated so that there isn't so much useless output when running tests. All these files should get updated automatically if you use the --update-ref flag to scons or util/regress, so this should only be an issue if you're updating the reference outputs manually.
Strangely, in the process of doing some updates using --update-ref, I got several errors like this: scons: *** [build/ALPHA/tests/opt/long/se/50.vortex/alpha/tru64/inorder-timing/_update] Source `tests/long/se/50.vortex/alpha/tru64/inorder-timing/stats.txt' not found, needed by target `build/ALPHA/tests/opt/long/se/50.vortex/alpha/tru64/inorder-timing/_update'. Note that that source path is not a valid path for either the new or old stats file (which would be 'tests/*opt*/long/se/50.vortex/alpha/tru64/inorder-timing/stats.txt' and 'tests/long/se/50.vortex/*ref*/alpha/tru64/inorder-timing/stats.txt' respectively). I'm pretty puzzled by this. The error only appears in some cases, and I've only seen it happen on 'long' runs. Worse, it goes away on subsequent runs, making it tricky to reproduce. It appears to be complaining about the new stats file. Looking into the code in tests/SConscript, I'm now wondering how it works in the first place, since I'm not sure where the 'opt' gets inserted in the path. I don't see any changes in the tests/SConscript history that look like they would impact this. Has anyone else seen this problem? A couple more questions while we're on the topic: - Now that we have a 'changed' status that's different from 'failed', we could consider using that to flag tests where there are changes in the non-stats.txt outputs even when the stats are unchanged (which in theory is very unlikely to happen, but would at least flag the situation where stats.txt got updated but the other files didn't). This would require having the diff 'ignore' regexps be very robust to insignificant changes like the compile and run dates, which I think is the case, but this would up the ante on getting those right. - Should we start including config.json in the refreence outputs too? It should be redundant to compare it, since we're comparing config.ini, but I thought I'd throw the idea out there. Steve _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
