On Wed, 2012-03-21 at 16:49 +0000, R. Diez wrote: > > > There are various places where the timeout is hard-coded. As noted in > > Ian Lance-Taylor's article above, DejaGnu is far from perfect. I have > > tried to fix these where I find them. If there are any places in the > > Or1ksim testsuite, then they should certainly be replaced by conditional > > code checking for a global value first. > > I searched for "set timeout" in all *.exp files and there are very few > hits. The only ones globally relevant are in or1ksim.exp and in > libsim.exp. However, for some reason, those timeouts are not being > taken into consideration. I would be grateful if you could remove > them, as they just cause confusion. I am not skilled enough in TCL to > write the conditional code you are suggesting.
Hi R Diez, Just delete the lines you don't like. I had to learn TCL in order to run DejaGnu, so I am not that far ahead of you. More work is needed on the test suite. A few years ago, all OpenRISC testing consisted of running 13 programs and checking they printed out 0xdeadbeef at the end. So I wrote the Or1ksim test suite to give us a reliable base reference, and now we have checks for around 2,500 points of correct behaviour. As a result Or1ksim is a much more accurate reflection of the behaviour of the architecture, and the architecture manual is much less ambiguous. That allowed Waqas Ahmed to build his MSc dissertation with an OVM test flow comparing RTL and Or1ksim, and as a result some 20 instructions were fixed. The old test suite for the design allowed the implementation to exist for nearly a decade before Waqas revealed several instructions were not actually implemented! Others have built further on that work. Now it is time for a new generation to make their contribution. And congratulations to you for stepping up to be part of that. > > Infinite timeout is a bad idea. There are some failure modes where tests > > loop forever. You really don't want to lose the remaining 2400 test > > results because test 100 hung. Just choose something appropriate. > > If we have any tests in or1ksim that should loop forever, they should > set their own individual timeout values. Choosing a default timeout > value for all tests is a hit-and-miss affair. For example, the > 'mycompress' test alone takes hours to run if GCC is not optimising it > with -O2, but that depends very much on the PC, and other tests run > much faster. I would set the default test suite timeout to infinite > and let the user specify something else in the command-line, in case > he really wants to run as many tests as possible within say 1 hour. > But I don't think they will be many people doing that with the or1ksim > test suite. That's how it is supposed to work already. I thought I'd set a global timeout that was good for Verilator. But if it doesn't work for you just change it. > By the way, TCL syntax errors do not make the test suite stop with an > error indication. Could you apply this patch to or1ksim? I have no idea how to do that. The subtleties of the TCL interpreter's error handling within Expect and DejaGnu are way beyond me! Jeremy -- Tel: +44 (1590) 610184 Cell: +44 (7970) 676050 SkypeID: jeremybennett Email: [email protected] Web: www.embecosm.com _______________________________________________ OpenRISC mailing list [email protected] http://lists.openrisc.net/listinfo/openrisc
