> >
> > I've also seen that the ORPSoC test suite uses grep on each test result
> log in order to find out whether the test was successful:
> >
> > /sim/bin/Makefile:
> > $(Q)echo "function check-test-log { if [ \`grep -c -i
> "$(TEST_OK_STRING)"
> "$(RTL_SIM_RESULTS_DIR)"/"$(TEST)$(TEST_OUT_FILE_SUFFIX)"\` -gt 0 ]; then
> return 0; else return 1; fi; }" >> $@
> >
> > Variable TEST_OK_STRING is just defined as the number "8000000d". I
> would think that this is not a very reliable way to determine if the test
> succeeded.
> >
>
Greping to see if the sim passed is a hard thing to do. All you need is for
some protocol checker to spit out the same string and your in trouble.
You need more information than that for your results. You need to know:
1) How many sims are in the entire test suite
2) How many actually ran ( syntax errors, licensing issues and disk full
problems will impact this number)
3) How many made it all the way to the end of the sim successfully (
timeouts will impact this number).
4) How many Errors occurred. The main task can finish successfully but you
can still have numerous errors in the subblocks that need fixed.
Your protocol checkers should never $finish a sim when they detect an
error. They should report it so that it is included in the error count but
the sim should continue to run. The or1200_monitor module doesn't do this.
Any number of cpu errors will stop the simulation. That is not what you
want to do in a multicore design. Go head and run the entire sim and report
that I have 127 Errors. It is my job to "peel the onion" that fix them one
at a time. If you halt the sim then you force me to make your error my
number one priority. System designers should never give component designers
that much power.
Every simulation needs a master clock and reset. In my socgen project I
have created a Testbench module called clock_gen_def that creates these
and I instantiate this model in all my testbenchs. The module has a clk and
reset output and parameters fro PERIOD and TIMEOUT. PERIOD sets the clk
frequency and if TIMEOUT is non zero then then sim will terminate with a
error if it reaches that number of clocks.
Rst is set high at the start of the sim and can be controlled by calling
the reset_on and reset_off tasks inside the module. There is also a next()
task that lets you wait any number of clocks before proceeding. The module
is always instantiated with the instance name of cg so that every sim
starts:
initial
TB.cg.next(12);
TB.cg.reset_off
...
There is a fail task that is called when anybody detects a error. It prints
out a message and increments a fail counter. A exit task is called when
the sim finishes. It prints out the error count and terminates. If the
cycle count reaches the TIMEOUT value before exit is called then it
terminates with a timeout error.
The nice thing about doing it this way is that everything the test
administrator cares about is controlled in one module and is easy to adapt
if you get a piece of IP that uses the same strings that you were relying
on to catch failures.
I would suggest that you use my clock_gen module in your sims
John Eaton
_______________________________________________
OpenRISC mailing list
[email protected]
http://lists.openrisc.net/listinfo/openrisc
