BY: Suraj Kurapati (snk) DATE: 2007-07-20 16:55 SUBJECT: RE: System level testing
On 7/20/07, Rob MacAulay wrote: > I have been having a quick play around with your Ruby-Vpi tools. > First of all, I'll just say that you seem to have been able to > implement a remarkable amount using a fairly lightweight > framework - always a good sign! Thanks. > I have a few general queries that are fairly broad in scope - so > broad, in fact, that you may ignore them if you wish! I know you > are busy with another release at the moment. Yeah, please accept my half-answers for now; I'll take a deeper look at this thread after the release. > 1) is it possible to use the ruby models of other parts of the > design? > It is currently possible to use a ruby prototype for the block > under test, but what about a larger system? Excellent question. I have been thinking about integration testing (with >1 verilog DUT and >1 ruby DUT prototype) in the back of my mind for some months now and I must say that the solution has not yet occurred to me in full clarity. However, the next release (hopefully this weekend -- the code was finished last week, but the documentation is not yet complete) does make steps toward supporting multiple DUTs & ruby prototypes by creating a separate namespace (Ruby module) for each DUT. As a result, we can load multiple DUTs into the simulation without having their proto.rb & design.rb files causing interference. However, I have not figured out how (or even if it is necessary) to load the spec.rb into the DUT namespace (since the RUnit or RSpec has its own global namespace...). This is just one small step forward, for now. > It might, for example, be useful to implement testbench models in > ruby. > There would seem to be two problems here: > 1.a Currently Ruby-VPI is in control, and then passes control over > to the single verilog instantiation. > If there are multiple ruby models, can they live in the master > ruby-vpi thread? Yes, with the separate namespaces, they can. > If so, I suppose they would need to be registered with the 'main' > control instance, which would need to call a 'simulate' method in > each independent module. Correct. The global "simulate" method is replaced by a DUT.cycle! method in the next release (where DUT is the name of the Verilog DUT). > 1.b At the moment there is an all-or-nothing PROTOTYPE switch - > this would have to have finer grained control such as a PROTOTYPE > level (or see later) I see, you would like to test a verilog DUT with a prototype, and vice versa. I don't have an answer right now, but we'll think of a way to specify such parameters. > 2) How does one cope with multiple clocks? In the current release, you would do this by editing the "simulate" method -- it can toggle/wiggle multiple clocks (any signal in general) each time it is called. > I guess this might be achieved by advancing time on a granularity > determined by the fastest clock. > Incidentally, where is the time advance increment set? I haven't > spotted it anywhere so far. Look at the advance_time() method in lib/ruby-vpi/vpi.rb The default argument for that method is 1. > 3) Is it possible to have multiple verilog builds? > I guess this would be done by having multiple targets in the main > rake file, so that one could have rtl, post-layout sdf with min, > max timings, etc.. These would replace the cver target If each of these targets can be achieved by specifying different command-line arguments to the simulator, then I think this may be possible already. You just have to define new tasks that modify/redefine the SIMULATOR_ARGUMENTS hash-table: desc "tests the RTL" task :rtl do SIMULATOR_ARGUMENTS[:cver] = "args for RTL stuff" end desc "runs the post layout stuff" task :post_layout do SIMULATOR_ARGUMENTS[:cver] = "args for post layout stuff" end And now, you would run rake with the name of these new tasks instead of just cver: rake post_layout rake --tasks # will show all targets + description > 4) At the moment, the use of the xUnit and RSpec frameworks is > really only oriented towards block-level testing. (well, that's > my impression at the moment, though I haven't really played > enough yet..) True, but that's mainly due to how Ruby-VPI has been organized so far. > What are your thoughts on system-level tests? I'd like to support them eventually. :-) The first step is to support integration tests, with multiple verilog DUTs and prototypes working together. I'm not sure what comes afterwards, so I'd like to ask you to guide me through your vision at that point. > 5) Is there a methodology for hierarchical application of > tests/specifications? > What I'm thinking of here, is that teams may develop IP blocks, > and their associated RSpec, RUnit tests. However, eventually, > these need to be assembled into a chip. > It would be useful to have some means whereby these tests could > be incorporated into the system level tests. There seems to be a > mechanism for this in RSpec, where one can specify behaviours as > being shared. > However, this may not be so simple here, since one may have some > block connected to a peripheral-type bus, which is in turn driven > by a system bus, which is in turn driven by code running on a > processor. Hence the tests must be 'propagated' upwards to run in > an environment that may be different from the original. If the chain of interaction is reasonably decoupled then there shouldn't be any problems. But I see your point here, and it would be good to incorporate ideas from SystemC to implement the decoupling behind the scenes. I'm not very familiar with SystemC and ESL but that is the general direction in which I want to drive this project. > Plus, here are a few things you might like to check out: [...] > C) Have a look at Teal and Truss: > http://www.trusster.com/ > Notice anything familiar? > The Teal VPI interface seems to cover a lot of the ground that > Ruby-VPI does. > I'd say it seems to be interfaced a little less elegantly, but it > does provide support for running multiple testbench threads, > which are used in the Truss framework. The latter is basically a > replacement for a verilog testbench - it is used to hook together > stimulators written in C++ to the device under test. Okay, I'll have a look at this again. > This is all fine and dandy, but what interests me is how to > provide better verification, not just to provide a better > framework for doing verification (if you see what I mean..) I think I understand: you want a (preferably invisible) light weight framework that won't get in the way of the actual task at hand: verification. > Hence the interest ins somehow promoting your test methodology up > to system level. Excellent! I have been waiting for someone knowledgeable in ESL to learn from in order to drive this project to larger scale verification. Let's talk some more after this next release and see what needs to be done. Thanks for your interest!
