On Mar 8, 2012, at 5:49 AM, Tristan Gingold wrote:
> Argh, that's an issue.  We don't run the gcc test suite natively on VMS
> because there is no port of Dejagnu (if ever doable) to VMS.  We haven't tried
> to test a cross-compiler (and running the executable on the VMS host) because
> an early attempt for another test suite pointed out slowness and reliability
> issues.

dejagnu slices through this type of testing just fine.  dejagnu is also adept 
at handling reliability issues, its history is littered with unreliability and 
it is usually fairly easy to work around any unreliability.  Selecting targets 
that happen to be in a `working' state, powercycling them, as needed, noticing 
when things go wrong, retrying things a few times, as sometimes, something 
doesn't just work and so on.  Also, the cross testing can come in many flavors, 
you can use a simulator (if you have one) and do cross and test on simulator.  
You can do this, without the simulator and just fail all the execute tests, you 
can do canadian cross controlling host to native host testing.  As for speed, 
well, it is all about latency and reliability, the lower the latency and the 
higher the reliability, the faster the testing, but, it is, what it is.  The 
modern testsuite might be 8 hour range or more, but overnight testing is better 
than no testing.  If you hide it behind a git send hook and stage everything 
through git and then push out from git as the testsuite passes...  you should 
be able to achieve a nice work-flow.

>  VMS machines could be considered as slow from today's standard POV.
> I haven't found a method to run only the compile tests and skip the executing 
> one.
> Is it possible to do that with the gcc test suite ?

If you configure a cross compiler and do a make check, you'll just get a fast 
fail on all the execute tests.  If you just look for regressions, you'll notice 
this works just fine.  Sit back, don't worry about the execution failures.  
When you wire up sim, just say the simulator is /bin/false or /bin/true 
(set_board_info sim /bin/false)

Feel free to email me directly if you need additional pointers.  It is fairly 
easy to setup, though, daunting, one has never done it before.

Reply via email to