>Martin Sebor wrote > >Travis Vitek wrote: >> >> >> Whatever timeout period you specify to --timeout-period=<N> >> will be the number of seconds that will elapse before the >> _rw_timeout_expired flag will be set to a non-zero value. >> [...] > >Right. But tests whose threads that don't check the flag will >continue running and, presumably, be killed by the infrastructure. >
Exactly. > >> If the user wants this timeout stuff in their multithreading >> test, then they have to add a hook in the command line arg >> processing stuff in main [just like they do for the --nthreads >> arg]. [...] > >How about implementing something more along the lines of what we >do with rw_opt_setlocales()? > Sorry, I gave a poor example. That is exactly what I mean. > >> From there I see a choice. Either the test can be expected >> to explicitly start/reset the alarm timer or it could be >> done implicitly within the rw_thread_pool() function. > >I don't follow this. Why would the test want to reset the alarm? >Or do you mean set it in response to the command line option? > Many of the multithreaded tests have several 'sections'. They create a pool of threads for each section. Take, for example, the test 22.locale.globals.mt.cpp. It has two sections. The first creates a pool of threads to call the function test_has_facet, and the other calls test_use_facet. Imagine that the first section would run for 30 seconds if given the time, and there was a 5 second timeout enabled. When the test is run, the first section would run for 5 seconds and the second section of the test would not run at all. That is no good. Even worse is the fact that the test would exit with a successful return code, so there would be no indication that the second part of the test has been skipped. I think that is a bad thing. >> Either >> way, before the threads are started the _rw_timeout_expired >flag would >> need to be cleared and the alarm would need to be set. After >all threads >> had exited, the alarm would need to be cancelled. >> >> This way each section of the test would use the provided timeout. The >> timeout could still have a default value of 30 seconds if we >wanted the >> tests to use this without needing to supply additional command line >> parameters to them from the build system. > >I see where you're going with this. You want the test to be able >to timeout, shut down all running threads when it does, and then >if there's more work to be done (i.e., another group of threads >to create), to reset the alarm and restart the whole process. >Right? Essentially. I don't mean restart the whole process in the literal sense. I mean it would go on to the next section of the test so at least that section would be exercised before the process exited. > >I guess I hadn't thought of the fact that some of these tests >create multiple thread pools. Hmm. I don't think the test should >be allowed to reset its own timeout but I'm not sure I see a good >solution. Yeah, ugh. >It certainly shouldn't exit successfully without doing >the rest of the work. Agreed. >Should it fail? Well, if we make the test fail aren't we essentially setting another hard limit? >Or should it just continue with the next pool and allow >itself to be killed? > If we do that, then this timeout idea is done. If the goal is to get the tests to run to completion sometime before the hard limit there are a few obvious changes we could make. In some cases we might be able to just reduce the default number of iterations. In other cases, we could combine 'sections' of the test. Some tests have a section to test a char specialization, then a wchar_t specialization section, and finally a combined section that tests both. We could potentially remove the first two sections, or at least disable them by default. Another option would be to continue testing these sections, but separate them out into their own tests. Travis
