+1
On Sat, Sep 21, 2013 at 8:48 AM, Johnson, Hans J <[email protected]> wrote: > Matt, > > I agree with this approach! > > Hans > > On 9/20/13 5:40 PM, "Matt McCormick" <[email protected]> wrote: > >>I did a visualization on test times [1] and it turns out that less >>than 1% of the tests (0.97%) take of half (48.7%) of the test times! >> >>It would be good to add a "RUNS_LONG" label [2] to these tests so that >>developers can run the test suite relatively quickly with >> >> ctest -LE RUNS_LONG >> >>and the Visual Studio Debug Nightly builds can exclude them with >> >> set(CTEST_TEST_ARGS EXCLUDE_LABEL RUNS_LONG) >> >>These tests are mostly integration tests that test complicated >>algorithms on semi-realistic data. They have value, but leaving them >>out in these situations should not do much harm relative to their time >>overhead. >> >>Thanks, >>Matt >> >>[1] http://review.source.kitware.com/#/c/12733/ >>[2] http://review.source.kitware.com/#/c/12734/ >> >> >>On Fri, Aug 2, 2013 at 11:13 AM, Bill Lorensen <[email protected]> >>wrote: >>> In general, we do want to make sure the tests produce "correct" results >>>and >>> make sure there are no regressions. By having the Release build run a >>>proper >>> set of iterations, the developer can verify the correctness. >>> >>> That said, using a lower resolution input dataset could probably help >>>most >>> the these tests and the release/debug times would both be reduced. >>> >>> >>> >>> On Fri, Aug 2, 2013 at 10:49 AM, David Cole <[email protected]> wrote: >>>> >>>> >>>> > I think it is OK to drop the iterations in the Debug tests. We are >>>>still >>>> > exercising the code. If we >>>> > need alternate baselines, we will still catch exceptions. This will >>>>also >>>> > allow more valgrind tests >>>> > to run to completion. >>>> >>>> >>>> Ha ha. Ironic. I just noticed the slowest 3 tests have the word “Fast” >>>>in >>>> the test name... Perhaps we should at the very least consider a >>>>rename. 😊 >>>> >>>> If it’s ok to drop the iterations in the Debug tests, then why not drop >>>> the iterations in the Release tests, too? There is something to be >>>>said for >>>> the “same” code being run for the test in both Debug and Release. If >>>>it’s >>>> not the same, then I should definitely be able to tell just by >>>>glancing at >>>> the source code for a test that there are Debug/non-Debug >>>>differences... >>>> >>>> I would say each individual test should be able to run in *seconds*, >>>>not >>>> minutes or hours. Ideally, less than a second, but I realize that’s >>>>overly >>>> ambitious with some ITK code. >>>> >>>> However, for the main purpose I have in mind here, namely reducing the >>>> time burden on my volunteer machine, I would be absolutely *ecstatic* >>>>if all >>>> of the 20 tests I listed in the original email were able to run in >>>>under 4 >>>> or 5 minutes each. >>>> >>>> Let me know if I can help out in some additional way (beyond just >>>> continuing to submit the build to CDash...) >>>> >>>> >>>> Thanks for the discussion, I appreciate it >>>> D >>>> >>> >>> >>> >>> >>> -- >>> Unpaid intern in BillsBasement at noware dot com >>> >>> _______________________________________________ >>> Powered by www.kitware.com >>> >>> Visit other Kitware open-source projects at >>> http://www.kitware.com/opensource/opensource.html >>> >>> Kitware offers ITK Training Courses, for more information visit: >>> http://kitware.com/products/protraining.php >>> >>> Please keep messages on-topic and check the ITK FAQ at: >>> http://www.itk.org/Wiki/ITK_FAQ >>> >>> Follow this link to subscribe/unsubscribe: >>> http://www.itk.org/mailman/listinfo/insight-developers >>> >>_______________________________________________ >>Powered by www.kitware.com >> >>Visit other Kitware open-source projects at >>http://www.kitware.com/opensource/opensource.html >> >>Kitware offers ITK Training Courses, for more information visit: >>http://kitware.com/products/protraining.php >> >>Please keep messages on-topic and check the ITK FAQ at: >>http://www.itk.org/Wiki/ITK_FAQ >> >>Follow this link to subscribe/unsubscribe: >>http://www.itk.org/mailman/listinfo/insight-developers > > > > ________________________________ > Notice: This UI Health Care e-mail (including attachments) is covered by the > Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential > and may be legally privileged. If you are not the intended recipient, you > are hereby notified that any retention, dissemination, distribution, or > copying of this communication is strictly prohibited. Please reply to the > sender that you have received the message in error, then delete it. Thank > you. > ________________________________ -- Unpaid intern in BillsBasement at noware dot com _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://www.itk.org/mailman/listinfo/insight-developers
