Re: [cmake-developers] Tests with custom launchers and Not Run
David Cole wrote: I don't like it. Existing tests that run and return, for example, a number of errors that occurred, will magically appear as not run when that number just so happens to be 77. If there are enough people who think this is simple and works and are not concerned about the accidental matching of an intentional return value of 77 that does NOT mean not run ... then I will relent and say, so be it, and allow it in. But only if there are some people who speak up here or add notes to the bug. It just seems wrong to me to treat 77 as some special number here. I would still go and make that a target property where you can set the return code where something is considered skipped. The only question is: how to name the property? Eike signature.asc Description: This is a digitally signed message part. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
On 19/09/12 08:18, Rolf Eike Beer wrote: David Cole wrote: I don't like it. Existing tests that run and return, for example, a number of errors that occurred, will magically appear as not run when that number just so happens to be 77. If there are enough people who think this is simple and works and are not concerned about the accidental matching of an intentional return value of 77 that does NOT mean not run ... then I will relent and say, so be it, and allow it in. But only if there are some people who speak up here or add notes to the bug. It just seems wrong to me to treat 77 as some special number here. I would still go and make that a target property where you can set the return code where something is considered skipped. The only question is: how to name the property? Eike -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers I already suggested to use a test property three years ago (see issue #8466). At that time David was very much against it, fearing to find a ***huueee*** can of worms right underneath the surface. Seems I'm slowly getting more fellow thinkers ;) Regards, Marcel Loose. attachment: loose.vcf-- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
On 09/12/2012 11:23 PM, Mathias Gaunard wrote: On 12/09/2012 22:39, Mathias Gaunard wrote: I'd be content with just interpreting a return value of 77 as a Not Run status; this is a fairly trivial change that should work well in any existing setups. I don't think it is really needed to make things as complex as in the bug report. Any intelligence, if required, can go in the test command itself. Simple patch attached. This fulfills my needs well enough and I've just configured my cross-compilation test servers to use it. This is simple and works. Is it satisfying? Any chance of having it included? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
I don't like it. Existing tests that run and return, for example, a number of errors that occurred, will magically appear as not run when that number just so happens to be 77. If there are enough people who think this is simple and works and are not concerned about the accidental matching of an intentional return value of 77 that does NOT mean not run ... then I will relent and say, so be it, and allow it in. But only if there are some people who speak up here or add notes to the bug. It just seems wrong to me to treat 77 as some special number here. On Tue, Sep 18, 2012 at 5:11 PM, Mathias Gaunard mathias.gaun...@ens-lyon.org wrote: On 09/12/2012 11:23 PM, Mathias Gaunard wrote: On 12/09/2012 22:39, Mathias Gaunard wrote: I'd be content with just interpreting a return value of 77 as a Not Run status; this is a fairly trivial change that should work well in any existing setups. I don't think it is really needed to make things as complex as in the bug report. Any intelligence, if required, can go in the test command itself. Simple patch attached. This fulfills my needs well enough and I've just configured my cross-compilation test servers to use it. This is simple and works. Is it satisfying? Any chance of having it included? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/** opensource/opensource.htmlhttp://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/**CMake_FAQhttp://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-**bin/mailman/listinfo/cmake-**developershttp://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
On 09/18/2012 11:37 PM, David Cole wrote: I don't like it. Existing tests that run and return, for example, a number of errors that occurred, will magically appear as not run when that number just so happens to be 77. If there are enough people who think this is simple and works and are not concerned about the accidental matching of an intentional return value of 77 that does NOT mean not run ... then I will relent and say, so be it, and allow it in. But only if there are some people who speak up here or add notes to the bug. It just seems wrong to me to treat 77 as some special number here. Well, you need to interpret a return value as a special value to be able to tell executables could not be tested. Without this, a whole category status is entirely unusable to whoever wants to use custom launchers! That's a pretty serious issue to me. 77 already exists in the autotools world, so it seems like a good idea to re-use it. Making the executable return the number of failing tests is a very bad idea: it doesn't even scale beyond 255 for the average operating system. Certain operating systems also have funny behaviour with exit status as well: with OpenVMS, all even values are considered failures and all odd values are considered successes. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
There's discussion about that in the following bug tracker entry: http://public.kitware.com/Bug/view.php?id=8466 That issue is presently in the 'backlog' though, which means nobody is actively looking at it. And there does not appear to be a consensus forming about what the right solution to the problem is. Although if everybody else involve converges on a reasonable consensus *and* the CDash results still make sense, I will withdraw my objections to the feature request. On Wed, Sep 12, 2012 at 1:26 PM, Mathias Gaunard mathias.gaun...@ens-lyon.org wrote: Hi, When not using a custom launcher for a test, if the executable doesn't exist then ctest will report Not Run. Is there a way to do this with a custom launcher as well? When the file doesn't exist it always gives me Failed instead. I need to run my tests in qemu because of cross-compilation, and because of this all my Not Run tests end up being marked as Failed, which is a bit annoying. Maybe CTest should be changed to interpret a certain return value as test does not exist? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Tests with custom launchers and Not Run
I'd be content with just interpreting a return value of 77 as a Not Run status; this is a fairly trivial change that should work well in any existing setups. I don't think it is really needed to make things as complex as in the bug report. Any intelligence, if required, can go in the test command itself. On 12/09/2012 20:59, David Cole wrote: There's discussion about that in the following bug tracker entry: http://public.kitware.com/Bug/view.php?id=8466 That issue is presently in the 'backlog' though, which means nobody is actively looking at it. And there does not appear to be a consensus forming about what the right solution to the problem is. Although if everybody else involve converges on a reasonable consensus *and* the CDash results still make sense, I will withdraw my objections to the feature request. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
