Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 20:09:09 Alexander Neundorf wrote: On Thursday 25 April 2013, Matthias Kretz wrote: On Thursday 25 April 2013 09:46:03 Bill Hoffman wrote: On 4/25/2013 9:15 AM, Matthias Kretz wrote: I have several checks in my project that make cmake error out with FATAL_ERROR. This is submitted as error to the dashboard. In reality it would be an error if this error would not appear - and it is the expected result if cmake errors out. Is there a possiblity to invert the result of the configure step for certain system setups? Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and just submit Notes that elaborate on the abort. But that doesn't show up on the dashboard at all. Any ideas? Is this for an individual test? If so, you can use the regex matching instead of return code to determine pass fail. Would that work? I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? so the good result is if the configure step of your project fails ? Exactly. And running some cmake code in the test phase is certainly possible, but not what I'm looking for. It would require the configure step of my project to pass, disable all targets, and enable a test that doesn't do anything useful anymore, because the test already happened at configure time. So I could also simply skip that and just let configure pass + disable all targets. -- Matthias Kretz (Germany) http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
Matthias Kretz wrote: On Thursday 25 April 2013 10:30:14 Bill Hoffman wrote: On 4/25/2013 10:05 AM, Matthias Kretz wrote: I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? But, if you have a FATAL_ERROR in the configure step, no build files will be created and you won't even get a build of the software to test. Which is the correct behavior. And I'd like my dashboard to tell me that it works. Without building or testing. The test in this case consists of making sure that my cmake script aborts with the right error message. See the RunCMake tests in the cmake test suite. Thanks, Steve. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 10:30:14 Bill Hoffman wrote: On 4/25/2013 10:05 AM, Matthias Kretz wrote: I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? But, if you have a FATAL_ERROR in the configure step, no build files will be created and you won't even get a build of the software to test. Which is the correct behavior. And I'd like my dashboard to tell me that it works. Without building or testing. The test in this case consists of making sure that my cmake script aborts with the right error message. I thought you might have test cases that were complete projects. I don't think you can change this. It is not just the error in the output, it is that cmake returns an error code when run. Basically the configure command fails. Yes. And if it wouldn't fail it would be an error. That's what I mean: I need to invert the logic of this part. Well, maybe it's better that those builds don't show on the dashboard at all... Cheers, Matthias -- Matthias Kretz (Germany) http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On 4/26/2013 4:35 AM, Matthias Kretz wrote: Which is the correct behavior. And I'd like my dashboard to tell me that it works. Without building or testing. The test in this case consists of making sure that my cmake script aborts with the right error message. I thought you might have test cases that were complete projects. I don't think you can change this. It is not just the error in the output, it is that cmake returns an error code when run. Basically the configure command fails. Yes. And if it wouldn't fail it would be an error. That's what I mean: I need to invert the logic of this part. Well, maybe it's better that those builds don't show on the dashboard at all... What I would do is turn this into a test case that is part of a successful build. add_test(FailConfig ... run some cmake script that does the config that fails) # use a regular expression to determine that the correct output shows up. You can do an out of source build so that does not mess with the tree that works. -- Bill Hoffman Kitware, Inc. 28 Corporate Drive Clifton Park, NY 12065 bill.hoff...@kitware.com http://www.kitware.com 518 881-4905 (Direct) 518 371-3971 x105 Fax (518) 371-4573 -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
Hi, I have several checks in my project that make cmake error out with FATAL_ERROR. This is submitted as error to the dashboard. In reality it would be an error if this error would not appear - and it is the expected result if cmake errors out. Is there a possiblity to invert the result of the configure step for certain system setups? Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and just submit Notes that elaborate on the abort. But that doesn't show up on the dashboard at all. Any ideas? Cheers, Matthias -- Matthias Kretz (Germany) http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 09:46:03 Bill Hoffman wrote: On 4/25/2013 9:15 AM, Matthias Kretz wrote: I have several checks in my project that make cmake error out with FATAL_ERROR. This is submitted as error to the dashboard. In reality it would be an error if this error would not appear - and it is the expected result if cmake errors out. Is there a possiblity to invert the result of the configure step for certain system setups? Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and just submit Notes that elaborate on the abort. But that doesn't show up on the dashboard at all. Any ideas? Is this for an individual test? If so, you can use the regex matching instead of return code to determine pass fail. Would that work? I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? Also note that CTEST_CUSTOM_ERROR_EXCEPTION has no effect (and it's documented to be used for ctest_build only). Cheers, Matthias -- Matthias Kretz (Germany) http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On 4/25/2013 10:05 AM, Matthias Kretz wrote: I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? But, if you have a FATAL_ERROR in the configure step, no build files will be created and you won't even get a build of the software to test. I thought you might have test cases that were complete projects. I don't think you can change this. It is not just the error in the output, it is that cmake returns an error code when run. Basically the configure command fails. -Bill -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013, Matthias Kretz wrote: On Thursday 25 April 2013 09:46:03 Bill Hoffman wrote: On 4/25/2013 9:15 AM, Matthias Kretz wrote: I have several checks in my project that make cmake error out with FATAL_ERROR. This is submitted as error to the dashboard. In reality it would be an error if this error would not appear - and it is the expected result if cmake errors out. Is there a possiblity to invert the result of the configure step for certain system setups? Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and just submit Notes that elaborate on the abort. But that doesn't show up on the dashboard at all. Any ideas? Is this for an individual test? If so, you can use the regex matching instead of return code to determine pass fail. Would that work? I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? so the good result is if the configure step of your project fails ? Alex -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
