Hi,
tl:dr
In your CMake code replace all remaining
add_test(<n> <c>)
with
add_test(NAME <n> COMMAND <c>)
to make sure test executables are still found once you bump the minimum ECM
version required >= 5.38.
And check your product on KDE CI, where not found test executables are now
properly reported as failure (everything got already up-to-date builds with
that change, if part of groups "Frameworks", "Applications" or "Extragear").
At least one project missed to see some unit-test regression on released
versions due to test executables not found and not run, without being
reported as failure on CI.
Long version:
It was found that on KDE CI some build states had been reported with a blue
ball (=all good), while actually most of its tests were not run, due to the
test executable not found. This was due to the logic on CI mapping ctest's
<notrun> to jenkins <skipped>, with the later not invalidating an "all good"
state. This has now been changed (D20874), <notrun> with 'Unable to find
executable' is now reported as <failure>, so will be treated like a failed
test.
(Locally a "make test" reports not found tests as failed, so we should have
ourselves run "make test" more often it seems :) ).
Tests executables not or no longer being found can happen when you bumped the
minimum required ECM version to 5.38 or above and use KDECMakeSettings.
Because this will trigger the placement of all built executable code into the
toplevel bin/ directory, not in the normal build dir.
Which breaks the inherit assumption of
add_executable(testfoo ${SRCS}) # add_executable(<target> ...)
add_test(foo-test testfoo) # add_test(<test-name> <command>)
where the command is taken as verbatim command and usually works as the name
of the executable target is also the name of the executable, which was placed
in the build dir, which is the default working directory on running the
tests.
To fix this, use the newer signature of add_test:
add_test(NAME foo-test COMMAND testfoo)
(so just insert "NAME" & "COMMAND" before the two arguments).
Because "[i]f <command> specifies an executable target (created by
add_executable()) it will automatically be replaced by the location of the
executable created at build time." Thus the test binaries being placed in
bin/ will no longer be an issue.
See https://cmake.org/cmake/help/latest/command/add_test.html
This signature exists already with cmake 2.8.12.
Any other related wrapper calls, like ecm_add_test, are fine and do not need
any work.
For more background why this is needed at all, see the docs about the efforts
trying to make tests & apps runnable uninstalled for developer convenience
(yes, no free lunch):
https://api.kde.org/ecm/kde-module/KDECMakeSettings.html
https://community.kde.org/Guidelines_and_HOWTOs/Making_apps_run_uninstalled
Cheers
Friedrich
