D28253: ECMPoQmToolsTest: have separate moc files for tr_thread_test 1 & 2
This revision was automatically updated to reflect the committed changes. Closed by commit R240:6136a9742626: ECMPoQmToolsTest: have separate moc files for tr_thread_test 1 2 (authored by kossebau). REPOSITORY R240 Extra CMake Modules CHANGES SINCE LAST UPDATE https://phabricator.kde.org/D28253?vs=78393=78399 REVISION DETAIL https://phabricator.kde.org/D28253 AFFECTED FILES tests/ECMPoQmToolsTest/CMakeLists.txt To: kossebau, #frameworks, #build_system, dfaure Cc: kde-frameworks-devel, kde-buildsystem, LeGast00n, cblack, GB_2, bencreasy, michaelh, ngraham, bruns
D28253: ECMPoQmToolsTest: have separate moc files for tr_thread_test 1 & 2
dfaure accepted this revision. dfaure added a comment. This revision is now accepted and ready to land. Thanks! REPOSITORY R240 Extra CMake Modules BRANCH fixtr_thread_tests REVISION DETAIL https://phabricator.kde.org/D28253 To: kossebau, #frameworks, #build_system, dfaure Cc: kde-frameworks-devel, kde-buildsystem, LeGast00n, cblack, GB_2, bencreasy, michaelh, ngraham, bruns
Re: ECM unittest fails, why?
Am Dienstag, 24. März 2020, 07:48:22 CET schrieb Friedrich W. H. Kossebau:
> Am Montag, 23. März 2020, 20:30:59 CET schrieb Friedrich W. H. Kossebau:
> > Am Sonntag, 22. März 2020, 10:50:39 CET schrieb David Faure:
> > > The CI for ECM has been failing ever since the tests were re-enabled.
> > >
> > > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.12/job/
> > > ex
> > > tr a-cmake-modules/job/kf5-qt5%20SUSEQt5.12/103/console shows that
> > > ECMPoQmToolsTest fails with "tr_thread_test.moc: No such file or
> > > directory".
> > >
> > > I have been trying to reproduce this locally, but no such luck.
> >
> > Reason seems to be that on CI the target tr_thread_test is handled before
> > tr_thread_test_2, while for us locally the order is the other way around.
> > And with tr_thread_test_2 the file tr_thread_test.moc is generated first,
> > so present in build dir afterwards, while with tr_thread_test the automoc
> > seems not happening, other than what the code hopes (perhaps some cmake
> > policy change since that code was written?).
> >
> > In general the test setup here seems bogus a bit, as one would expect that
> > the two different test cases (1 & 2) should not share the same url for the
> > generated moc file and have them in respective exclusive include dirs.
> >
> > I might have a closer look later tonight, if no-one else would work on
> > this.
> Seems the reason is that the same source file is reused for both targets,
> and the variant without automoc, explicitly calling qt5_generate_moc() on
> the source file, with target tr_thread_test_2, also sees this call (with
> infile being the source file):
> set_source_files_properties(${infile} PROPERTIES SKIP_AUTOMOC ON)
>
> That property is not bound to any target, but global for the source file.
> And as result enabling automoc with the other target tr_thread_test, by
> set_target_properties(tr_thread_test PROPERTIES AUTOMOC ON)
> will still keep cmake not doing any automoc for the very source file.
>
> So when tr_thread_test is handled first, the moc file is not yet generated,
> and thus the include of it fails.
https://phabricator.kde.org/D28253 would be some approach to this, without
changing things too much, and just ensuring working flags.
Pretty messy, so not really happy, but no really better/cleaner idea right
now.
Cheers
Friedrich
D28253: ECMPoQmToolsTest: have separate moc files for tr_thread_test 1 & 2
kossebau created this revision. kossebau added reviewers: Frameworks, Build System, dfaure. Herald added projects: Frameworks, Build System. Herald added subscribers: kde-buildsystem, kde-frameworks-devel. kossebau requested review of this revision. REPOSITORY R240 Extra CMake Modules BRANCH fixtr_thread_tests REVISION DETAIL https://phabricator.kde.org/D28253 AFFECTED FILES tests/ECMPoQmToolsTest/CMakeLists.txt To: kossebau, #frameworks, #build_system, dfaure Cc: kde-frameworks-devel, kde-buildsystem, LeGast00n, cblack, GB_2, bencreasy, michaelh, ngraham, bruns
Re: ECM unittest fails, why?
On Tue, Mar 24, 2020 at 4:07 PM Friedrich W. H. Kossebau wrote: > > Am Dienstag, 24. März 2020, 08:37:56 CET schrieb Ben Cooksley: > > On Tue, Mar 24, 2020 at 7:48 PM Friedrich W. H. Kossebau > > > > wrote: > > > Why CI does this order, but not locally for us, no clue so far. > > > > Could this ordering be a race condition by any chance? > > > > Two of the three nodes that perform the builds on the CI system are > > 3rd generation Ryzen 7 systems (Octa Cores) with NVMe storage, so > > there is a possibility this is the case... > > There is only one thread/job queue here when make is run if I saw correctly. > So no race here I think. It might be rather something which determines the > order of otherwise same-level targets handled, something like unsorted hash > table layout, random number generation seed, something of that kind, > Perhaps the order in which directory entries are returned, which would depend on inode creation? Regards, - Johan
Re: ECM unittest fails, why?
Am Dienstag, 24. März 2020, 08:37:56 CET schrieb Ben Cooksley: > On Tue, Mar 24, 2020 at 7:48 PM Friedrich W. H. Kossebau > > wrote: > > Why CI does this order, but not locally for us, no clue so far. > > Could this ordering be a race condition by any chance? > > Two of the three nodes that perform the builds on the CI system are > 3rd generation Ryzen 7 systems (Octa Cores) with NVMe storage, so > there is a possibility this is the case... There is only one thread/job queue here when make is run if I saw correctly. So no race here I think. It might be rather something which determines the order of otherwise same-level targets handled, something like unsorted hash table layout, random number generation seed, something of that kind, > > No idea right now how to properly fix this (possibly need to have > > different > > source files then, perhaps a symlink), would look again in the evening. > > So feel invited to beat me on a proper solution :) > > Could we add a dependency between the targets to force the correct > ordering be followed? Those two targets should be independent and not have effects on each other, from what I had understood the design of the tests so far. So CI behaviour actually uncovers a flaw of the current cmakelists.txt code (generated moc file visible to build of other target, target specific code with side effects to other targets). The question about order of handling is more of general curiosity, to have full understanding of what we see. Cheers Friedrich
Re: ECM unittest fails, why?
On Tue, Mar 24, 2020 at 7:48 PM Friedrich W. H. Kossebau
wrote:
>
> Am Montag, 23. März 2020, 20:30:59 CET schrieb Friedrich W. H. Kossebau:
> > Am Sonntag, 22. März 2020, 10:50:39 CET schrieb David Faure:
> > > The CI for ECM has been failing ever since the tests were re-enabled.
> > >
> > > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.12/job/ex
> > > tr a-cmake-modules/job/kf5-qt5%20SUSEQt5.12/103/console shows that
> > > ECMPoQmToolsTest fails with "tr_thread_test.moc: No such file or
> > > directory".
> > >
> > > I have been trying to reproduce this locally, but no such luck.
> >
> > Reason seems to be that on CI the target tr_thread_test is handled before
> > tr_thread_test_2, while for us locally the order is the other way around.
> > And with tr_thread_test_2 the file tr_thread_test.moc is generated first, so
> > present in build dir afterwards, while with tr_thread_test the automoc
> > seems not happening, other than what the code hopes (perhaps some cmake
> > policy change since that code was written?).
> >
> > In general the test setup here seems bogus a bit, as one would expect that
> > the two different test cases (1 & 2) should not share the same url for the
> > generated moc file and have them in respective exclusive include dirs.
> >
> > I might have a closer look later tonight, if no-one else would work on this.
>
> Seems the reason is that the same source file is reused for both targets,
> and the variant without automoc, explicitly calling qt5_generate_moc() on the
> source file, with target tr_thread_test_2, also sees this call (with infile
> being the source file):
> set_source_files_properties(${infile} PROPERTIES SKIP_AUTOMOC ON)
>
> That property is not bound to any target, but global for the source file. And
> as result enabling automoc with the other target tr_thread_test, by
> set_target_properties(tr_thread_test PROPERTIES AUTOMOC ON)
> will still keep cmake not doing any automoc for the very source file.
>
> So when tr_thread_test is handled first, the moc file is not yet generated,
> and thus the include of it fails.
> Why CI does this order, but not locally for us, no clue so far.
Could this ordering be a race condition by any chance?
Two of the three nodes that perform the builds on the CI system are
3rd generation Ryzen 7 systems (Octa Cores) with NVMe storage, so
there is a possibility this is the case...
>
> No idea right now how to properly fix this (possibly need to have different
> source files then, perhaps a symlink), would look again in the evening.
> So feel invited to beat me on a proper solution :)
Could we add a dependency between the targets to force the correct
ordering be followed?
>
> Cheers
> Friedrich
>
>
Cheers,
Ben
Re: ECM unittest fails, why?
Am Montag, 23. März 2020, 20:30:59 CET schrieb Friedrich W. H. Kossebau:
> Am Sonntag, 22. März 2020, 10:50:39 CET schrieb David Faure:
> > The CI for ECM has been failing ever since the tests were re-enabled.
> >
> > https://build.kde.org/job/Frameworks/view/Platform%20-%20SUSEQt5.12/job/ex
> > tr a-cmake-modules/job/kf5-qt5%20SUSEQt5.12/103/console shows that
> > ECMPoQmToolsTest fails with "tr_thread_test.moc: No such file or
> > directory".
> >
> > I have been trying to reproduce this locally, but no such luck.
>
> Reason seems to be that on CI the target tr_thread_test is handled before
> tr_thread_test_2, while for us locally the order is the other way around.
> And with tr_thread_test_2 the file tr_thread_test.moc is generated first, so
> present in build dir afterwards, while with tr_thread_test the automoc
> seems not happening, other than what the code hopes (perhaps some cmake
> policy change since that code was written?).
>
> In general the test setup here seems bogus a bit, as one would expect that
> the two different test cases (1 & 2) should not share the same url for the
> generated moc file and have them in respective exclusive include dirs.
>
> I might have a closer look later tonight, if no-one else would work on this.
Seems the reason is that the same source file is reused for both targets,
and the variant without automoc, explicitly calling qt5_generate_moc() on the
source file, with target tr_thread_test_2, also sees this call (with infile
being the source file):
set_source_files_properties(${infile} PROPERTIES SKIP_AUTOMOC ON)
That property is not bound to any target, but global for the source file. And
as result enabling automoc with the other target tr_thread_test, by
set_target_properties(tr_thread_test PROPERTIES AUTOMOC ON)
will still keep cmake not doing any automoc for the very source file.
So when tr_thread_test is handled first, the moc file is not yet generated,
and thus the include of it fails.
Why CI does this order, but not locally for us, no clue so far.
No idea right now how to properly fix this (possibly need to have different
source files then, perhaps a symlink), would look again in the evening.
So feel invited to beat me on a proper solution :)
Cheers
Friedrich
