Re: [cmake-developers] CTest script and many subprojects

[David Cole via cmake-developers]

I think your analysis is correct.

You could try doing a ctest_submit after each ctest_test call. Not
sure if you could get "correct" results with that technique. I run
some scripts that do something similar without subproject involved,
and it mostly works, but messes up the +/- tests passed and failed
since last submission data on the instance of CDash I'm using.

If you would like it to work differently than it presently does, it
would probably involve some changes to ctest. Making it do what you
want it to do and becoming a contributor is probably the quickest way
to achieve that.


HTH,
David C.



On Fri, Sep 1, 2017 at 2:36 PM, Wouter Klouwen
 wrote:
> Hi all,
>
> I've been having some success with CTest and am interested in using
> CDash as well.
>
> In order to generate some nice output, sub projects seem to be the best
> way of presenting the data in CDash for our setup. This setup is perhaps
> summed up by having one projects that contains a few hundred projects.
> Most of these "sub projects" are normal CMake projects.
>
> Now, in the normal build, we have a `check` target. This is powered by
> having tests added with add_test() and this check target invokes CTest
> with a script that contains the setting of the SubProject property.
>
> This produces the expected Test.xml with the SubProject tag. When
> submitting these to CDash, it produces the right subproject view and the
> expected aggregate view.
>
> The next step is that we have code on devices for which we cross compile
> and we want to use the same mechanism here in order to execute tests
> when installed on our target devices.
>
> Before I started the CDash work, I was able to generate CTest files
> which contain the add_test() and set_tests_properties() in order to run
> the tests on the devices. This was then invoked with the
> DartConfiguration.tcl and ctest -T Test
>
> Perhaps not ideal as it involves generated many CTestTestfile.cmakes,
> but this was able to execute the several hundreds of tests with the
> appropriate load balancing and produce one giant Test.xml.
>
> In order to switch to generate Test.xml files with SubProject tags this
> mechanism needs to be switched to using a CTest script.
>
> As SubProject is a global property, I assume this needs to be unique for
> each invocation of ctest_test().
> Unfortunately putting multiple ctest_test() calls in one CTest script
> seems to result in one Test.xml, which is the last project run. The test
> of the output is overwritten.
>
> It seems the only way I can make this work is by having a script which
> effectively creates a new CTest script for each subproject, and moves
> the produced Test.xml out of the way for the next run.
> Is this right?
>
> I hope this all makes sense.
>
> Thanks in advance,
> W
> This transmission contains information that may be confidential and contain
> personal views which are not necessarily those of YouView TV Ltd. YouView TV
> Ltd (Co No:7308805) is a limited liability company registered in England and
> Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower
> Thames Street, London, EC3R 6YT. For details see our web site at
> http://www.youview.com
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] How should config packages handle components?

[Alex Turbov]

I've started to use CMake a quite long time ago, and do not search for
"standards" or "guidelines" anymore %)
But I could mention some chapters in the official documentation (in the
"Reference Manuals" section) which are really full of "secret knowledge",
but the problem is that they are too complicated for newcomers %) but
personally I read them many many times before and each time found smth
"new" to me :)

I would say that after a couple of years I started to feel comfortable w/
CMake -- when I've used almost all offered features in a different set of
platform and generators.

On Fri, Sep 1, 2017 at 9:49 PM, Robert Dailey 
wrote:

> On Fri, Sep 1, 2017 at 1:40 PM, Alex Turbov  wrote:
> > Hi Robert,
> >
> >
> > On Fri, Sep 1, 2017 at 9:21 PM, Robert Dailey 
> > wrote:
> >>
> >>
> >> One problem I thought of with the former (one big target.cmake with
> >> all import targets in there) is that if you only ask for a subset of
> >> components in find_package(), you will still get all of them since all
> >> imports are defined in a single file.
> >
> >
> > In my project I have a bunch of components and do one exported target per
> > component
> > exactly by the mentioned reason -- user didn't ask for others...
> >
> >>
> >> Does this go against any design
> >> principles?
> >
> >
> > As far as I know, there are no clear design principles :) (yet, at least
> > nowadays) -- at least doing
> > a lot of CMake projects since 2009, I've never seen an explicit list of
> them
> > %)
> > IMHO, there is a lack of "official guildelines" (or it is really hard to
> > search for 'em)
> >
> >> Assuming this really happens, are there any negative side
> >> effects?
> >
> >
> > I could see the impact on build time only in this case... and for me the
> > most obvious is increasing
> > time to process the lists (which is for some reasons really slow on
> Windows,
> > at least in our
> > build farm which uses vargant and VirtualBox images)
> > (but I don't have any particular numbers, cuz never implemented the first
> > approach)
>
> Thanks for the quick response. The "official guidelines" or "package
> standard" is really exactly what we need I think. What worries me the
> most is that it seems like this is deep knowledge that is stuck in the
> brains of folks like Brad King and David Cole. I think somehow getting
> a knowledge dump from them into a documentation page would be a
> valuable task. I think for something as complex and variable as
> packages in CMake (install process in general) deserves some
> standardization, because we need the ability to distinguish between
> practices that we should follow for legacy (backward compatibility)
> reasons, non-cmake project reasons, and fully "modern" cmake packages.
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] How should config packages handle components?

[Robert Dailey]

On Fri, Sep 1, 2017 at 1:40 PM, Alex Turbov  wrote:
> Hi Robert,
>
>
> On Fri, Sep 1, 2017 at 9:21 PM, Robert Dailey 
> wrote:
>>
>>
>> One problem I thought of with the former (one big target.cmake with
>> all import targets in there) is that if you only ask for a subset of
>> components in find_package(), you will still get all of them since all
>> imports are defined in a single file.
>
>
> In my project I have a bunch of components and do one exported target per
> component
> exactly by the mentioned reason -- user didn't ask for others...
>
>>
>> Does this go against any design
>> principles?
>
>
> As far as I know, there are no clear design principles :) (yet, at least
> nowadays) -- at least doing
> a lot of CMake projects since 2009, I've never seen an explicit list of them
> %)
> IMHO, there is a lack of "official guildelines" (or it is really hard to
> search for 'em)
>
>> Assuming this really happens, are there any negative side
>> effects?
>
>
> I could see the impact on build time only in this case... and for me the
> most obvious is increasing
> time to process the lists (which is for some reasons really slow on Windows,
> at least in our
> build farm which uses vargant and VirtualBox images)
> (but I don't have any particular numbers, cuz never implemented the first
> approach)

Thanks for the quick response. The "official guidelines" or "package
standard" is really exactly what we need I think. What worries me the
most is that it seems like this is deep knowledge that is stuck in the
brains of folks like Brad King and David Cole. I think somehow getting
a knowledge dump from them into a documentation page would be a
valuable task. I think for something as complex and variable as
packages in CMake (install process in general) deserves some
standardization, because we need the ability to distinguish between
practices that we should follow for legacy (backward compatibility)
reasons, non-cmake project reasons, and fully "modern" cmake packages.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] How should config packages handle components?

[Alex Turbov]

Hi Robert,


On Fri, Sep 1, 2017 at 9:21 PM, Robert Dailey 
wrote:

>
> One problem I thought of with the former (one big target.cmake with
> all import targets in there) is that if you only ask for a subset of
> components in find_package(), you will still get all of them since all
> imports are defined in a single file.


In my project I have a bunch of components and do one exported target per
component
exactly by the mentioned reason -- user didn't ask for others...


> Does this go against any design
> principles?


As far as I know, there are no clear design principles :) (yet, at least
nowadays) -- at least doing
a lot of CMake projects since 2009, I've never seen an explicit list of
them %)
IMHO, there is a lack of "official guildelines" (or it is really hard to
search for 'em)

Assuming this really happens, are there any negative side
> effects?
>

I could see the impact on build time only in this case... and for me the
most obvious is increasing
time to process the lists (which is for some reasons really slow on
Windows, at least in our
build farm which uses vargant and VirtualBox images)
(but I don't have any particular numbers, cuz never implemented the first
approach)




--
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] CTest script and many subprojects

[Wouter Klouwen]

Hi all,

I've been having some success with CTest and am interested in using
CDash as well.

In order to generate some nice output, sub projects seem to be the best
way of presenting the data in CDash for our setup. This setup is perhaps
summed up by having one projects that contains a few hundred projects.
Most of these "sub projects" are normal CMake projects.

Now, in the normal build, we have a `check` target. This is powered by
having tests added with add_test() and this check target invokes CTest
with a script that contains the setting of the SubProject property.

This produces the expected Test.xml with the SubProject tag. When
submitting these to CDash, it produces the right subproject view and the
expected aggregate view.

The next step is that we have code on devices for which we cross compile
and we want to use the same mechanism here in order to execute tests
when installed on our target devices.

Before I started the CDash work, I was able to generate CTest files
which contain the add_test() and set_tests_properties() in order to run
the tests on the devices. This was then invoked with the
DartConfiguration.tcl and ctest -T Test

Perhaps not ideal as it involves generated many CTestTestfile.cmakes,
but this was able to execute the several hundreds of tests with the
appropriate load balancing and produce one giant Test.xml.

In order to switch to generate Test.xml files with SubProject tags this
mechanism needs to be switched to using a CTest script.

As SubProject is a global property, I assume this needs to be unique for
each invocation of ctest_test().
Unfortunately putting multiple ctest_test() calls in one CTest script
seems to result in one Test.xml, which is the last project run. The test
of the output is overwritten.

It seems the only way I can make this work is by having a script which
effectively creates a new CTest script for each subproject, and moves
the produced Test.xml out of the way for the next run.
Is this right?

I hope this all makes sense.

Thanks in advance,
W
This transmission contains information that may be confidential and contain 
personal views which are not necessarily those of YouView TV Ltd. YouView TV 
Ltd (Co No:7308805) is a limited liability company registered in England and 
Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower Thames 
Street, London, EC3R 6YT. For details see our web site at http://www.youview.com
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] How should config packages handle components?

[Robert Dailey]

First of all, I want to apologize for including the developer list.
Maybe I'm not being patient enough, but it seems like every post I've
made on the normal users list doesn't get any attention.

Secondly, the cmake-packages portion of the cmake documentation
doesn't go into a ton of detail about components, but it does give an
example towards the bottom of how you export targets for components.
This leads to my questions:

When defining the target exports via install(TARGET foo EXPORT
foo-export), is it recommended for all components to collectively
export as 1 target.cmake script? Or is it better to have 1
target.cmake script per component? If we use Boost as an example, the
latter would mean having:

boost-config.cmake
boost-target-filesystem.cmake
boost-target-thread.cmake
...etc...

This means that boost-config.cmake would "include()" only the relevant
target cmake scripts based on the provided COMPONENTS list, I assume?

Which is the better approach here and why?

One problem I thought of with the former (one big target.cmake with
all import targets in there) is that if you only ask for a subset of
components in find_package(), you will still get all of them since all
imports are defined in a single file. Does this go against any design
principles? Assuming this really happens, are there any negative side
effects?
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers