On 11. Jul 2019, at 14:45, Christian Kandeler <[email protected]> wrote:
>
> On Thu, 11 Jul 2019 12:18:30 +0000
> Maximilian Hrabowski <[email protected]> wrote:
>
>> I have a fairly large project with a root projects that pulls in several
>> SubProjects, altogether with unit tests its about 313 qbs files that are
>> pulled in.
>
> The number of project files is not all that important by itself. Qt Creator,
> for instance, has a comparable number, and resolves much faster than your
> project.
Can you give any guidance what to avoid, i.e. what makes resolving slow if QtC
is much faster resolved?
>
>> Let root.qbs be the root qbs file of my project and depend.qbs a subproject
>> that many other subprojects depend on.
>>
>> With a single profile and single config I encounter the following duration
>> for qbs resolve on my macbook pro with 12 logical CPUs:
>>
>> 1) qbs resolve (clean, new build directory, so includes creating a build
>> graph): ~1m 25s
>
> qbs --log-time might give some hints as to where that time is spent.
Adding —log-time revealed that indeed the ModuleLoader uses up nearly about 70%
of the total time.
>
>> 2) qbs resolve (no changes): ~0.5 s
>> 3) qbs resolve after "touch root.qbs" (restores build graph): ~1m 15 s
>> 4) qbs resolve after “touch depend.qbs” (restores build graph): ~1m 15s
>>
>> From the durations i would expect that there is some room for improvement.
>> To me it seems (by looking at 3 and 4) that qbs rebuilds the whole build
>> graph if it detects any change to any qbs file in the project.
>
> Yes, there is no partial re-resolve.
>
>> If this is the case i wonder how difficult it would be to improve this and
>> what the right approach would be. Maybe looking at “Depends {}” would be
>> enough to determine the “dirty path”.
>
> I suspect this is hopeless. In general, there are too many possible
> interdependencies for a statement like "if file x has changed, we know what
> only product y can be affected". At the very least, much more now-temporary
> data from the resolve stage would have to be stored for such logic to work.
>
>> Fortunately QtC 4.9.x seems to cache something now since opening a qbs
>> project will no longer cause a full resolve.
>
> I don't think anything has changed there recently. qbs re-resolved when it
> thinks that's necessary, i.e. a project file or a property or the environment
> has changed.
Well, before some event (I consider QtC update to 4.9.x as this event) opening
the root project always resulted in about 2 minutes wait time. since then
opening is always quite fast (~5sec).
>
>> Any ideas or maybe even someone looking at that already?
>
> The only remote possibility that I see is somehow making use of concurrency
> when resolving, e.g. have one thread per product. It would not be trivial,
> though, due to inter-product dependencies etc; also, you'd need a dedicated
> script engine per thread. I don't remember the details, but whenever I though
> about this topic, I quickly stopped again after examining what we do in the
> ModuleLoader, which is by far the most expensive stage of project resolving
> these days, and thus the only one worth optimizing.
Indeed, a quick profiling using Instruments does not show any obvious culprit
to optimize in ModuleLoader but show 70% of time goes there.
> (I certainly don't want to to dissuade anyone from trying, it's just that you
> need to be prepared for a disappointing outcome. For a much lower-hanging
> fruit in the area of performance improvement, take a look at
> https://bugreports.qt.io/browse/QBS-1448.)
_______________________________________________
Qbs mailing list
[email protected]
https://lists.qt-project.org/listinfo/qbs
