On Thu, 11 Jul 2019 12:18:30 +0000
Maximilian Hrabowski <[email protected]> wrote:
> I have a fairly large project with a root projects that pulls in several
> SubProjects, altogether with unit tests its about 313 qbs files that are
> pulled in.
The number of project files is not all that important by itself. Qt Creator,
for instance, has a comparable number, and resolves much faster than your
project.
> Let root.qbs be the root qbs file of my project and depend.qbs a subproject
> that many other subprojects depend on.
>
> With a single profile and single config I encounter the following duration
> for qbs resolve on my macbook pro with 12 logical CPUs:
>
> 1) qbs resolve (clean, new build directory, so includes creating a build
> graph): ~1m 25s
qbs --log-time might give some hints as to where that time is spent.
> 2) qbs resolve (no changes): ~0.5 s
> 3) qbs resolve after "touch root.qbs" (restores build graph): ~1m 15 s
> 4) qbs resolve after “touch depend.qbs” (restores build graph): ~1m 15s
>
> From the durations i would expect that there is some room for improvement. To
> me it seems (by looking at 3 and 4) that qbs rebuilds the whole build graph
> if it detects any change to any qbs file in the project.
Yes, there is no partial re-resolve.
> If this is the case i wonder how difficult it would be to improve this and
> what the right approach would be. Maybe looking at “Depends {}” would be
> enough to determine the “dirty path”.
I suspect this is hopeless. In general, there are too many possible
interdependencies for a statement like "if file x has changed, we know what
only product y can be affected". At the very least, much more now-temporary
data from the resolve stage would have to be stored for such logic to work.
> Fortunately QtC 4.9.x seems to cache something now since opening a qbs
> project will no longer cause a full resolve.
I don't think anything has changed there recently. qbs re-resolved when it
thinks that's necessary, i.e. a project file or a property or the environment
has changed.
> Any ideas or maybe even someone looking at that already?
The only remote possibility that I see is somehow making use of concurrency
when resolving, e.g. have one thread per product. It would not be trivial,
though, due to inter-product dependencies etc; also, you'd need a dedicated
script engine per thread. I don't remember the details, but whenever I though
about this topic, I quickly stopped again after examining what we do in the
ModuleLoader, which is by far the most expensive stage of project resolving
these days, and thus the only one worth optimizing.
(I certainly don't want to to dissuade anyone from trying, it's just that you
need to be prepared for a disappointing outcome. For a much lower-hanging fruit
in the area of performance improvement, take a look at
https://bugreports.qt.io/browse/QBS-1448.)
Christian
_______________________________________________
Qbs mailing list
[email protected]
https://lists.qt-project.org/listinfo/qbs
