Perhaps QThreadPooler <https://code.woboq.org/qt5/qt3d/src/core/jobs/qthreadpooler_p.h.html> can help accelerating building products with inter-product dependencies.
On Thu, 11 Jul 2019 at 20:46, Christian Kandeler <[email protected]> wrote: > On Thu, 11 Jul 2019 12:18:30 +0000 > Maximilian Hrabowski <[email protected]> wrote: > > > I have a fairly large project with a root projects that pulls in several > SubProjects, altogether with unit tests its about 313 qbs files that are > pulled in. > > The number of project files is not all that important by itself. Qt > Creator, for instance, has a comparable number, and resolves much faster > than your project. > > > Let root.qbs be the root qbs file of my project and depend.qbs a > subproject that many other subprojects depend on. > > > > With a single profile and single config I encounter the following > duration for qbs resolve on my macbook pro with 12 logical CPUs: > > > > 1) qbs resolve (clean, new build directory, so includes creating a build > graph): ~1m 25s > > qbs --log-time might give some hints as to where that time is spent. > > > 2) qbs resolve (no changes): ~0.5 s > > 3) qbs resolve after "touch root.qbs" (restores build graph): ~1m 15 s > > 4) qbs resolve after “touch depend.qbs” (restores build graph): ~1m 15s > > > > From the durations i would expect that there is some room for > improvement. To me it seems (by looking at 3 and 4) that qbs rebuilds the > whole build graph if it detects any change to any qbs file in the project. > > Yes, there is no partial re-resolve. > > > If this is the case i wonder how difficult it would be to improve this > and what the right approach would be. Maybe looking at “Depends {}” would > be enough to determine the “dirty path”. > > I suspect this is hopeless. In general, there are too many possible > interdependencies for a statement like "if file x has changed, we know what > only product y can be affected". At the very least, much more now-temporary > data from the resolve stage would have to be stored for such logic to work. > > > Fortunately QtC 4.9.x seems to cache something now since opening a qbs > project will no longer cause a full resolve. > > I don't think anything has changed there recently. qbs re-resolved when it > thinks that's necessary, i.e. a project file or a property or the > environment has changed. > > > Any ideas or maybe even someone looking at that already? > > The only remote possibility that I see is somehow making use of > concurrency when resolving, e.g. have one thread per product. It would not > be trivial, though, due to inter-product dependencies etc; also, you'd need > a dedicated script engine per thread. I don't remember the details, but > whenever I though about this topic, I quickly stopped again after examining > what we do in the ModuleLoader, which is by far the most expensive stage of > project resolving these days, and thus the only one worth optimizing. > (I certainly don't want to to dissuade anyone from trying, it's just that > you need to be prepared for a disappointing outcome. For a much > lower-hanging fruit in the area of performance improvement, take a look at > https://bugreports.qt.io/browse/QBS-1448.) > > > Christian > _______________________________________________ > Qbs mailing list > [email protected] > https://lists.qt-project.org/listinfo/qbs >
_______________________________________________ Qbs mailing list [email protected] https://lists.qt-project.org/listinfo/qbs
