02.10.2020, 15:53, "Toni Saario" <toni.saa...@qt.io>: > As Iikka said that there has not been plans to use Conan when provisioning > the images for CI.
That's sad, I hoped plan was to use Conan for QtBase dependencies eventually to avoid custom bundling code and 3rd-party sources checked into the repo. And also eliminate potential for conflicts when user's project wants to use the same libraries as bundled into QtBase > > However there has been some discussions how to make the provisioning easier > to manage and have some documentability in them. Currently the provisioning > is just a pile of scripts which have zero value for documentation tools. > Easiest solution for this would be to list the dependencies on some other > file in common format and then install listed dependencies with simple > scripts. About using Conan there, I cannot really say anything about it with > my knowledge about it. > > Another thing is that there are many items built during provisioning which > become dependencies, can we reduce the provisioning workload by having these > cached in Conan storage? Yes, I've done this for QtWebKit dependencies. Packages of dependencies are built on public CI services for all necessary compilers and uploaded to https://bintray.com/qtproject/, then during provisioning they are downloaded and verified. > One thing to check is that how much the provisioning it is possible to > translate to Conan, I would believe that in the end it does not work for > everything. Most of provisioning scripts fall into 2 categories: 1) Download installer (or other binary package) of 3rd party stuff, verify checksum, install. These scripts are quite simple. 2) Build stuff from sources in provisioning time, sometimes in a sophisticated way (e.g. https://code.qt.io/cgit/qt/qt5.git/tree/coin/provisioning/common/windows/android-openssl.ps1). These can be (and IMO should be) replaced with Conan, to reduce provisioning time and complexity. > > As for provisioning and why the CI images are different from what are > documented, biggest problem there is that the images are used by other > components also e.g. RTA testing. Those require some additional stuff and > provisioning currently installs everything. AFAIK also all the optional > dependencies are installed, this makes it hard to distinguish between them > and required dependencies because both are always present. These make it easy > that new dependencies etc. slip through unnoticed. Note that when Conan packages are used via conanfiles (i.e. as it's originally intended by Conan developers), there cannot be any dependency slips because Conan dependencies don't become visible to any project which doesn't require them via conanfile (directly or indirectly). I can see another possible way to use Conan in CI - install packages into the system via "deploy" generator so that Qt build system can find them without any knowledge of Conan. I guess this way may be called "Ossi-friendly because Ossi was adamantly opposed to any explicit support of Conan in build system, see e.g. https://codereview.qt-project.org/c/qt/qtbase/+/184241, however this way indeed may cause dependency slips. > > -Toni > > ---------------------------------------- > From: Konstantin Tokarev <annu...@yandex.ru> > Sent: Friday, October 2, 2020 3:20 PM > To: Iikka Eklund <iikka.ekl...@qt.io>; Richard Weickelt > <rich...@weickelt.de>; development@qt-project.org > <development@qt-project.org>; Toni Saario <toni.saa...@qt.io> > Subject: Re: [Development] Building additional components with Conan for Qt 6 > > 02.10.2020, 14:47, "Iikka Eklund" <iikka.ekl...@qt.io>: >> Hi Konstantin, all, >> >>>> thanks for the heads up. I have some further questions: >>>> >>>> 1. Will Conan be used to manage dependencies of Qt as well? >>>> >>>> If this is the plan, note that there is infrastructure [1] for using Conan >>>> in provisioning images, >>>> with verification of downloaded file checksums and build reproducibility >>>> for each qt5 commit [2]. >> >> Regarding Qt CI @Toni Saario is the best one to answer on this, though I >> haven't heard >> if there are plans to use Conan more extensively by CI/provisioning at this >> point? > > From my experience, when the way project is built in CI is radically > different from a way suggested to > the end user, it inevitably leads to troubles. Users hit unexpected errors > (in this case they may be > caused by changes in 3rd party Conan recipes/packages, or by contributors to > Qt module which > don't use Conan), and they come back to me asking for solution. > >> >> What comes to add-ons >> (https://doc-snapshots.qt.io/qt6-dev/qtmodules.html#qt-add-ons) as mentioned >> earlier the idea is to release add-ons as src packages accompanied with >> conanfile.py i.e. the build recipe. >> If the add-on depends on 3rd party libraries which are available e.g. in >> conan-center the recipe could >> naturally utilize packages from there (openssl, zlib, etc.). >> >>> 2. How will the recipes be managed and where on code.qt.io can I find >>>them? >> >> The recipes of the first two add-ons are not part of the source repositories >> yet, probably should be. >> Though there are some drawbacks in keeping the recipes and sources in the >> same repo >> (some related discussion https://github.com/conan-io/conan/issues/4734). >> >> Currently the recipes are part of an internal packaging repository but the >> current recipes can be found from the src >> installation directory: "6.0.0/Src/qtnetworkauth/conanfile.py" after you >> have installed an add-on package. >> >> Iikka Eklund >> Senior Software Engineer >> >> iikka.ekl...@qt.io >> http://qt.io >> >> ---------------------------------------- >> From: Konstantin Tokarev <annu...@yandex.ru> >> Sent: Friday, October 2, 2020 12:15 PM >> To: Richard Weickelt <rich...@weickelt.de>; Iikka Eklund >> <iikka.ekl...@qt.io>; development@qt-project.org <development@qt-project.org> >> Subject: Spam: Re: [Development] Building additional components with Conan >> for Qt 6 >> >> 01.10.2020, 16:16, "Konstantin Tokarev" <annu...@yandex.ru>: >>> 01.10.2020, 16:07, "Richard Weickelt" <rich...@weickelt.de>: >>>> Hello Ilkka, >>>> >>>> thanks for the heads up. I have some further questions: >>>> >>>> 1. Will Conan be used to manage dependencies of Qt as well? >>> >>> If this is the plan, note that there is infrastructure [1] for using Conan >>> in provisioning images, >>> with verification of downloaded file checksums and build reproducibility >>> for each qt5 commit [2]. >>> >>> If you are interested in details we could discuss them in depth in IRC. >>> >>> [1] >>> https://code.qt.io/cgit/qt/qt5.git/tree/coin/provisioning/common/windows/conan.ps1 >>> [2] requires https://codereview.qt-project.org/c/qt/qt5/+/313930 to handle >>> situations when the same package version is rebuilt without version bump >> >> Iikka, any comments? >> >> -- >> Regards, >> Konstantin > > -- > Regards, > Konstantin -- Regards, Konstantin _______________________________________________ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
