Hi, I would like to provide an update to this (old) thread based on the development in the past months and weeks.
Am 15.02.19 um 10:30 schrieb Frederik Gladhorn: > Hi, > > On fredag 15. februar 2019 07:31:33 CET Lars Knoll wrote: >> Summing up the discussion here. It looks like people overall agree that the >> pinned dependency approach (option 3) sounds better than what we currently >> have. The main concern was CI capacity, but Frederik believes that with >> enough storage capacity for build artefacts this will not be worse than >> what we have today with all the failed qt5.git integrations. > >> The only other option that was seriously considered was a more monolithic >> repo (option 4). Disadvantages here are that it would require more work on >> the CI front to make testing times bearable, and it doesn’t give us a very >> flexible approach with regards to adding/removing modules. This is >> something that option 3 will nicely give us, as it treats all repositories >> the same way. Option 3 does btw not exclude that we later on merge some >> repositories if we feel that makes sense. > >> So let’s go with option 3. Frederik will be leading the work to get this >> implemented. > >> As far as I can see, it requires some changes to CI so that we have the >> dependencies encoded in each repository, a bot to automatically push sha1’s >> of dependencies forward and some monitoring to alert us if modules get left >> behind. > > I have indeed been playing with this, and I think I have an implementation > that works when it comes to resolving dependencies based on the idea. > > After some pondering and starting with ini files, I settled on yaml, since we > probably want to extend the data later on and I would like this to be easily > human and machine read- and writable. > > My current files look like this: > dependencies: > qt/qtbase: > ref: 698078680fc5a6870177af285fa50c0d8a7c0bc3 > url: code.qt.io > qt/qtsvg: > ref: 2ae3c52fc061f574c9582bf58963fb3996724fbf > url: code.qt.io > > > This is for qt/qtdeclarative, sha1's taken from qt5.git at some point. > Then there is the question of what the file name should be. Should it be at > the top-level or in a coin subdirectory? dependencies.yaml? > > > I have not yet worked on the bot doing the module updates (which may turn out > to be the harder part). > I don't think this can go live before we have that written as well. Thanks to Frederik and Aapo, Coin has learned to read and utilize the dependencies.yaml files in repositories when present. This permits pinning dependencies to specific sha1s. The second part was a tool to maintain these permanent pinnings, by posting updates to Gerrit through the graph of dependencies. This tool is now ready for use and is based on a fairly simple algorithm. I'd like to put this tool into Gerrit in the near future. Meanwhile, here's a description of how it works from the README: ## Algorithm The process of updating dependencies starts by collecting a list of all repositories and determining the root of the graph. That's typically qtbase. From there one, updates to all repositories are posted that only depend on the root. All other repositories remain in a "todo" list. The root is remembered in a "done" list and all repositories that we are currently trying to bring up-to-date are in a "pending" list. Once this process is started, the program saves its state in a ```.json``` file and terminates. The next time the Qt Module Updater is started, it resumes the state and begins checking the state of all pending updates. If an update succeeded, then the corresponding repository is added to the "done" list and we can prepare updates for repositories that have now their dependencies satisfied by picking them from the "todo" list. If the update failed, the repository is dropped from the batch of updates and all other repositories that directly or indirectly depend on the failed one are also removed. After every such iteration of processing pending updates and pushing new ones to Gerrit, the process terminates and saves its state. When the todo list is empty and there are no more pending updates, the batch update is complete. If during that update there were no failures, the Qt Module Updater will also push a change to qt5.git with an update to all submodule sha1s of the new consistent set of modules. Since this is needed to facilitate private API changes in Qt 6, the objective is to take this into production in either the wip/qt6 branch or dev in the coming days. If you see these files appearing, then you know why :-) There is no immediate plan to take this into use for Qt 5 development. That's not a "never ever!" statement though, more an expression of "let's try this in Qt 6 first". Simon _______________________________________________ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
