Re: [Development] atomic commits across submodules
On Mon, Dec 18, 2017 at 09:22:31AM -0500, Adam Treat wrote: > 3) Private API being changed out from under > 3a) yet unreleased public api being revised. there is just no reasonable way to claim that the pain inflicted by the 5-stage workaround is a good thing. it's a barely acceptable thing given the constraint of atomic commits being unrealistic for the time being. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] atomic commits across submodules
On 12/18/2017 05:24 AM, Jędrzej Nowacki wrote: On czwartek, 7 grudnia 2017 12:17:16 CET Adam Treat wrote: Hi, I think it is high time that we fix the underlying problem: supporting atomic commits across submodules. As I'm not against the idea, I'm not really fan of it either. The problem with atomic commits across submodules is that they encourage API breakages. Without supporting both code paths even for a little while, one is not aware of the pain that our users needs to go through every time we break API. So your position is we shouldn't need atomic commits across submodules because the changes that are leading to breakage are themselves the problem. Let's examine types of changes: 1) Build system changes that require cross-module modifications 2) New public API in base module where dependent module takes advantage of it 3) Private API being changed out from under 4) Merges of the above 3 types from release branches to dev Of these, only #2 should be solvable without atomic commits. In that case, we can introduce API first and when it is integrated, only then update dependent modules. This is still a problem for #4 though, but more about that later. For the other types, atomic commits are at least sometimes going to be required. Right now, we are experiencing breakage. And it *is* painful. But I have not seen a postmortem of said breakages that shows the current processes are up to preventing such breakage. Maybe the answer is to move away from private API and a build system that does not need to be bootstrapped from Qt... Now, back to #4, I do know that those responsible for merging from release branches into dev face a mind field of cross-module modifications. Consider #2 again and say such a change across modules has made it into release branches. It is my understanding that it is *just such a set of changes* that led to Liang Qi's request to add labels to API changes because the ordering info gets lost. Merges from release to dev are going to be problematic until we have some way of ensuring cross-module dependencies are handled. Which is why I propose that we stop merging from release to dev and require all changes to hit dev first. But, more on that in another email thread later. Maybe after the holidays ;) Once this is done we should revert our CI to test changes against latest version of all modules. No. First, how these two are connected, that is a big jump to a solution, without providing reasoning. I know that you didn't know about the fact that we pin submodules in qt5 and you got burned by this. That __is__ an argument, but an alternative solution would be a better documentation or UI. Second, the most important, that doesn't scale. By compiling against latest Qt5 we reduced CI load by ~80%. Fair enough, let's table this. I have more to say about this, but it really is related to my suggestion above. Again, more about that later. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] atomic commits across submodules (was: Suggestion to add labels when changing API)
On czwartek, 7 grudnia 2017 12:17:16 CET Adam Treat wrote: > Hi, > > I think it is high time that we fix the underlying problem: supporting > atomic commits across submodules. As I'm not against the idea, I'm not really fan of it either. The problem with atomic commits across submodules is that they encourage API breakages. Without supporting both code paths even for a little while, one is not aware of the pain that our users needs to go through every time we break API. Even private API modifications have side effects. If you are working in qtbase then it is easy, but otherwise you are forced to recompile quite often or risk debugging subtle binary incompatibilities. It is not as crazy as during qt5.0 but still happens. In addition we were thinking about splitting submodules even more. Reduce amount of private API usage or even make them use only public API (quite nice but a controversial dream, please do not start discussion about that now :-) ). Make them more independent libraries, which would have many positive aspects. If you keep all these in mind, cross sumodule modifications are not encouraged and our process was never optimized for the case. Just to be very clear if you want to simplify changing many submodules at once, then you would get my full support, but keep in mind that it should not be disruptive nor for other Qt developers nor for Qt users. > Once this is done we should revert our CI to test changes against latest > version of all modules. No. First, how these two are connected, that is a big jump to a solution, without providing reasoning. I know that you didn't know about the fact that we pin submodules in qt5 and you got burned by this. That __is__ an argument, but an alternative solution would be a better documentation or UI. Second, the most important, that doesn't scale. By compiling against latest Qt5 we reduced CI load by ~80%. > * Adopt something like Google's repo tool: > https://code.google.com/archive/p/git-repo/ > * Stop using submodules and use a monolithic repo > * Git subtree > * Implement atomic commit across submodules not in Git, but in the > gerrit/COIN layer so that COIN effectively locks integrations until > commits that span submodules are finished > * Something else? I think there is nothing in our infrastructure that would block us from implementing atomic cross submodules commits. It is matter of Coin calling few more functions. Cheers, Jędrek ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
