Re: [Development] Deprecation of QByteArray operator const char *()?
Am 26.01.2019 um 08:22 schrieb Olivier Goffart: On 25.01.19 21:11, Christian Ehrlicher wrote: Hello, The two implicit conversions from QByteArray to const char*/void* (QByteArray::operator const char *() and QByteArray::operator const void*()) were marked as obsolete with Qt5.0 but never get decorated as such. This means their usage even inside QtBase is huge (I would say at least 70-80 locations). Therefore my question before I'm pushing a patch to add QT_DEPRECATED_X to them - is the deprecation still valid? I count 4331 uses https://code.woboq.org/data/symbol.html?root=../qt5/=_ZNK10QByteArraycvPKcEv That's a bit more than 70-80 I talked about QtBase, not Qt5 :) But this even makes the un-deprecation more useful. Christian ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Deprecation of QByteArray operator const char *()?
On 25.01.19 21:11, Christian Ehrlicher wrote: Hello, The two implicit conversions from QByteArray to const char*/void* (QByteArray::operator const char *() and QByteArray::operator const void*()) were marked as obsolete with Qt5.0 but never get decorated as such. This means their usage even inside QtBase is huge (I would say at least 70-80 locations). Therefore my question before I'm pushing a patch to add QT_DEPRECATED_X to them - is the deprecation still valid? I count 4331 uses https://code.woboq.org/data/symbol.html?root=../qt5/=_ZNK10QByteArraycvPKcEv That's a bit more than 70-80 Given somebody would have to fix all of these before marking the function as depracated, that sounds like a lot of work! the const void* operator has only 4 uses and should probably be deprecated. https://code.woboq.org/data/symbol.html?root=../qt5/=_ZNK10QByteArraycvPKvEv -- Olivier Woboq - Qt services and support - https://woboq.com - https://code.woboq.org ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
On 25.01.19 14:12, Frederik Gladhorn wrote: Hi all, I'd like to start another discussion around our development workflow. We arrived at our current model of Qt modules (in the git repository sense) and using qt5.git as a container for all of them through a series of steps and changes. Mix in the evolution of the testing environment over time and we have something that has grown in interesting ways. I will try to describe the problem in this mail. I also have discussed with a bunch of people (inside The Qt Company) about potential solutions. After brainstorming and some back and forth, we had five suggestions on a whiteboard and picked our favorite. In a separate mail I'll try to describe these and what we concluded with as our favorite. All of this is up for discussion, so I'm hoping for someone to come up with even better ideas. In my opinion, I think it would be much easier to merge a bunch of repository together. Maybe modularisation went to far. QtQuick is a very important part of Qt, and therefore it would make sense to merge qtdeclarative into qtbase. Also small repository could go back in qtbase. Of course, I wouldn't want to merge big repository such as qtwebengine into qtbase. IMHO there are these factor that should come into play: - How much is that module important for Qt? - How big is that module? - How much of private API is that module using (and is it possible to use less of private API?) - ... I think the "monorepo" is clearly a good approach. And git is evolving with shadow clones and partial checkout. LLVM/Clang recently choose the monorepo approach as it moves to git. Of one problem is that this will make CI integration slower because each CI run now have to test both qtbase and qtdeclarative tests. We have to say how much this is a problem compared the the simplicity of development. I'd claim this in some respect, it is actually an advantage, for changes in qtcore. Maybe CI bot could have a simple list of directory that white, or blacklist tests to run depending on the touched directories. -- Olivier Woboq - Qt services and support - https://woboq.com - https://code.woboq.org ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
On Friday, 25 January 2019 13:39:49 PST Konstantin Tokarev wrote: > > All living languages are supposed to be stored in the BMP, which means no > > UTF-16 surrogate pairs to encode them. > > AFAIK all emojis are encoded with surrogate pairs Emojis are not part of a living language. They're drawings. But yes, they're outside the BMP. In any case, they're often represented by more than one codepoint anyway, so whether we used N*2 UTF-16 code units to represent them or N UTF-32 code units, it makes no difference. Your code needs to know how to deal with them, where to properly break, how to combine them, how to calculate the width, etc. Also note how they'd be represented by N*4 bytes in UTF-8, which means all three representations take exactly the same amount of memory. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Deprecation of QByteArray operator const char *()?
On Friday, 25 January 2019 13:36:04 PST Christian Ehrlicher wrote: > Here the commit which deprecated the functions: > https://codereview.qt-project.org/#/c/17343/ > I found one place inside QtBase which hit this problem: > https://codereview.qt-project.org/#/c/251028/ I'm not questioning the usefulness of the macro to disable the operator. I am questioning deprecating the operators. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
25.01.2019, 23:33, "Thiago Macieira" : > On Friday, 25 January 2019 04:54:22 PST Edward Welbourne wrote: >> we >> fail to properly support cultures whose scripts are relegated to the >> outer planes of Unicode - as, for example, the Chakma language's number >> system > > All living languages are supposed to be stored in the BMP, which means no > UTF-16 surrogate pairs to encode them. AFAIK all emojis are encoded with surrogate pairs > > That doesn't mean a single code unit, mind you. Think of combining characters. > > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center > > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development -- Regards, Konstantin ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Deprecation of QByteArray operator const char *()?
Am 25.01.2019 um 21:23 schrieb Thiago Macieira: On Friday, 25 January 2019 12:11:33 PST Christian Ehrlicher wrote: Hello, The two implicit conversions from QByteArray to const char*/void* (QByteArray::operator const char *() and QByteArray::operator const void*()) were marked as obsolete with Qt5.0 but never get decorated as such. This means their usage even inside QtBase is huge (I would say at least 70-80 locations). Therefore my question before I'm pushing a patch to add QT_DEPRECATED_X to them - is the deprecation still valid? I'd un-deprecate them. The use to casting to char* is way too useful. But I wasn't aware we event had a void* cast. Here the commit which deprecated the functions: https://codereview.qt-project.org/#/c/17343/ I found one place inside QtBase which hit this problem: https://codereview.qt-project.org/#/c/251028/ Christian ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
On Friday, 25 January 2019 04:30:52 PST Shawn Rutledge wrote: > Could we simplify testing for some types of patches? E.g. if the patch only > touches documentation, don’t run tests, except those that involve building > docs, if we have any of those. I think that's a good idea to investigate, but remember that you *can* break things with doc changes. Maybe a way for a "low-priority stage", which stages a change but doesn't trigger the integration unless a timeout comes around or a regular priority change is also staged. One time back in the TT/Nokia days Frans asked me for a quick review of a "### Qt5" change he wanted to add and I gave him my approval verbally (that was before code review systems). So he pushed it. 5 minutes later, Paul waked into our office saying we had broken the build: we forgot the // to *start* the comment. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
On Friday, 25 January 2019 01:10:35 PST Simon Hausmann wrote: > I do quite like what Allan suggested: We could try the cherry-pick the other > way around. Volker, Lars, Thiago etc. surely remember the p4i script we > used to have when we were using Perforce. Imagine that with more > automation. I vaguely remember it. But your bringing it up is *not* an argument in favour of cherry-picking. Doing backporting properly in the Perforce days was a nightmare. We did not consistently backport fixes to earlier releases, even when we had a somewhat long stable series (4.3 went through 4.3.5). Also, unlike Git, Perforce "merges" *were* a series of cherry-picks and it did know which commits had been cherry-picked and which ones hadn't. So if you told it to merge a range, it would tell you everything that was missing. More importantly, the range did not have to be contiguous, so you could cherry-pick only your own changes and let others deal with theirs. It also helped p4 had a pretty good automated conflict resolution in the command-line (today, I use kdiff3 for that, via git mergetool). -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
On Friday, 25 January 2019 04:54:22 PST Edward Welbourne wrote: > we > fail to properly support cultures whose scripts are relegated to the > outer planes of Unicode - as, for example, the Chakma language's number > system All living languages are supposed to be stored in the BMP, which means no UTF-16 surrogate pairs to encode them. That doesn't mean a single code unit, mind you. Think of combining characters. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
On Friday, 25 January 2019 08:54:38 PST Konstantin Tokarev wrote: > > How often do you need that, oustide of QString itself? And maybe a few > > efficient QtCore classes? (QCborValue comes to mind) > > Each time I need to interact efficiently with extenal code which isn't > Qt-based, e.g. WebKit, ICU. In particular, this extra copy would certainly > degrade performance of QtWebKit. > > Oh and you've mentioned CBOR, this implies that it won't be possible for Qt > users to make efficient implementation of a different serialization format. I didn't say we shouldn't have it. I was just trying to gather information about the need. So it looks like we do need it, if we ever change the encoding. My worry is that people will fail to handle the combinations properly. Which is why I dislike different encodings even more than changing it wholesale with an API- breaking change. However, one of my pending Qt 6 changes is to store a flag in QString that says "this UTF-16 string is known to contain only US-ASCII characters". That way, toUtf8() can use the faster toLatin1() algorithm (the flag is set by toUtf8() and toLatin1() the first time they're called). The problem is that it needs to clear that flag in all detach() calls. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
Il 25/01/19 10:49, Dominik Haumann ha scritto: Sidenote: Such a QStringIterator would also be helpful for KTextEditor, where we likely have some bugs we usually never see since we never have > UTF16 or composed characters. I've managed to merge it in QtCore some 5 years ago, comes with docs and tests: https://codereview.qt-project.org/#/c/77136/ You can use it today: CONFIG += core-private #include It's still missing a couple of bits and bolts to turn it public -- most notably, ranged for / STL loop support support. I'd also like to investigate more how it overlaps with SG16 / Boost.Text / etc. efforts before publishing the current API. My 2 c, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Deprecation of QByteArray operator const char *()?
On Friday, 25 January 2019 12:11:33 PST Christian Ehrlicher wrote: > Hello, > > The two implicit conversions from QByteArray to const char*/void* > (QByteArray::operator const char *() and QByteArray::operator const > void*()) were marked as obsolete with Qt5.0 but never get decorated as > such. This means their usage even inside QtBase is huge (I would say at > least 70-80 locations). > Therefore my question before I'm pushing a patch to add QT_DEPRECATED_X > to them - is the deprecation still valid? I'd un-deprecate them. The use to casting to char* is way too useful. But I wasn't aware we event had a void* cast. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] Deprecation of QByteArray operator const char *()?
Hello, The two implicit conversions from QByteArray to const char*/void* (QByteArray::operator const char *() and QByteArray::operator const void*()) were marked as obsolete with Qt5.0 but never get decorated as such. This means their usage even inside QtBase is huge (I would say at least 70-80 locations). Therefore my question before I'm pushing a patch to add QT_DEPRECATED_X to them - is the deprecation still valid? Thx, Christian ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Archiving is working
Den fre 25 jan. 2019 kl 19:23 skrev Giuseppe D'Angelo via Development
:
>
> Il 25/01/19 18:41, Edward Welbourne ha scritto:
> > The pages have all been updated and had their URLs changed.
> > Or are you trying to tell me something else ?
> > I'm not sure I fully understand.
>
> Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the
> mailing list are found in commit messages, on the bugtracker, and so on.
> Breaking them is a big no-no-no.
>
>
> As I said before, the new archive is corrupted, and thus must be fixed.
> Look at the numbering of emails from the new archives and note at how it
> "jumps":
>
> > https://lists.qt-project.org/pipermail/development/2011-October/date.html
>
> Compare it to the old numbering (progressive):
>
> > https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/
>
>
>
> Then take a look at the current archive:
>
> > https://lists.qt-project.org/pipermail/development/2011-October.txt.gz
>
> Compare it with the old archive:
>
> > https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/2011-October.txt.gz
>
> The new archive is corrupted; there are duplicated emails in there.
> What's needed now is to make pipermail produce a result identical to the
> old archive.
+1 https://www.w3.org/Provider/Style/URI ("Cool URIs don't change")
If you're going accept this as "solved", be aware that you're breaking
links in a *lot* of places, not just on Qt infra like QUIPs and
whatnot, but lots of other public forums, mailing lists, code bases,
commit messages, issue trackers etc.
I really think you need to find a way to make existing links work again.
Elvis
>
>
> My 2 c,
> --
> Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
> KDAB (France) S.A.S., a KDAB Group company
> Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
> KDAB - The Qt, C++ and OpenGL Experts
>
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] Archiving is working
Il 25/01/19 18:41, Edward Welbourne ha scritto: The pages have all been updated and had their URLs changed. Or are you trying to tell me something else ? I'm not sure I fully understand. Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the mailing list are found in commit messages, on the bugtracker, and so on. Breaking them is a big no-no-no. As I said before, the new archive is corrupted, and thus must be fixed. Look at the numbering of emails from the new archives and note at how it "jumps": https://lists.qt-project.org/pipermail/development/2011-October/date.html Compare it to the old numbering (progressive): https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/ Then take a look at the current archive: https://lists.qt-project.org/pipermail/development/2011-October.txt.gz Compare it with the old archive: https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/2011-October.txt.gz The new archive is corrupted; there are duplicated emails in there. What's needed now is to make pipermail produce a result identical to the old archive. My 2 c, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Archiving is working
Liang Qi (25 January 2019 14:17) > Looks like the index was also regenerated for October 2016, > > [Development] Coin news - Jedrzej Nowacki > Tue Oct 25 10:22:41 EEST 2016 > > Current link: > https://lists.qt-project.org/pipermail/development/2016-October/055084.html > > It was: > http://lists.qt-project.org/pipermail/development/2016-October/027542.html > > via > https://web.archive.org/web/20161108191742/http://lists.qt-project.org/pipermail/development/2016-October/thread.html The pages have all been updated and had their URLs changed. Or are you trying to tell me something else ? I'm not sure I fully understand. I've just pushed a change for review, that updates all existing QUIPs to use URLs in the new archive; and commented on all pending QUIPs in review on my best guesses at which URLs in the new archive match the ones they refer to in the old one. See various reviews at: * https://codereview.qt-project.org/#/q/status:open+project:meta/quips,n,z I note, in passing, that there are several outstanding reviews that seem not to have moved in a long time. Please think about what you're doing with those QUIPs, authors and reviewers. As part of that, I've also updated the heroku instance that displays the current state of everything; it uses the most recent patch set from each outstanding review, plus some clean-up of mine (renaming quip-*.txt to quip-*.rst and fixing the URLs to those I suggested in reviews). If you find any remaining broken links in those, please let me know. * https://quips-qt-io.herokuapp.com Now I shall go home and have a week-end that feels over-due, Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
25.01.2019, 01:02, "Thiago Macieira" : > On Thursday, 24 January 2019 05:06:58 PST Konstantin Tokarev wrote: >> I will be officially pissed off if possibility to access raw data of QString >> without extra copy is gone It would be better if there is a way to figure >> out internal storage encoding (e.g. isUtf16()) and access raw data > > How often do you need that, oustide of QString itself? And maybe a few > efficient QtCore classes? (QCborValue comes to mind) Each time I need to interact efficiently with extenal code which isn't Qt-based, e.g. WebKit, ICU. In particular, this extra copy would certainly degrade performance of QtWebKit. Oh and you've mentioned CBOR, this implies that it won't be possible for Qt users to make efficient implementation of a different serialization format. -- Regards, Konstantin ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
Am 25.01.2019 um 11:08 schrieb Lars Knoll: > > > On 25 Jan 2019, at 10:10, Simon Hausmann > mailto:simon.hausm...@qt.io>> wrote: > > > I think it's worthwhile to develop the tooling to automate cherry-picking. > That tooling is something that is perhaps best tried on a release branch > first. > > I do quite like what Allan suggested: We could try the cherry-pick the other > way around. Volker, Lars, Thiago etc. surely remember the p4i script we used > to have when we were using Perforce. Imagine that with more automation. > > The risk with this is that we might end up having fixes in a stable branch > that don’t make it do dev for various reasons, so it’ll regress in the next > minor release. > > > (1) Liang's work load gets reduced dramatically - conflict resolution is > distributed. > (2) Changes hit the customer relevant branch ASAP. > (3) The load on the CI does not change for anybody working in dev. > > The CI problem comes from the fact that if we have a high rate of stages to > qtbase/dev, we at some point get into a deadlock situation, even if we > disregard any flakiness in the system. That’s because higher rates imply that > more changes are tested together. This in turn increasing the risk of > rejection of all changes because of one bad change in the set. So the current > system doesn’t scale and basically rate limits the amount of changes going > into a branch (or worse, we end up getting traffic jams where the actual rate > actually goes down to zero). > > To me this means we need to seriously rethink that part of our CI system, and > ideally test changes (or patch series) individually and in parallel. So maybe > we should adjust our CI system that way: > > * test changes (or patch series) individually > * If they pass CI merge or cherry-pick them into the branch > * reject only if the merge/cherry-pick gives conflicts > > This adds a very small risk that two parallel changes don’t conflict during > the merge/cherry-pick process, but cause a test regression together. To help > with that, we can simply run a regular status check on the repo. If this > happens the repo will be blocked for further testing until someone submits a > fix or reverts an offending change, which is acceptable. Could that be solved by testing the combinations of changes again? * test changes (or patch series) individually * If they pass CI merge or cherry-pick them into some local branch * reject if the merge/cherry-pick gives conflicts * when time period x has passed or the local branch contains y changes, test the local branch good: push the local branch to the public one bad: repeat step four with a subset of the changes it had before Assuming that y is significantly greater than 1, the added overhead for one more test run seems relatively small to me. Cheers, Robert > > Another advantage is that this would pave the road towards a system where CI > testing happens before human review, so we can in the longer term avoid > duplicated review/approval work. > > Cheers, > Lars > > > > > Simon > > From: Lars Knoll > Sent: Friday, January 25, 2019 9:05:45 AM > To: Ville Voutilainen > Cc: Simon Hausmann; Qt development mailing list > Subject: Re: [Development] Proposal: New branch model > >> On 24 Jan 2019, at 22:29, Ville Voutilainen >> mailto:ville.voutilai...@gmail.com>> wrote: >> >> On Thu, 24 Jan 2019 at 20:26, Simon Hausmann >> mailto:simon.hausm...@qt.io>> wrote: >>> >>> >>> I would see the biggest long term impact with the massive amount of cherry >>> picks from dev to qt6 over a long period of time. >>> >>> Git rerere works locally, so it doesn’t help in this setup I think. >> >> Completely seriously, without any preference to either branching >> approach: aren't we going to be in some sort of trouble >> with the qt6 branch anyway, no matter what we do? Following on a bit: >> >> Pardon me if I missed some important part of the motivation of all of >> this, but I *CAN'T* see how this should, could, >> or needs to be an approach that helps with the qt6 branch >> merge/cherry-pick/rebase. I don't think that's going to >> be a pleasant operation whatever we do. >> >> I would like a "push to trunk, backport to release-branches" approach >> going forward. As in, once we have the usual >> umpteen X.y branches, it's a simple approach. >> >> But I don't see how going from merge-forward (except also >> merge-backward sometimes) to cherry-pick-backward >> (except also cherry-pick forward, or kinda sideways to qt6, and maybe >> sometimes merge in some direction) is going >> to help us with qt6. These matters seem orthogonal. > > After some more thinking, I tend to agree. Qt6 is somewhat special and maybe > needs to be treated differently. > > But except for Qt 6 I like the proposed policy for a couple of reasons: > > * I think it makes the life of casual/new contributors easier. Simply always > develop and push against the
Re: [Development] Qt6: Adding UTF-8 storage support to QString
> By all means, let's make sure the internals are efficient for the more > common languages and scripts; but it's way past time to start doing > Unicode properly, so that all cultures are well-served by default, when > the software folk are using is built on Qt, I don't think anyone knows what "properly" is. But the more I think about it, the more I like the idea I expressed as a list of sequences of various character sizes. I think it is a good balance between space and efficiency. To recap that: A class that stores a list of list of same-width characters. For the most naive case the list is 1 list long and contains only 8bit characters. This performs identically to QByteArray. Non-ASCII languages requiring 16-bit storage are as QStrings are now. Then, in the more complicated scenarios, it breaks out 8-bit segments and 16-bit segments and makes them appear contiguous. (Emoji in ASCII text). Of course there could be functions to collapse it all to the uniform largest used width (maximize()) or break it apart to minimize() space (for very long 8-bit strings with occasional characters), and there can even be a bestFit() heuristic. And as always you can get it serialized as UTF-8 or 16... All the above also extends to 32-bit as well. I think this blends handles the average case very well (all characters of same width) and has reasonable cost for occasional exotic characters. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
25.01.2019, 18:11, "Allan Sandfeld Jensen" : > I think for something like qt6, I would like to suggest I think I have brought > up before in relation to testing qt5.git changes on more platforms or > configuations: A system for non blocking CI failures. > > In WebKit they what they called the CI-waterfall. It was a table with commits > as rows and platforms as columns. https://build.webkit.org/waterfall > Only some platforms were blocking and could > prevent patches from integrating(Apple's own of course), the rest was non > blocking and would just turn red on the waterfall when they broke. There is another set of build bots called EWS, they build each revision of submitted patch, so that both patch authors reviewers can be sure that patch builds on all major platforms when discussing code changes. In Qt it sometimes happens that problems are only discovered after +2, leading to anpother review cycle. -- Regards, Konstantin ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
On Freitag, 25. Januar 2019 14:12:11 CET Frederik Gladhorn wrote: > Hi all, > > I'd like to start another discussion around our development workflow. > We arrived at our current model of Qt modules (in the git repository sense) > and using qt5.git as a container for all of them through a series of steps > and changes. Mix in the evolution of the testing environment over time and > we have something that has grown in interesting ways. > > I will try to describe the problem in this mail. I also have discussed with > a bunch of people (inside The Qt Company) about potential solutions. After > brainstorming and some back and forth, we had five suggestions on a > whiteboard and picked our favorite. In a separate mail I'll try to describe > these and what we concluded with as our favorite. All of this is up for > discussion, so I'm hoping for someone to come up with even better ideas. > > > The problem: > > qt5.git serves several purposes today, for example: > - it contains a few binaries needed for Windows development > - it gives a definition of which modules make up a Qt release > - it collects sets of revisions which work together (because they were > tested with each other) > - it has a bit of build system infrastructure to build all of this > > In my opinion there are a few issues with what we have: > - updating qt5.git (and thus making releases) is cumbersome > - it's unclear for many people what a module is tested against in the CI > (its dependencies are checked out at the revisions specified in qt5.git, > not the latest revisions of the corresponding branch) > - we have more products, such as Qt for Automotive, Automation, Device > Creation, should those have qt5-special-purpose repositories? > - modules outside of qt5.git get different treatment, making it hard to > include new modules > > In the light of Qt 6, there is also the question of how we can allow making > source incompatible changes in an easy fashion. > Why is it hard to do source incompatible changes? Let's say we want to > rename a function in qtbase. qtdeclarative is the only user of that > function in our code-base. Today we can do that in five easy steps: > > 1) add function with new name in qtbase, do not remove the old one yet > 2) update qt5.git > 3+4) move qtdeclarative to the new function name; remove the old function > from qtbase > 5) after 3 and 4 are in, update qt5.git > > Five commits, two of which are qt5.git updates, is a lot to ask for. And > this is assuming that the contributor knows of this work-flow and does > everything by the unwritten book. In the unhappy case, we do an attempt of > changing qtbase directly, then we notice qt5.git doesn't update and we need > to bring back the old function name first. Another aspect is that even > after 5, if this happened to be a branch other than dev, we might face all > of these problems while doing merges to the next branch again (been there, > done that, and so has Liang). > > This has led to a mess in how we create releases and how we test things. It > creates a very hard and artificial line of what's part of the core product > and what is outside and testing in the CI is also majorly influenced by > this. Modules are not self-contained (since the fine-grained dependencies > are specified in qt5.git). We don't know how to cleanly handle dependencies > for modules outside. We have trouble getting some of the releases out, > since Qt for Device Creation follows slightly different routines than Qt > for Application Development > I think for something like qt6, I would like to suggest I think I have brought up before in relation to testing qt5.git changes on more platforms or configuations: A system for non blocking CI failures. In WebKit they what they called the CI-waterfall. It was a table with commits as rows and platforms as columns. Only some platforms were blocking and could prevent patches from integrating(Apple's own of course), the rest was non blocking and would just turn red on the waterfall when they broke. If we had something like this for qt6, and instead of having a qt6.git to begin with just tested the qt6 head of all modules, a source incompatible change would turn the waterfall red for all the modules that broke, we would know which commit caused it, but could work on making the original change more source compatible in qtbase, or fix the modules depending on it one at a time with too much overhead. Once we had a system like that, we could also use to monitor if changes in modules causes other modules to break before qt5.git (or qt6.git) is updated, and test more platforms and configurations without blocking integration more than we already do. best regards 'Allan ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
Am 24.01.2019 um 10:39 schrieb Volker Hilsheimer: >> On 24 Jan 2019, at 09:22, Jaroslaw Kobus wrote: "All(**) changes would go to dev. From which they would be automatically cherry-picked by a bot to other branches. The decision to which branch cherry- pick, would be taken based on a tag in the commit message. We could add a footer that marks the change risk level as in quip-5" No sure I understand the above correctly. Let's say in dev branch some source file got refactored completely, so that no single line match the old version anymore, e.g. Qt 5.9. Now you need to fix the old code, which is in 5.9 branch - how in this case you may try to push your fix to dev? Jarek >>> That’s where the “with exception of branch specific changes” clause (which >>> the ** points at) kicks in. >>> >>> Is the fix needed in dev (or is the bug fixed by the refactoring)? >>> >>> If yes, then fix it in dev, and then make a separate fix in the relevant >>> LTS branches (perhaps starting with the cherry-pick’ed change). Or just >>> accept that this bug won’t/can’t be fixed in the pre-refactoring codebase. >>> >>> If no, then push the fix for the newest branch where it’s needed, from >>> where it can be cherry picked further; don’t do anything in dev (including >>> “don’t expect someone that knows nothing about your change to deal with the >>> merge conflict”). >>> >> In this case there is additional step then. You would be forced now to check >> first both 5.9 and dev branches and detect if a fix would be "applyable" to >> the dev or not. > > > Testing whether the bug that I’m fixing exists in dev or not is part of the > drill of fixing bug, isn’t it? Why would you spend time on fixing something > in 5.12 without checking whether the issue is still present in the latest > codebase? Perhaps the issue has been fixed already, just without the author > considering it relevant for 5.12 (perhaps for good reasons). Why would somebody who maintains a project on Qt5 but made the decision to not migrate it to Qt6 care about fixing the latter? Cheers, Robert >> Besides, if you decide that your fix goes only to the 5.9 branch - in this >> case you follow the current model anyway, right? So, having two models isn't >> better than having just one I think. > > > With the proposed model there are no more automatic merges from more stable > to less stable, so once my change made it into 5.9, that is it. > > > Volker > > > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development > ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
> -Original Message- > From: Development On Behalf Of > Frederik Gladhorn > Sent: Friday, 25 January 2019 2:12 PM > To: development@qt-project.org > Subject: [Development] Qt modules, API changes and Qt 6 > > Hi all, > > I'd like to start another discussion around our development workflow. > We arrived at our current model of Qt modules (in the git repository sense) > and using qt5.git as a container for all of them through a series of steps and > changes. Mix in the evolution of the testing environment over time and we > have something that has grown in interesting ways. > > I will try to describe the problem in this mail. I also have discussed with a > bunch of people (inside The Qt Company) about potential solutions. After > brainstorming and some back and forth, we had five suggestions on a > whiteboard and picked our favorite. In a separate mail I'll try to describe > these and what we concluded with as our favorite. All of this is up for > discussion, so I'm hoping for someone to come up with even better ideas. > > > The problem: > > qt5.git serves several purposes today, for example: > - it contains a few binaries needed for Windows development > - it gives a definition of which modules make up a Qt release > - it collects sets of revisions which work together (because they were tested > with each other) > - it has a bit of build system infrastructure to build all of this > > In my opinion there are a few issues with what we have: > - updating qt5.git (and thus making releases) is cumbersome > - it's unclear for many people what a module is tested against in the CI > (its dependencies are checked out at the revisions specified in qt5.git, not > the latest revisions of the corresponding branch) > - we have more products, such as Qt for Automotive, Automation, Device > Creation, should those have qt5-special-purpose repositories? > - modules outside of qt5.git get different treatment, making it hard to > include new modules > > In the light of Qt 6, there is also the question of how we can allow making > source incompatible changes in an easy fashion. > Why is it hard to do source incompatible changes? Let's say we want to > rename a function in qtbase. qtdeclarative is the only user of that function > in > our code-base. Today we can do that in five easy steps: > > 1) add function with new name in qtbase, do not remove the old one yet > 2) update qt5.git > 3+4) move qtdeclarative to the new function name; remove the old > 3+function from > qtbase > 5) after 3 and 4 are in, update qt5.git > > Five commits, two of which are qt5.git updates, is a lot to ask for. And this > is > assuming that the contributor knows of this work-flow and does everything > by the unwritten book. In the unhappy case, we do an attempt of changing > qtbase directly, then we notice qt5.git doesn't update and we need to bring > back the old function name first. Another aspect is that even after 5, if this > happened to be a branch other than dev, we might face all of these > problems while doing merges to the next branch again (been there, done > that, and so has Liang). > > This has led to a mess in how we create releases and how we test things. It > creates a very hard and artificial line of what's part of the core product and > what is outside and testing in the CI is also majorly influenced by this. > Modules are not self-contained (since the fine-grained dependencies are > specified in qt5.git). We don't know how to cleanly handle dependencies for > modules outside. We have trouble getting some of the releases out, since Qt > for Device Creation follows slightly different routines than Qt for > Application > Development > > On a semi-related note, running git-bisect on anything but qtbase is a > science, hard enough to at least put me off. Indeed. For anyone who might be interested, I've found the best way to bisect leaf modules like qtquickcontrols2 (which relies on qtdeclarative and qtbase) is to use qt5.git submodule updates: 1. Clone qt5.git, init only the modules you're interested in. 2. Look for qt5.git submodule updates (e.g. by searching in Gerrit) that happened before and after the issue started happening. Use these as the good/bad commits for the bisect. 3. Run "git submodule update --init qtbase qtxmlpatterns qtdeclarative" (passing in each module you're interested in) after each bisect round to checkout the relevant submodule commits for that qt5.git commit. 4. Wipe the Qt build, configure and build it, then test. 5. Once you've found the bad qt5.git commit, look through (bisect if possible) the range of commits for that in the relevant submodules. This is super slow, but it's the most reliable way, avoiding compiler errors etc. that you get from mixing and matching commits across modules. Luckily it's not always necessary. > Cheers, > Frederik > > > > ___ > Development mailing list > Development@qt-project.org >
Re: [Development] Proposal: New branch model
On 25 Jan 2019, at 10:10, Simon Hausmann mailto:simon.hausm...@qt.io>> wrote: >> I think it's worthwhile to develop the tooling to automate >> cherry-picking. That tooling is something that is perhaps best tried >> on a release branch first. >> >> I do quite like what Allan suggested: We could try the cherry-pick >>the other way around. Volker, Lars, Thiago etc. surely remember the >>p4i script we used to have when we were using Perforce. Imagine that >>with more automation. Lars Knoll (25 January 2019 11:08) > The risk with this is that we might end up having fixes in a stable > branch that don’t make it do dev for various reasons, so it’ll regress > in the next minor release. It should be fairly trivial to ensure we tag the cherry-picks in such a way that some routine script can alert relevant folk to ones that haven't been sorted out within a month, or before the next release on the branch they targeted. I think this is less of a reason to worry than you suppose. Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Archiving is working
Looks like the index was also regenerated for October 2016, [Development] Coin news - Jedrzej Nowacki Tue Oct 25 10:22:41 EEST 2016 Current link: https://lists.qt-project.org/pipermail/development/2016-October/055084.html It was: http://lists.qt-project.org/pipermail/development/2016-October/027542.html via https://web.archive.org/web/20161108191742/http://lists.qt-project.org/pipermail/development/2016-October/thread.html --Liang On Fri, 25 Jan 2019 at 14:00, Edward Welbourne wrote: > > Hi all, > > Again, our sysadmins claim they've fixed what was broken in the > archives. Please check up on any archives in which you've seen holes > lately and let us know if there are still any problems. > > If all's well, I'll hope our archives are stable again and begin going > through all our QUIPs working out where the mails they've linked have > ended up after all of this - please do likewise for other links you know > about. > > Eddy. > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] Qt modules, API changes and Qt 6
Hi all, I'd like to start another discussion around our development workflow. We arrived at our current model of Qt modules (in the git repository sense) and using qt5.git as a container for all of them through a series of steps and changes. Mix in the evolution of the testing environment over time and we have something that has grown in interesting ways. I will try to describe the problem in this mail. I also have discussed with a bunch of people (inside The Qt Company) about potential solutions. After brainstorming and some back and forth, we had five suggestions on a whiteboard and picked our favorite. In a separate mail I'll try to describe these and what we concluded with as our favorite. All of this is up for discussion, so I'm hoping for someone to come up with even better ideas. The problem: qt5.git serves several purposes today, for example: - it contains a few binaries needed for Windows development - it gives a definition of which modules make up a Qt release - it collects sets of revisions which work together (because they were tested with each other) - it has a bit of build system infrastructure to build all of this In my opinion there are a few issues with what we have: - updating qt5.git (and thus making releases) is cumbersome - it's unclear for many people what a module is tested against in the CI (its dependencies are checked out at the revisions specified in qt5.git, not the latest revisions of the corresponding branch) - we have more products, such as Qt for Automotive, Automation, Device Creation, should those have qt5-special-purpose repositories? - modules outside of qt5.git get different treatment, making it hard to include new modules In the light of Qt 6, there is also the question of how we can allow making source incompatible changes in an easy fashion. Why is it hard to do source incompatible changes? Let's say we want to rename a function in qtbase. qtdeclarative is the only user of that function in our code-base. Today we can do that in five easy steps: 1) add function with new name in qtbase, do not remove the old one yet 2) update qt5.git 3+4) move qtdeclarative to the new function name; remove the old function from qtbase 5) after 3 and 4 are in, update qt5.git Five commits, two of which are qt5.git updates, is a lot to ask for. And this is assuming that the contributor knows of this work-flow and does everything by the unwritten book. In the unhappy case, we do an attempt of changing qtbase directly, then we notice qt5.git doesn't update and we need to bring back the old function name first. Another aspect is that even after 5, if this happened to be a branch other than dev, we might face all of these problems while doing merges to the next branch again (been there, done that, and so has Liang). This has led to a mess in how we create releases and how we test things. It creates a very hard and artificial line of what's part of the core product and what is outside and testing in the CI is also majorly influenced by this. Modules are not self-contained (since the fine-grained dependencies are specified in qt5.git). We don't know how to cleanly handle dependencies for modules outside. We have trouble getting some of the releases out, since Qt for Device Creation follows slightly different routines than Qt for Application Development On a semi-related note, running git-bisect on anything but qtbase is a science, hard enough to at least put me off. Cheers, Frederik ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Archiving is working
Hi all, Again, our sysadmins claim they've fixed what was broken in the archives. Please check up on any archives in which you've seen holes lately and let us know if there are still any problems. If all's well, I'll hope our archives are stable again and begin going through all our QUIPs working out where the mails they've linked have ended up after all of this - please do likewise for other links you know about. Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt6: Adding UTF-8 storage support to QString
Arnaud Clère (25 January 2019 10:59) wrote: > Most user code I have written or seen handles text data naively and is > incorrect in some respect but I think only a minority of if is leading > to real problems because input data will rarely trigger them. That depends a lot on who's supplying your data. The same rationale was given for "making do" with old 8-bit encodings, which meant programs worked for various rich nations' primary languages and didn't for anyone else's. Then we switched to UTF-16, which let us continue not thinking about what we're really doing, while reaching a larger slice of the world. Still, that leaves us complicit in suppressing various minority cultures by making software that works for the dominant culture around them, but not for them. Until we get into the habit of thinking of text properly (and I still don't even know the terminology, so I have a way to go on this, just like anyone) instead of as a sequence of evenly-sized units, we're going to continue either being inefficient (because we use units that are bigger than needed for many use-cases - arguably true of UTF-16) or we fail to properly support cultures whose scripts are relegated to the outer planes of Unicode - as, for example, the Chakma language's number system, which QLocale currently can't represent (QTBUG-69324) because the digits don't fit in a single UTF-16 unit (as QLocaleData expects of digits, signs and quotes, though it understands most of its other locale-specific texts might be longer). As a result, we can't support any Chakma locale. By all means, let's make sure the internals are efficient for the more common languages and scripts; but it's way past time to start doing Unicode properly, so that all cultures are well-served by default, when the software folk are using is built on Qt, Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
> On 25 Jan 2019, at 09:43, Martin Smith wrote: > >> It is the absolute exception that a change goes into qtbase on first attempt. > > But many rejections have nothing to do with any change at all. I often submit > documentation-only changes to QtBase, and they often get tested alone, with > no other changes. They still get rejected multiple times because test X fails > this time and test Y fails next time, despite the fact that my change only > fixes typos in the qdoc comments. Sometimes there is a compiler failure too, > which is not caused by the change. Could we simplify testing for some types of patches? E.g. if the patch only touches documentation, don’t run tests, except those that involve building docs, if we have any of those. (Even building the code is dubious in that case, as long as the bot that decides that it’s a doc-only patch actually verifies that the only changes are inside comments.) If it only touches widgets, some time-consuming and failure-prone tests like network tests could be skipped. And so on. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Nominating Kimmo Ollila for Approver
On 25.1.2019 12.43, Ville Voutilainen wrote: > On Fri, 25 Jan 2019 at 10:22, Teemu Holappa wrote: >> >> Hello All, >> >> I'd like to nominate Kimmo Ollila for Approver. He has joined The Qt Company >> more than five years ago. Lately he has been working on INTEGRITY RTOS >> support and maintenance. > > > +1. +1, with the disclaimer that I'm in the neighboring room in the office. -- Kari ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
> On 25. Jan 2019, at 11:08, Lars Knoll wrote: > > > >> On 25 Jan 2019, at 10:10, Simon Hausmann wrote: >> >> >> I think it's worthwhile to develop the tooling to automate cherry-picking. >> That tooling is something that is perhaps best tried on a release branch >> first. >> >> I do quite like what Allan suggested: We could try the cherry-pick the other >> way around. Volker, Lars, Thiago etc. surely remember the p4i script we used >> to have when we were using Perforce. Imagine that with more automation. > > The risk with this is that we might end up having fixes in a stable branch > that don’t make it do dev for various reasons, so it’ll regress in the next > minor release. Note that this risk exists partially even if fixes are first pushed into dev and then from dev directly to multiple “stable” branches. Fix goes into dev. Fix is cherry-picked into 5.9 without issues. Fix is cherry-picked into 5.12 and for whatever reason there are issues with that. There is a risk that the bug will not be fixed in the next (or any) 5.12 release even though it already is fixed in 5.9. Granted, it will be fixed in 5.13 again. This is also not ideal and requires release management to keep track of these changes to avoid it. (Some of this risk exists also for 5.12.x branch vs 5.12 branch, if we directly pick from dev to 5.12.x) > >> (1) Liang's work load gets reduced dramatically - conflict resolution is >> distributed. >> (2) Changes hit the customer relevant branch ASAP. >> (3) The load on the CI does not change for anybody working in dev. > > The CI problem comes from the fact that if we have a high rate of stages to > qtbase/dev, we at some point get into a deadlock situation, even if we > disregard any flakiness in the system. That’s because higher rates imply that > more changes are tested together. This in turn increasing the risk of > rejection of all changes because of one bad change in the set. So the current > system doesn’t scale and basically rate limits the amount of changes going > into a branch (or worse, we end up getting traffic jams where the actual rate > actually goes down to zero). > > To me this means we need to seriously rethink that part of our CI system, and > ideally test changes (or patch series) individually and in parallel. So maybe > we should adjust our CI system that way: > > * test changes (or patch series) individually > * If they pass CI merge or cherry-pick them into the branch > * reject only if the merge/cherry-pick gives conflicts > > This adds a very small risk that two parallel changes don’t conflict during > the merge/cherry-pick process, but cause a test regression together. To help > with that, we can simply run a regular status check on the repo. If this > happens the repo will be blocked for further testing until someone submits a > fix or reverts an offending change, which is acceptable. > > Another advantage is that this would pave the road towards a system where CI > testing happens before human review, so we can in the longer term avoid > duplicated review/approval work. > > Cheers, > Lars > >> >> >> >> Simon >> From: Lars Knoll >> Sent: Friday, January 25, 2019 9:05:45 AM >> To: Ville Voutilainen >> Cc: Simon Hausmann; Qt development mailing list >> Subject: Re: [Development] Proposal: New branch model >> >> > On 24 Jan 2019, at 22:29, Ville Voutilainen >> > wrote: >> > >> > On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: >> >> >> >> >> >> I would see the biggest long term impact with the massive amount of >> >> cherry picks from dev to qt6 over a long period of time. >> >> >> >> Git rerere works locally, so it doesn’t help in this setup I think. >> > >> > Completely seriously, without any preference to either branching >> > approach: aren't we going to be in some sort of trouble >> > with the qt6 branch anyway, no matter what we do? Following on a bit: >> > >> > Pardon me if I missed some important part of the motivation of all of >> > this, but I *CAN'T* see how this should, could, >> > or needs to be an approach that helps with the qt6 branch >> > merge/cherry-pick/rebase. I don't think that's going to >> > be a pleasant operation whatever we do. >> > >> > I would like a "push to trunk, backport to release-branches" approach >> > going forward. As in, once we have the usual >> > umpteen X.y branches, it's a simple approach. >> > >> > But I don't see how going from merge-forward (except also >> > merge-backward sometimes) to cherry-pick-backward >> > (except also cherry-pick forward, or kinda sideways to qt6, and maybe >> > sometimes merge in some direction) is going >> > to help us with qt6. These matters seem orthogonal. >> >> After some more thinking, I tend to agree. Qt6 is somewhat special and maybe >> needs to be treated differently. >> >> But except for Qt 6 I like the proposed policy for a couple of reasons: >> >> * I think it makes the life of casual/new contributors easier. Simply
Re: [Development] Nominating Sami Nurmenniemi for Approver status
On Fri, 25 Jan 2019 at 10:31, Simon Hausmann wrote: > > > +1 without any hesitation. (I thought he was an approver already :) +1. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Nominating Kimmo Ollila for Approver
On Fri, 25 Jan 2019 at 10:22, Teemu Holappa wrote: > > Hello All, > > I'd like to nominate Kimmo Ollila for Approver. He has joined The Qt Company > more than five years ago. Lately he has been working on INTEGRITY RTOS > support and maintenance. +1. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
On 25 Jan 2019, at 10:10, Simon Hausmann mailto:simon.hausm...@qt.io>> wrote: I think it's worthwhile to develop the tooling to automate cherry-picking. That tooling is something that is perhaps best tried on a release branch first. I do quite like what Allan suggested: We could try the cherry-pick the other way around. Volker, Lars, Thiago etc. surely remember the p4i script we used to have when we were using Perforce. Imagine that with more automation. The risk with this is that we might end up having fixes in a stable branch that don’t make it do dev for various reasons, so it’ll regress in the next minor release. (1) Liang's work load gets reduced dramatically - conflict resolution is distributed. (2) Changes hit the customer relevant branch ASAP. (3) The load on the CI does not change for anybody working in dev. The CI problem comes from the fact that if we have a high rate of stages to qtbase/dev, we at some point get into a deadlock situation, even if we disregard any flakiness in the system. That’s because higher rates imply that more changes are tested together. This in turn increasing the risk of rejection of all changes because of one bad change in the set. So the current system doesn’t scale and basically rate limits the amount of changes going into a branch (or worse, we end up getting traffic jams where the actual rate actually goes down to zero). To me this means we need to seriously rethink that part of our CI system, and ideally test changes (or patch series) individually and in parallel. So maybe we should adjust our CI system that way: * test changes (or patch series) individually * If they pass CI merge or cherry-pick them into the branch * reject only if the merge/cherry-pick gives conflicts This adds a very small risk that two parallel changes don’t conflict during the merge/cherry-pick process, but cause a test regression together. To help with that, we can simply run a regular status check on the repo. If this happens the repo will be blocked for further testing until someone submits a fix or reverts an offending change, which is acceptable. Another advantage is that this would pave the road towards a system where CI testing happens before human review, so we can in the longer term avoid duplicated review/approval work. Cheers, Lars Simon From: Lars Knoll Sent: Friday, January 25, 2019 9:05:45 AM To: Ville Voutilainen Cc: Simon Hausmann; Qt development mailing list Subject: Re: [Development] Proposal: New branch model > On 24 Jan 2019, at 22:29, Ville Voutilainen > mailto:ville.voutilai...@gmail.com>> wrote: > > On Thu, 24 Jan 2019 at 20:26, Simon Hausmann > mailto:simon.hausm...@qt.io>> wrote: >> >> >> I would see the biggest long term impact with the massive amount of cherry >> picks from dev to qt6 over a long period of time. >> >> Git rerere works locally, so it doesn’t help in this setup I think. > > Completely seriously, without any preference to either branching > approach: aren't we going to be in some sort of trouble > with the qt6 branch anyway, no matter what we do? Following on a bit: > > Pardon me if I missed some important part of the motivation of all of > this, but I *CAN'T* see how this should, could, > or needs to be an approach that helps with the qt6 branch > merge/cherry-pick/rebase. I don't think that's going to > be a pleasant operation whatever we do. > > I would like a "push to trunk, backport to release-branches" approach > going forward. As in, once we have the usual > umpteen X.y branches, it's a simple approach. > > But I don't see how going from merge-forward (except also > merge-backward sometimes) to cherry-pick-backward > (except also cherry-pick forward, or kinda sideways to qt6, and maybe > sometimes merge in some direction) is going > to help us with qt6. These matters seem orthogonal. After some more thinking, I tend to agree. Qt6 is somewhat special and maybe needs to be treated differently. But except for Qt 6 I like the proposed policy for a couple of reasons: * I think it makes the life of casual/new contributors easier. Simply always develop and push against the development branch. The more experienced reviewer can then easily decide that the fix should be cherry-picked into a stable branch. * The system should be able to handle most of the cherry-picks automatically * In case something goes wrong, it’s up to the developer/reviewer/maintainer to fix the cherry-pick and get it into the old branch. This leaves conflict resolution with the people that have the knowledge about that part of the code, not one central person that has to babysit merges * It will somewhat naturally limit fixes going into the stable branches to bug fixes. In most cases, we do not want long patch series in stable branches, as we shouldn’t have feature development there. A possible solution for Qt 6 could be to treat it as a feature branch from dev, and keep merging from dev, at
Re: [Development] Proposal: New branch model
First the change would need to land in the targeted branch, then the cherry-picks to other branches. Batching and controlled intervals are possible, but taken to an extreme they would replicate the current load as a group of commits would be taken in a single build as with merges currently. If there's less batching, there are more integrations and thus more load. Could be only a little more, of course. A lot of the stated advantages have also been about how creators of the changes can take ownership of them and how changes can proceed into other branches indepently. If you are can (and are responsible for) staging the cherry-picks if the initial stage is not successful, how would you ensure the batching? Sometimes there are conflicts in the cherry-picks. Will other changes in the same batch wait for them to be fixed? Or is the batching smart enough to understand whether there is a dependency or not? I think there is a risk of reinventing the merges here. Or if not reinvention, at least creating a new tool that is going to see feature creep like Sergio's mail hinted from experience. -- Kari On 25.1.2019 11.50, Simon Hausmann wrote: > > The first attempt at getting a change in is distributed over multiple > branches. > The staging of the cherry-pick can happen in batches and controlled intervals > as > with merges. > > > > Simon > > > *From:* Kari Oikarinen > *Sent:* Friday, January 25, 2019 10:41:21 AM > *To:* Simon Hausmann; Lars Knoll; Ville Voutilainen > *Cc:* Qt development mailing list > *Subject:* Re: [Development] Proposal: New branch model > How would the CI load not change? Rather than one merge bringing multiple > commits, each change would still be picked individually. > > I also think the cherry-picking model makes more sense when branches with less > action get the cherry-picks. The cherry-picking conflict issues grow as the > amount of changes grow. > > If we want more cherry-picking, I think we should rather pick the most frozen > branches not yet in cherry-picking to convert. That of course would only help > with (1). > > -- > Kari > > On 25.1.2019 11.10, Simon Hausmann wrote: >> >> I think it's worthwhile to develop the tooling to automate cherry-picking. >> That >> tooling is something that is perhaps best tried on a release branch first. >> >> >> I do quite like what Allan suggested: We could try the cherry-pick the other >> way >> around. Volker, Lars, Thiago etc. surely remember the p4i script we used to >> have >> when we were using Perforce. Imagine that with more automation. >> >> >> (1) Liang's work load gets reduced dramatically - conflict resolution is >> distributed. >> >> (2) Changes hit the customer relevant branch ASAP. >> >> (3) The load on the CI does not change for anybody working in dev. >> >> >> >> >> Simon >> >> >> *From:* Lars Knoll >> *Sent:* Friday, January 25, 2019 9:05:45 AM >> *To:* Ville Voutilainen >> *Cc:* Simon Hausmann; Qt development mailing list >> *Subject:* Re: [Development] Proposal: New branch model >>> On 24 Jan 2019, at 22:29, Ville Voutilainen >>> wrote: >>> >>> On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: I would see the biggest long term impact with the massive amount of cherry picks from dev to qt6 over a long period of time. Git rerere works locally, so it doesn’t help in this setup I think. >>> >>> Completely seriously, without any preference to either branching >>> approach: aren't we going to be in some sort of trouble >>> with the qt6 branch anyway, no matter what we do? Following on a bit: >>> >>> Pardon me if I missed some important part of the motivation of all of >>> this, but I *CAN'T* see how this should, could, >>> or needs to be an approach that helps with the qt6 branch >>> merge/cherry-pick/rebase. I don't think that's going to >>> be a pleasant operation whatever we do. >>> >>> I would like a "push to trunk, backport to release-branches" approach >>> going forward. As in, once we have the usual >>> umpteen X.y branches, it's a simple approach. >>> >>> But I don't see how going from merge-forward (except also >>> merge-backward sometimes) to cherry-pick-backward >>> (except also cherry-pick forward, or kinda sideways to qt6, and maybe >>> sometimes merge in some direction) is going >>> to help us with qt6. These matters seem orthogonal. >> >> After some more thinking, I tend to agree. Qt6 is somewhat special and maybe >> needs to be treated differently. >> >> But except for Qt 6 I like the proposed policy for a couple of reasons: >> >> * I think it makes the life of casual/new contributors easier. Simply always >> develop and push against the development branch. The more experienced >> reviewer >> can then easily decide that the fix should be cherry-picked into a stable >> branch.
Re: [Development] Qt6: Adding UTF-8 storage support to QString
> Original Message- > From: Thiago Macieira > > But we WILL NOT change from UTF-16 in the next 2 years. From a user standpoint, this seems perfectly Ok to me. I do not buy the argument that if switching QString to utf8 make developer bugs appear sooner, this is a good thing. Most user code I have written or seen handles text data naively and is incorrect in some respect but I think only a minority of if is leading to real problems because input data will rarely trigger them. Although not perfect, using 16 bits "characters" for QString and Windows API is good approximation that helped a lot make user code more robust without requiring understanding charsets and encodings. At least, it saved me a lot of time if I remember correctly the kind of bugs I was dealing with in the 90's. So, IMHO, accessing QString content in utf8 "character" units should remain an explicit choice, not the default one. Even choosing utf8 internally QString for performance reasons seems dubious to me, at least for a good half of the world... ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
The first attempt at getting a change in is distributed over multiple branches. The staging of the cherry-pick can happen in batches and controlled intervals as with merges. Simon From: Kari Oikarinen Sent: Friday, January 25, 2019 10:41:21 AM To: Simon Hausmann; Lars Knoll; Ville Voutilainen Cc: Qt development mailing list Subject: Re: [Development] Proposal: New branch model How would the CI load not change? Rather than one merge bringing multiple commits, each change would still be picked individually. I also think the cherry-picking model makes more sense when branches with less action get the cherry-picks. The cherry-picking conflict issues grow as the amount of changes grow. If we want more cherry-picking, I think we should rather pick the most frozen branches not yet in cherry-picking to convert. That of course would only help with (1). -- Kari On 25.1.2019 11.10, Simon Hausmann wrote: > > I think it's worthwhile to develop the tooling to automate cherry-picking. > That > tooling is something that is perhaps best tried on a release branch first. > > > I do quite like what Allan suggested: We could try the cherry-pick the other > way > around. Volker, Lars, Thiago etc. surely remember the p4i script we used to > have > when we were using Perforce. Imagine that with more automation. > > > (1) Liang's work load gets reduced dramatically - conflict resolution is > distributed. > > (2) Changes hit the customer relevant branch ASAP. > > (3) The load on the CI does not change for anybody working in dev. > > > > > Simon > > > *From:* Lars Knoll > *Sent:* Friday, January 25, 2019 9:05:45 AM > *To:* Ville Voutilainen > *Cc:* Simon Hausmann; Qt development mailing list > *Subject:* Re: [Development] Proposal: New branch model >> On 24 Jan 2019, at 22:29, Ville Voutilainen >> wrote: >> >> On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: >>> >>> >>> I would see the biggest long term impact with the massive amount of cherry >>> picks from dev to qt6 over a long period of time. >>> >>> Git rerere works locally, so it doesn’t help in this setup I think. >> >> Completely seriously, without any preference to either branching >> approach: aren't we going to be in some sort of trouble >> with the qt6 branch anyway, no matter what we do? Following on a bit: >> >> Pardon me if I missed some important part of the motivation of all of >> this, but I *CAN'T* see how this should, could, >> or needs to be an approach that helps with the qt6 branch >> merge/cherry-pick/rebase. I don't think that's going to >> be a pleasant operation whatever we do. >> >> I would like a "push to trunk, backport to release-branches" approach >> going forward. As in, once we have the usual >> umpteen X.y branches, it's a simple approach. >> >> But I don't see how going from merge-forward (except also >> merge-backward sometimes) to cherry-pick-backward >> (except also cherry-pick forward, or kinda sideways to qt6, and maybe >> sometimes merge in some direction) is going >> to help us with qt6. These matters seem orthogonal. > > After some more thinking, I tend to agree. Qt6 is somewhat special and maybe > needs to be treated differently. > > But except for Qt 6 I like the proposed policy for a couple of reasons: > > * I think it makes the life of casual/new contributors easier. Simply always > develop and push against the development branch. The more experienced reviewer > can then easily decide that the fix should be cherry-picked into a stable > branch. > * The system should be able to handle most of the cherry-picks automatically > * In case something goes wrong, it’s up to the developer/reviewer/maintainer > to > fix the cherry-pick and get it into the old branch. This leaves conflict > resolution with the people that have the knowledge about that part of the > code, > not one central person that has to babysit merges > * It will somewhat naturally limit fixes going into the stable branches to bug > fixes. In most cases, we do not want long patch series in stable branches, as > we > shouldn’t have feature development there. > > A possible solution for Qt 6 could be to treat it as a feature branch from > dev, > and keep merging from dev, at least as long as possible. Once it’s not > possible > anymore, we should probably make Qt 6 the new development branch and move all > Qt > 5 related branches into a stable mode where we only focus on bug fixes. > > Let’s also not forget that Qt 6 is to some extent a temporary problem that > we’ll > have for maximum one year. After that all of Qt 5 is in stable mode with bug > fixing only. Then we can basically go back to a slightly modified cherry-pick > mode: Push to dev (Qt6) and cherry-pick into Qt 5.15 for bug fixes. If the > code > base has diverged and the fix needs to go into Qt 5, we might end up doing two > changes, one for Qt6 and one for Qt5. That’s not
Re: [Development] Qt6: Adding UTF-8 storage support to QString
On Thu, Jan 24, 2019 at 10:57 PM Thiago Macieira wrote: > > On Wednesday, 23 January 2019 23:32:28 PST Olivier Goffart wrote: > > - Introduce some iterator that iterates over unicode code points. > > I wrote that about a decade ago. It's called QStringIterator and it's inside > our sources, but in a private header. > > But we may want to make it iterate over grapheme clusters instead of Unicode > codepoints. That is, make it use QTextBoundaryFinder to iterate, instead of > decode the storage to UTF-32. > [...] Sidenote: Such a QStringIterator would also be helpful for KTextEditor, where we likely have some bugs we usually never see since we never have > UTF16 or composed characters. Greetings Dominik ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
How would the CI load not change? Rather than one merge bringing multiple commits, each change would still be picked individually. I also think the cherry-picking model makes more sense when branches with less action get the cherry-picks. The cherry-picking conflict issues grow as the amount of changes grow. If we want more cherry-picking, I think we should rather pick the most frozen branches not yet in cherry-picking to convert. That of course would only help with (1). -- Kari On 25.1.2019 11.10, Simon Hausmann wrote: > > I think it's worthwhile to develop the tooling to automate cherry-picking. > That > tooling is something that is perhaps best tried on a release branch first. > > > I do quite like what Allan suggested: We could try the cherry-pick the other > way > around. Volker, Lars, Thiago etc. surely remember the p4i script we used to > have > when we were using Perforce. Imagine that with more automation. > > > (1) Liang's work load gets reduced dramatically - conflict resolution is > distributed. > > (2) Changes hit the customer relevant branch ASAP. > > (3) The load on the CI does not change for anybody working in dev. > > > > > Simon > > > *From:* Lars Knoll > *Sent:* Friday, January 25, 2019 9:05:45 AM > *To:* Ville Voutilainen > *Cc:* Simon Hausmann; Qt development mailing list > *Subject:* Re: [Development] Proposal: New branch model >> On 24 Jan 2019, at 22:29, Ville Voutilainen >> wrote: >> >> On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: >>> >>> >>> I would see the biggest long term impact with the massive amount of cherry >>> picks from dev to qt6 over a long period of time. >>> >>> Git rerere works locally, so it doesn’t help in this setup I think. >> >> Completely seriously, without any preference to either branching >> approach: aren't we going to be in some sort of trouble >> with the qt6 branch anyway, no matter what we do? Following on a bit: >> >> Pardon me if I missed some important part of the motivation of all of >> this, but I *CAN'T* see how this should, could, >> or needs to be an approach that helps with the qt6 branch >> merge/cherry-pick/rebase. I don't think that's going to >> be a pleasant operation whatever we do. >> >> I would like a "push to trunk, backport to release-branches" approach >> going forward. As in, once we have the usual >> umpteen X.y branches, it's a simple approach. >> >> But I don't see how going from merge-forward (except also >> merge-backward sometimes) to cherry-pick-backward >> (except also cherry-pick forward, or kinda sideways to qt6, and maybe >> sometimes merge in some direction) is going >> to help us with qt6. These matters seem orthogonal. > > After some more thinking, I tend to agree. Qt6 is somewhat special and maybe > needs to be treated differently. > > But except for Qt 6 I like the proposed policy for a couple of reasons: > > * I think it makes the life of casual/new contributors easier. Simply always > develop and push against the development branch. The more experienced > reviewer > can then easily decide that the fix should be cherry-picked into a stable > branch. > * The system should be able to handle most of the cherry-picks automatically > * In case something goes wrong, it’s up to the developer/reviewer/maintainer > to > fix the cherry-pick and get it into the old branch. This leaves conflict > resolution with the people that have the knowledge about that part of the > code, > not one central person that has to babysit merges > * It will somewhat naturally limit fixes going into the stable branches to > bug > fixes. In most cases, we do not want long patch series in stable branches, as > we > shouldn’t have feature development there. > > A possible solution for Qt 6 could be to treat it as a feature branch from > dev, > and keep merging from dev, at least as long as possible. Once it’s not > possible > anymore, we should probably make Qt 6 the new development branch and move all > Qt > 5 related branches into a stable mode where we only focus on bug fixes. > > Let’s also not forget that Qt 6 is to some extent a temporary problem that > we’ll > have for maximum one year. After that all of Qt 5 is in stable mode with bug > fixing only. Then we can basically go back to a slightly modified cherry-pick > mode: Push to dev (Qt6) and cherry-pick into Qt 5.15 for bug fixes. If the > code > base has diverged and the fix needs to go into Qt 5, we might end up doing > two > changes, one for Qt6 and one for Qt5. That’s not very different from the > situation we had with Qt 4 some years ago. > > Cheers, > Lars > >> >> Qt6 and dev are going to diverge. Drastically, eventually. I don't >> know how to solve that. A new branching policy >> is not going to help with that. >> ___ >> Development mailing list >> Development@qt-project.org >>
Re: [Development] Proposal: New branch model
I think it's worthwhile to develop the tooling to automate cherry-picking. That tooling is something that is perhaps best tried on a release branch first. I do quite like what Allan suggested: We could try the cherry-pick the other way around. Volker, Lars, Thiago etc. surely remember the p4i script we used to have when we were using Perforce. Imagine that with more automation. (1) Liang's work load gets reduced dramatically - conflict resolution is distributed. (2) Changes hit the customer relevant branch ASAP. (3) The load on the CI does not change for anybody working in dev. Simon From: Lars Knoll Sent: Friday, January 25, 2019 9:05:45 AM To: Ville Voutilainen Cc: Simon Hausmann; Qt development mailing list Subject: Re: [Development] Proposal: New branch model > On 24 Jan 2019, at 22:29, Ville Voutilainen > wrote: > > On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: >> >> >> I would see the biggest long term impact with the massive amount of cherry >> picks from dev to qt6 over a long period of time. >> >> Git rerere works locally, so it doesn’t help in this setup I think. > > Completely seriously, without any preference to either branching > approach: aren't we going to be in some sort of trouble > with the qt6 branch anyway, no matter what we do? Following on a bit: > > Pardon me if I missed some important part of the motivation of all of > this, but I *CAN'T* see how this should, could, > or needs to be an approach that helps with the qt6 branch > merge/cherry-pick/rebase. I don't think that's going to > be a pleasant operation whatever we do. > > I would like a "push to trunk, backport to release-branches" approach > going forward. As in, once we have the usual > umpteen X.y branches, it's a simple approach. > > But I don't see how going from merge-forward (except also > merge-backward sometimes) to cherry-pick-backward > (except also cherry-pick forward, or kinda sideways to qt6, and maybe > sometimes merge in some direction) is going > to help us with qt6. These matters seem orthogonal. After some more thinking, I tend to agree. Qt6 is somewhat special and maybe needs to be treated differently. But except for Qt 6 I like the proposed policy for a couple of reasons: * I think it makes the life of casual/new contributors easier. Simply always develop and push against the development branch. The more experienced reviewer can then easily decide that the fix should be cherry-picked into a stable branch. * The system should be able to handle most of the cherry-picks automatically * In case something goes wrong, it’s up to the developer/reviewer/maintainer to fix the cherry-pick and get it into the old branch. This leaves conflict resolution with the people that have the knowledge about that part of the code, not one central person that has to babysit merges * It will somewhat naturally limit fixes going into the stable branches to bug fixes. In most cases, we do not want long patch series in stable branches, as we shouldn’t have feature development there. A possible solution for Qt 6 could be to treat it as a feature branch from dev, and keep merging from dev, at least as long as possible. Once it’s not possible anymore, we should probably make Qt 6 the new development branch and move all Qt 5 related branches into a stable mode where we only focus on bug fixes. Let’s also not forget that Qt 6 is to some extent a temporary problem that we’ll have for maximum one year. After that all of Qt 5 is in stable mode with bug fixing only. Then we can basically go back to a slightly modified cherry-pick mode: Push to dev (Qt6) and cherry-pick into Qt 5.15 for bug fixes. If the code base has diverged and the fix needs to go into Qt 5, we might end up doing two changes, one for Qt6 and one for Qt5. That’s not very different from the situation we had with Qt 4 some years ago. Cheers, Lars > > Qt6 and dev are going to diverge. Drastically, eventually. I don't > know how to solve that. A new branching policy > is not going to help with that. > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
On Friday, 25 January 2019 09:05:45 CET Lars Knoll wrote: > * I think it makes the life of casual/new contributors easier. Simply always > develop and push against the development branch. The more experienced > reviewer can then easily decide that the fix should be cherry-picked into a > stable branch. This could also be accomplished by moving the change to the correct branch in gerrit as part of the review process. Currently, only the owner can ask the bot to move the change, but since this is our bot I assume that we can change that policy. - Paul ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
>It is the absolute exception that a change goes into qtbase on first attempt. But many rejections have nothing to do with any change at all. I often submit documentation-only changes to QtBase, and they often get tested alone, with no other changes. They still get rejected multiple times because test X fails this time and test Y fails next time, despite the fact that my change only fixes typos in the qdoc comments. Sometimes there is a compiler failure too, which is not caused by the change. Maybe we should have a regular cron job that just runs an empty change through the system to see what tests, compilers, etc are failing and flag them for fixing. And I suppose the cherrypicks to stable branches should not be triggered until the change has passed in the dev branch. martin From: Development on behalf of Simon Hausmann Sent: Friday, January 25, 2019 9:25:16 AM To: Allan Sandfeld Jensen; Jedrzej Nowacki Cc: development@qt-project.org Subject: Re: [Development] Proposal: New branch model I'm somewhat attracted to the proposed model, in conjunction with automation and by treating Qt6 differently. However Allan's last point is what sticks to me most, the load on the CI and the resulting impact on productivity: If all it would take to get changes distributed is a cherry-pick, then I can totally see how this can fly. However with a blocking CI and the rate of flakyness we are experiencing at least in qtbase, I think this carries a high risk of slowing down development. It is the absolute exception that a change goes into qtbase on first attempt. I'd say that in average perhaps three attempts of staging are needed. Currently the batches of integrations are distributed among the release branch (5.12.x), 5.12 and dev. With all changes going to dev first, the size of batches is going to increase as well as the frequency of integration attempts for the same branch. This either increases the change of rejection of your change to due an unrelated change (as the batches are bigger) or it will take longer before your change gets tried. That would seem like a slow down to me and also not something that helps casual first-time contributors. I would love to hear the opinion from frequent qtbase submitters on the proposal and how they feel about this. Liang's email weighs a lot in my opinion. And Sergio's and Ville's experience with this model as well. I think that we should try this with a release branch first before we go all in. It would seem that quite a bit of tooling needs to be developed first anyway? Simon From: Development on behalf of Allan Sandfeld Jensen Sent: Thursday, January 24, 2019 1:33:28 PM To: Jedrzej Nowacki Cc: development@qt-project.org Subject: Re: [Development] Proposal: New branch model On Donnerstag, 24. Januar 2019 13:27:41 CET Jedrzej Nowacki wrote: > Dnia środa, 23 stycznia 2019 22:04:16 CET Allan Sandfeld Jensen pisze: > > > On Mittwoch, 23. Januar 2019 16:51:10 CET Jedrzej Nowacki wrote: > > > > > Proposal in short: let's use cherry-pick mode everywhere. > > > > > > All(**) changes would go to dev. From which they would be > > > automatically > > > > > > > > > cherry-picked by a bot to other branches. The decision to which branch > > > cherry- > > > > > > > > pick, would be taken based on a tag in the commit message. We > > > > > > > > > could add a footer that marks the change risk level as in quip-5 > > > (http://quips-qt-io.herokuapp.com/quip-0005.html), so for example > > > "dev", > > > "stable", "LTS". By default everything would be cherry-picked to qt6 > > > branch > > > unless "no-future" tag would be given. Of course we can bike-shed about > > > the > > > tag names. > > > > > > I don't see any advantage to this what so ever. The same amount of work > > and refactoring needs to be done, all you have done is made development > > more prone to human error, and fixes less likely to reach their intended > > target, and made getting point releases out on time harder as they need > > to go through more steps before they have all their patches in. > > > > 'Allan > > > > It is hard to answer this because you have forgotten to write why you think > that development would be more error prone or fixes less likely to reach > the destination or more steps would be involved in releasing. So I will try > shooting in the dark :-) > > The current merging strategy involves couple of people, one merge master and > a random reviewer that looks at conflicts if there are any. Usually it is > not even close to quality level we have for single patches in which we have > more reviewers and gerrit shows the real diff. In practice we assume that > no conflicts == good merge. > > Fixes are less likely to hit target branches, true, but only if nobody > cares. Mark that a fix can magically disappear during broken mere, nobody > will see it. The proposed solution allows at
Re: [Development] Nominating Sami Nurmenniemi for Approver status
+1 without any hesitation. (I thought he was an approver already :) Simon From: Development on behalf of Kari Oikarinen Sent: Friday, January 25, 2019 9:06:29 AM To: development@qt-project.org Subject: [Development] Nominating Sami Nurmenniemi for Approver status Hi! I'd like to nominate Sami Nurmenniemi for Approver. He has already been working in The Qt Company in the same team as I am for a good while. Sami has worked on (among other things): - Improving flaky tests - CI coverage for ARM platforms by using user mode QEMU - Demos for embedded devices He is a careful and competent developer and reviewer. I trust he would wield his new powers with good judgment. His changes: https://codereview.qt-project.org/#/q/owner:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z His reviews: https://codereview.qt-project.org/#/q/reviewer:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z -- Kari ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
I'm somewhat attracted to the proposed model, in conjunction with automation and by treating Qt6 differently. However Allan's last point is what sticks to me most, the load on the CI and the resulting impact on productivity: If all it would take to get changes distributed is a cherry-pick, then I can totally see how this can fly. However with a blocking CI and the rate of flakyness we are experiencing at least in qtbase, I think this carries a high risk of slowing down development. It is the absolute exception that a change goes into qtbase on first attempt. I'd say that in average perhaps three attempts of staging are needed. Currently the batches of integrations are distributed among the release branch (5.12.x), 5.12 and dev. With all changes going to dev first, the size of batches is going to increase as well as the frequency of integration attempts for the same branch. This either increases the change of rejection of your change to due an unrelated change (as the batches are bigger) or it will take longer before your change gets tried. That would seem like a slow down to me and also not something that helps casual first-time contributors. I would love to hear the opinion from frequent qtbase submitters on the proposal and how they feel about this. Liang's email weighs a lot in my opinion. And Sergio's and Ville's experience with this model as well. I think that we should try this with a release branch first before we go all in. It would seem that quite a bit of tooling needs to be developed first anyway? Simon From: Development on behalf of Allan Sandfeld Jensen Sent: Thursday, January 24, 2019 1:33:28 PM To: Jedrzej Nowacki Cc: development@qt-project.org Subject: Re: [Development] Proposal: New branch model On Donnerstag, 24. Januar 2019 13:27:41 CET Jedrzej Nowacki wrote: > Dnia środa, 23 stycznia 2019 22:04:16 CET Allan Sandfeld Jensen pisze: > > > On Mittwoch, 23. Januar 2019 16:51:10 CET Jedrzej Nowacki wrote: > > > > > Proposal in short: let's use cherry-pick mode everywhere. > > > > > > All(**) changes would go to dev. From which they would be > > > automatically > > > > > > > > > cherry-picked by a bot to other branches. The decision to which branch > > > cherry- > > > > > > > > pick, would be taken based on a tag in the commit message. We > > > > > > > > > could add a footer that marks the change risk level as in quip-5 > > > (http://quips-qt-io.herokuapp.com/quip-0005.html), so for example > > > "dev", > > > "stable", "LTS". By default everything would be cherry-picked to qt6 > > > branch > > > unless "no-future" tag would be given. Of course we can bike-shed about > > > the > > > tag names. > > > > > > I don't see any advantage to this what so ever. The same amount of work > > and refactoring needs to be done, all you have done is made development > > more prone to human error, and fixes less likely to reach their intended > > target, and made getting point releases out on time harder as they need > > to go through more steps before they have all their patches in. > > > > 'Allan > > > > It is hard to answer this because you have forgotten to write why you think > that development would be more error prone or fixes less likely to reach > the destination or more steps would be involved in releasing. So I will try > shooting in the dark :-) > > The current merging strategy involves couple of people, one merge master and > a random reviewer that looks at conflicts if there are any. Usually it is > not even close to quality level we have for single patches in which we have > more reviewers and gerrit shows the real diff. In practice we assume that > no conflicts == good merge. > > Fixes are less likely to hit target branches, true, but only if nobody > cares. Mark that a fix can magically disappear during broken mere, nobody > will see it. The proposed solution allows at least track such cases. > > Releases branches already use cherry-pick mode so from that perspective > there is no big changes, the only one is the origin branch from which the > cherry- pick should happen. > Yeah, this was no the best of comments on the issue. But I believe I wrote my objections elsewhere: - Cherry-picks are more error prone in git than merges and requires manual intervention more frequently - Even if we accept the always and instantly cherry-pick model, there is no reason to push to dev, continuing to push to the same branches as now could achieve all the same things with less room for error (because where it is supposed to be merged to is both implicit and unambigious) - By having all commits go to the same branch you make that branch overloaded, and will break our CI even more than it already is. - I don't see making it harder to fix bugs in the stable branch as a benefit. Best regards 'Allan ___ Development mailing list Development@qt-project.org
[Development] Nominating Kimmo Ollila for Approver
Hello All, I'd like to nominate Kimmo Ollila for Approver. He has joined The Qt Company more than five years ago. Lately he has been working on INTEGRITY RTOS support and maintenance. Here is the list of his changes: https://codereview.qt-project.org/#/q/owner:+kiollila,n,z and the list of his reviews: https://codereview.qt-project.org/#/q/reviewer:+kiollila,n,z Br, Teemu Holappa ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Nominating Sami Nurmenniemi for Approver status
+1 Thank Sami for lots of help on QEMU build and etc. —Liang > On 25 Jan 2019, at 09:06, Kari Oikarinen wrote: > > Hi! > > I'd like to nominate Sami Nurmenniemi for Approver. He has already been > working > in The Qt Company in the same team as I am for a good while. > > Sami has worked on (among other things): > > - Improving flaky tests > - CI coverage for ARM platforms by using user mode QEMU > - Demos for embedded devices > > He is a careful and competent developer and reviewer. I trust he would wield > his > new powers with good judgment. > > His changes: > > https://codereview.qt-project.org/#/q/owner:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z > > His reviews: > > https://codereview.qt-project.org/#/q/reviewer:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z > > -- > Kari > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
It's a fair point to raise. The closest scenario that I can think of that we've had in the past with a long living branch was 5.6 before it entered cherry-picking mode. The first change to qtbase 5.6 was uploaded to Gerrit on August 11th 2015. The last merge that I can see away from 5.6 was on November 25th 2016. That's ~16 months of merging. Short of other options, I think the method of using git merge is the option that's least painful for a long lived qt 6 branch. Simon From: Ville Voutilainen Sent: Thursday, January 24, 2019 10:29:13 PM To: Simon Hausmann Cc: Jedrzej Nowacki; development@qt-project.org Subject: Re: [Development] Proposal: New branch model On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: > > > I would see the biggest long term impact with the massive amount of cherry > picks from dev to qt6 over a long period of time. > > Git rerere works locally, so it doesn’t help in this setup I think. Completely seriously, without any preference to either branching approach: aren't we going to be in some sort of trouble with the qt6 branch anyway, no matter what we do? Following on a bit: Pardon me if I missed some important part of the motivation of all of this, but I *CAN'T* see how this should, could, or needs to be an approach that helps with the qt6 branch merge/cherry-pick/rebase. I don't think that's going to be a pleasant operation whatever we do. I would like a "push to trunk, backport to release-branches" approach going forward. As in, once we have the usual umpteen X.y branches, it's a simple approach. But I don't see how going from merge-forward (except also merge-backward sometimes) to cherry-pick-backward (except also cherry-pick forward, or kinda sideways to qt6, and maybe sometimes merge in some direction) is going to help us with qt6. These matters seem orthogonal. Qt6 and dev are going to diverge. Drastically, eventually. I don't know how to solve that. A new branching policy is not going to help with that. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] Nominating Sami Nurmenniemi for Approver status
Hi! I'd like to nominate Sami Nurmenniemi for Approver. He has already been working in The Qt Company in the same team as I am for a good while. Sami has worked on (among other things): - Improving flaky tests - CI coverage for ARM platforms by using user mode QEMU - Demos for embedded devices He is a careful and competent developer and reviewer. I trust he would wield his new powers with good judgment. His changes: https://codereview.qt-project.org/#/q/owner:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z His reviews: https://codereview.qt-project.org/#/q/reviewer:%22Sami+Nurmenniemi+%253Csami.nurmenniemi%2540qt.io%253E%22,n,z -- Kari ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: New branch model
> On 24 Jan 2019, at 22:29, Ville Voutilainen > wrote: > > On Thu, 24 Jan 2019 at 20:26, Simon Hausmann wrote: >> >> >> I would see the biggest long term impact with the massive amount of cherry >> picks from dev to qt6 over a long period of time. >> >> Git rerere works locally, so it doesn’t help in this setup I think. > > Completely seriously, without any preference to either branching > approach: aren't we going to be in some sort of trouble > with the qt6 branch anyway, no matter what we do? Following on a bit: > > Pardon me if I missed some important part of the motivation of all of > this, but I *CAN'T* see how this should, could, > or needs to be an approach that helps with the qt6 branch > merge/cherry-pick/rebase. I don't think that's going to > be a pleasant operation whatever we do. > > I would like a "push to trunk, backport to release-branches" approach > going forward. As in, once we have the usual > umpteen X.y branches, it's a simple approach. > > But I don't see how going from merge-forward (except also > merge-backward sometimes) to cherry-pick-backward > (except also cherry-pick forward, or kinda sideways to qt6, and maybe > sometimes merge in some direction) is going > to help us with qt6. These matters seem orthogonal. After some more thinking, I tend to agree. Qt6 is somewhat special and maybe needs to be treated differently. But except for Qt 6 I like the proposed policy for a couple of reasons: * I think it makes the life of casual/new contributors easier. Simply always develop and push against the development branch. The more experienced reviewer can then easily decide that the fix should be cherry-picked into a stable branch. * The system should be able to handle most of the cherry-picks automatically * In case something goes wrong, it’s up to the developer/reviewer/maintainer to fix the cherry-pick and get it into the old branch. This leaves conflict resolution with the people that have the knowledge about that part of the code, not one central person that has to babysit merges * It will somewhat naturally limit fixes going into the stable branches to bug fixes. In most cases, we do not want long patch series in stable branches, as we shouldn’t have feature development there. A possible solution for Qt 6 could be to treat it as a feature branch from dev, and keep merging from dev, at least as long as possible. Once it’s not possible anymore, we should probably make Qt 6 the new development branch and move all Qt 5 related branches into a stable mode where we only focus on bug fixes. Let’s also not forget that Qt 6 is to some extent a temporary problem that we’ll have for maximum one year. After that all of Qt 5 is in stable mode with bug fixing only. Then we can basically go back to a slightly modified cherry-pick mode: Push to dev (Qt6) and cherry-pick into Qt 5.15 for bug fixes. If the code base has diverged and the fix needs to go into Qt 5, we might end up doing two changes, one for Qt6 and one for Qt5. That’s not very different from the situation we had with Qt 4 some years ago. Cheers, Lars > > Qt6 and dev are going to diverge. Drastically, eventually. I don't > know how to solve that. A new branching policy > is not going to help with that. > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
