On Fri, Apr 28, 2023 at 11:38 AM Mark Michelson <[email protected]> wrote: > > Sorry for the top-post, but trying to write my responses inline at this > point is basically impossible for me if I want to actually make any > salient points :) Instead, I'll address the various topics one-by-one > here at the top. > > First, I'll weigh in on some of the easier topics: > > * More frequent minor releases > > I 100% agree that we should have more frequent minor releases, > especially on the LTS branches. I made a proposal on this list in > November 2021 here > https://mail.openvswitch.org/pipermail/ovs-dev/2021-November/389581.html > that would have made monthly point releases happen. The email got no > response, so I assumed it was unwanted and moved on. At this point, > maybe we don't necessarily need to put out new minor releases that > frequently, but having it be more regular would make it more worthwhile > to backport fixes to supported branches. > > * Less frequent major releases > > I agree with this too. When OVN split from OVS, we wanted a more > frequent release cadence since we were adding lots of new features and > optimizations into OVN, and CMSes were eager to consume these changes. > Three years later and the number of massive features and optimizations > in OVN has slowed down. The larger changes that we would like to make > could benefit from a longer lead time before release. It's been > suggested to move down to three major releases a year, but I'd even be > happy with two.
+1 for two major releases a year. > > * Shorter LTS support window > > I, personally, would largely be in favor of this since supporting old > versions is hard and can be risky. However, I think CMSes would not be > thrilled about being forced to upgrade more frequently. In fact, I'm > pretty sure Red Hat's OpenStack team was already not happy that they > *only* get two years of full support on an LTS. If the concern about the > length of LTS support is about the number of backports required, then I > think having more infrequent major releases would solve the problem for > us instead, and we can keep our current support windows. > > * Fewer unnecessary refactors > > I'm not sure this happens much, tbh. Typically, when a refactor happens, > it's also introducing new functionality or an optimization. The refactor > is there in order to more easily facilitate the accompanying change. > While there may be some refactors that exist solely for code "niceness" > (and I may have even contributed some), I can't think of any. I'm > hesitant to forbid refactors that don't add anything new, since they may > help maintainability of the code in the future. I think the best way to > approach this is a case-by-case basis. > > > OK, now for the harder topic. First, I'll make a hypothetical release > timeline. Pretend for a moment that It's December 2024. With the current > release schedule, the following releases would exist: > > 24.12 > 24.09 > 24.06 > 24.03 (LTS) > 23.12 > 23.09 > 23.06 > 23.03 > 22.12 > 22.09 > 22.06 > 22.03 (LTS) > > Based on our current documentation, the following releases would be > under support (based on the "backporting" documentation cited by Ilya): > > 24.12 > 24.09 > 24.06 > 24.03 (LTS) > > The following releases would have "critical" bug fix support > > 22.03 (LTS) > > And the following releases would be considered unsupported > > 23.12 > 23.09 > 23.06 > 23.03 > 22.12 > 22.09 > 22.06 > > So now let's consider some hypotheticals. > > Hypothetical 1: A bug is reported against OVN. Its root cause is > discovered, and it turns out it's been in every release of OVN ever. > Where should the fix be backported? > > Answer 1: Based on current policy, the bug fix should be backported to > all branches currently under support (24.03, 24.06, 24.09, and 24.12). > Even though it may be possible to backport the patches further, there's > not much point to it since we will never make any more minor releases on > the unsupported branches. However, I don't think anyone would NACK a > backport to an unsupported branch if it applies cleanly and does not > risk regression. > Agree. Just want to emphasize that the major point of the discussion for this case is when we are at the point of 25.12 (or 23.12), we will need to backport to 8 branches. Changing to two major releases per year would reduce the load. I think if we agree with this, we need to change as soon as we can. Even if we change it now, i.e. let the next release be 23.09, we will still need to maintain 6 branches for a period of time and decrease to 4 gradually. > Hypothetical 2: A critical security issues is reported against OVN. Its > root cause is discovered, and it turns out it's been in every release of > OVN ever. Where should the fix be backported? > > Answer 2: By the letter of the documentation, the fix would be applied > to all versions in Hypothetical 1, as well as 22.03. The discussion in > this thread has questioned whether the unsupported branches between the > LTSes should also have the fix applied, since this is not explicitly > covered by the documentation. My intention when writing the LTS policy > was that the answer was NO. The reason for this is that if a newer LTS > exists, then the preferred upgrade path for someone running an old LTS > would be to upgrade to the next LTS directly. And those that run any of > the releases between LTSes are also encouraged to upgrade to the newest > LTS. While that was my intention, it sounds like other devs in this > thread disagree and think that the fix should also be applied to the > unsupported releases between the LTSes. > Other than the consideration for upgrades, Ilya also mentioned that skipping branches "makes it harder to bisect in which branch the issue was introduced". I think it makes sense, but I don't remember facing this issue for OVN, so I'm not sure how important this factor will be. > > One last point I want to bring up is that there is nothing in our > documentation that says how long a "standard" release is supported. > Based on the backporting document that Ilya cited, it makes it sound as > though standard releases are supported up to the point of releasing a > new LTS. I think this should be rethought. Consider in the above example > the 23.12 release. It exists for three months, then the 24.03 LTS > release is made. Does that mean that a three month old release > immediately becomes unsupported because there's now a new LTS? > Meanwhile, the 22.06 release will have received 18 months of bug fixes. > To me, that makes little sense. I think we should codify standard > releases as follows: > > "Standard releases will receive full support for one year from the time > of their release. This means that bug fixes and security fixes will be > applied to this version, and minor releases will be made periodically > for one year after the release is made. Once the support window ends, > bugfixes and security fixes *may* be applied to the branches from which > standard releases are created, but no further minor releases will be > made. The reason that fixes may be applied to the unsupported branches > is to help facilitate backports to older supported LTS releases." > Thanks for bringing up this point. For a branch like 22.06, it would actually be supported for 21 months (much longer than 1 year), because we don't skip branches when backporting down to the last LTS release. So even if we have this one-year policy, the non-LTS releases still are not created equal. I think this may need to be clarified clearly, so that users understand that releases closer to LTS have shorter support time, while the minimum is one year. If I understand correctly, OVS is doing this differently and making each release supported for two years. Thanks, Han > Note that this now conflicts with the "backporting" documentation since > it says we will backport to all branches back to the most recent LTS. I > think we should still make a best effort to backport fixes to all > branches since the most recent LTS. However, I don't think it should be > required if the effort or risk of the backport is too much. So I think > the backporting documentation should be updated to reflect that if we > agree on the proposed lifetime of standard releases. > > > On 4/18/23 13:35, Han Zhou wrote: > > > > On Tue, Apr 18, 2023 at 10:33 AM Han Zhou <[email protected] > > <mailto:[email protected]>> wrote: > > > > > > > > > > > > On Tue, Apr 18, 2023 at 1:50 AM Dumitru Ceara <[email protected] > > <mailto:[email protected]>> wrote: > > > > > > > > On 4/17/23 21:46, Han Zhou wrote: > > > > > On Mon, Apr 17, 2023 at 12:01 PM Ilya Maximets > > <[email protected] <mailto:[email protected]>> wrote: > > > > >> > > > > >> On 4/17/23 19:20, Han Zhou wrote: > > > > >>> > > > > >>> > > > > >>> On Mon, Apr 17, 2023 at 9:24 AM Numan Siddique <[email protected] > > <mailto:[email protected]> <mailto: > > > > > [email protected] <mailto:[email protected]>>> wrote: > > > > >>>> > > > > >>>> On Mon, Apr 17, 2023 at 5:57 AM Dumitru Ceara > > <[email protected] <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > >>>>> > > > > >>>>> On 4/17/23 11:55, Dumitru Ceara wrote: > > > > >>>>>> On 4/14/23 18:01, Han Zhou wrote: > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> On Fri, Apr 14, 2023 at 2:16 AM Dumitru Ceara > > <[email protected] <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]>> > > > > >>>>>>> <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > >>>>>>>> > > > > >>>>>>>> On 4/14/23 04:15, Han Zhou wrote: > > > > >>>>>>>>> On Thu, Apr 13, 2023 at 12:44 PM Han Zhou <[email protected] > > <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]>> > > > > >>>>>>> <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>> I backported this series to the last release branch-23.03 and > > > > > the LTS > > > > >>>>>>>>> branch-22.03. > > > > >>>>>>>>> The code base has changed a lot so was mostly manually > > > > > backported > > > > >>>>>>> instead > > > > >>>>>>>>> of cherry-pick. > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> Thanks for that but I think we shouldn't skip stable > > branches in > > > > >>>>>>>> between. We likely still have users on those that won't > > upgrade > > > > > to the > > > > >>>>>>>> latest 23.03 but instead to a new z-stream release (when that > > > > > happens) > > > > >>>>>>>> of the stable branch. > > > > >>>>>>>> > > > > >>>>>>>> I cherry picked your backports and made sure the tests > > pass and > > > > > then > > > > >>>>>>>> pushed them to branches 22.12, 22.09 and 22.06. > > > > >>>>>>> > > > > >>>>>>> Thanks Dumitru for helping backporting. I was uncertain about > > > > > whether we > > > > >>>>>>> should continue backporting without skipping branches, as our > > > > > growing > > > > >>>>>>> number of released branches makes this approach increasingly > > > > > difficult > > > > >>>>>>> to sustain. > > > > >>>>>>> > > > > >>>>>>> The release process document > > > > >>>>>>> > > (https://docs.ovn.org/en/latest/internals/release-process.html > > <https://docs.ovn.org/en/latest/internals/release-process.html> < > > > > > https://docs.ovn.org/en/latest/internals/release-process.html > > <https://docs.ovn.org/en/latest/internals/release-process.html>> > > > > >>>>>>> > > <https://docs.ovn.org/en/latest/internals/release-process.html > > <https://docs.ovn.org/en/latest/internals/release-process.html> < > > > > > https://docs.ovn.org/en/latest/internals/release-process.html > > <https://docs.ovn.org/en/latest/internals/release-process.html>>>) doesn't > > > > >>>>>>> specify how long we should maintain non-LTS branches. My > > > > > understanding > > > > >>>>>>> was that we should primarily maintain LTS and the latest > > releases > > > > > for > > > > >>>>>>> regular bug fixes, while critical/security fixes would be > > applied > > > > > to all > > > > >>>>>>> branches. It appears we have different interpretations of this > > > > > process. > > > > >>>>>>> While it's beneficial to backport to all branches, I > > thought this > > > > > was > > > > >>>>>>> more of a convenience when there weren't too many branches to > > > > > manage. In > > > > >>>>>>> the worst case, a patch may require manual backporting to each > > > > > branch if > > > > >>>>>>> new features make the codebase in each branch unique. I also > > > > > question > > > > >>>>>>> what differentiates an LTS branch if we're backporting to all > > > > > branches > > > > >>>>>>> anyway. My assumption was that users opting for a non-LTS > > branch > > > > > should > > > > >>>>>>> be prepared to upgrade to newer releases, given the > > "short-term" > > > > >>>>>>> maintenance of these branches. Is my understanding incorrect? > > > > >>>>>>> > > > > >>>>>> Hi Han, > > > > >>>>>> > > > > >>>>>> I changed the subject of the email to better reflect the new > > > > > discussion. > > > > >>>>>> > > > > >>>>>> There's this statement in the release-process document: "LTS > > > > > releases > > > > >>>>>> will receive bug fixes until the point that another LTS is > > > > > released. At > > > > >>>>>> that point, the old LTS will receive an additional year of > > > > > critical and > > > > >>>>>> security fixes." > > > > >>>>>> > > > > >>>>>> Which further differentiates an LTS release from a non-LTS > > release. > > > > >>> > > > > >>> If you are saying "the old LTS will receive an additional year of > > > > > critical and security fixes" is the different part, I agree. But > > still, in > > > > > this case, for critical and security fixes, when applied to the > > LTS for the > > > > > *additional* year, do you agree that we will skip non-LTS > > releases? If not > > > > > skipping, then it makes no difference. > > > > >>> > > > > >>>>>> > > > > >>>>>>> Perhaps we should discuss and formalize our backporting > > practice > > > > > going > > > > >>>>>>> forward, as the number of branches will only continue to > > grow in > > > > > the > > > > >>>>>>> coming months and years. cc @Mark Michelson > > > > >>>>>>> <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> @Numan > > > > > Siddique <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> > > > > >>>>>>> > > > > >>>>>> > > > > >>>>>> Checking the OVS/OVN release-process documents again it's indeed > > > > > not > > > > >>>>>> explicitly specified that non-LTS stable release branches are > > > > > supposed > > > > >>>>>> to get all bug fixes. But the OVS document has this addition: > > > > >>>>>> > > > > >>>>>> "While branches other than LTS and the latest release are not > > > > > formally > > > > >>>>>> maintained, the OVS project usually provides stable releases for > > > > > these > > > > >>>>>> branches for at least 2 years, i.e. stable releases are provided > > > > > for the > > > > >>>>>> last 4 release branches. However, these branches may not > > include > > > > > all > > > > >>>>>> the fixes that LTS has in case backporting is not > > straightforward > > > > > and > > > > >>>>>> developers are not willing to spend their time on that (this > > mostly > > > > >>>>>> affects branches that are older than the LTS, because > > backporting > > > > > to LTS > > > > >>>>>> implies backporting to all intermediate branches)." > > > > >>>>>> > > > > >>>>>> This last part really makes me think that we SHOULD backport > > to all > > > > >>>>>> intermediate branches: "... because backporting to LTS > > > > >>>>>> implies backporting to all intermediate branches". > > > > >>>>>> > > > > >>>> > > > > >>>> I agree with this. > > > > >>>> > > > > >>> If we consistently backport to all intermediate releases, the LTS > > > > > release appears to function more as a reference point in the > > series of > > > > > releases, rather than a version with additional maintenance > > support. This > > > > > is ok if we all agree to it. > > > > >>> > > > > >>>>>> Maybe we should make this rule (or some other backport > > strategy we > > > > >>>>>> decide to use from this point on) explicit indeed. > > > > >> > > > > >> First of all, the backporting process is already documented in OVN: > > > > >> > > > > > > > https://docs.ovn.org/en/latest/internals/contributing/backporting-patches.html < https://docs.ovn.org/en/latest/internals/contributing/backporting-patches.html > > > > > >> > > > > >> "The maintainer first applies the patch to master, then > > backports the > > > > >> patch to each older affected tree, as far back as it goes or at > > least > > > > >> to all currently supported branches. This is usually each > > branch back > > > > >> to the most recent LTS release branch." > > > > > > > > > > Thanks for the pointer, and sorry for missing it earlier. Probably we > > > > > should combine these two documents for OVN because both mentioned > > some part > > > > > of backport processes, and remove the irrelevant part (like > > > > > userspace/kernel related discussion for OVS). > > > > > > > > > > > > > I think this still needs clarification: "... then backports the > > patch to > > > > each older affected tree, as far back as it goes or at least to all > > > > currently supported branches". We still don't mention that all > > branches > > > > between main/master and the LTS are currently supported. I always > > > > assumed they are but it's obviously not clear since we're having this > > > > conversation. > > > > > > > > >> > > > > >> No-one updated the documentation in general since migration from > > master > > > > >> to main, but that's not the point. > > > > >> > > > > >> Backports are done for each intermediate branch to avoid regressions > > > > >> on upgrades, which may be very bad for users and not a great > > thing to > > > > >> have in general (also makes it harder to bisect in which branch the > > > > >> issue was introduced). > > > > > > > > +1 > > > > > > > > That's exactly the reason why I added the missing backports of the > > > > original bug fix. > > > > > > > > >> > > > > >> This should remain this way, I think, unless OVN decides that > > intermediate > > > > >> branches are not supported at all and never get any backports. > > i.e. the > > > > >> release schema similar to DPDK, where all xx.11 releases are > > supported for > > > > >> a few years, but intermediate releases are fully abandoned the > > moment the > > > > >> next one is out. > > > > >> > > > > >> > > > > >> "the LTS release appears to function more as a reference point > > in the > > > > >> series of releases, rather than a version with additional > > maintenance > > > > >> support" > > > > >> > > > > >> The reason why it appears this way is a bit orthogonal. The point > > > > >> of extra maintenance support is preparing of official minor > > releases. > > > > >> Minor releases on LTS and the latest stable should be much more > > > > >> frequent than on other branches that are not fully supported. > > > > >> For example, I'm trying to release a new minor version of OVS > > LTS and > > > > >> the latest stable every ~2 months, while other "unsupported" > > branches > > > > >> may receive minor releases once in 4-6 months. > > > > >> > > > > >> The problem with OVN project that makes us ask these questions > > is that > > > > >> OVN doesn't really prepare any minor releases any regularly at all, > > > > >> while it should. OVN 22.03 LTS received 2 stables releases > > since release > > > > >> in March 2022 after people asking for them. In comparison, OVS > > 2.17 LTS > > > > >> received 6 new minor versions within approximately same time frame. > > > > >> > > > > > > > > It does seem like we should have more predictable minor release > > schedules. > > > > > > > > >> So, the original point of a long term support was to keep > > backporting > > > > >> fixes all the way down and prepare minor releases more frequently on > > > > >> this one particular branch. And backporting all the way down, > > according > > > > >> to the current documentation, means backporting to all intermediate > > > > >> branches. > > > > >> > > > > > Thanks for the explanation. Now I see the point of a LTS release. > > Releasing > > > > > minor versions more frequently does make it more useful. > > > > > So according to the current documented process we should not skip > > branches > > > > > even for the "additional" year of a LTS, although the DPDK > > approach seems > > > > > more attractive to me considering the amount of release branches > > between > > > > > two LTS in OVN. > > > > > > > > > > > > > If we're careful in backporting only "very important fixes" (as Ilya > > > > called them below) to the older LTS then even if we need to backport to > > > > all intermediate branches the maintainer effort should still be > > reasonable. > > > > > > > This may sound reasonable but I would not count on it. What we can do > > is try our best to ensure quality and avoid bugs as much as we can, but > > beyond that there is not much we can control. > > > When bugs need to be fixed, the importance of the bug fix mostly > > depends on the bug itself (how users are impacted), not how we evaluate > > it. (We may tend to backport less bug fixes when there are more branches > > to maintain, but if we do that it would be wrong.) > > > > > > > >>>> > > > > >>>> +1 on for this. > > > > >>>> > > > > >>>> If we have these branches > > > > >>>> > > > > >>>> - Latest release (R9) > > > > >>>> - R8 > > > > >>>> - R7 > > > > >>>> - R6 (LTS) > > > > >>>> - R5 > > > > >>>> - R4 > > > > >>>> - R3 > > > > >>>> - R2 (LTS) > > > > >>>> - R1 > > > > >>>> > > > > >>>> IMO it should be enough to backport bug fixes up to the latest LTS > > > > >>>> (which is R6) in the above example. > > > > >>> > > > > >>> This example looks not bad, but it is not how OVN would be > > released, > > > > > because for OVN "LTS releases are scheduled to be released once > > every two > > > > > years". So it will look like: > > > > >>> - Latest release (R17) > > > > >>> ... > > > > >>> - R12 > > > > >>> - R11 > > > > >>> - R10 (LTS) > > > > >>> ... > > > > >>> - R4 > > > > >>> - R3 > > > > >>> - R2 (LTS) > > > > >>> - R1 > > > > >>> > > > > >>> We will always have to backport from R17 down to R10 in this > > example (8 > > > > > branches). > > > > >>> > > > > >>>> In case we decide to backport to R2 as well, then I think we > > should > > > > >>>> not skip intermediate branches (R5, R4 and R3) > > > > >>>> and instead backport upto R2. > > > > >>> > > > > >>> I don't think we will ever want to backport to R2 in this case. > > If we > > > > > do, it would mean from R17 - R2, which includes 16 branches. > > > > >>> > > > > >>> But look at another example: > > > > >>> - Latest release (R13) > > > > >>> - R12 > > > > >>> - R11 > > > > >>> - R10 (LTS) > > > > >>> ... > > > > >>> - R4 > > > > >>> - R3 > > > > >>> - R2 (LTS) > > > > >>> - R1 > > > > >>> > > > > >>> Because the new LTS (R10) in this example is released less than > > a year, > > > > > according to the rule, we will support the older LTS (R2) for > > critical / > > > > > security bug fixes. > > > > >> > > > > >> > > > > >> Right, that's why only very important fixes should go to R2. > > > > >> i.e. these should actually be fixes for very severe issues that > > > > >> majority of users is experiencing. Fixes for some minor leaks > > > > >> or broken communication in corner cases should not be backported > > > > >> to R2. > > > > >> > > > > >> In a worst case scenario, it will be ~12 branches to backport to, > > > > >> because an LTS release will be alive for at most 3 years. > > > > >> 12 is a lot, but again, backporting that far should only happen > > > > >> in exceptional cases. > > > > >> > > > > >> In case of a single LTS branch, there should be maximum of 8 > > > > >> branches to backport to. That is a lot too, but that's what > > > > >> the current process suggests. > > > > >> > > > > >> The issue of backporting too far can be mitigated by being more > > > > >> conscious about what kind of changes are getting backported, > > > > >> how severe the issue they are trying to fix in comparison to the > > > > >> amount of work backporting and validation of stable branches > > > > >> requires. That's partially what frequent major releases are for. > > > > >> If users want to stick to stable releases instead and still want > > > > >> every minor issue to be fixed, maybe this is time to reduce the > > > > >> frequency of major releases to e.g. 3 times a year instead of 4. > > > > >> Or reduce the support time of LTS to e.g. 1.5 years of full support > > > > >> plus 0.5 years of security / critical fixes and designating every > > > > >> xx.03 release as an LTS. > > > > >> > > > > > I like this last proposal. Shall we discuss this again in OVN > > meeting? I > > > > > think the overhead of maintenance was not fully understood when > > we agreed > > > > > on this LTS cycle initially (I could be only me). > > > > > > > > > > > > > +1 > > > > > > > > Counting how many new features/news items we got per stable (.0) > > release > > > > since 22.03 (LTS) I get an average of 5 items per release. Moving to 3 > > > > releases per year instead of 4 shouldn't make a huge difference in the > > > > maintenance effort. > > > > > > > Agree that moving to 3 releases per year doesn't help much. What I > > liked is this part: > > > > > > ... reduce the support time of LTS to e.g. 1.5 years of full support > > > plus 0.5 years of security / critical fixes and designating every > > > xx.03 release as an LTS. > > > > > > > >> Also, OVN sometimes may be more conscious about constantly > > > > >> re-factoring everything. :D Sometimes it's necessary, true, but > > > > >> sometimes the same functionality can be achieved without reworking > > > > >> a ton of code just for looks of it. This may greatly reduce the > > > > >> need in manual backports. > > > > > > > > > > Strongly agree (but I also insist that my recent refactor for I-P > > should be > > > > > exempted :D) > > > > > > > > > > > > > We went through a period in which we realized some things need to be > > > > redesigned in order for the whole OVN model to scale (e.g., datapath > > > > groups, LB groups, generating LB logical flows, northd I-P). That > > > > would've been impossible without a certain amount of refactoring. What > > > > I hope is that the rate of refactoring goes down in the near future. > > > > > > Forgot to comment on this. I think we had this hope ~5 year ago, and now > > I have zero hope for this :D > > > > > > Maybe a compromise (to reduce the maintainer load) is to be more strict > > > > in extracting the refactor part in explicit commits and allowing those > > > > to be backported. > > > > > > > Backporting for refactoring itself would be a big burden and > > sometimes impossible when there are too many branches (this is also a > > conflict principle of "backport only very important fixes" discussed > > earlier). So I am not sure if this would help in practice. > > > > > > Thanks a lot for the discussion. Would love to discuss more in the > > next meeting. > > > > > > Han > > > > > > > >> > > > > >> My 2c. > > > > >> > > > > >> /me mumbles something inaudible and goes back to PTO. > > > > > > > > Enjoy and thanks for the feedback! > > > > > > > > > > > > > > Thanks again for educating me during your PTO. > > > > > Han > > > > > > > > > >> > > > > >> Best regards, Ilya Maximets. > > > > >> > > > > >> > > > > >>> I think we need to agree on whether we should skip R3 - R9. > > > > >>> > > > > >>> Thanks, > > > > >>> Han > > > > >>> > > > > >>>> > > > > >>>> +1 from to have a formal process for backports. > > > > >>>> > > > > >>>> Numan > > > > >>>> > > > > > > > > Regards, > > > > Dumitru > > > > > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
