Hi Vihang, We only have 4 tickets remaining to be backported from branch-3.1 to branch-3. It will be completed next week.
But there are a lot of new tickets that will go into release 3.2.0 on top of this. I was thinking of not cutting a release candidate now since it would mean that we only backport changes into that release candidate branch. This would again mean that if people commit only to branch-3 or the release branch, there will again be a lot of difference in these two branches when someone picks up the next release. Instead I am thinking that we should backport new changes to branch-3 and then only cut the release candidate. Please let me know your thoughts. If we agree that changes need to go into the new release candidate branch only, I am okay with that (I do not prefer it btw) Thanks, Aman. Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: vihang karajgaonkar <vihan...@apache.org> Sent: Monday, May 8, 2023 4:57:24 AM To: dev@hive.apache.org <dev@hive.apache.org> Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability Hi Aman, I know you are backporting the branch-3.1 commits to branch-3. How close are you with finishing with them. Is there anything that we can help with to get it over the finish line? I am interested to know how close are we to cutting the branch for 3.2.0? Do you think we can have a release candidate this week? Thanks, Vihang On Thu, Mar 30, 2023 at 2:18 AM Stamatis Zampetakis <zabe...@gmail.com> wrote: > Huge thanks to everyone involved it is great to see the branch-3 in stable > state. As other people mentioned let's keep it that way! > > As far as it concerns back ports please be particularly cautious with > anything that touches the metastore schema and Thrift APIs. > > Best, > Stamatis > > On Wed, Mar 29, 2023, 4:36 AM vihang karajgaonkar <vihan...@apache.org> > wrote: > > > Thanks a lot Aman for all your efforts on this. Really appreciate the > > initiative and all your hard work on this. > > > > I would like to request that all the committers should follow the merge > > process of master branch to merge PRs in branch-3. If there are any test > > failures which seem unrelated, please do not ignore them. One can run the > > flaky > > test runner > > <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fjob%2Fhive-flaky-check%2F&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NFcfXybje7zooPXkkN9UnSQIexxjKJ9wQR%2B0nrXk4e0%3D&reserved=0<http://ci.hive.apache.org/job/hive-flaky-check/>> > > to make > sure > > that test is indeed flaky. If the test is found to be flaky a > > ticket should be created to disable it. A separate ticket should be > created > > to deflake it and you can mention the original author or previous commit > > author who changed the test on that ticket to get help since they likely > > have the most context around that test. Once the flaky test is disabled > and > > we have a green CI job run, we should merge the PR. If others have any > > suggestions to improve this process please chime in. > > > > Thanks, > > Vihang > > > > On Tue, Mar 28, 2023 at 10:55 PM Aman Raj <raja...@microsoft.com.invalid > > > > wrote: > > > > > Hi community, > > > > > > This is to notify that we have a green branch-3 now. The entire effort > of > > > fixing branch-3 test cases took around 4 months and as a team we > managed > > to > > > fix 2900+ test failures on branch-3. The entire effort can be tracked > > here > > > HIVE-26836<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26836&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DXjdb%2FLcUztu6Kr2phfhuVgdP652Yvlddq1QmtTndAw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-26836>>. > > > We are > > > ready to push new features and improvements on branch-3 now. > > > > > > I really want to thank Vihang Karajgaonkar, Chris Nauroth, Lazlo Bodor, > > > Stamatis Zampetakis and Sankar Hariappan without whom this would not at > > all > > > have been possible. As a team we stuck together and participated in > > reviews > > > and actively suggested improvements which really helped in fixing some > > > major test failures. > > > > > > I would sincerely request that going further it should be made a point > to > > > merge things into branch-3 only if we have a green Jenkins pipeline. > > > > > > The next step would be to backport changes from branch-3.1 (From where > > > Hive-3.1.3 release was made) to branch-3. This would ensure that we do > > not > > > miss any specific ticket which went into Hive-3.1.3. I will take care > of > > > this. We can parallelly start pushing additional changes on branch-3. > > There > > > are approximately 25 tickets that need to be backported in this effort > > (Of > > > backporting changes from branch-3.1). I have made a note here< > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1K0U-vxLRZEs13oBzYBlVyK8dMMNthgXL5VEgzLRbeKs%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UB3VwQkRFsNlPsZYuGuzt1uQwp20L%2BmgLAD8CvopzUw%3D&reserved=0<https://docs.google.com/spreadsheets/d/1K0U-vxLRZEs13oBzYBlVyK8dMMNthgXL5VEgzLRbeKs/edit?usp=sharing> > > > > > > > > > > Again, thanks a lot to everyone who supported and participated in this > > > effort. Lets make this 3.2.0 Hive release happen!! > > > > > > Thanks, > > > Aman. > > > > > > ________________________________ > > > From: Aman Raj <raja...@microsoft.com.INVALID> > > > Sent: Monday, March 20, 2023 9:21 AM > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > Hi Vihang/community, > > > > > > Found the ticket which broke mm_all.q. This issue comes because of > > > HIVE-20182. Works in my local and on the Jenkins pipeline as well. > Link : > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhive%2Fpull%2F4127&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZRb8agmteUTddKJaBEtUC3EMd6ZibPfIu2nB9yOptk%3D&reserved=0<https://github.com/apache/hive/pull/4127> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhive%2Fpull%2F4127&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aZRb8agmteUTddKJaBEtUC3EMd6ZibPfIu2nB9yOptk%3D&reserved=0<https://github.com/apache/hive/pull/4127>> > > > Reverting this commit for > > now. > > > > > > Thanks, > > > Aman. > > > ________________________________ > > > From: Aman Raj <raja...@microsoft.com.INVALID> > > > Sent: Monday, March 20, 2023 8:28 AM > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > Sure Vihang, will look at the other ones. You can pick this up. > > > > > > Thanks, > > > Aman. > > > > > > Get Outlook for Android< > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg>>> > > > ________________________________ > > > From: vihang karajgaonkar <vihan...@apache.org> > > > Sent: Monday, March 20, 2023 7:58:48 AM > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > I think we should revert offending commits first to unblock the branch. > > We > > > can create followup tickets to determine if these fixes are blockers > for > > > 3.2 release and if yes, we should merge them the right way with a green > > > test run. Fixing forward always comes with the risk that it introduces > > new > > > test failures. > > > > > > Thanks for all your efforts on this Aman. > > > > > > I can take a look at > testBootstrapReplLoadRetryAfterFailureForPartitions > > if > > > you haven’t already started on it. > > > > > > Thanks, > > > Vihang > > > > > > On Sun, Mar 19, 2023 at 10:09 PM Aman Raj > <raja...@microsoft.com.invalid > > > > > > wrote: > > > > > > > Hi Vihang/community, > > > > > > > > Thanks a lot Vihang for working on the major test failure. This > blocked > > > > more than 35 test cases. Now we are down to the final 4 failures. I > > have > > > > analyzed some of them and here they are (Link : > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-4067%2F12%2Ftests&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Vjjsob4PG0%2FU5uDRUEf23WCCfuM622SWcBLEhuX1qKo%3D&reserved=0<http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4067/12/tests> > > > )< > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-4067%2F12%2Ftests&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Vjjsob4PG0%2FU5uDRUEf23WCCfuM622SWcBLEhuX1qKo%3D&reserved=0<http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-4067/12/tests> > > > > > > > > : > > > > > > > > 1. > > > > multi_in_clause - This was committed in HIVE-21685 without validating > > the > > > > scenario. > > > > This fails because Hive is not able to parse > > > > explain cbo > > > > select * from very_simple_table_for_in_test where name IN('g','r') > AND > > > > name IN('a','b') > > > > If we want this to work, I am able to do it in my local. We have 2 > > > options > > > > : > > > > a. Either revert HIVE-21685 since this scenario was not validated > back > > > > then before adding this test. > > > > b. This fix was present in > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UyGzmPpRJjFM1AciK0W8TMZ3Y%2BNrAtfQcyCEyrFu7uM%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-20718> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UyGzmPpRJjFM1AciK0W8TMZ3Y%2BNrAtfQcyCEyrFu7uM%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-20718>> > > > but to cherry pick > > this > > > > we need to cherry pick > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pwa7HtgXaeSKVrDgb6Bh8Fg2tBYdDKmh1O6NCA0qPrw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-17040> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pwa7HtgXaeSKVrDgb6Bh8Fg2tBYdDKmh1O6NCA0qPrw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-17040>> > > > > since HIVE-20718< > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UyGzmPpRJjFM1AciK0W8TMZ3Y%2BNrAtfQcyCEyrFu7uM%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-20718> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-20718&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UyGzmPpRJjFM1AciK0W8TMZ3Y%2BNrAtfQcyCEyrFu7uM%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-20718>>> > > > has a > > > > lot of merge conflicts with HIVE-17040< > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pwa7HtgXaeSKVrDgb6Bh8Fg2tBYdDKmh1O6NCA0qPrw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-17040> > > > ><https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-17040&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pwa7HtgXaeSKVrDgb6Bh8Fg2tBYdDKmh1O6NCA0qPrw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-17040>>. > > > > But after cherry > > > > picking these we have other failures to fix. > > > > 2. > > > > current_date_timestamp.q - This breaking change was committed in > > > > HIVE-21388 without validation. > > > > The failure is because again Hive is not able to parse > > > > explain cbo select current_timestamp() from alltypesorc > > > > The solution or revert option is same as point 1. > > > > 3. > > > > testBootstrapReplLoadRetryAfterFailureForPartitions() - This I have > not > > > > investigated till now. > > > > 4. > > > > mm_all.q - This I have not investigated till now. > > > > > > > > Thanks, > > > > Aman. > > > > ________________________________ > > > > From: vihang karajgaonkar <vihan...@apache.org> > > > > Sent: Friday, March 17, 2023 8:42 PM > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > > > Just wanted to close the loop on the TestMiniSparkOnYarnCliDriver > test > > > > failures. We will be able to re-enable most of them back on branch-3. > > The > > > > ones which were disabled are being tracked separately in a different > > > ticket > > > > < > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27146&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kMZtW1GSTclD0RjnALCDs6KoT%2F%2B13PFbNvD8psiU%2BCQ%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-27146> > > > ><https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27146&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kMZtW1GSTclD0RjnALCDs6KoT%2F%2B13PFbNvD8psiU%2BCQ%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-27146>> > > > > but they don't look like > > > > a blocker. > > > > > > > > Hi Aman, > > > > > > > > Do you know how close are we to reopening branch-3? > > > > > > > > Thanks, > > > > Vihang > > > > > > > > On Sat, Mar 4, 2023 at 7:23 PM Aman Raj > <raja...@microsoft.com.invalid > > > > > > > wrote: > > > > > > > > > Or you can cd into itests and run the command you are using. Just > > > another > > > > > way I run. > > > > > > > > > > Thanks, > > > > > Aman. > > > > > Get Outlook for Android< > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg>> > > > > > > > > > > ________________________________ > > > > > From: Aman Raj <raja...@microsoft.com> > > > > > Sent: Saturday, March 4, 2023 7:20:36 PM > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > > > > > Hi Vihang, > > > > > > > > > > Thanks a lot for working on this. Can you try using > -Pqsplits,itests. > > > > > Also, I usually give a -o option after doing a clean install. > > > > > > > > > > Thanks, > > > > > Aman. > > > > > > > > > > Get Outlook for Android< > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gG0C%2BmQpxQOV%2BCQChCGbnlMv5e9BCS7YZ5xO9y9zaw%3D&reserved=0<https://aka.ms/AAb9ysg>> > > > > > > > > > > > > > > > ________________________________ > > > > > From: vihang karajgaonkar <vihan...@apache.org> > > > > > Sent: Saturday, 4 March, 2023, 11:35 > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build stability > > > > > > > > > > [You don't often get email from vihan...@apache.org. Learn why > this > > is > > > > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > > > Just to update on the HoS test failures for > > > > TestMiniSparkOnYarnCliDriver, I > > > > > think I was finally able to resolve them (at least on local). I had > > to > > > > > revert HIVE-21044 because it was causing OOM for those tests. Also, > > in > > > > > order for these tests to work we will have to downgrade netty from > > > > > 4.1.69.Final to 4.1.51.Final. I understand that we had upgraded > netty > > > > from > > > > > 4.1.17.Final to 4.1.69.Final for CVEs but the highest netty version > > > that > > > > we > > > > > can support without breaking HoS is 4.1.51.Final. Note that > > > 4.1.51.Final > > > > > includes many of the CVEs which affected 4.1.17.Final so we are > still > > > in > > > > a > > > > > better place than branch-3.1. Unfortunately, there is no good way > to > > > make > > > > > HoS work with a higher netty version so I think we should downgrade > > the > > > > > netty version to 4.1.51.Final for now and look at more options to > > > upgrade > > > > > it 4.1.69.Final in a separate ticket. > > > > > > > > > > I still need to understand why the tests which are working for me > > > locally > > > > > don't work on the PR job. I tried running the split test classes > > using > > > > the > > > > > following command. Is that the right way to simulate builds from > the > > PR > > > > > job? Let me know if anyone has more ideas. > > > > > > > > > > mvn test > > > > > > -Dtest=org.apache.hadoop.hive.cli.split2.TestMiniSparkOnYarnCliDriver > > > > > -Pqsplits > > > > > > > > > > Thanks, > > > > > Vihang > > > > > > > > > > > > > > > On Fri, Feb 17, 2023 at 4:01 AM Stamatis Zampetakis < > > zabe...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > Thanks Aman for bringing this up and also for cleaning up after > > > others > > > > (I > > > > > > saw that you raised tickets and PRs for addressing the failures). > > > > > > > > > > > > Many thanks to Vihang as well for helping out. Regarding flaky > > tests, > > > > yes > > > > > > we should disable them as soon as we see them. > > > > > > There have been some other discussions on how to approach flaky > > tests > > > > the > > > > > > more recent I could find is here [1]. > > > > > > > > > > > > Best, > > > > > > Stamatis > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Flv3bhlfoq8fwd9dwyjf7g4nx32wtrygv&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ym1WBuGKRt%2FuTHitzCwR%2FzckJGLPO1XinHZm6kt%2BgKk%3D&reserved=0<https://lists.apache.org/thread/lv3bhlfoq8fwd9dwyjf7g4nx32wtrygv> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Flv3bhlfoq8fwd9dwyjf7g4nx32wtrygv&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988697872822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ym1WBuGKRt%2FuTHitzCwR%2FzckJGLPO1XinHZm6kt%2BgKk%3D&reserved=0<https://lists.apache.org/thread/lv3bhlfoq8fwd9dwyjf7g4nx32wtrygv>> > > > > > > > > > > > > On Fri, Feb 17, 2023 at 4:37 AM Aman Raj > > > <raja...@microsoft.com.invalid > > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi team, > > > > > > > > > > > > > > Thanks Vihang for looking into this. I have commented on the > JIRA > > > you > > > > > > > created. > > > > > > > > > > > > > > Just to bring everyone's notice, I have seen that there has > been > > a > > > > > couple > > > > > > > of pushes to branch-3, which has lead to 5 more new test > > failures. > > > > The > > > > > > test > > > > > > > failures are in orc_merge1, orc_merge2, orc_merge3, orc_merge4 > > and > > > > > > > orc_merge10. These tests did not use to fail before. I would > > > > sincerely > > > > > > urge > > > > > > > the community to raise a PR against branch-3, so that the > Jenkins > > > > > > pipeline > > > > > > > can run and then only merge things to branch-3. We had 2900+ > > > failures > > > > > > when > > > > > > > we started 2 months back and now having brought it down to less > > > than > > > > > 15, > > > > > > > new failures again has pushed us back in this effort. > > > > > > > > > > > > > > I would like to thank everyone who has participated in this > > effort > > > > and > > > > > > > made it possible till this stage. Also, if the contributors can > > > take > > > > > > > ownership of these new test case failures and fix them, it will > > be > > > of > > > > > > great > > > > > > > help. > > > > > > > > > > > > > > Thanks, > > > > > > > Aman. > > > > > > > ________________________________ > > > > > > > From: vihang karajgaonkar <vihan...@apache.org> > > > > > > > Sent: Friday, February 17, 2023 6:10 AM > > > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build > > stability > > > > > > > > > > > > > > [You don't often get email from vihan...@apache.org. Learn why > > > this > > > > is > > > > > > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > > > > > > > Hi Aman, > > > > > > > > > > > > > > I created > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6vJLQ2iZLyEfQv%2By8C8se%2F%2BABdUmnOlnmCSOfTEByWg%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-27087> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6vJLQ2iZLyEfQv%2By8C8se%2F%2BABdUmnOlnmCSOfTEByWg%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-27087>> > > > > > > > to look into > > > > > > > TestMiniSparkOnYarnCliDriver failures. I have a working theory > of > > > > what > > > > > > > might be going on there. I am still investigating what is the > > right > > > > way > > > > > > to > > > > > > > fix it though. > > > > > > > > > > > > > > Thanks, > > > > > > > Vihang > > > > > > > > > > > > > > On Fri, Feb 10, 2023 at 10:26 AM Aman Raj > > > > > <raja...@microsoft.com.invalid > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Vihang, > > > > > > > > > > > > > > > > Yes the tests are failing locally as well with the same > issue. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Aman. > > > > > > > > > > > > > > > > Get Outlook for Android< > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FU380OPfIwF6BHqGG85T8XQlWjG6QmDuQfwLytzoE6w%3D&reserved=0<https://aka.ms/AAb9ysg> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FU380OPfIwF6BHqGG85T8XQlWjG6QmDuQfwLytzoE6w%3D&reserved=0<https://aka.ms/AAb9ysg>> > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > From: Vihang Karajgaonkar > > > > <vihang.karajgaon...@databricks.com.INVALID > > > > > > > > > > > > > > Sent: Friday, February 10, 2023 11:22:15 PM > > > > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > > > > Subject: Re: [EXTERNAL] Re: Branch-3 backports and build > > > stability > > > > > > > > > > > > > > > > [You don't often get email from > > > > > > > vihang.karajgaon...@databricks.com.invalid. > > > > > > > > Learn why this is important at > > > > > > > > https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > > > > > > > > > Thanks a lot Stamatis for starting this thread. I really > > > appreciate > > > > > all > > > > > > > the > > > > > > > > efforts to stabilize branch-3 to get it to a releasable state > > > and I > > > > > > agree > > > > > > > > that we should get it to a green state before opening it for > > PRs > > > > not > > > > > > > > related to test failures. I can help with the effort as well. > > > > > > > > > > > > > > > > If we want to get the branch back to green state soon, have > we > > > > > > considered > > > > > > > > disabling the tests which are clearly flaky? (e.g pass on > some > > > > builds > > > > > > and > > > > > > > > fail on the other build with no new code changes). If we > don't > > do > > > > > that, > > > > > > > we > > > > > > > > will keep playing whack a mole with those tests. I propose > for > > > such > > > > > > tests > > > > > > > > we should disable them and create tickets to unflake them > > > > separately. > > > > > > > This > > > > > > > > will help us get back to a green state faster. > > > > > > > > > > > > > > > > Hi Aman, > > > > > > > > For TestMiniSparkOnYarnCliDriver failures, you probably > should > > > also > > > > > > look > > > > > > > > into the spark driver/application logs and see if there are > > > > > > > infrastructure > > > > > > > > errors (e.g OOMs). Are these tests failing when you run > > locally? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Vihang > > > > > > > > > > > > > > > > On Tue, Feb 7, 2023 at 10:05 PM Aman Raj > > > > > <raja...@microsoft.com.invalid > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1, > > > > > > > > > Thanks Stamatis and Lazlo for helping in the test case > fixes > > > till > > > > > > now. > > > > > > > > > > > > > > > > > > Team, > > > > > > > > > I need help in fixing the following tests in Hive. I have > > tried > > > > > > > different > > > > > > > > > approaches but no luck till now. > > > > > > > > > I am facing some issues in fixing the following tests : > > > > > > > > > org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver > > > > > > > > > > > > > > > > > > Issue : > > > > > > > > > PREHOOK: Input: default@src > > > > > > > > > PREHOOK: Output: default@src > > > > > > > > > Failed to monitor Job[-1] with exception > > > > > > > > > 'java.lang.IllegalStateException(Connection to remote Spark > > > > driver > > > > > > was > > > > > > > > > lost)' Last known state = SENT > > > > > > > > > Failed to execute spark task, with exception > > > > > > > > > 'java.lang.IllegalStateException(RPC channel is closed.)' > > > > > > > > > FAILED: Execution Error, return code 1 from > > > > > > > > > org.apache.hadoop.hive.ql.exec.spark.SparkTask. RPC channel > > is > > > > > > closed. > > > > > > > > > > > > > > > > > > History : > > > > > > > > > Initially the tests had failed with errors which I fixed in > > the > > > > > > > following > > > > > > > > > task : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=33sxNakmV8J0QuhB4tPbe2qBtO%2B6pe9ViImnh3cWN%2Fw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-26940> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=33sxNakmV8J0QuhB4tPbe2qBtO%2B6pe9ViImnh3cWN%2Fw%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-26940>> > > > > > > > > > > > > > > > > > > Does anyone know what the issue is here ? There are 6-7 > > > failures > > > > > > > because > > > > > > > > > of this test case. Link to the failed test cases for the > > > > > stacktrace : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-3949%2F2%2Ftests%2F&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6YsAKbLGSDWISDgOIS8kDmsaRGQsLuMCa1Z9Mg%2FNb68%3D&reserved=0<http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3949/2/tests/> > > > < > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-3949%2F2%2Ftests%2F&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6YsAKbLGSDWISDgOIS8kDmsaRGQsLuMCa1Z9Mg%2FNb68%3D&reserved=0<http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3949/2/tests/> > > > > > > > > > > > > > Thanks, > > > > > > > > > Aman. > > > > > > > > > > > > > > > > > > ________________________________ > > > > > > > > > From: László Bodor <bodorlaszlo0...@gmail.com> > > > > > > > > > Sent: Tuesday, February 7, 2023 4:46 PM > > > > > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > > > > > Subject: [EXTERNAL] Re: Branch-3 backports and build > > stability > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > also, if I merged something that I thought was for test > > > stability > > > > > > (but > > > > > > > > > instead it was a feature), excuse me :) > > > > > > > > > for reference, the whole green test initiative is tracked > > under > > > > > this > > > > > > > > > umbrella: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26836&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5yJtrMl80d%2BKYqlLuoG4vLalBHflWqsJN8zpua4tplY%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-26836> > > > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26836&data=05%7C01%7Crajaman%40microsoft.com%7C9f071a96a03e491757e908db4f52a621%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638190988698028895%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5yJtrMl80d%2BKYqlLuoG4vLalBHflWqsJN8zpua4tplY%3D&reserved=0<https://issues.apache.org/jira/browse/HIVE-26836>> > > > > > > > > > > > > > > > > > > Stamatis Zampetakis <zabe...@gmail.com> ezt írta (időpont: > > > 2023. > > > > > > febr. > > > > > > > > 7., > > > > > > > > > K, 12:09): > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > The build in branch-3 is not yet green; there are ~25 > test > > > > > > failures. > > > > > > > It > > > > > > > > > is > > > > > > > > > > a common practice that we shouldn't push changes on top > of > > a > > > > > broken > > > > > > > > build > > > > > > > > > > unless they are addressing test failures. > > > > > > > > > > > > > > > > > > > > Some people (mainly Aman Raj, Chris Nauroth, and Laszlo > > > Bodor) > > > > > are > > > > > > > > > working > > > > > > > > > > hard to stabilize the build for quite some time now. If > you > > > > want > > > > > to > > > > > > > > help > > > > > > > > > > out then start by reviewing, merging, and fixing things > > > around > > > > > test > > > > > > > > > > failures. > > > > > > > > > > > > > > > > > > > > It's not yet the time to bring new features, upgrades, > > bugs, > > > > > etc., > > > > > > in > > > > > > > > > > branch-3. I would encourage committers to not approve > such > > > > > changes > > > > > > > > till > > > > > > > > > we > > > > > > > > > > get back to a stable branch. > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > Stamatis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
