Folks, Let’s take a deep breath here, everyone is aiming for a good release.
With 4.6 we are trying a new way of creating the release, it may not be the best, but I think we need to stick with the current process and release. We can then have a post-mortem and see what worked and what did not work. David and co have been working on hardware setup to finally get a CI running under ASF. This will help. For now, we need to focus on the release. Reverting a 6 months old feature is in my view a no go, and I don’t have the feeling that it is the actual problem. I suggest we all jump on a hangout tomorrow and try to figure this out. * Wilder and co have done great work to test pending PR * We have a set of blockers * We have two great RMs. The one thing that is very troubling to me are the statements that “this used to work 3 weeks ago”. I don’t understand how things can be broken now if only the RMs can merge to master. I will have a look tomorrow. So please, let’s take a deep breath, only RMs can touch master, and let’s work it out tomorrow. Cheers, -sebastien > On Sep 24, 2015, at 10:27 PM, Wilder Rodrigues > <wrodrig...@schubergphilis.com> wrote: > > Thinking about being disrespectful when one doesn’t read the emails, or does > but filters parts of the message, and keeps storming about unclear things. > > Yes, time to move on. We have to get a cloud running. > > Cheers, > Wilder > >> On 24 Sep 2015, at 20:29, Raja Pullela <raja.pull...@citrix.com> wrote: >> >> this is very disrespectful... Sorry to say that you don't understand the >> complexity and impact of this.. Let's not discuss this over an email and >> agree to disagree with each other... move on! >> >>> On Sep 24, 2015, at 10:20 PM, Wilder Rodrigues >>> <wrodrig...@schubergphilis.com> wrote: >>> >>> Raja, >>> >>> Do you actually know the amount of blockers we have and how many are VR >>> related? Because I have seen emails from Rajani around concerning the >>> blockers and I don’t see many. So, yes, I really do think your approach is >>> non-sense. >>> >>> I mentioned it before, about 1 week ago, but I think you just ignored the >>> content of the email. We have 7 blockers, from which 4 are VR related but >>> probably only 2 are related to the refactor of the router side (python >>> code). You created 2 of the blockers. So, I think would be better to focus >>> on fixing them other than making a storm out of it. >>> >>> You can see the list here: >>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12326765 >>> >>> The java part of the router refactor was released on 4.5, quite some time >>> ago. So, please have a look at git log before mentioned the refactor as a >>> whole. >>> >>> Another thing is: master is unstable - not because VR changes - and nobody >>> could tests the PRs that should fix the VR issues. When we suggested to >>> stabilise Master, people kept pushing features through PRs thinking that it >>> would help - even after we said only BLOCKER issues would be merged. >>> >>> So, please stop this storm around the VR because we are trying to work. >>> >>> Cheers, >>> Wilder >>> >>> >>> On 24 Sep 2015, at 17:21, Raja Pullela >>> <raja.pull...@citrix.com<mailto:raja.pull...@citrix.com>> wrote: >>> >>> @wilder, Not sure why you would think it as a nonsense approach? sure, you >>> realize amount of code churn and blockers we are dealing with when 4.6 is >>> ready to go out. >>> >>> Agreed, the refactoring happened several months ago and we could have taken >>> a closer look then- the recent blockers filed have uncovered areas where >>> in the implementation didn't exist. I will post the bug details around >>> this. >>> >>> Obviously, the refactoring changes missed multiple critical test scenarios >>> and will take substantial time to test/stabilize. >>> >>> The BVTs are coming good for basic zone and Adv zone there are still a >>> number of failures and it will take us good time to get those fixed. >>> >>> Besides the BVTs, regression tests are in a very bad shape. Hope to get to >>> these starting next week. >>> >>> Please see my latest bvt report, I will post in another 2 hrs, waiting for >>> a new run to complete. >>> >>> On Sep 24, 2015, at 7:00 PM, sebgoa >>> <run...@gmail.com<mailto:run...@gmail.com>> wrote: >>> >>> >>> On Sep 24, 2015, at 3:17 PM, Remi Bergsma >>> <rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote: >>> >>> Are you serious? You consider to revert a PR that was merged over 6 months >>> ago? And expect it to become more stable? >>> >>> I have not followed all the latest development, but if we are talking about >>> the VR refactoring, indeed it happened several months back. Reverting it >>> now does not seem like a good idea. >>> >>> I am probably missed a beat here, but the latest BVT results sent by Raja >>> showed XS tests almost at 100%, there were only some issues with KVM. >>> >>> The problem, in MHO, is not that we find bugs that we consider blockers. >>> The problem is we are unable to resolve them effectively because master is >>> unstable. There currently isn’t a single PR that solves it, hence there is >>> no way to test PRs. This is because we have many PRs open and they were all >>> branched off of a master that doesn’t work. I simply can't test proposed >>> PRs. >>> >>> This problem occurred about 3 weeks ago, because before that master worked >>> and we could solve issues and merge PRs. I’m not saying it was bug-free, >>> but at least we could work on stabilising it. Most likely, we accepted a >>> “fix” that made things worse. Probably even multiple of them. >>> >>> Master seemed stable and PR where being merged towards 4.6 with success (it >>> seemed), so indeed if we have issues now of stability, we should identify >>> what caused it >>> >>> To get out of this, I think we need to combine a few PRs that make it >>> stable. I’ll have a look today with Wilder and Funs to see if what fixes we >>> need to combine to make it work again. O >>> nce we merge it and master actually works again, we can rebase any open PR >>> with current master and work from there. >>> >>> Potentially, if you identify the commit or commits that brought the >>> instability you could revert to that point and play forward PRs that did >>> not render master unstable. >>> >>> Thanks for looking into it. >>> >>> -seb >>> >>> Regards, >>> Remi >>> >>> >>> >>> >>> On 24/09/15 14:00, "Ramanath Katru" >>> <ramanath.ka...@citrix.com<mailto:ramanath.ka...@citrix.com>> wrote: >>> >>> My vote is for the approach no.1 - to backout completely. Most of VR >>> functionalities are broken and are in a mess to say the least. It >>> definitely will take some time and effort from several folks to get it to a >>> stable state. >>> >>> Ram Katru >>> >>> >>> -----Original Message----- >>> From: Raja Pullela [mailto:raja.pull...@citrix.com] >>> Sent: Thursday, September 24, 2015 2:06 PM >>> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> >>> Subject: VR refactoring, concerns and a way out ? >>> >>> Hi, >>> >>> >>> >>> I understand a concern on the VR changes was raised earlier. My apologies >>> to restart this thread again. >>> >>> >>> >>> However, my last conversation with Jayapal, who has fixed/have been fixing >>> lot of VR issues, about the VR issues and he is pretty concerned about the >>> refactoring that has happened. I have had the same concern for sometime >>> now (VR issues have been on the list of issues to be looked into for at >>> least 4+ weeks) and wanted to see a good solution for this- with VR being >>> very fundamental to the system. >>> >>> >>> >>> Couple of solutions/proposals – >>> >>> 1) Back out the VR changes – Pros: VR has been stable for some time >>> and it is working well. >>> >>> 2) Continue to fix/stability VR changes - Concerns: is the unknowns, >>> what we will find out and how long this will take to stabilize the VR >>> functionality. >>> >>> >>> >>> Please chime in if you have any thoughts or concerns around this, >>> >>> >>> >>> best, >>> >>> Raja >>> >>> >
