Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Igor Marnat]

Dmitry, Aleksandra,
thank you for help and support!

Regards,
Igor Marnat

On Fri, Mar 4, 2016 at 1:20 AM, Aleksandra Fedorova 
wrote:

> As we agreed, we have switched ISO builds to latest CentOS 7.2 snapshots.
>
> You can see now that each ISO build (see for ex. [1]) produces several
> *_id.txt artifacts.
> Note that centos_mirror_id.txt points to CentOS snapshot at
> http://mirror.fuel-infra.org/pkgs/
>
> BVT test is stable, see [2], and nightly system tests results are
> basically the same as they were before.
>
>
> [1] https://ci.fuel-infra.org/job/9.0-community.all/3868/
> [2]
> https://ci.fuel-infra.org/view/ISO/job/9.0.fuel_community.ubuntu.bvt_2/
>
> On Thu, Mar 3, 2016 at 3:46 AM, Dmitry Borodaenko
>  wrote:
> > Thanks for the detailed explanation, very helpful!
> >
> > Considering that this change is atomic and easily revertable, lets
> > proceed with the change, the sooner we do that the more time we'll have
> > to confirm that there is no impact and revert if necessary.
> >
> > --
> > Dmitry Borodaenko
> >
> > On Thu, Mar 03, 2016 at 03:40:22AM +0300, Aleksandra Fedorova wrote:
> >> Hi,
> >>
> >> let me add some details about the change:
> >>
> >> 1) There are two repositories used to build Fuel ISO: base OS
> >> repository [1], and mos repository [2], where we put Fuel dependencies
> >> and packages we rebuild due to certain version requirements.
> >>
> >> The CentOS 7.2 feature is related to the upstream repo only. Packages
> >> like RabbitMQ, MCollective, Puppet, MySQL and PostgreSQL live in mos
> >> repository, which has higher priority then upstream.
> >>
> >> I think we need to setup a separate discussion about our policy
> >> regarding these packages, but for now they are fixed and won't be
> >> updated by CentOS 7.2 switch.
> >>
> >> 2) This change doesn't affect Fuel codebase.
> >>
> >> The upstream mirror we use for ISO build is controlled by environment
> >> variable which is set via Jenkins [3] and can be changed anytime.
> >>
> >> As we have daily snapshots of CentOS repository available at [4], in
> >> case of regression in upstream we can pin our builds to stable
> >> snapshot and work on the issue without blocking the main development
> >> flow.
> >
> > Please make sure that the current snapshot of CentOS 7.1 is not rotated
> > away so that we don't loose the point we can revert to.
> >
> >> 3) The "improve snapshotting" work item which is at the moment in
> >> progress, will prevent any possibility to "accidentally" migrate to
> >> CentOS 7.3, when it becomes available.
> >> Thus the only changes which we can fetch from upstream are changes
> >> which are published to updates/ component of CentOS 7.2 repo.
> >>
> >>
> >> As latest BVT on master is green
> >>https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/69/
> >> I think we should proceed with Jenkins reconfiguration [5] and switch
> >> to latest snapshots by default.
> >>
> >> [1] currently http://vault.centos.org/7.1.1503/
> >> [2]
> http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7-fuel/os/x86_64/
> >> [3]
> https://github.com/fuel-infra/jenkins-jobs/blob/76b5cdf1828b7db1957f7967180d20be099b0c63/common/scripts/all.sh#L84
> >> [4] http://mirror.fuel-infra.org/pkgs/
> >> [5] https://review.fuel-infra.org/#/c/17712/
> >>
> >> On Wed, Mar 2, 2016 at 9:22 PM, Mike Scherbakov
> >>  wrote:
> >> > It is not just about BVT. I'd suggest to monitor situation overall,
> >> > including failures of system tests [1]. If we see regressions there,
> or some
> >> > test cases will start flapping (what is even worse), then we'd have to
> >> > revert back to CentOS 7.1.
> >> >
> >> > [1] https://github.com/openstack/fuel-qa
> >> >
> >> > On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko <
> dborodae...@mirantis.com>
> >> > wrote:
> >> >>
> >> >> I agree with Mike's concerns, and propose to make these limitations
> (4
> >> >> weeks before FF for OS upgrades, 2 weeks for upgrades of key
> >> >> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
> >> >> anything else?) official for 10.0/Newton.
> >> >>
> >> >> For 9.0/Mitaka, it is too late to impose them, so we just have to be
> >> >> very careful and conservative with this upgrade. First of all, we
> need
> >> >> to have a green BVT before and after switching to the CentOS 7.2 repo
> >> >> snapshot, so while I approved the spec, we can't move forward with
> this
> >> >> until BVT is green again, and right now it's red:
> >> >>
> >> >> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
> >> >>
> >> >> If we get it back to green but it becomes red after the upgrade, you
> >> >> must switch back to CentOS 7.1 *immediately*. If you are able to
> stick
> >> >> to this plan, there is still time to complete the transition today
> >> >> without requiring an FFE.
> >> >>
> >> >> --
> >> >> Dmitry Borodaenko
> >> >>
> >> >>
> >> >> On Wed, Mar 02, 2016 at 05:53:53PM +, 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Aleksandra Fedorova]

As we agreed, we have switched ISO builds to latest CentOS 7.2 snapshots.

You can see now that each ISO build (see for ex. [1]) produces several
*_id.txt artifacts.
Note that centos_mirror_id.txt points to CentOS snapshot at
http://mirror.fuel-infra.org/pkgs/

BVT test is stable, see [2], and nightly system tests results are
basically the same as they were before.


[1] https://ci.fuel-infra.org/job/9.0-community.all/3868/
[2]
https://ci.fuel-infra.org/view/ISO/job/9.0.fuel_community.ubuntu.bvt_2/

On Thu, Mar 3, 2016 at 3:46 AM, Dmitry Borodaenko
 wrote:
> Thanks for the detailed explanation, very helpful!
>
> Considering that this change is atomic and easily revertable, lets
> proceed with the change, the sooner we do that the more time we'll have
> to confirm that there is no impact and revert if necessary.
>
> --
> Dmitry Borodaenko
>
> On Thu, Mar 03, 2016 at 03:40:22AM +0300, Aleksandra Fedorova wrote:
>> Hi,
>>
>> let me add some details about the change:
>>
>> 1) There are two repositories used to build Fuel ISO: base OS
>> repository [1], and mos repository [2], where we put Fuel dependencies
>> and packages we rebuild due to certain version requirements.
>>
>> The CentOS 7.2 feature is related to the upstream repo only. Packages
>> like RabbitMQ, MCollective, Puppet, MySQL and PostgreSQL live in mos
>> repository, which has higher priority then upstream.
>>
>> I think we need to setup a separate discussion about our policy
>> regarding these packages, but for now they are fixed and won't be
>> updated by CentOS 7.2 switch.
>>
>> 2) This change doesn't affect Fuel codebase.
>>
>> The upstream mirror we use for ISO build is controlled by environment
>> variable which is set via Jenkins [3] and can be changed anytime.
>>
>> As we have daily snapshots of CentOS repository available at [4], in
>> case of regression in upstream we can pin our builds to stable
>> snapshot and work on the issue without blocking the main development
>> flow.
>
> Please make sure that the current snapshot of CentOS 7.1 is not rotated
> away so that we don't loose the point we can revert to.
>
>> 3) The "improve snapshotting" work item which is at the moment in
>> progress, will prevent any possibility to "accidentally" migrate to
>> CentOS 7.3, when it becomes available.
>> Thus the only changes which we can fetch from upstream are changes
>> which are published to updates/ component of CentOS 7.2 repo.
>>
>>
>> As latest BVT on master is green
>>https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/69/
>> I think we should proceed with Jenkins reconfiguration [5] and switch
>> to latest snapshots by default.
>>
>> [1] currently http://vault.centos.org/7.1.1503/
>> [2] 
>> http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7-fuel/os/x86_64/
>> [3] 
>> https://github.com/fuel-infra/jenkins-jobs/blob/76b5cdf1828b7db1957f7967180d20be099b0c63/common/scripts/all.sh#L84
>> [4] http://mirror.fuel-infra.org/pkgs/
>> [5] https://review.fuel-infra.org/#/c/17712/
>>
>> On Wed, Mar 2, 2016 at 9:22 PM, Mike Scherbakov
>>  wrote:
>> > It is not just about BVT. I'd suggest to monitor situation overall,
>> > including failures of system tests [1]. If we see regressions there, or 
>> > some
>> > test cases will start flapping (what is even worse), then we'd have to
>> > revert back to CentOS 7.1.
>> >
>> > [1] https://github.com/openstack/fuel-qa
>> >
>> > On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko 
>> > 
>> > wrote:
>> >>
>> >> I agree with Mike's concerns, and propose to make these limitations (4
>> >> weeks before FF for OS upgrades, 2 weeks for upgrades of key
>> >> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
>> >> anything else?) official for 10.0/Newton.
>> >>
>> >> For 9.0/Mitaka, it is too late to impose them, so we just have to be
>> >> very careful and conservative with this upgrade. First of all, we need
>> >> to have a green BVT before and after switching to the CentOS 7.2 repo
>> >> snapshot, so while I approved the spec, we can't move forward with this
>> >> until BVT is green again, and right now it's red:
>> >>
>> >> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
>> >>
>> >> If we get it back to green but it becomes red after the upgrade, you
>> >> must switch back to CentOS 7.1 *immediately*. If you are able to stick
>> >> to this plan, there is still time to complete the transition today
>> >> without requiring an FFE.
>> >>
>> >> --
>> >> Dmitry Borodaenko
>> >>
>> >>
>> >> On Wed, Mar 02, 2016 at 05:53:53PM +, Mike Scherbakov wrote:
>> >> > Formally, we can merge it today. Historically, every update of OS caused
>> >> > us
>> >> > instability for some time: from days to a couple of month.
>> >> > Taking this into account and number of other exceptions requested,
>> >> > overall
>> >> > stability of code, my opinion would be to postpone this to 10.0.
>> >> >
>> >> > Also, 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Dmitry Borodaenko]

Thanks for the detailed explanation, very helpful!

Considering that this change is atomic and easily revertable, lets
proceed with the change, the sooner we do that the more time we'll have
to confirm that there is no impact and revert if necessary.

-- 
Dmitry Borodaenko

On Thu, Mar 03, 2016 at 03:40:22AM +0300, Aleksandra Fedorova wrote:
> Hi,
> 
> let me add some details about the change:
> 
> 1) There are two repositories used to build Fuel ISO: base OS
> repository [1], and mos repository [2], where we put Fuel dependencies
> and packages we rebuild due to certain version requirements.
> 
> The CentOS 7.2 feature is related to the upstream repo only. Packages
> like RabbitMQ, MCollective, Puppet, MySQL and PostgreSQL live in mos
> repository, which has higher priority then upstream.
> 
> I think we need to setup a separate discussion about our policy
> regarding these packages, but for now they are fixed and won't be
> updated by CentOS 7.2 switch.
> 
> 2) This change doesn't affect Fuel codebase.
> 
> The upstream mirror we use for ISO build is controlled by environment
> variable which is set via Jenkins [3] and can be changed anytime.
> 
> As we have daily snapshots of CentOS repository available at [4], in
> case of regression in upstream we can pin our builds to stable
> snapshot and work on the issue without blocking the main development
> flow.

Please make sure that the current snapshot of CentOS 7.1 is not rotated
away so that we don't loose the point we can revert to.

> 3) The "improve snapshotting" work item which is at the moment in
> progress, will prevent any possibility to "accidentally" migrate to
> CentOS 7.3, when it becomes available.
> Thus the only changes which we can fetch from upstream are changes
> which are published to updates/ component of CentOS 7.2 repo.
> 
> 
> As latest BVT on master is green
>https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/69/
> I think we should proceed with Jenkins reconfiguration [5] and switch
> to latest snapshots by default.
> 
> [1] currently http://vault.centos.org/7.1.1503/
> [2] 
> http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7-fuel/os/x86_64/
> [3] 
> https://github.com/fuel-infra/jenkins-jobs/blob/76b5cdf1828b7db1957f7967180d20be099b0c63/common/scripts/all.sh#L84
> [4] http://mirror.fuel-infra.org/pkgs/
> [5] https://review.fuel-infra.org/#/c/17712/
> 
> On Wed, Mar 2, 2016 at 9:22 PM, Mike Scherbakov
>  wrote:
> > It is not just about BVT. I'd suggest to monitor situation overall,
> > including failures of system tests [1]. If we see regressions there, or some
> > test cases will start flapping (what is even worse), then we'd have to
> > revert back to CentOS 7.1.
> >
> > [1] https://github.com/openstack/fuel-qa
> >
> > On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko 
> > wrote:
> >>
> >> I agree with Mike's concerns, and propose to make these limitations (4
> >> weeks before FF for OS upgrades, 2 weeks for upgrades of key
> >> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
> >> anything else?) official for 10.0/Newton.
> >>
> >> For 9.0/Mitaka, it is too late to impose them, so we just have to be
> >> very careful and conservative with this upgrade. First of all, we need
> >> to have a green BVT before and after switching to the CentOS 7.2 repo
> >> snapshot, so while I approved the spec, we can't move forward with this
> >> until BVT is green again, and right now it's red:
> >>
> >> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
> >>
> >> If we get it back to green but it becomes red after the upgrade, you
> >> must switch back to CentOS 7.1 *immediately*. If you are able to stick
> >> to this plan, there is still time to complete the transition today
> >> without requiring an FFE.
> >>
> >> --
> >> Dmitry Borodaenko
> >>
> >>
> >> On Wed, Mar 02, 2016 at 05:53:53PM +, Mike Scherbakov wrote:
> >> > Formally, we can merge it today. Historically, every update of OS caused
> >> > us
> >> > instability for some time: from days to a couple of month.
> >> > Taking this into account and number of other exceptions requested,
> >> > overall
> >> > stability of code, my opinion would be to postpone this to 10.0.
> >> >
> >> > Also, I'd suggest to change the process, and have freeze date for all OS
> >> > updates no later than a month before official FF date. This will give us
> >> > time to stabilize, and ensure that base on which all new code is being
> >> > developed is stable when approaching FF.
> >> >
> >> > I'd also propose to have freeze for major upgrades of 3rd party packages
> >> > no
> >> > later than 2 weeks before FF, which Fuel depends heavily upon. For
> >> > instance, such will include RabbitMQ, MCollective, Puppet.
> >> >
> >> > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat  wrote:
> >> >
> >> > > Igor,
> >> > > couple of points from my side.
> >> > >
> >> > > CentOS 7.2 will be 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Aleksandra Fedorova]

Hi,

let me add some details about the change:

1) There are two repositories used to build Fuel ISO: base OS
repository [1], and mos repository [2], where we put Fuel dependencies
and packages we rebuild due to certain version requirements.

The CentOS 7.2 feature is related to the upstream repo only. Packages
like RabbitMQ, MCollective, Puppet, MySQL and PostgreSQL live in mos
repository, which has higher priority then upstream.

I think we need to setup a separate discussion about our policy
regarding these packages, but for now they are fixed and won't be
updated by CentOS 7.2 switch.

2) This change doesn't affect Fuel codebase.

The upstream mirror we use for ISO build is controlled by environment
variable which is set via Jenkins [3] and can be changed anytime.

As we have daily snapshots of CentOS repository available at [4], in
case of regression in upstream we can pin our builds to stable
snapshot and work on the issue without blocking the main development
flow.

3) The "improve snapshotting" work item which is at the moment in
progress, will prevent any possibility to "accidentally" migrate to
CentOS 7.3, when it becomes available.
Thus the only changes which we can fetch from upstream are changes
which are published to updates/ component of CentOS 7.2 repo.


As latest BVT on master is green
   https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/69/
I think we should proceed with Jenkins reconfiguration [5] and switch
to latest snapshots by default.

[1] currently http://vault.centos.org/7.1.1503/
[2] http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7-fuel/os/x86_64/
[3] 
https://github.com/fuel-infra/jenkins-jobs/blob/76b5cdf1828b7db1957f7967180d20be099b0c63/common/scripts/all.sh#L84
[4] http://mirror.fuel-infra.org/pkgs/
[5] https://review.fuel-infra.org/#/c/17712/

On Wed, Mar 2, 2016 at 9:22 PM, Mike Scherbakov
 wrote:
> It is not just about BVT. I'd suggest to monitor situation overall,
> including failures of system tests [1]. If we see regressions there, or some
> test cases will start flapping (what is even worse), then we'd have to
> revert back to CentOS 7.1.
>
> [1] https://github.com/openstack/fuel-qa
>
> On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko 
> wrote:
>>
>> I agree with Mike's concerns, and propose to make these limitations (4
>> weeks before FF for OS upgrades, 2 weeks for upgrades of key
>> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
>> anything else?) official for 10.0/Newton.
>>
>> For 9.0/Mitaka, it is too late to impose them, so we just have to be
>> very careful and conservative with this upgrade. First of all, we need
>> to have a green BVT before and after switching to the CentOS 7.2 repo
>> snapshot, so while I approved the spec, we can't move forward with this
>> until BVT is green again, and right now it's red:
>>
>> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
>>
>> If we get it back to green but it becomes red after the upgrade, you
>> must switch back to CentOS 7.1 *immediately*. If you are able to stick
>> to this plan, there is still time to complete the transition today
>> without requiring an FFE.
>>
>> --
>> Dmitry Borodaenko
>>
>>
>> On Wed, Mar 02, 2016 at 05:53:53PM +, Mike Scherbakov wrote:
>> > Formally, we can merge it today. Historically, every update of OS caused
>> > us
>> > instability for some time: from days to a couple of month.
>> > Taking this into account and number of other exceptions requested,
>> > overall
>> > stability of code, my opinion would be to postpone this to 10.0.
>> >
>> > Also, I'd suggest to change the process, and have freeze date for all OS
>> > updates no later than a month before official FF date. This will give us
>> > time to stabilize, and ensure that base on which all new code is being
>> > developed is stable when approaching FF.
>> >
>> > I'd also propose to have freeze for major upgrades of 3rd party packages
>> > no
>> > later than 2 weeks before FF, which Fuel depends heavily upon. For
>> > instance, such will include RabbitMQ, MCollective, Puppet.
>> >
>> > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat  wrote:
>> >
>> > > Igor,
>> > > couple of points from my side.
>> > >
>> > > CentOS 7.2 will be getting updates for several more months, and we
>> > > have
>> > > snapshots and all the mechanics in place to switch to the next version
>> > > when
>> > > needed.
>> > >
>> > > Speaking of getting this update into 9.0, we actually don't need FFE,
>> > > we
>> > > can merge remaining staff today. It has enough reviews, so if you add
>> > > your
>> > > +1 today, we don't need FFE.
>> > >
>> > > https://review.openstack.org/#/c/280338/
>> > > https://review.fuel-infra.org/#/c/17400/
>> > >
>> > >
>> > >
>> > > Regards,
>> > > Igor Marnat
>> > >
>> > > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin
>> > > 
>> > > wrote:
>> > >
>> > >> Igor,
>> > >>
>> > 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Mike Scherbakov]

It is not just about BVT. I'd suggest to monitor situation overall,
including failures of system tests [1]. If we see regressions there, or
some test cases will start flapping (what is even worse), then we'd have to
revert back to CentOS 7.1.

[1] https://github.com/openstack/fuel-qa

On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko 
wrote:

> I agree with Mike's concerns, and propose to make these limitations (4
> weeks before FF for OS upgrades, 2 weeks for upgrades of key
> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
> anything else?) official for 10.0/Newton.
>
> For 9.0/Mitaka, it is too late to impose them, so we just have to be
> very careful and conservative with this upgrade. First of all, we need
> to have a green BVT before and after switching to the CentOS 7.2 repo
> snapshot, so while I approved the spec, we can't move forward with this
> until BVT is green again, and right now it's red:
>
> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
>
> If we get it back to green but it becomes red after the upgrade, you
> must switch back to CentOS 7.1 *immediately*. If you are able to stick
> to this plan, there is still time to complete the transition today
> without requiring an FFE.
>
> --
> Dmitry Borodaenko
>
>
> On Wed, Mar 02, 2016 at 05:53:53PM +, Mike Scherbakov wrote:
> > Formally, we can merge it today. Historically, every update of OS caused
> us
> > instability for some time: from days to a couple of month.
> > Taking this into account and number of other exceptions requested,
> overall
> > stability of code, my opinion would be to postpone this to 10.0.
> >
> > Also, I'd suggest to change the process, and have freeze date for all OS
> > updates no later than a month before official FF date. This will give us
> > time to stabilize, and ensure that base on which all new code is being
> > developed is stable when approaching FF.
> >
> > I'd also propose to have freeze for major upgrades of 3rd party packages
> no
> > later than 2 weeks before FF, which Fuel depends heavily upon. For
> > instance, such will include RabbitMQ, MCollective, Puppet.
> >
> > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat  wrote:
> >
> > > Igor,
> > > couple of points from my side.
> > >
> > > CentOS 7.2 will be getting updates for several more months, and we have
> > > snapshots and all the mechanics in place to switch to the next version
> when
> > > needed.
> > >
> > > Speaking of getting this update into 9.0, we actually don't need FFE,
> we
> > > can merge remaining staff today. It has enough reviews, so if you add
> your
> > > +1 today, we don't need FFE.
> > >
> > > https://review.openstack.org/#/c/280338/
> > > https://review.fuel-infra.org/#/c/17400/
> > >
> > >
> > >
> > > Regards,
> > > Igor Marnat
> > >
> > > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin <
> dtesel...@mirantis.com>
> > > wrote:
> > >
> > >> Igor,
> > >>
> > >> Your statement about updates for 7.2 isn't correct - it will receive
> > >> updates,  because it's the latest release ATM. There is *no* pinning
> inside
> > >> ISO, and the only place where it was 8.0 were docker containers just
> > >> because we had to workaround some issues. But there are no docker
> > >> containers in 9.0, so that's not the case.
> > >> The proposed solution to switch to CentOS-7.2 in fact is based on
> > >> selecting the right snapshot with packages. There is no pinning in
> ISO (it
> > >> was in earlier versions of the spec but was removed).
> > >>
> > >> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky <
> ikalnit...@mirantis.com>
> > >> wrote:
> > >>
> > >>> Dmitry, Igor,
> > >>>
> > >>> > Very important thing is that CentOS 7.1 which master node is based
> now
> > >>> > don't get updates any longer.
> > >>>
> > >>> If you are using "fixed" release you must be ready that you won't get
> > >>> any updates. So with CentOS 7.2 the problem still the same.
> > >>>
> > >>> However, let's wait for Fuel PTL decision. I only shared my POV:
> > >>> that's not a critical feature, and taking into account the risks of
> > >>> regression - I'd prefer to do not accept it in 9.0.
> > >>>
> > >>> Regards,
> > >>> Igor
> > >>>
> > >>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat 
> > >>> wrote:
> > >>> > Igor,
> > >>> > please note that this is pretty much not like update of master node
> > >>> which we
> > >>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which
> team
> > >>> > tested for more than 2 months already.
> > >>> >
> > >>> > We don't expect it to require any additional efforts from core or
> qa
> > >>> team.
> > >>> >
> > >>> > Very important thing is that CentOS 7.1 which master node is based
> now
> > >>> don't
> > >>> > get updates any longer. Updates are only provided for CentOS 7.2.
> > >>> >
> > >>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
> > >>> >
> > >>> > We can do it now for more or less free, later in release 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Dmitry Borodaenko]

I agree with Mike's concerns, and propose to make these limitations (4
weeks before FF for OS upgrades, 2 weeks for upgrades of key
dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
anything else?) official for 10.0/Newton. 

For 9.0/Mitaka, it is too late to impose them, so we just have to be
very careful and conservative with this upgrade. First of all, we need
to have a green BVT before and after switching to the CentOS 7.2 repo
snapshot, so while I approved the spec, we can't move forward with this
until BVT is green again, and right now it's red:

https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/

If we get it back to green but it becomes red after the upgrade, you
must switch back to CentOS 7.1 *immediately*. If you are able to stick
to this plan, there is still time to complete the transition today
without requiring an FFE.

-- 
Dmitry Borodaenko


On Wed, Mar 02, 2016 at 05:53:53PM +, Mike Scherbakov wrote:
> Formally, we can merge it today. Historically, every update of OS caused us
> instability for some time: from days to a couple of month.
> Taking this into account and number of other exceptions requested, overall
> stability of code, my opinion would be to postpone this to 10.0.
> 
> Also, I'd suggest to change the process, and have freeze date for all OS
> updates no later than a month before official FF date. This will give us
> time to stabilize, and ensure that base on which all new code is being
> developed is stable when approaching FF.
> 
> I'd also propose to have freeze for major upgrades of 3rd party packages no
> later than 2 weeks before FF, which Fuel depends heavily upon. For
> instance, such will include RabbitMQ, MCollective, Puppet.
> 
> On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat  wrote:
> 
> > Igor,
> > couple of points from my side.
> >
> > CentOS 7.2 will be getting updates for several more months, and we have
> > snapshots and all the mechanics in place to switch to the next version when
> > needed.
> >
> > Speaking of getting this update into 9.0, we actually don't need FFE, we
> > can merge remaining staff today. It has enough reviews, so if you add your
> > +1 today, we don't need FFE.
> >
> > https://review.openstack.org/#/c/280338/
> > https://review.fuel-infra.org/#/c/17400/
> >
> >
> >
> > Regards,
> > Igor Marnat
> >
> > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin 
> > wrote:
> >
> >> Igor,
> >>
> >> Your statement about updates for 7.2 isn't correct - it will receive
> >> updates,  because it's the latest release ATM. There is *no* pinning inside
> >> ISO, and the only place where it was 8.0 were docker containers just
> >> because we had to workaround some issues. But there are no docker
> >> containers in 9.0, so that's not the case.
> >> The proposed solution to switch to CentOS-7.2 in fact is based on
> >> selecting the right snapshot with packages. There is no pinning in ISO (it
> >> was in earlier versions of the spec but was removed).
> >>
> >> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky 
> >> wrote:
> >>
> >>> Dmitry, Igor,
> >>>
> >>> > Very important thing is that CentOS 7.1 which master node is based now
> >>> > don't get updates any longer.
> >>>
> >>> If you are using "fixed" release you must be ready that you won't get
> >>> any updates. So with CentOS 7.2 the problem still the same.
> >>>
> >>> However, let's wait for Fuel PTL decision. I only shared my POV:
> >>> that's not a critical feature, and taking into account the risks of
> >>> regression - I'd prefer to do not accept it in 9.0.
> >>>
> >>> Regards,
> >>> Igor
> >>>
> >>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat 
> >>> wrote:
> >>> > Igor,
> >>> > please note that this is pretty much not like update of master node
> >>> which we
> >>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
> >>> > tested for more than 2 months already.
> >>> >
> >>> > We don't expect it to require any additional efforts from core or qa
> >>> team.
> >>> >
> >>> > Very important thing is that CentOS 7.1 which master node is based now
> >>> don't
> >>> > get updates any longer. Updates are only provided for CentOS 7.2.
> >>> >
> >>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
> >>> >
> >>> > We can do it now for more or less free, later in release cycle for
> >>> higher
> >>> > risk and QA efforts and after the release for 2x price because of
> >>> additional
> >>> > QA cycle we'll need to pass through.
> >>> >
> >>> >
> >>> >
> >>> > Regards,
> >>> > Igor Marnat
> >>> >
> >>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin <
> >>> dtesel...@mirantis.com>
> >>> > wrote:
> >>> >>
> >>> >> Hi Igor,
> >>> >>
> >>> >> Postponing this till Fuel 10 means we have to elaborate a plan to do
> >>> such
> >>> >> upgrade for Fuel 9 after the release - the underlying system will not
> >>> get
> >>> >> updated on it's own, and the security issues will 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Mike Scherbakov]

Formally, we can merge it today. Historically, every update of OS caused us
instability for some time: from days to a couple of month.
Taking this into account and number of other exceptions requested, overall
stability of code, my opinion would be to postpone this to 10.0.

Also, I'd suggest to change the process, and have freeze date for all OS
updates no later than a month before official FF date. This will give us
time to stabilize, and ensure that base on which all new code is being
developed is stable when approaching FF.

I'd also propose to have freeze for major upgrades of 3rd party packages no
later than 2 weeks before FF, which Fuel depends heavily upon. For
instance, such will include RabbitMQ, MCollective, Puppet.

On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat  wrote:

> Igor,
> couple of points from my side.
>
> CentOS 7.2 will be getting updates for several more months, and we have
> snapshots and all the mechanics in place to switch to the next version when
> needed.
>
> Speaking of getting this update into 9.0, we actually don't need FFE, we
> can merge remaining staff today. It has enough reviews, so if you add your
> +1 today, we don't need FFE.
>
> https://review.openstack.org/#/c/280338/
> https://review.fuel-infra.org/#/c/17400/
>
>
>
> Regards,
> Igor Marnat
>
> On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin 
> wrote:
>
>> Igor,
>>
>> Your statement about updates for 7.2 isn't correct - it will receive
>> updates,  because it's the latest release ATM. There is *no* pinning inside
>> ISO, and the only place where it was 8.0 were docker containers just
>> because we had to workaround some issues. But there are no docker
>> containers in 9.0, so that's not the case.
>> The proposed solution to switch to CentOS-7.2 in fact is based on
>> selecting the right snapshot with packages. There is no pinning in ISO (it
>> was in earlier versions of the spec but was removed).
>>
>> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky 
>> wrote:
>>
>>> Dmitry, Igor,
>>>
>>> > Very important thing is that CentOS 7.1 which master node is based now
>>> > don't get updates any longer.
>>>
>>> If you are using "fixed" release you must be ready that you won't get
>>> any updates. So with CentOS 7.2 the problem still the same.
>>>
>>> However, let's wait for Fuel PTL decision. I only shared my POV:
>>> that's not a critical feature, and taking into account the risks of
>>> regression - I'd prefer to do not accept it in 9.0.
>>>
>>> Regards,
>>> Igor
>>>
>>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat 
>>> wrote:
>>> > Igor,
>>> > please note that this is pretty much not like update of master node
>>> which we
>>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
>>> > tested for more than 2 months already.
>>> >
>>> > We don't expect it to require any additional efforts from core or qa
>>> team.
>>> >
>>> > Very important thing is that CentOS 7.1 which master node is based now
>>> don't
>>> > get updates any longer. Updates are only provided for CentOS 7.2.
>>> >
>>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
>>> >
>>> > We can do it now for more or less free, later in release cycle for
>>> higher
>>> > risk and QA efforts and after the release for 2x price because of
>>> additional
>>> > QA cycle we'll need to pass through.
>>> >
>>> >
>>> >
>>> > Regards,
>>> > Igor Marnat
>>> >
>>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin <
>>> dtesel...@mirantis.com>
>>> > wrote:
>>> >>
>>> >> Hi Igor,
>>> >>
>>> >> Postponing this till Fuel 10 means we have to elaborate a plan to do
>>> such
>>> >> upgrade for Fuel 9 after the release - the underlying system will not
>>> get
>>> >> updated on it's own, and the security issues will not close
>>> themselves. The
>>> >> problem here is that such upgrade of deployed master node requires a
>>> lot of
>>> >> QA work also.
>>> >>
>>> >> Since we are not going to update package we build on our own (they
>>> still
>>> >> targeted 7.1) switching master node base to CentOS-72 is not that
>>> dangerous
>>> >> as it seems.
>>> >>
>>> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky <
>>> ikalnit...@mirantis.com>
>>> >> wrote:
>>> >>>
>>> >>> Hey Dmitry,
>>> >>>
>>> >>> No offence, but I rather against that exception. We have too many
>>> >>> things to do in Mitaka, and moving to CentOS 7.2 means
>>> >>>
>>> >>> * extra effort from core team
>>> >>> * extra effort from qa team
>>> >>>
>>> >>> Moreover, it might block development by introducing unpredictable
>>> >>> regressions. Remember 8.0? So I think it'd be better to reduce risk
>>> of
>>> >>> regressions that affects so many developers by postponing CentOS 7.2
>>> >>> till Fuel 10.
>>> >>>
>>> >>> Thanks,
>>> >>> Igor
>>> >>>
>>> >>>
>>> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin <
>>> dtesel...@mirantis.com>
>>> >>> wrote:
>>> >>> > I'd like to ask for a feature freeze exception 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Igor Marnat]

Igor,
couple of points from my side.

CentOS 7.2 will be getting updates for several more months, and we have
snapshots and all the mechanics in place to switch to the next version when
needed.

Speaking of getting this update into 9.0, we actually don't need FFE, we
can merge remaining staff today. It has enough reviews, so if you add your
+1 today, we don't need FFE.

https://review.openstack.org/#/c/280338/
https://review.fuel-infra.org/#/c/17400/



Regards,
Igor Marnat

On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin 
wrote:

> Igor,
>
> Your statement about updates for 7.2 isn't correct - it will receive
> updates,  because it's the latest release ATM. There is *no* pinning inside
> ISO, and the only place where it was 8.0 were docker containers just
> because we had to workaround some issues. But there are no docker
> containers in 9.0, so that's not the case.
> The proposed solution to switch to CentOS-7.2 in fact is based on
> selecting the right snapshot with packages. There is no pinning in ISO (it
> was in earlier versions of the spec but was removed).
>
> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky 
> wrote:
>
>> Dmitry, Igor,
>>
>> > Very important thing is that CentOS 7.1 which master node is based now
>> > don't get updates any longer.
>>
>> If you are using "fixed" release you must be ready that you won't get
>> any updates. So with CentOS 7.2 the problem still the same.
>>
>> However, let's wait for Fuel PTL decision. I only shared my POV:
>> that's not a critical feature, and taking into account the risks of
>> regression - I'd prefer to do not accept it in 9.0.
>>
>> Regards,
>> Igor
>>
>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat  wrote:
>> > Igor,
>> > please note that this is pretty much not like update of master node
>> which we
>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
>> > tested for more than 2 months already.
>> >
>> > We don't expect it to require any additional efforts from core or qa
>> team.
>> >
>> > Very important thing is that CentOS 7.1 which master node is based now
>> don't
>> > get updates any longer. Updates are only provided for CentOS 7.2.
>> >
>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
>> >
>> > We can do it now for more or less free, later in release cycle for
>> higher
>> > risk and QA efforts and after the release for 2x price because of
>> additional
>> > QA cycle we'll need to pass through.
>> >
>> >
>> >
>> > Regards,
>> > Igor Marnat
>> >
>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin > >
>> > wrote:
>> >>
>> >> Hi Igor,
>> >>
>> >> Postponing this till Fuel 10 means we have to elaborate a plan to do
>> such
>> >> upgrade for Fuel 9 after the release - the underlying system will not
>> get
>> >> updated on it's own, and the security issues will not close
>> themselves. The
>> >> problem here is that such upgrade of deployed master node requires a
>> lot of
>> >> QA work also.
>> >>
>> >> Since we are not going to update package we build on our own (they
>> still
>> >> targeted 7.1) switching master node base to CentOS-72 is not that
>> dangerous
>> >> as it seems.
>> >>
>> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky <
>> ikalnit...@mirantis.com>
>> >> wrote:
>> >>>
>> >>> Hey Dmitry,
>> >>>
>> >>> No offence, but I rather against that exception. We have too many
>> >>> things to do in Mitaka, and moving to CentOS 7.2 means
>> >>>
>> >>> * extra effort from core team
>> >>> * extra effort from qa team
>> >>>
>> >>> Moreover, it might block development by introducing unpredictable
>> >>> regressions. Remember 8.0? So I think it'd be better to reduce risk of
>> >>> regressions that affects so many developers by postponing CentOS 7.2
>> >>> till Fuel 10.
>> >>>
>> >>> Thanks,
>> >>> Igor
>> >>>
>> >>>
>> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin <
>> dtesel...@mirantis.com>
>> >>> wrote:
>> >>> > I'd like to ask for a feature freeze exception for switching to
>> >>> > CentOS-7.2
>> >>> > feature [0].
>> >>> >
>> >>> > CentOS-7.2 ISO's have been tested periodically since the beginning
>> of
>> >>> > the
>> >>> > year, and all major issues were addressed / fixed at the moment.
>> During
>> >>> > the
>> >>> > last weekend I've made 70 BVT runs to verify that the  solution [2]
>> for
>> >>> > the
>> >>> > last issue - e1000 transmit unit hangs works. And it works, 0 tests
>> of
>> >>> > 35
>> >>> > failed [3].
>> >>> >
>> >>> > Benefits of switching to CentOS-7.2 are quite obvious - we will
>> return
>> >>> > to
>> >>> > latest supported CentOS release, will fix a lot of bugs / security
>> >>> > issues
>> >>> > [4] and will make further updates easier.
>> >>> >
>> >>> > Risk of regression still exists, but it's quite low, 35 successful
>> BVTs
>> >>> > can't be wrong.
>> >>> >
>> >>> > To finish that feature the following should be done:
>> >>> > * review and merge e1000 workaround [2]
>> 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Dmitry Teselkin]

Igor,

Your statement about updates for 7.2 isn't correct - it will receive
updates,  because it's the latest release ATM. There is *no* pinning inside
ISO, and the only place where it was 8.0 were docker containers just
because we had to workaround some issues. But there are no docker
containers in 9.0, so that's not the case.
The proposed solution to switch to CentOS-7.2 in fact is based on selecting
the right snapshot with packages. There is no pinning in ISO (it was in
earlier versions of the spec but was removed).

On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky 
wrote:

> Dmitry, Igor,
>
> > Very important thing is that CentOS 7.1 which master node is based now
> > don't get updates any longer.
>
> If you are using "fixed" release you must be ready that you won't get
> any updates. So with CentOS 7.2 the problem still the same.
>
> However, let's wait for Fuel PTL decision. I only shared my POV:
> that's not a critical feature, and taking into account the risks of
> regression - I'd prefer to do not accept it in 9.0.
>
> Regards,
> Igor
>
> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat  wrote:
> > Igor,
> > please note that this is pretty much not like update of master node
> which we
> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
> > tested for more than 2 months already.
> >
> > We don't expect it to require any additional efforts from core or qa
> team.
> >
> > Very important thing is that CentOS 7.1 which master node is based now
> don't
> > get updates any longer. Updates are only provided for CentOS 7.2.
> >
> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
> >
> > We can do it now for more or less free, later in release cycle for higher
> > risk and QA efforts and after the release for 2x price because of
> additional
> > QA cycle we'll need to pass through.
> >
> >
> >
> > Regards,
> > Igor Marnat
> >
> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin 
> > wrote:
> >>
> >> Hi Igor,
> >>
> >> Postponing this till Fuel 10 means we have to elaborate a plan to do
> such
> >> upgrade for Fuel 9 after the release - the underlying system will not
> get
> >> updated on it's own, and the security issues will not close themselves.
> The
> >> problem here is that such upgrade of deployed master node requires a
> lot of
> >> QA work also.
> >>
> >> Since we are not going to update package we build on our own (they still
> >> targeted 7.1) switching master node base to CentOS-72 is not that
> dangerous
> >> as it seems.
> >>
> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky  >
> >> wrote:
> >>>
> >>> Hey Dmitry,
> >>>
> >>> No offence, but I rather against that exception. We have too many
> >>> things to do in Mitaka, and moving to CentOS 7.2 means
> >>>
> >>> * extra effort from core team
> >>> * extra effort from qa team
> >>>
> >>> Moreover, it might block development by introducing unpredictable
> >>> regressions. Remember 8.0? So I think it'd be better to reduce risk of
> >>> regressions that affects so many developers by postponing CentOS 7.2
> >>> till Fuel 10.
> >>>
> >>> Thanks,
> >>> Igor
> >>>
> >>>
> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin <
> dtesel...@mirantis.com>
> >>> wrote:
> >>> > I'd like to ask for a feature freeze exception for switching to
> >>> > CentOS-7.2
> >>> > feature [0].
> >>> >
> >>> > CentOS-7.2 ISO's have been tested periodically since the beginning of
> >>> > the
> >>> > year, and all major issues were addressed / fixed at the moment.
> During
> >>> > the
> >>> > last weekend I've made 70 BVT runs to verify that the  solution [2]
> for
> >>> > the
> >>> > last issue - e1000 transmit unit hangs works. And it works, 0 tests
> of
> >>> > 35
> >>> > failed [3].
> >>> >
> >>> > Benefits of switching to CentOS-7.2 are quite obvious - we will
> return
> >>> > to
> >>> > latest supported CentOS release, will fix a lot of bugs / security
> >>> > issues
> >>> > [4] and will make further updates easier.
> >>> >
> >>> > Risk of regression still exists, but it's quite low, 35 successful
> BVTs
> >>> > can't be wrong.
> >>> >
> >>> > To finish that feature the following should be done:
> >>> > * review and merge e1000 workaround [2]
> >>> > * review and merge spec [0]
> >>> > * review and merge request that switches build CI to CentOS-7.2 [5]
> >>> >
> >>> > I expect the last day it could be done is March, 4.
> >>> >
> >>> > [0] https://review.openstack.org/#/c/280338/
> >>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544
> >>> > [2] https://review.openstack.org/#/c/285306/
> >>> > [3]
> https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
> >>> > [4]
> https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
> >>> > [5] https://review.fuel-infra.org/#/c/17400/
> >>> >
> >>> >
> >>> > --
> >>> > Thanks,
> >>> > Dmitry Teselkin
> >>> > Mirantis
> >>> > http://www.mirantis.com
> >>> >
> >>> >
> >>> >
> 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Igor Kalnitsky]

Dmitry, Igor,

> Very important thing is that CentOS 7.1 which master node is based now
> don't get updates any longer.

If you are using "fixed" release you must be ready that you won't get
any updates. So with CentOS 7.2 the problem still the same.

However, let's wait for Fuel PTL decision. I only shared my POV:
that's not a critical feature, and taking into account the risks of
regression - I'd prefer to do not accept it in 9.0.

Regards,
Igor

On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat  wrote:
> Igor,
> please note that this is pretty much not like update of master node which we
> had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
> tested for more than 2 months already.
>
> We don't expect it to require any additional efforts from core or qa team.
>
> Very important thing is that CentOS 7.1 which master node is based now don't
> get updates any longer. Updates are only provided for CentOS 7.2.
>
> So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
>
> We can do it now for more or less free, later in release cycle for higher
> risk and QA efforts and after the release for 2x price because of additional
> QA cycle we'll need to pass through.
>
>
>
> Regards,
> Igor Marnat
>
> On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin 
> wrote:
>>
>> Hi Igor,
>>
>> Postponing this till Fuel 10 means we have to elaborate a plan to do such
>> upgrade for Fuel 9 after the release - the underlying system will not get
>> updated on it's own, and the security issues will not close themselves. The
>> problem here is that such upgrade of deployed master node requires a lot of
>> QA work also.
>>
>> Since we are not going to update package we build on our own (they still
>> targeted 7.1) switching master node base to CentOS-72 is not that dangerous
>> as it seems.
>>
>> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky 
>> wrote:
>>>
>>> Hey Dmitry,
>>>
>>> No offence, but I rather against that exception. We have too many
>>> things to do in Mitaka, and moving to CentOS 7.2 means
>>>
>>> * extra effort from core team
>>> * extra effort from qa team
>>>
>>> Moreover, it might block development by introducing unpredictable
>>> regressions. Remember 8.0? So I think it'd be better to reduce risk of
>>> regressions that affects so many developers by postponing CentOS 7.2
>>> till Fuel 10.
>>>
>>> Thanks,
>>> Igor
>>>
>>>
>>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin 
>>> wrote:
>>> > I'd like to ask for a feature freeze exception for switching to
>>> > CentOS-7.2
>>> > feature [0].
>>> >
>>> > CentOS-7.2 ISO's have been tested periodically since the beginning of
>>> > the
>>> > year, and all major issues were addressed / fixed at the moment. During
>>> > the
>>> > last weekend I've made 70 BVT runs to verify that the  solution [2] for
>>> > the
>>> > last issue - e1000 transmit unit hangs works. And it works, 0 tests of
>>> > 35
>>> > failed [3].
>>> >
>>> > Benefits of switching to CentOS-7.2 are quite obvious - we will return
>>> > to
>>> > latest supported CentOS release, will fix a lot of bugs / security
>>> > issues
>>> > [4] and will make further updates easier.
>>> >
>>> > Risk of regression still exists, but it's quite low, 35 successful BVTs
>>> > can't be wrong.
>>> >
>>> > To finish that feature the following should be done:
>>> > * review and merge e1000 workaround [2]
>>> > * review and merge spec [0]
>>> > * review and merge request that switches build CI to CentOS-7.2 [5]
>>> >
>>> > I expect the last day it could be done is March, 4.
>>> >
>>> > [0] https://review.openstack.org/#/c/280338/
>>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544
>>> > [2] https://review.openstack.org/#/c/285306/
>>> > [3] https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
>>> > [4] https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
>>> > [5] https://review.fuel-infra.org/#/c/17400/
>>> >
>>> >
>>> > --
>>> > Thanks,
>>> > Dmitry Teselkin
>>> > Mirantis
>>> > http://www.mirantis.com
>>> >
>>> >
>>> > __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> --
>> Thanks,
>> Dmitry Teselkin
>> Mirantis
>> http://www.mirantis.com
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Igor Marnat]

Igor,
please note that this is pretty much not like update of master node which
we had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which team
tested for more than 2 months already.

We don't expect it to require any additional efforts from core or qa team.

Very important thing is that CentOS 7.1 which master node is based now
don't get updates any longer. Updates are only provided for CentOS 7.2.

So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.

We can do it now for more or less free, later in release cycle for higher
risk and QA efforts and after the release for 2x price because of
additional QA cycle we'll need to pass through.



Regards,
Igor Marnat

On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin 
wrote:

> Hi Igor,
>
> Postponing this till Fuel 10 means we have to elaborate a plan to do such
> upgrade for Fuel 9 after the release - the underlying system will not get
> updated on it's own, and the security issues will not close themselves. The
> problem here is that such upgrade of deployed master node requires a lot of
> QA work also.
>
> Since we are not going to update package we build on our own (they still
> targeted 7.1) switching master node base to CentOS-72 is not that dangerous
> as it seems.
>
> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky 
> wrote:
>
>> Hey Dmitry,
>>
>> No offence, but I rather against that exception. We have too many
>> things to do in Mitaka, and moving to CentOS 7.2 means
>>
>> * extra effort from core team
>> * extra effort from qa team
>>
>> Moreover, it might block development by introducing unpredictable
>> regressions. Remember 8.0? So I think it'd be better to reduce risk of
>> regressions that affects so many developers by postponing CentOS 7.2
>> till Fuel 10.
>>
>> Thanks,
>> Igor
>>
>>
>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin 
>> wrote:
>> > I'd like to ask for a feature freeze exception for switching to
>> CentOS-7.2
>> > feature [0].
>> >
>> > CentOS-7.2 ISO's have been tested periodically since the beginning of
>> the
>> > year, and all major issues were addressed / fixed at the moment. During
>> the
>> > last weekend I've made 70 BVT runs to verify that the  solution [2] for
>> the
>> > last issue - e1000 transmit unit hangs works. And it works, 0 tests of
>> 35
>> > failed [3].
>> >
>> > Benefits of switching to CentOS-7.2 are quite obvious - we will return
>> to
>> > latest supported CentOS release, will fix a lot of bugs / security
>> issues
>> > [4] and will make further updates easier.
>> >
>> > Risk of regression still exists, but it's quite low, 35 successful BVTs
>> > can't be wrong.
>> >
>> > To finish that feature the following should be done:
>> > * review and merge e1000 workaround [2]
>> > * review and merge spec [0]
>> > * review and merge request that switches build CI to CentOS-7.2 [5]
>> >
>> > I expect the last day it could be done is March, 4.
>> >
>> > [0] https://review.openstack.org/#/c/280338/
>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544
>> > [2] https://review.openstack.org/#/c/285306/
>> > [3] https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
>> > [4] https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
>> > [5] https://review.fuel-infra.org/#/c/17400/
>> >
>> >
>> > --
>> > Thanks,
>> > Dmitry Teselkin
>> > Mirantis
>> > http://www.mirantis.com
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Thanks,
> Dmitry Teselkin
> Mirantis
> http://www.mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Dmitry Teselkin]

Hi Igor,

Postponing this till Fuel 10 means we have to elaborate a plan to do such
upgrade for Fuel 9 after the release - the underlying system will not get
updated on it's own, and the security issues will not close themselves. The
problem here is that such upgrade of deployed master node requires a lot of
QA work also.

Since we are not going to update package we build on our own (they still
targeted 7.1) switching master node base to CentOS-72 is not that dangerous
as it seems.

On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky 
wrote:

> Hey Dmitry,
>
> No offence, but I rather against that exception. We have too many
> things to do in Mitaka, and moving to CentOS 7.2 means
>
> * extra effort from core team
> * extra effort from qa team
>
> Moreover, it might block development by introducing unpredictable
> regressions. Remember 8.0? So I think it'd be better to reduce risk of
> regressions that affects so many developers by postponing CentOS 7.2
> till Fuel 10.
>
> Thanks,
> Igor
>
>
> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin 
> wrote:
> > I'd like to ask for a feature freeze exception for switching to
> CentOS-7.2
> > feature [0].
> >
> > CentOS-7.2 ISO's have been tested periodically since the beginning of the
> > year, and all major issues were addressed / fixed at the moment. During
> the
> > last weekend I've made 70 BVT runs to verify that the  solution [2] for
> the
> > last issue - e1000 transmit unit hangs works. And it works, 0 tests of 35
> > failed [3].
> >
> > Benefits of switching to CentOS-7.2 are quite obvious - we will return to
> > latest supported CentOS release, will fix a lot of bugs / security issues
> > [4] and will make further updates easier.
> >
> > Risk of regression still exists, but it's quite low, 35 successful BVTs
> > can't be wrong.
> >
> > To finish that feature the following should be done:
> > * review and merge e1000 workaround [2]
> > * review and merge spec [0]
> > * review and merge request that switches build CI to CentOS-7.2 [5]
> >
> > I expect the last day it could be done is March, 4.
> >
> > [0] https://review.openstack.org/#/c/280338/
> > [1] https://bugs.launchpad.net/fuel/+bug/1526544
> > [2] https://review.openstack.org/#/c/285306/
> > [3] https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
> > [4] https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
> > [5] https://review.fuel-infra.org/#/c/17400/
> >
> >
> > --
> > Thanks,
> > Dmitry Teselkin
> > Mirantis
> > http://www.mirantis.com
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Thanks,
Dmitry Teselkin
Mirantis
http://www.mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Feature Freeze Exception Request - switching to CentOS-7.2

[Igor Kalnitsky]

Hey Dmitry,

No offence, but I rather against that exception. We have too many
things to do in Mitaka, and moving to CentOS 7.2 means

* extra effort from core team
* extra effort from qa team

Moreover, it might block development by introducing unpredictable
regressions. Remember 8.0? So I think it'd be better to reduce risk of
regressions that affects so many developers by postponing CentOS 7.2
till Fuel 10.

Thanks,
Igor


On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin  wrote:
> I'd like to ask for a feature freeze exception for switching to CentOS-7.2
> feature [0].
>
> CentOS-7.2 ISO's have been tested periodically since the beginning of the
> year, and all major issues were addressed / fixed at the moment. During the
> last weekend I've made 70 BVT runs to verify that the  solution [2] for the
> last issue - e1000 transmit unit hangs works. And it works, 0 tests of 35
> failed [3].
>
> Benefits of switching to CentOS-7.2 are quite obvious - we will return to
> latest supported CentOS release, will fix a lot of bugs / security issues
> [4] and will make further updates easier.
>
> Risk of regression still exists, but it's quite low, 35 successful BVTs
> can't be wrong.
>
> To finish that feature the following should be done:
> * review and merge e1000 workaround [2]
> * review and merge spec [0]
> * review and merge request that switches build CI to CentOS-7.2 [5]
>
> I expect the last day it could be done is March, 4.
>
> [0] https://review.openstack.org/#/c/280338/
> [1] https://bugs.launchpad.net/fuel/+bug/1526544
> [2] https://review.openstack.org/#/c/285306/
> [3] https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
> [4] https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
> [5] https://review.fuel-infra.org/#/c/17400/
>
>
> --
> Thanks,
> Dmitry Teselkin
> Mirantis
> http://www.mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev