Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi Client,

> Don't do it? Evolve v1 as needed, bud I'd recommend keeping whatever users 
> you currently have functional for as long as possible. Work with the API-WG 
> to make sure what you're doing is driving toward an API that fits inside 
> their best practices.
Thanks for your advice, surely I will remember it.

> We should write this down somewhere if we haven't already but these are the 
> basic principles that will let live upgrades happen in your
> database:
>
> * Add columns with default values or make them nullable
> * Drop columns and tables only after all releases that reference them are EOL.
> * Never rename anything. Create new, migrate data, drop old after EOL.
> * Make new code able to detect a partial migration and fall back to old
>  behavior.
I got it, but I just consider one thing about deleting(altering) table/column. 
Could you please refer this mailing-list [1] for more detail.

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113073.html 

-Original Message-
From: Clint Byrum [mailto:cl...@fewbar.com] 
Sent: Wednesday, March 01, 2017 1:57 AM
To: openstack-dev
Subject: Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

Excerpts from na...@vn.fujitsu.com's message of 2017-02-28 09:52:13 +:
> Hi everyone,
> 
> Recently, there are many emails to discuss a topic that "Why are projects 
> trying to avoid Barbican, still?" [0]. That is very an interesting topic. Now 
> I would like to make a new topic related to Rolling upgrade. I am trying to 
> find  information about the strategy to support rolling-upgrade for Barbican 
> project. Unfortunately, there is not any results, if any, could you please 
> update it for me. 
> 
> In my point of view, in order to support this feature, there will be three 
> impacts which need to solve:
> 
> 1. API versioning:
> 
> Maybe, Barbican will have version two [1] so I think we should have a plan to 
> still support version 1 on version 2. What do you about this point?
> 

Don't do it? Evolve v1 as needed, bud I'd recommend keeping whatever users you 
currently have functional for as long as possible. Work with the API-WG to make 
sure what you're doing is driving toward an API that fits inside their best 
practices.

> 2. Database
> 
> - For OVO (Oslo version object) [2], I am wondering we should use OVO for 
> this project. In my option, I am afraid that OVO is overkill for this project.
> - For DB upgrading, currently there are some files [3-4] to drop and alter 
> column. This really should not have because there is not still have a 
> solution in case of delete or alter DB. That why there is a rule that don't 
> allow somebody to delete/alter DB in Nova and Neutron :) Do you think we 
> should prepare for this?
> 

We should write this down somewhere if we haven't already but these are the 
basic principles that will let live upgrades happen in your
database:

* Add columns with default values or make them nullable
* Drop columns and tables only after all releases that reference them are EOL.
* Never rename anything. Create new, migrate data, drop old after EOL.
* Make new code able to detect a partial migration and fall back to old
  behavior.
  

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi Dave,

Thanks for your reply. I would like to answer your email like below:

> Offline rolling upgrades is part of the current Barbican project
Do you mean that Barbican supported to upgrade? Currently, there are 4 main 
level during upgrade [1]
1. Support upgrade
2. Support rolling upgrade
3. Support zero downtime
4. Support zero impact

> 2) Database Upgrades
>
> I think we have good support for doing database upgrades when upgrading the 
> version of Barbican.  We offer a barbican-db-manage script to handles 
> upgrades. [1] It would be good to improve the documentation on this process.  
> It would also be good to add a gate job to automatically test upgrading 
> Barbican.
Yes, I tested upgrading DB and it works well. However, I am considering a point 
that Barbican still allow our to delete/alter table/column in DB by upgrading, 
that is a problem if we want to support rolling upgrade. For example, in 
Neutron, there was a blueprint [2] and a spec to describe this.  You can see 
the description of the blueprint is """Currently pretty much every major 
upgrade requires full shutdown for all neutron-server instances for the time 
while upgrade process is running. The downtime is due to the need to run 
alembic scripts that modify schema and transform data. Neutron-server instances 
are currently not resilient to working with older schema. We also don't make an 
effort to avoid 'contract' migrations""".

> 3) RPC version
So I am wondering that Barbican have a plan to do this on this cycle(Pike)?

[1] https://governance.openstack.org/tc/reference/tags/#project-assertion-tags
[2] https://blueprints.launchpad.net/neutron/+spec/online-upgrades 
[3] https://review.openstack.org/#/c/386685/ 


-Original Message-
From: Dave McCowan (dmccowan) [mailto:dmcco...@cisco.com] 
Sent: Tuesday, February 28, 2017 9:07 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

Hi Nam--
   Thanks for writing.  Offline rolling upgrades is part of the current 
Barbican project.  Better support and documentation for upgrades would be a 
welcome addition.

1) API Versioning

Currently, Barbican only has one API version.  The wiki you reference is an old 
list of ideas that we started collecting for when/if we create a V2.  I agree, 
that as part of any new API version, we'll need to consider upgrades and 
backwards compatibility.

2) Database Upgrades

I think we have good support support for doing database upgrades when upgrading 
the version of Barbican.  We offer a barbican-db-manage script to handles 
upgrades. [1] It would be good to improve the documentation on this process.  
It would also be good to add a gate job to automatically test upgrading 
Barbican.

3) RPC Versioning

Adding versioning to our RPC message would be good to future-proof our design.  
Perhaps we should leave the message objects stable for now, and add a version 
field in the future when/if a scenario occurs that requires us to change these 
message objects.


[1]
https://docs.openstack.org/developer/barbican/contribute/database_migration
s.html

--Dave (dave-mccowan)



On 2/28/17, 4:52 AM, "na...@vn.fujitsu.com" <na...@vn.fujitsu.com> wrote:

>Hi everyone,
>
>Recently, there are many emails to discuss a topic that "Why are 
>projects trying to avoid Barbican, still?" [0]. That is very an interesting 
>topic.
>Now I would like to make a new topic related to Rolling upgrade. I am 
>trying to find  information about the strategy to support 
>rolling-upgrade for Barbican project. Unfortunately, there is not any 
>results, if any, could you please update it for me.
>
>In my point of view, in order to support this feature, there will be 
>three impacts which need to solve:
>
>1. API versioning:
>
>Maybe, Barbican will have version two [1] so I think we should have a 
>plan to still support version 1 on version 2. What do you about this 
>point?
>
>2. Database
>
>- For OVO (Oslo version object) [2], I am wondering we should use OVO 
>for this project. In my option, I am afraid that OVO is overkill for 
>this project.
>- For DB upgrading, currently there are some files [3-4] to drop and 
>alter column. This really should not have because there is not still 
>have a solution in case of delete or alter DB. That why there is a rule 
>that don't allow somebody to delete/alter DB in Nova and Neutron :) Do 
>you think we should prepare for this?
>
>3. RPC
>
>There is not version parameter when sending AMQP [5]. This parameter is 
>really necessary to distinguish the message is Ocata or Pike.
>
>That is some points in my option, what do you think we should have to 
>plan to support this feature and I am very glad to discuss this topic 

Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[Clint Byrum]

Excerpts from na...@vn.fujitsu.com's message of 2017-02-28 09:52:13 +:
> Hi everyone,
> 
> Recently, there are many emails to discuss a topic that "Why are projects 
> trying to avoid Barbican, still?" [0]. That is very an interesting topic. Now 
> I would like to make a new topic related to Rolling upgrade. I am trying to 
> find  information about the strategy to support rolling-upgrade for Barbican 
> project. Unfortunately, there is not any results, if any, could you please 
> update it for me. 
> 
> In my point of view, in order to support this feature, there will be three 
> impacts which need to solve:
> 
> 1. API versioning:
> 
> Maybe, Barbican will have version two [1] so I think we should have a plan to 
> still support version 1 on version 2. What do you about this point?
> 

Don't do it? Evolve v1 as needed, bud I'd recommend keeping whatever
users you currently have functional for as long as possible. Work with
the API-WG to make sure what you're doing is driving toward an API that
fits inside their best practices.

> 2. Database
> 
> - For OVO (Oslo version object) [2], I am wondering we should use OVO for 
> this project. In my option, I am afraid that OVO is overkill for this project.
> - For DB upgrading, currently there are some files [3-4] to drop and alter 
> column. This really should not have because there is not still have a 
> solution in case of delete or alter DB. That why there is a rule that don't 
> allow somebody to delete/alter DB in Nova and Neutron :) Do you think we 
> should prepare for this?
> 

We should write this down somewhere if we haven't already but these are
the basic principles that will let live upgrades happen in your
database:

* Add columns with default values or make them nullable
* Drop columns and tables only after all releases that reference them are EOL.
* Never rename anything. Create new, migrate data, drop old after EOL.
* Make new code able to detect a partial migration and fall back to old
  behavior.
  

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[Dave McCowan (dmccowan)]

Hi Nam--
   Thanks for writing.  Offline rolling upgrades is part of the current
Barbican project.  Better support and documentation for upgrades would be
a welcome addition.

1) API Versioning

Currently, Barbican only has one API version.  The wiki you reference is
an old list of ideas that we started collecting for when/if we create a
V2.  I agree, that as part of any new API version, we'll need to consider
upgrades and backwards compatibility.

2) Database Upgrades

I think we have good support support for doing database upgrades when
upgrading the version of Barbican.  We offer a barbican-db-manage script
to handles upgrades. [1]
It would be good to improve the documentation on this process.  It would
also be good to add a gate job to automatically test upgrading Barbican.

3) RPC Versioning

Adding versioning to our RPC message would be good to future-proof our
design.  Perhaps we should leave the message objects stable for now, and
add a version field in the future when/if a scenario occurs that requires
us to change these message objects.


[1] 
https://docs.openstack.org/developer/barbican/contribute/database_migration
s.html

--Dave (dave-mccowan)



On 2/28/17, 4:52 AM, "na...@vn.fujitsu.com"  wrote:

>Hi everyone,
>
>Recently, there are many emails to discuss a topic that "Why are projects
>trying to avoid Barbican, still?" [0]. That is very an interesting topic.
>Now I would like to make a new topic related to Rolling upgrade. I am
>trying to find  information about the strategy to support rolling-upgrade
>for Barbican project. Unfortunately, there is not any results, if any,
>could you please update it for me.
>
>In my point of view, in order to support this feature, there will be
>three impacts which need to solve:
>
>1. API versioning:
>
>Maybe, Barbican will have version two [1] so I think we should have a
>plan to still support version 1 on version 2. What do you about this
>point?
>
>2. Database
>
>- For OVO (Oslo version object) [2], I am wondering we should use OVO for
>this project. In my option, I am afraid that OVO is overkill for this
>project.
>- For DB upgrading, currently there are some files [3-4] to drop and
>alter column. This really should not have because there is not still have
>a solution in case of delete or alter DB. That why there is a rule that
>don't allow somebody to delete/alter DB in Nova and Neutron :) Do you
>think we should prepare for this?
>
>3. RPC
>
>There is not version parameter when sending AMQP [5]. This parameter is
>really necessary to distinguish the message is Ocata or Pike.
>
>That is some points in my option, what do you think we should have to
>plan to support this feature and I am very glad to discuss this topic
>with you on this thread mailing-list.
>
>[0] 
>http://lists.openstack.org/pipermail/openstack-dev/2017-January/thread.htm
>l#110192 
>[1] https://wiki.openstack.org/wiki/Barbican/v2
>[2] https://www.slideshare.net/davanum/ovo-deep-dive
>[3] 
>https://github.com/openstack/barbican/blob/master/barbican/model/migration
>/alembic_migrations/versions/3041b53b95d7_remove_size_limits_on_meta_table
>_values.py 
>[4] 
>https://github.com/openstack/barbican/blob/master/barbican/model/migration
>/alembic_migrations/versions/254495565185_removing_redundant_fields_from_o
>rder.py 
>[5] 
>https://github.com/openstack/barbican/blob/master/barbican/queue/client.py
>#L49 
>
> --
>Best Regard,
>
>Nam NH
>OpenStack team
>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi everyone,

Recently, there are many emails to discuss a topic that "Why are projects 
trying to avoid Barbican, still?" [0]. That is very an interesting topic. Now I 
would like to make a new topic related to Rolling upgrade. I am trying to find  
information about the strategy to support rolling-upgrade for Barbican project. 
Unfortunately, there is not any results, if any, could you please update it for 
me. 

In my point of view, in order to support this feature, there will be three 
impacts which need to solve:

1. API versioning:

Maybe, Barbican will have version two [1] so I think we should have a plan to 
still support version 1 on version 2. What do you about this point?

2. Database

- For OVO (Oslo version object) [2], I am wondering we should use OVO for this 
project. In my option, I am afraid that OVO is overkill for this project.
- For DB upgrading, currently there are some files [3-4] to drop and alter 
column. This really should not have because there is not still have a solution 
in case of delete or alter DB. That why there is a rule that don't allow 
somebody to delete/alter DB in Nova and Neutron :) Do you think we should 
prepare for this?

3. RPC

There is not version parameter when sending AMQP [5]. This parameter is really 
necessary to distinguish the message is Ocata or Pike.   

That is some points in my option, what do you think we should have to plan to 
support this feature and I am very glad to discuss this topic with you on this 
thread mailing-list. 

[0] 
http://lists.openstack.org/pipermail/openstack-dev/2017-January/thread.html#110192
 
[1] https://wiki.openstack.org/wiki/Barbican/v2 
[2] https://www.slideshare.net/davanum/ovo-deep-dive 
[3] 
https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/3041b53b95d7_remove_size_limits_on_meta_table_values.py
 
[4] 
https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/254495565185_removing_redundant_fields_from_order.py
 
[5] 
https://github.com/openstack/barbican/blob/master/barbican/queue/client.py#L49 

 --
Best Regard,

Nam NH
OpenStack team


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev