[openstack-dev] [barbican] Hangout Barbican team

[na...@vn.fujitsu.com]

Hi Barbican team,

In order to be easy for reviewing some patch sets in Barbican, we propose that 
it should have a hangout meeting on 10pm EDT - Monday 30 April. So i would like 
to send an email to notify everyone that feel free to join with us by leaving 
your email.


Cheers,
Nam?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [bifrost][helm][OSA][barbican] Switching from fedora-26 to fedora-27

[na...@vn.fujitsu.com]

Hello Paul,

I am Nam from Barbican team. I would like to notify a problem when using 
fedora-27. 

Currently, fedora-27 is using mariadb at 10.2.12. But there is a bug in this 
version and it is the main reason for failure Barbican database upgrading [1], 
the bug was fixed at 10.2.13 [2]. Would you mind updating the version of 
mariadb before removing fedora-26.

[1] https://bugs.launchpad.net/barbican/+bug/1734329 
[2] https://jira.mariadb.org/browse/MDEV-13508 

Thanks,
Nam

> -Original Message-
> From: Paul Belanger [mailto:pabelan...@redhat.com]
> Sent: Tuesday, March 13, 2018 9:54 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [bifrost][helm][OSA][barbican] Switching from
> fedora-26 to fedora-27
> 
> On Mon, Mar 05, 2018 at 06:45:13PM -0500, Paul Belanger wrote:
> > Greetings,
> >
> > A quick search of git shows your projects are using fedora-26 nodes for
> testing.
> > Please take a moment to look at gerrit[1] and help land patches.  We'd
> > like to remove fedora-26 nodes in the next week and to avoid broken
> > jobs you'll need to approve these patches.
> >
> > If you jobs are failing under fedora-27, please take the time to fix
> > any issue or update said patches to make them non-voting.
> >
> > We (openstack-infra) aim to only keep the latest fedora image online,
> > which changes aprox every 6 months.
> >
> > Thanks for your help and understanding, Paul
> >
> > [1] https://review.openstack.org/#/q/topic:fedora-27+status:open
> >
> Greetings,
> 
> This is a friendly reminder, about moving jobs to fedora-27. I'd like to 
> remove
> our fedora-26 images next week and if jobs haven't been migrated you may
> start to see NODE_FAILURE messages while running jobs.  Please take a
> moment to merge the open changes or update them to be non-voting while
> you work on fixes.
> 
> Thanks again,
> Paul
> 
> __
> 
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [barbican] weekly meeting time

[na...@vn.fujitsu.com]

Hi Ade,

The two options are good to me. I choose the second time.

Thanks,
Nam

> -Original Message-
> From: Ade Lee [mailto:a...@redhat.com]
> Sent: Tuesday, February 13, 2018 10:18 PM
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Subject: [openstack-dev] [barbican] weekly meeting time
> 
> Hi all,
> 
> The Barbican weekly meeting has been fairly sparsely attended for a little
> while now, and the most active contributors these days appear to be in Asia.
> 
> Its time to consider moving the weekly meeting to a time when more
> contributors can attend.  I'm going to propose a couple times below to start
> out.
> 
> 2 am UTC Tuesday == 9 pm EST Monday == 10 am CST (China) Tuesday
> 3 am UTC Tuesday == 10 pm EST Monday == 11 am CST (China) Tuesday
> 
> Feel free to propose other days/times.
> 
> Thanks,
> Ade
> 
> P.S. Until decided otherwise, the Barbican meeting remains on Mondays at
> 2000 UTC
> 
> 
> 
> 
> __
> 
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [barbican][nova][cinder][tacker][glance] Remove Certificate Orders and CAs from API

[na...@vn.fujitsu.com]

Hi all,

Barbican's team are considering whether the Certificate Orders and CAs should 
be removed or not [1]. And we would like to hear information from other 
projects. If you are using this feature for your project, please raise your 
hand. We will discuss about this.

[1] https://review.openstack.org/#/c/462363/

Cheers,
Nam


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi Client,

> Don't do it? Evolve v1 as needed, bud I'd recommend keeping whatever users 
> you currently have functional for as long as possible. Work with the API-WG 
> to make sure what you're doing is driving toward an API that fits inside 
> their best practices.
Thanks for your advice, surely I will remember it.

> We should write this down somewhere if we haven't already but these are the 
> basic principles that will let live upgrades happen in your
> database:
>
> * Add columns with default values or make them nullable
> * Drop columns and tables only after all releases that reference them are EOL.
> * Never rename anything. Create new, migrate data, drop old after EOL.
> * Make new code able to detect a partial migration and fall back to old
>  behavior.
I got it, but I just consider one thing about deleting(altering) table/column. 
Could you please refer this mailing-list [1] for more detail.

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113073.html 

-Original Message-
From: Clint Byrum [mailto:cl...@fewbar.com] 
Sent: Wednesday, March 01, 2017 1:57 AM
To: openstack-dev
Subject: Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

Excerpts from na...@vn.fujitsu.com's message of 2017-02-28 09:52:13 +:
> Hi everyone,
> 
> Recently, there are many emails to discuss a topic that "Why are projects 
> trying to avoid Barbican, still?" [0]. That is very an interesting topic. Now 
> I would like to make a new topic related to Rolling upgrade. I am trying to 
> find  information about the strategy to support rolling-upgrade for Barbican 
> project. Unfortunately, there is not any results, if any, could you please 
> update it for me. 
> 
> In my point of view, in order to support this feature, there will be three 
> impacts which need to solve:
> 
> 1. API versioning:
> 
> Maybe, Barbican will have version two [1] so I think we should have a plan to 
> still support version 1 on version 2. What do you about this point?
> 

Don't do it? Evolve v1 as needed, bud I'd recommend keeping whatever users you 
currently have functional for as long as possible. Work with the API-WG to make 
sure what you're doing is driving toward an API that fits inside their best 
practices.

> 2. Database
> 
> - For OVO (Oslo version object) [2], I am wondering we should use OVO for 
> this project. In my option, I am afraid that OVO is overkill for this project.
> - For DB upgrading, currently there are some files [3-4] to drop and alter 
> column. This really should not have because there is not still have a 
> solution in case of delete or alter DB. That why there is a rule that don't 
> allow somebody to delete/alter DB in Nova and Neutron :) Do you think we 
> should prepare for this?
> 

We should write this down somewhere if we haven't already but these are the 
basic principles that will let live upgrades happen in your
database:

* Add columns with default values or make them nullable
* Drop columns and tables only after all releases that reference them are EOL.
* Never rename anything. Create new, migrate data, drop old after EOL.
* Make new code able to detect a partial migration and fall back to old
  behavior.
  

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi Dave,

Thanks for your reply. I would like to answer your email like below:

> Offline rolling upgrades is part of the current Barbican project
Do you mean that Barbican supported to upgrade? Currently, there are 4 main 
level during upgrade [1]
1. Support upgrade
2. Support rolling upgrade
3. Support zero downtime
4. Support zero impact

> 2) Database Upgrades
>
> I think we have good support for doing database upgrades when upgrading the 
> version of Barbican.  We offer a barbican-db-manage script to handles 
> upgrades. [1] It would be good to improve the documentation on this process.  
> It would also be good to add a gate job to automatically test upgrading 
> Barbican.
Yes, I tested upgrading DB and it works well. However, I am considering a point 
that Barbican still allow our to delete/alter table/column in DB by upgrading, 
that is a problem if we want to support rolling upgrade. For example, in 
Neutron, there was a blueprint [2] and a spec to describe this.  You can see 
the description of the blueprint is """Currently pretty much every major 
upgrade requires full shutdown for all neutron-server instances for the time 
while upgrade process is running. The downtime is due to the need to run 
alembic scripts that modify schema and transform data. Neutron-server instances 
are currently not resilient to working with older schema. We also don't make an 
effort to avoid 'contract' migrations""".

> 3) RPC version
So I am wondering that Barbican have a plan to do this on this cycle(Pike)?

[1] https://governance.openstack.org/tc/reference/tags/#project-assertion-tags
[2] https://blueprints.launchpad.net/neutron/+spec/online-upgrades 
[3] https://review.openstack.org/#/c/386685/ 


-Original Message-
From: Dave McCowan (dmccowan) [mailto:dmcco...@cisco.com] 
Sent: Tuesday, February 28, 2017 9:07 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [barbican] Rolling upgrade in Barbican project

Hi Nam--
   Thanks for writing.  Offline rolling upgrades is part of the current 
Barbican project.  Better support and documentation for upgrades would be a 
welcome addition.

1) API Versioning

Currently, Barbican only has one API version.  The wiki you reference is an old 
list of ideas that we started collecting for when/if we create a V2.  I agree, 
that as part of any new API version, we'll need to consider upgrades and 
backwards compatibility.

2) Database Upgrades

I think we have good support support for doing database upgrades when upgrading 
the version of Barbican.  We offer a barbican-db-manage script to handles 
upgrades. [1] It would be good to improve the documentation on this process.  
It would also be good to add a gate job to automatically test upgrading 
Barbican.

3) RPC Versioning

Adding versioning to our RPC message would be good to future-proof our design.  
Perhaps we should leave the message objects stable for now, and add a version 
field in the future when/if a scenario occurs that requires us to change these 
message objects.


[1]
https://docs.openstack.org/developer/barbican/contribute/database_migration
s.html

--Dave (dave-mccowan)



On 2/28/17, 4:52 AM, "na...@vn.fujitsu.com" <na...@vn.fujitsu.com> wrote:

>Hi everyone,
>
>Recently, there are many emails to discuss a topic that "Why are 
>projects trying to avoid Barbican, still?" [0]. That is very an interesting 
>topic.
>Now I would like to make a new topic related to Rolling upgrade. I am 
>trying to find  information about the strategy to support 
>rolling-upgrade for Barbican project. Unfortunately, there is not any 
>results, if any, could you please update it for me.
>
>In my point of view, in order to support this feature, there will be 
>three impacts which need to solve:
>
>1. API versioning:
>
>Maybe, Barbican will have version two [1] so I think we should have a 
>plan to still support version 1 on version 2. What do you about this 
>point?
>
>2. Database
>
>- For OVO (Oslo version object) [2], I am wondering we should use OVO 
>for this project. In my option, I am afraid that OVO is overkill for 
>this project.
>- For DB upgrading, currently there are some files [3-4] to drop and 
>alter column. This really should not have because there is not still 
>have a solution in case of delete or alter DB. That why there is a rule 
>that don't allow somebody to delete/alter DB in Nova and Neutron :) Do 
>you think we should prepare for this?
>
>3. RPC
>
>There is not version parameter when sending AMQP [5]. This parameter is 
>really necessary to distinguish the message is Ocata or Pike.
>
>That is some points in my option, what do you think we should have to 
>plan to support this feature and I am very glad to discuss this topic 

[openstack-dev] [barbican] Rolling upgrade in Barbican project

[na...@vn.fujitsu.com]

Hi everyone,

Recently, there are many emails to discuss a topic that "Why are projects 
trying to avoid Barbican, still?" [0]. That is very an interesting topic. Now I 
would like to make a new topic related to Rolling upgrade. I am trying to find  
information about the strategy to support rolling-upgrade for Barbican project. 
Unfortunately, there is not any results, if any, could you please update it for 
me. 

In my point of view, in order to support this feature, there will be three 
impacts which need to solve:

1. API versioning:

Maybe, Barbican will have version two [1] so I think we should have a plan to 
still support version 1 on version 2. What do you about this point?

2. Database

- For OVO (Oslo version object) [2], I am wondering we should use OVO for this 
project. In my option, I am afraid that OVO is overkill for this project.
- For DB upgrading, currently there are some files [3-4] to drop and alter 
column. This really should not have because there is not still have a solution 
in case of delete or alter DB. That why there is a rule that don't allow 
somebody to delete/alter DB in Nova and Neutron :) Do you think we should 
prepare for this?

3. RPC

There is not version parameter when sending AMQP [5]. This parameter is really 
necessary to distinguish the message is Ocata or Pike.   

That is some points in my option, what do you think we should have to plan to 
support this feature and I am very glad to discuss this topic with you on this 
thread mailing-list. 

[0] 
http://lists.openstack.org/pipermail/openstack-dev/2017-January/thread.html#110192
 
[1] https://wiki.openstack.org/wiki/Barbican/v2 
[2] https://www.slideshare.net/davanum/ovo-deep-dive 
[3] 
https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/3041b53b95d7_remove_size_limits_on_meta_table_values.py
 
[4] 
https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/254495565185_removing_redundant_fields_from_order.py
 
[5] 
https://github.com/openstack/barbican/blob/master/barbican/queue/client.py#L49 

 --
Best Regard,

Nam NH
OpenStack team


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo.db] [CC neutron] CIDR overlap functionality and constraints

[na...@vn.fujitsu.com]

Mike Bayer wrote:

> Well let me reiterate the idea, which is that:
>
> 1. we add features to oslo.db so that the use of a custom stored
> function is not a big deal
>
> 2. we add features to oslo.db that are based on using triggers, special
> constraints, or Gist indexes, so that the use of a database constraint
> that needs this kind of thing is not a big deal
>
> 3. the first proof of concept for this, is a CIDR function / trigger for
> this one reported issue in Neutron.
>
> Now the question is, "can I think of any operation in openstack, besides
> this one, that would benefit from a custom stored function or a
> specialized constraint".   The answer for me is "not specifically but i
> bet if I started looking, I would".  Anytime there's an application
> loading some rows of data out of a table, doing some calculations on it,
> then dealing with a subset of those rows as a result, is a candidate for
> #1 (in fact I have some vague recollection of seeing some patch in
> Neutron that had this issue, it was the reason that compare-and-swap
> could not be used).   Anytime an application is trying to insert rows
> into a table which should be rejected based on some criteria beyond
> "unique key", that's a candidate for #2 - perhaps the plethora of
> UUID-based recipes throughout openstack in some cases could be better
> stated by more data-driven constraints.
>
> If we were to decide that the Neutron issue right here doesn't need any
> changes, then I would be fine abandoning this initiative for now.  But
> as it stands, there seems to be a need to either do this change, *or*
> add a new UUID column to the subnets table, and basically I'm hoping to
> start steering the boat away from the island of
> add-a-new-column-everytime-theres-a-concurrency-problem.

Hello Mike,

Thanks for your idea, If this feature can be moved to Oslo_db then
that's better because Oslo_db has more people with experience of Database
than Neutron has. Do you have any plan for this in Newton cycle?
If yes would you mind sharing me about your plan?

In case we couldn't move this one in Newton cycle then we should have
an alternative solution which Kevin mentioned as "compare-and-swap".
In next cycle, we will come back your idea. How do you think?


Kevin Benton wrote:

> we can solve this particular bug by doing a compare and swap
> operation on some network scoped value (at the cost of all subnet creates
> on the same network becoming serialized via conflicts and retries).

Hello Kevin,

In my understanding, after your patch set [1] is merged, we can add one
line of code so that the systems will increase the revision_number value
of a network which contain a subnet before adding/deleting the subnet,
Am I right?

[1] https://review.openstack.org/#/c/303966/

Thanks and best regards,
NamNH
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Add an extension "btree_gist" to Postgresql

[na...@vn.fujitsu.com]

Ihar Hrachyshka wrote:

> That’s an interesting precedent. I understand that since we gate on  
> postgresql and mysql only, the solution is good enough to pass in the gate.  
> But are we ok fixing a bug just for those two backends, knowing that it’s  
> left exposed for other backends? Could you think of a solution that would  
> be backend agnostic?

Thank you for your reply and sorry for my late because I got many comments
from core-reviews including you so I need to investigate their one. By the way, 
they are discussing on my patch set so shall I close this thread.
Please check my one [1] for more detail.

Thanks and best regards,
Nam

[1] https://review.openstack.org/#/c/314054/ 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] Add an extension "btree_gist" to Postgresql

[na...@vn.fujitsu.com]

Hi everyone,

I am trying to fix a bug that created two subnets on *one* network with 
overlap CIDR [1]. My solution is to focus on Database interaction [2].

I am encountering/facing a problem with Postgresql 
(There is no problem with MySQL).

For Postgresql, I used a feature called GiST that I strong believe 
it can fix this bug. However, this feature needs to be created 
an extension "btree_gist" that we have to need "root" permission.
In my point of view, we cannot do this on Neutron 
because it only has normal user permission.

Currently, There are two ways that may be fix this issue in my opinion:

1. I will try to find a solution to let Neutron can create an extension 
in Postgresql with normal user. But I'm not sure about this yet.

2. I will try to implement a create an extension "btree_gist" during 
Postgresql Database setting up when deploying OpenStack. 
If so, an "Installation Guide" section must be included.

How do you think about that or do you have any advice? 
Any help is very appreciated!

[1] https://bugs.launchpad.net/neutron/+bug/1532695 
[2] https://review.openstack.org/#/c/314054/ 

Thanks,
Nam

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Plan for changes affecting DB schema

[na...@vn.fujitsu.com]

Thanks for your answer. 

-Original Message-
From: Henry Gessau [mailto:hen...@gessau.net] 
Sent: Tuesday, March 22, 2016 8:34 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Neutron] Plan for changes affecting DB schema

na...@vn.fujitsu.com <na...@vn.fujitsu.com> wrote:
> Two weeks ago, I received an information about changing affecting DB 
> schema [1] from Henry Gessau just a day before the deadline. I was so 
> surprised about this and could not change my plan for my patch sets. 
> Do you know any plan for this ?

There should be no surprises. Neutron follows the OpenStack release schedule 
[1]. For Mitaka, it looked like [2].

> In the future, do you have plan for this in Netwon cycle?

The Newton release schedule is at [3], although the details are still being 
planned. The detailed dates should be available soo.

[1] http://releases.openstack.org
[2] http://releases.openstack.org/mitaka/schedule.html
[3] http://releases.openstack.org/newton/schedule.html


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev