[Openstack-operators] FIPS Compliance

[Sean McGinnis]

I'm interested in some feedback from the community, particularly those running
OpenStack deployments, as to whether FIPS compliance [0][1] is something folks
are looking for.

I've been seeing small changes starting to be proposed here and there for
things like MD5 usage related to its incompatibility to FIPS mode. But looking
across a wider stripe of our repos, it appears like it would be a wider effort
to be able to get all OpenStack services compatible with FIPS mode.

This should be a fairly easy thing to test, but before we put in much effort
into updating code and figuring out testing, I'd like to see some input on
whether something like this is needed.

Thanks for any input on this.

Sean

[0] https://en.wikipedia.org/wiki/FIPS_140-2
[1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.140-2.pdf

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Openstack-sigs] [openstack-dev] [horizon][nova][cinder][keystone][glance][neutron][swift] Horizon feature gaps

[Sean McGinnis]

On Wed, Oct 17, 2018 at 10:41:36AM -0500, Matt Riedemann wrote:
> On 10/17/2018 9:24 AM, Ivan Kolodyazhny wrote:
> > 
> > As you may know, unfortunately, Horizon doesn't support all features
> > provided by APIs. That's why we created feature gaps list [1].
> > 
> > I'd got a lot of great conversations with projects teams during the PTG
> > and we tried to figure out what should be done prioritize these tasks.
> > It's really helpful for Horizon to get feedback from other teams to
> > understand what features should be implemented next.
> > 
> > While I'm filling launchpad with new bugs and blueprints for [1], it
> > would be good to review this list again and find some volunteers to
> > decrease feature gaps.
> > 
> > [1] https://etherpad.openstack.org/p/horizon-feature-gap
> > 
> > Thanks everybody for any of your contributions to Horizon.
> 
> +openstack-sigs
> +openstack-operators
> 
> I've left some notes for nova. This looks very similar to the compute API
> OSC gap analysis I did [1]. Unfortunately it's hard to prioritize what to
> really work on without some user/operator feedback - maybe we can get the
> user work group involved in trying to help prioritize what people really
> want that is missing from horizon, at least for compute?
> 
> [1] https://etherpad.openstack.org/p/compute-api-microversion-gap-in-osc
> 
> -- 
> 
> Thanks,
> 
> Matt

I also have a cinderclient OSC gap analysis I've started working on. It might
be useful to add a Horizon column to this list too.

https://ethercalc.openstack.org/cinderclient-osc-gap

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Forum Schedule - Seeking Community Review

[Sean McGinnis]

On Mon, Oct 15, 2018 at 03:01:07PM -0500, Jimmy McArthur wrote:
> Hi -
> 
> The Forum schedule is now up
> (https://www.openstack.org/summit/berlin-2018/summit-schedule/#track=262).
> If you see a glaring content conflict within the Forum itself, please let me
> know.
> 

I have updated the Forum wiki page in preparation for the topic etherpads:

https://wiki.openstack.org/wiki/Forum/Berlin2018

Please add your working session etherpad links once they are available so
everyone has one spot to go to to find all relevant links.

Thanks!
Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [SIGS] Ops Tools SIG

[Sean McGinnis]

On Fri, Oct 12, 2018 at 11:25:20AM +0200, Martin Magr wrote:
> Greetings guys,
> 
> On Thu, Oct 11, 2018 at 4:19 PM, Miguel Angel Ajo Pelayo <
> majop...@redhat.com> wrote:
> 
> > Adding the mailing lists back to your reply, thank you :)
> >
> > I guess that +melvin.hills...@huawei.com  can
> > help us a little bit organizing the SIG,
> > but I guess the first thing would be collecting a list of tools which
> > could be published
> > under the umbrella of the SIG, starting by the ones already in Osops.
> >
> > Publishing documentation for those tools, and the catalog under
> > docs.openstack.org
> > is possibly the next step (or a parallel step).
> >
> >
> > On Wed, Oct 10, 2018 at 4:43 PM Rob McAllister 
> > wrote:
> >
> >> Hi Miguel,
> >>
> >> I would love to join this. What do I need to do?
> >>
> >> Sent from my iPhone
> >>
> >> On Oct 9, 2018, at 03:17, Miguel Angel Ajo Pelayo 
> >> wrote:
> >>
> >> Hello
> >>
> >> Yesterday, during the Oslo meeting we discussed [6] the possibility
> >> of creating a new Special Interest Group [1][2] to provide home and release
> >> means for operator related tools [3] [4] [5]
> >>
> >>
>   all of those tools have python dependencies related to openstack such as
> python-openstackclient or python-pbr. Which is exactly the reason why we
> moved osops-tools-monitoring-oschecks packaging away from OpsTools SIG to
> Cloud SIG. AFAIR we had some issues of having opstools SIG being dependent
> on openstack SIG. I believe that Cloud SIG is proper home for tools like
> [3][4][5] as they are related to OpenStack anyway. OpsTools SIG contains
> general tools like fluentd, sensu, collectd.
> 
> 
> Hope this helps,
> Martin
> 

Hey Martin,

I'm not sure I understand the issue with these tools have dependencies on other
packages and the relationship to SIG ownership. Is your concern (or the history
of a concern you are pointing out) that the tools would have a more difficult
time if they required updates to dependencies if they are owned by a different
group?

Thanks!
Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [sean.mcgin...@gmx.com: [openstack-dev] [ptl][release] Proposed changes for cycle-with-milestones deliverables]

[Sean McGinnis]

Hello operators!

I'm trying to reach out to more folks that this might impact. There are nitty
gritty details below, but to summarize - the release team is looking at
changing the way we currently do releases for the main service projects. So
far, we have been doing three milestone releases throughout a development
cycle, one or more release candidates, then the final release for the cycle.

From what we could tell, these milestone releases were really not being used by
anyone (mostly) so to a degree, they were just busy work that we were imposing
on the project teams.

We are thinking of changing the release model to get rid of the milestones and
just do the release candidates as we finalize the release. Those can be
considered close to done for the purposes of those wanting to get an early
start on testing deployments and doing any kind of packaging.

The only problem we've seen is without the milestone releases during the cycle,
there is quite a gap before the version numbers get incremented for anyone
doing more frequent testing. We wanted to reach out the operator community
since I know there are at least a few of you that do "roll your own" for
deploying upstream code.

If anyone knows this change would have a negative impact for you, please do let
us know so we can take that into consideration or can make any tweaks to our
plans. We want to reduce work put on the teams, but we don't want to do that at
the expense of preventing others from being able to get their work done.

Thanks!
Sean

- Forwarded message from Sean McGinnis  -

Date: Wed, 26 Sep 2018 09:22:30 -0500
From: Sean McGinnis 
To: openstack-...@lists.openstack.org
Subject: [openstack-dev] [ptl][release] Proposed changes for 
cycle-with-milestones deliverables
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 

User-Agent: Mutt/1.10.1 (2018-07-13)

During the Stein PTG in Denver, the release management team talked about ways
we can make things simpler and reduce the "paper pushing" work that all teams
need to do right now. One topic that came up was the usefulness of pushing tags
around milestones during the cycle.

There were a couple of needs identified for doing such "milestone releases":
1) It tests the release automation machinery to identify problems before
   the RC and final release crunch time.
2) It creates a nice cadence throughout the cycle to help teams stay on
   track and focus on the right things for each phase of the cycle.
3) It gives us an indication that teams are healthy, active, and planning
   to include their components in the final release.

One of the big motivators in the past was also to have output that downstream
distros and users could pick up for testing and early packaging. Based on our
admittedly anecdotal small sample, it doesn't appear this is actually a big
need, so we propose to stop tagging milestone releases for the
cycle-with-milestone projects.

We would still have "milestones" during the cycle to facilitate work
organization and create a cadence: teams should still be aware of them, and we
will continue to communicate those dates in the schedule and in the release
countdown emails. But you would no longer be required to request a release for 
each milestone.

Beta releases would be optional: if teams do want to have some beta version
tags before the final release they can still request them - whether on one of
the milestone dates, or whenever there is the need for the project.

Release candidates would still require a tag. To facilitate that step and
guarantee we have a release candidate for every deliverable, the release team
proposes to automatically generate a release request early in the week of the
RC deadline. That patch would be used as a base to communicate with the team:
if a team wants to wait for a specific patch to make it to the RC, someone from
the team can -1 the patch to have it held, or update that patch with a
different commit SHA. If there are no issues, ideally we would want a +1 from
the PTL and/or release liaison to indicate approval, but we would also consider
no negative feedback as an indicator that the automatically proposed patches
without a -1 can all be approved at the end of the RC deadline week.

To cover point (3) above, and clearly know that a project is healthy and should
be included in the coordinated release, we are thinking of requiring a person 
for each team to add their name to a "manifest" of sorts for the release cycle.
That "final release liaison" person would be the designated person to follow
through on finishing out the releases for that team, and would be designated
ahead of the final release phases.

With all these changes, we would rename the cycle-with-milestones release
model to something like cycle-with-rc.

FAQ:
Q: Does this mean I don't need to pay attention to releases any more and the
   release team will just take

Re: [Openstack-operators] [openstack-dev] [all] Consistent policy names

[Sean McGinnis]

On Fri, Sep 28, 2018 at 01:54:01PM -0500, Lance Bragstad wrote:
> On Fri, Sep 28, 2018 at 1:03 PM Harry Rybacki  wrote:
> 
> > On Fri, Sep 28, 2018 at 1:57 PM Morgan Fainberg
> >  wrote:
> > >
> > > Ideally I would like to see it in the form of least specific to most
> > specific. But more importantly in a way that there is no additional
> > delimiters between the service type and the resource. Finally, I do not
> > like the change of plurality depending on action type.
> > >
> > > I propose we consider
> > >
> > > ::[:]
> > >
> > > Example for keystone (note, action names below are strictly examples I
> > am fine with whatever form those actions take):
> > > identity:projects:create
> > > identity:projects:delete
> > > identity:projects:list
> > > identity:projects:get
> > >
> > > It keeps things simple and consistent when you're looking through
> > overrides / defaults.
> > > --Morgan
> > +1 -- I think the ordering if `resource` comes before
> > `action|subaction` will be more clean.
> >
> 

Great idea. This is looking better and better.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [all] Consistent policy names

[Sean McGinnis]

> On Fri, Sep 28, 2018 at 8:48 AM Lance Bragstad  wrote:
> 
> > Bumping this thread again and proposing two conventions based on the
> > discussion here. I propose we decide on one of the two following
> > conventions:
> >
> > *::*
> >
> > or
> >
> > *:_*
> >
> > Where  is the corresponding service type of the project [0],
> > and  is either create, get, list, update, or delete. I think
> > decoupling the method from the policy name should aid in consistency,
> > regardless of the underlying implementation. The HTTP method specifics can
> > still be relayed using oslo.policy's DocumentedRuleDefault object [1].
> >
> > I think the plurality of the resource should default to what makes sense
> > for the operation being carried out (e.g., list:foobars, create:foobar).
> >
> > I don't mind the first one because it's clear about what the delimiter is
> > and it doesn't look weird when projects have something like:
> >
> > :::
> >

My initial preference was the second format, but you make a good point here
about potential subactions. Either is fine with me - the main thing I would
love to see is consistency in format. But based on this part, I vote for option
2.

> > If folks are ok with this, I can start working on some documentation that
> > explains the motivation for this. Afterward, we can figure out how we want
> > to track this work.
> >

+1 thanks for working on this!


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] Capturing Feedback/Input

[Sean McGinnis]

On Thu, Sep 20, 2018 at 05:30:32PM -0500, Melvin Hillsman wrote:
> Hey everyone,
> 
> During the TC meeting at the PTG we discussed the ideal way to capture
> user-centric feedback; particular from our various groups like SIGs, WGs,
> etc.
> 
> Options that were mentioned ranged from a wiki page to a standalone
> solution like discourse.
> 
> While there is no perfect solution it was determined that Storyboard could
> facilitate this. It would play out where there is a project group
> openstack-uc? and each of the SIGs, WGs, etc would have a project under
> this group; if I am wrong someone else in the room correct me.
> 
> The entire point is a first step (maybe final) in centralizing user-centric
> feedback that does not require any extra overhead be it cost, time, or
> otherwise. Just kicking off a discussion so others have a chance to chime
> in before anyone pulls the plug or pushes the button on anything and we
> settle as a community on what makes sense.
> 
> -- 
> Kind regards,
> 
> Melvin Hillsman

I think Storyboard would be a good place to manage SIG/WG feedback. It will
take some time before the majority of projects have moved over from Launchpad,
but once they do, this will make it much easier to track SIG initiatives all
the way through to code implementation.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Openstack-sigs] Are we ready to put stable/ocata into extended maintenance mode?

[Sean McGinnis]

On Tue, Sep 18, 2018 at 02:27:03PM -0500, Matt Riedemann wrote:
> The release page says Ocata is planned to go into extended maintenance mode
> on Aug 27 [1]. There really isn't much to this except it means we don't do
> releases for Ocata anymore [2]. There is a caveat that project teams that do
> not wish to maintain stable/ocata after this point can immediately end of
> life the branch for their project [3]. We can still run CI using tags, e.g.
> if keystone goes ocata-eol, devstack on stable/ocata can still continue to
> install from stable/ocata for nova and the ocata-eol tag for keystone.
> Having said that, if there is no undue burden on the project team keeping
> the lights on for stable/ocata, I would recommend not tagging the
> stable/ocata branch end of life at this point.
> 
> So, questions that need answering are:
> 
> 1. Should we cut a final release for projects with stable/ocata branches
> before going into extended maintenance mode? I tend to think "yes" to flush
> the queue of backports. In fact, [3] doesn't mention it, but the resolution
> said we'd tag the branch [4] to indicate it has entered the EM phase.
> 
> 2. Are there any projects that would want to skip EM and go directly to EOL
> (yes this feels like a Monopoly question)?
> 
> [1] https://releases.openstack.org/
> [2] 
> https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases
> [3] 
> https://docs.openstack.org/project-team-guide/stable-branches.html#extended-maintenance
> [4] 
> https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.html#end-of-life
> 
> -- 
> 
> Thanks,
> 
> Matt

I have a patch that's been pending for marking it as extended maintenance:

https://review.openstack.org/#/c/598164/

That's just the state for Ocata. You raise some other good points here that I
am curious to see input on.

Sean


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova][cinder][neutron] Cross-cell cold migration

[Sean McGinnis]

> >
> > Yeah it's already on the PTG agenda [1][2]. I started the thread because I
> > wanted to get the ball rolling as early as possible, and with people that
> > won't attend the PTG and/or the Forum, to weigh in on not only the known
> > issues with cross-cell migration but also the things I'm not thinking about.
> >
> > [1] https://etherpad.openstack.org/p/nova-ptg-stein
> > [2] https://etherpad.openstack.org/p/nova-ptg-stein-cells
> >
> > --
> >
> > Thanks,
> >
> > Matt
> >
> 
> Should we also add the topic to the Thursday Cinder-Nova slot in case
> there are some questions where the Cinder team can assist?
> 
> Cheers,
> Gorka.
> 

Good idea. That will be a good time to circle back between the teams to see if
any Cinder needs come up that we can still have time to talk through and see if
we can get work started.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova][cinder][neutron] Cross-cell cold migration

[Sean McGinnis]

On Wed, Aug 22, 2018 at 08:23:41PM -0500, Matt Riedemann wrote:
> Hi everyone,
> 
> I have started an etherpad for cells topics at the Stein PTG [1]. The main
> issue in there right now is dealing with cross-cell cold migration in nova.
> 
> At a high level, I am going off these requirements:
> 
> * Cells can shard across flavors (and hardware type) so operators would like
> to move users off the old flavors/hardware (old cell) to new flavors in a
> new cell.
> 
> * There is network isolation between compute hosts in different cells, so no
> ssh'ing the disk around like we do today. But the image service is global to
> all cells.
> 
> Based on this, for the initial support for cross-cell cold migration, I am
> proposing that we leverage something like shelve offload/unshelve
> masquerading as resize. We shelve offload from the source cell and unshelve
> in the target cell. This should work for both volume-backed and
> non-volume-backed servers (we use snapshots for shelved offloaded
> non-volume-backed servers).
> 
> There are, of course, some complications. The main ones that I need help
> with right now are what happens with volumes and ports attached to the
> server. Today we detach from the source and attach at the target, but that's
> assuming the storage backend and network are available to both hosts
> involved in the move of the server. Will that be the case across cells? I am
> assuming that depends on the network topology (are routed networks being
> used?) and storage backend (routed storage?). If the network and/or storage
> backend are not available across cells, how do we migrate volumes and ports?
> Cinder has a volume migrate API for admins but I do not know how nova would
> know the proper affinity per-cell to migrate the volume to the proper host
> (cinder does not have a routed storage concept like routed provider networks
> in neutron, correct?). And as far as I know, there is no such thing as port
> migration in Neutron.
> 

Just speaking to iSCSI storage, I know some deployments do not route their
storage traffic. If this is the case, then both cells would need to have access
to the same subnet to still access the volume.

I'm also referring to the case where the migration is from one compute host to
another compute host, and not from one storage backend to another storage
backend.

I haven't gone through the workflow, but I thought shelve/unshelve could detach
the volume on shelving and reattach it on unshelve. In that workflow, assuming
the networking is in place to provide the connectivity, the nova compute host
would be connecting to the volume just like any other attach and should work
fine. The unknown or tricky part is making sure that there is the network
connectivity or routing in place for the compute host to be able to log in to
the storage target.

If it's the other scenario mentioned where the volume needs to be migrated from
one storage backend to another storage backend, then that may require a little
more work. The volume would need to be retype'd or migrated (storage migration)
from the original backend to the new backend.

Again, in this scenario at some point there needs to be network connectivity
between cells to copy over that data.

There is no storage-offloaded migration in this situation, so Cinder can't
currently optimize how that data gets from the original volume backend to the
new one. It would require a host copy of all the data on the volume (an often
slow and expensive operation) and it would require that the host doing the data
copy has access to both the original backend and then new backend.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova][cinder] Disabling nova volume-update (aka swap volume; aka cinder live migration)

[Sean McGinnis]

> 
> The solution is conceptually simple.  We add a new API microversion in
> Cinder that adds and optional parameter called "generic_keep_source"
> (defaults to False) to both migrate and retype operations.
> 
> This means that if the driver optimized migration cannot do the
> migration and the generic migration code is the one doing the migration,
> then, instead of our final step being to swap the volume id's and
> deleting the source volume, what we would do is to swap the volume id's
> and move all the snapshots to reference the new volume.  Then we would
> create a user message with the new ID of the volume.
> 

How would you propose to "move all the snapshots to reference the new volume"?
Most storage does not allow a snapshot to be moved from one volume to another.
really the only way a migration of a snapshot can work across all storage types
would be to incrementally copy the data from a source to a destination up to
the point of the oldest snapshot, create a new snapshot on the new volume, then
proceed through until all snapshots have been rebuilt on the new volume.


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Community Documentation - first anchor point

[Sean McGinnis]

On Tue, Aug 21, 2018 at 03:14:48PM -0400, Jonathan Proulx wrote:
> Hi All...
> 
> I'm still a little confused by the state of this :)
> 
> I know I made some promises then got distracted the looks like Sean
> stepped up and got things a bit further, but where is it now?  Do we
> have an active repo?
> 
> It would be nice to have the repo in place before OPs meetup.
> 
> -Jon
> 

Hey Jon,

Pretty much everything is in place now. There is one outstanding patch to
officially add things under the SIG governance here:

https://review.openstack.org/#/c/591248/

That's a formality that needs to be done, but we do have the content being
published to the site:

https://docs.openstack.org/operations-guide/

And we have the repo set up and ready for updates to be proposed:

http://git.openstack.org/cgit/openstack/operations-guide

Our next step is to start encouraging contributions from the community. This
would be a great things to discuss at the PTG, and I now realize that I didn't
add this obvious thing to our ops PTG planning etherpad. I will add it there
and hopefully we can go over some of the content and get some more interest in
contributing to it.

Thanks!
Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Community Documentation - first anchor point

[Sean McGinnis]

> > Plan
> > 
> > So to recap above, I would propose the following actions be taken:
> > 
> > 1. Create sig-operators as a group to manage operator efforts at least 
> > related
> >to what needs to be done in repos.
> > 2. Create an openstack/operations-guide repo to be the new home of the
> >operations documentation.
> 
> One correction to this - that repo already exists. It has been retired, so I
> think the action here would just be to "un-retire" the repo and get things
> updated to start publishing again.
> 
> > 3. Create a new StoryBoard project to help track work in these repos

Update on progress for this:

Step 1
--

For step 1, the reason for creating a SIG is there is a policy that anything
publishing to docs.openstack.org is owned by some sort of official team. I had
proposed an Operator SIG (https://review.openstack.org/578408) but Thierry
rightly points out that the scope of the way I proposed it is perhaps a little
too broad. I wanted to leave room for ownership of other non-documentation
efforts, but perhaps that is not the best plan.

I can change that, but I think the better approach right now is just to see if
the UC is willing to be the owners of this repo since they are already the
owners for a few others:

http://git.openstack.org/cgit/openstack/governance/tree/reference/user-committee-repos.yaml#n14

I will propose that soon.

Step 2
--

I have gone through the infra steps to unretire the openstack/operations-guide
repo and set up docs build jobs to run on proposed patches and a publishing job
to publish merged patches to docs.openstack.org. That should give us
https://docs.openstack.org/operations-guide/ once we merge an update.

I've also restored the content by pulling the latest out of the
openstack-manuals repo just prior to when it was removed. Sorry, but I was not
able to preserve the git history for all of it, but I do not the source of the
content in the commit message:

https://review.openstack.org/578946

We've updated the "core" group that has rights to merge patches for that repo
and Melvin has sent out an email to see if any of the existing members still
want to be involved. Hopefully we can regrow that list over time.

Step 3
--

I do still need to look into the StoryBoard project creation. This is lower
priority than the other tasks, but I will try to get to this step soon.

Thanks,
Sean


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Community Documentation - first anchor point

[Sean McGinnis]

> 
> Plan
> 
> So to recap above, I would propose the following actions be taken:
> 
> 1. Create sig-operators as a group to manage operator efforts at least related
>to what needs to be done in repos.
> 2. Create an openstack/operations-guide repo to be the new home of the
>operations documentation.

One correction to this - that repo already exists. It has been retired, so I
think the action here would just be to "un-retire" the repo and get things
updated to start publishing again.

> 3. Create a new StoryBoard project to help track work in these repos
> x. Document all this.
> 9. Profit!
> 
> I'm willing to work through the steps to get these things set up. Please give
> feedback if this proposed plan makes sense or if there is anything different
> that would be preferred.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Community Documentation - first anchor point

[Sean McGinnis]

Reviving this thread with a fresh start. See below for the original.

To recap, the ops community is willing to take over some of the operator
documentation that is no longer available due to the loss of documentation team
resources. From discussions, there needs to be some official governance over
this operator owned repo (or repos) so it is recommended that a sig be formed.
The repos can be created in the meantime, but consideration needs to be taken
about naming as by default, the repo name is what is reflected in the
documentation publishing location.

SIG Formation
-
There were a couple suggestions on naming and focus for this sig, but I would
like to make a slightly different proposal. I would actually like to see a
sig-operator group formed. We have repos for operator tools and other useful
things and we have a mix of operators, vendors, and others that work together
on things like the ops meetup. I think it would make sense to make this into an
official SIG that could have a broader scope than just documentation.

Docs Repos
--
Doug made a good suggestion that we may want these things published under
something like docs.openstack.org/operations-guide. So based on this, I think
for now at least we should create an opestack/operations-guide repo that will
end up being owned by this SIG. I would expect most documentation generated or
owned by this group would just be located somewhere under that repo, but if the
need arises we can add additional repos.

There are other ops repos out there right now. I would expect the ownership of
those to move under this sig as well, but that is a seperate and less pressing
concern at this point.

Bug Tracking

There should be some way to track tasks and needs for this documentation and
any other repos that are moved under this sig. Since it is the currently
planned direction for all OpenStack projects (or at least there is a vocal
desire for it to be) I think a Storyboard project should be created for this
SIG's activities.

Plan

So to recap above, I would propose the following actions be taken:

1. Create sig-operators as a group to manage operator efforts at least related
   to what needs to be done in repos.
2. Create an openstack/operations-guide repo to be the new home of the
   operations documentation.
3. Create a new StoryBoard project to help track work in these repos
x. Document all this.
9. Profit!

I'm willing to work through the steps to get these things set up. Please give
feedback if this proposed plan makes sense or if there is anything different
that would be preferred.

Thanks,
Sean

On Wed, May 23, 2018 at 06:38:32PM -0700, Chris Morgan wrote:
> Hello Everyone,
> 
> In the Ops Community documentation working session today in Vancouver, we
> made some really good progress (etherpad here:
> https://etherpad.openstack.org/p/YVR-Ops-Community-Docs but not all of the
> good stuff is yet written down).
> 
> In short, we're going to course correct on maintaining the Operators Guide,
> the HA Guide and Architecture Guide, not edit-in-place via the wiki and
> instead try still maintaining them as code, but with a different, new set
> of owners, possibly in a new Ops-focused repo. There was a strong consensus
> that a) code workflow >> wiki workflow and that b) openstack core docs
> tools are just fine.
> 
> There is a lot still to be decided on how where and when, but we do have an
> offer of a rewrite of the HA Guide, as long as the changes will be allowed
> to actually land, so we expect to actually start showing some progress.
> 
> At the end of the session, people wanted to know how to follow along as
> various people work out how to do this... and so for now that place is this
> very email thread. The idea is if the code for those documents goes to live
> in a different repo, or if new contributors turn up, or if a new version we
> will announce/discuss it here until such time as we have a better home for
> this initiative.
> 
> Cheers
> 
> Chris
> 
> -- 
> Chris Morgan 

> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [TC] Stein Goal Selection

[Sean McGinnis]

Adding back the openstack-operators list that Matt added.


On 06/04/2018 05:13 PM, Sean McGinnis wrote:

On 06/04/2018 04:17 PM, Doug Hellmann wrote:

Excerpts from Matt Riedemann's message of 2018-06-04 15:38:48 -0500:

On 6/4/2018 1:07 PM, Sean McGinnis wrote:

Python 3 First
==

One of the things brought up in the session was picking things that 
bring
excitement and are obvious benefits to deployers and users of 
OpenStack
services. While this one is maybe not as immediately obvious, I 
think this
is something that will end up helping deployers and also falls into 
the tech

debt reduction category that will help us move quicker long term.

Python 2 is going away soon, so I think we need something to help 
compel folks
to work on making sure we are ready to transition. This will also 
be a good
point to help switch the mindset over to Python 3 being the default 
used
everywhere, with our Python 2 compatibility being just to continue 
legacy

support.

I still don't really know what this goal means - we have python 3
support across the projects for the most part don't we? Based on that,
this doesn't seem like much to take an entire "goal slot" for the 
release.

We still run docs, linters, functional tests, and other jobs under
python 2 by default. Perhaps a better framing would be to call this
"Python 3 by default", because the point is to change all of those jobs
to use Python 3, and to set up all future jobs using Python 3 unless we
specifically need to run them under Python 2.

This seems like a small thing, but when we did it for Oslo we did find
code issues because the linters apply different rules and we did find
documentation build issues. The fixes were all straightforward, so I
don't expect it to mean a lot of work, but it's more than a single patch
per project. I also think using a goal is a good way to start shifting
the mindset of the contributor base into this new perspective.

Yes, that's probably a better way to word it to properly convey the goal.
Basically, all things running under Python3, project code and tooling, as
the default unless specifically geared towards Python2.


Cold Upgrade Support


The other suggestion in the Forum session related to upgrades was 
the addition
of "upgrade check" CLIs for each project, and I was tempted to 
suggest that as
my second strawman choice. For some projects that would be a very 
minimal or
NOOP check, so it would probably be easy to complete the goal. But 
ultimately
what I think would bring the most value would be the work on 
supporting cold
upgrade, even if it will be more of a stretch for some projects to 
accomplish.

I think you might be mixing two concepts here.
Not so much mixing as discussing the two and the reason why I 
personally thought

the one was a better goal, if you read through what was said about it.


The cold upgrade support, per my understanding, is about getting the
assert:supports-upgrade tag:

https://governance.openstack.org/tc/reference/tags/assert_supports-upgrade.html 



Which to me basically means the project runs a grenade job. There was
discussion in the room about grenade not being a great tool for all
projects, but no one is working on a replacement for that, so I don't
think it's really justification at this point for *not* making it a 
goal.


The "upgrade check" CLIs is a different thing though, which is more
about automating as much of the upgrade release notes as possible. See
the nova docs for examples on how we have used it:

https://docs.openstack.org/nova/latest/cli/nova-status.html

I'm not sure what projects you had in mind when you said, "For some
projects that would be a very minimal or NOOP check, so it would
probably be easy to complete the goal." I would expect that projects
aren't meeting the goal if they are noop'ing everything. But what 
can be

automated like this isn't necessarily black and white either.

What I remember from the discussion in the room was that not all
projects are going to have anything to do by hand that would block
an upgrade, but we still want all projects to have the test command.
That means many of those commands could potentially be no-ops,
right? Unless they're all going to do something like verify the
schema has been updated somehow?


Yes, exactly what I meant by the NOOP. I'm not sure what Cinder would
check here. We don't have to see if placement has been set up or if cell0
has been configured. Maybe once we have the facility in place we would
find some things worth checking, but at present I don't know what that
would be.

Which also makes me wonder, should this be an oslo thing that projects
just plug in to for their specific checks?

Upgrades have been a major focus of discussion lately, especially 
as our
operators have been trying to get closer to the latest work 
upstream. This has

been an ongoing challenge.

There has also been a lot of talk about LTS releases. We

Re: [Openstack-operators] Proposing no Ops Meetups team meeting this week

[Sean McGinnis]

On 05/29/2018 06:14 AM, Chris Morgan wrote:
Some of us will be only just returning to work today after being away 
all week last week for the (successful) OpenStack Summit, therefore I 
propose we skip having a meeting today but regroup next week?


Chris


Makes sense to me. I know I have a lot of catching up to do.


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Strange behaviour change in cinder with a Dell compellent backend

[Sean McGinnis]

On Tue, Apr 24, 2018 at 09:58:26AM +0900, Jean-Philippe Méthot wrote:
> Hi,
> 
> This is a very strange behaviour that has causing me issues with my SAN ever 
> since we upgraded to Mitaka or Ocata I believe, several months ago. 
> Essentially, I used to be able to change the ID of a disk in the SAN to swap 
> the disk in Openstack. So, for example, I had disk 1. I could restore a 
> snapshot of disk 1 on disk 2, rename disk 1 to disk 1-bak and rename disk 2 
> to disk 1 and the VM would start booting off of the new disk I had just made. 
> This has changed. Now, somehow, Openstack sticks to the original disk even if 
> I rename the disk. 
> 
> While annoying, this behaviour didn’t really cause me that many issues. 
> However, I have discovered that if I migrate VMs on which such an operation 
> happened, the VM will try to boot off the original disk from several months 
> ago, despite the new disk being there with the correct ID. How can Openstack 
> find the old disk even if its ID in the SAN has changed? 
> 

If I remember right, this was a fix in the driver to be able to track the
volume by its native array ID and not its name. This is how things should work.
Reliance on the name of the volume is not safe, and as seen here, can be
misused to do things that are not really supported and can cause some
unintended side effects.

You can update the database to get the same (mis)behavior you were using, but I
am not suggesting that is a good thing to do.


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] 4K block size

[Sean McGinnis]

On Mon, Apr 23, 2018 at 05:46:40PM +, Tim Bell wrote:
> 
> Has anyone experience of working with local disks or volumes with 
> physical/logical block sizes of 4K rather than 512?
> 
> There seems to be KVM support for this 
> (http://fibrevillage.com/sysadmin/216-how-to-make-qemu-kvm-accept-4k-sector-sized-disks)
>  but I could not see how to get the appropriate flavors/volumes in an 
> OpenStack environment?
> 
> Is there any performance improvement from moving to 4K rather than 512 byte 
> sectors?
> 

I haven't seen much of a performance difference between drives with one
exception that I don't think will apply here. For backward compatiblity, there
is something that is called "512e mode". This basically takes a 4k sector size
and, using software abstraction, presents it to the host as a 512 byte sector
drive. So with this abstraction being done in between, there can be a slight
performance hit as things are translated.

As far as I know, you shouldn't need to have specific volume types and the use
of these drives should be transparent to the upper layers. At least coming from
the Cinder side. I'm not sure if there are any special considerations on the
Nova side though, so it would be great to hear from anyone that has any
experience with this and Nova.

I did find a decent write up on some of the differences with 4k vs 512. It is
focusing on SQL workloads, but I think the basics are generally applicable to
most workloads:

http://en.community.dell.com/techcenter/enterprise-solutions/w/sql_solutions/12102.performance-comparison-between-4k-and-512e-hard-drives

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Meetup, Co-Location options, and User Feedback

[Sean McGinnis]

On Mon, Apr 02, 2018 at 03:15:56PM -0500, Melvin Hillsman wrote:
> Unless anyone has any objections I believe we have quorum Jimmy.
> 

I agree, I think the feedback I've heard so far is that all parties are willing
to give this a shot. I think we should go ahead.

> On Mon, Apr 2, 2018 at 12:53 PM, Melvin Hillsman 
> wrote:
> 
> > +1
> >
> > On Mon, Apr 2, 2018 at 11:39 AM, Jimmy McArthur 
> > wrote:
> >
> >> Hi all -
> >>
> >> I'd like to check in to see if we've come to a consensus on the
> >> colocation of the Ops Meetup.  Please let us know as soon as possible as we
> >> have to alert our events team.
> >>
> >> Thanks!
> >> Jimmy

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] RFC: Next minimum libvirt / QEMU versions for "Solar" release

[Sean McGinnis]

> While at it, we should also discuss about what will be the NEXT_MIN   
>  
> libvirt and QEMU versions for the "Solar" release.  To that end, I've 
>  
> spent going through different distributions and updated the   
>  
> DistroSupportMatrix Wiki[2].   
> 
> Taking the DistroSupportMatrix into picture, for the sake of discussion,
> how about the following NEXT_MIN versions for "Solar" release:
>  
> 
Correction - for the "Stein" release. :)


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] OpenStack User Survey: Identity Service, Networking and Block Storage Drivers Answer Options

[Sean McGinnis]

Hey Allison,

I have a few comments below about the Cinder drivers. Would love to hear
everyone's input too.


On Thu, Mar 29, 2018 at 12:22:05PM -0500, Allison Price wrote:
> Hi everyone,
> 
> We are opening the OpenStack User Survey submission process next month and 
> wanted to collect operator feedback on the answer choices for three 
> particular questions: Identity Service (Keystone) drivers, Network (Neutron) 
> drivers and Block Storage (Cinder) drivers. We want to make sure that we have 
> a list of the most commonly used drivers so that we can collect the 
> appropriate data from OpenStack users. Each of the questions will have a free 
> text “Other” option, so they don’t need to be comprehensive, but if you think 
> that there is a driver that should be included, please reply on this email 
> thread or contact me directly. 
> 
> Thanks!
> Allison


> Which OpenStack Block Storage (Cinder) drivers are you using? 
> Ceph RBD
Coraid - This was removed from Cinder several releases ago after only a short 
time in-tree.
Dell EqualLogic - May want to indicate an alias for its current name - Dell EMC 
PS Series
EMC - This one may be confusing as there are several EMC drivers, and now that 
it is Dell EMC, a couple more (see previous entry)
GlusterFS - This was also removed, by Red Hat. But that may be a good reason to 
include it to see if anyone is still using it.
> HDS
> HP 3PAR
> HP LeftHand
> Huawei
> IBM GPFS
> IBM NAS
> IBM Storwize
> IBM XIV / DS8000
> LVM (default) 
> Mellanox
> NetApp
> Nexenta 
> NFS
> ProphetStor
SAN / Solaris - Not sure what this is.
> Scality
> Sheepdog
SolidFire - now one of several NetApp drivers
> VMware VMDK
Windows Server 2012 - SMBFS driver?
Xenapi NFS - no idea what this is.
XenAPI Storage Manager - Or this one.
> Zadara 
> Other 

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Tokyo Ops Meetup - attendees muster!

[Sean McGinnis]

I have both set up already on my phone. Either WhatsApp or Hangouts work fine 
for me.

> On Mar 5, 2018, at 23:31, Shintaro Mizuno  
> wrote:
> 
> Hi Chris,
> 
> Hangout, WhatsApp is fine, too :)


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Tokyo Ops Meetup - attendees muster!

[Sean McGinnis]

I’m at the Hotel Gracery. Really rather not install slack, but if that works 
well for everyone then that’s fine.

I’ll probably wander around the area to find dinner tonight. If I don’t meet up 
with others, see you at the meet up tomorrow.

Sean

> On Mar 5, 2018, at 22:14, Shintaro Mizuno  
> wrote:
> 
> Hi Chris,
> 
> Welcome to Tokyo!
> I hope your flight didn't get affected by the storm we had last night.
> 
> We had a slack channel in MEX OpsMeetup and it worked well for general 
> announcement for attendees.
> Etherpad may be enough for the purpose, but with my phone, slack works better.
> 
> Regards,
> Shintaro
> 
> On 2018/03/06 11:04, Chris Morgan wrote:
>> I’m in Tokyo for the Ops Meetup tomorrow and I wonder who else is around.  I 
>> was wondering if people want to make a discussion or chat room? I’ve got 
>> WhatsApp, Slack,  Google Hangouts etc or we can just respond to this thread ?
>> 
>> I’m at hotel villa fontaine across the street from granpark
>> 
>> Chris
>> 
>> Sent from my iPhone
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> 
> 
> 
> -- 
> Shintaro MIZUNO (水野伸太郎)
> NTT Software Innovation Center
> TEL: 0422-59-4977
> E-mail: mizuno.shint...@lab.ntt.co.jp
> 
> 
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [openstack-dev] [all] TL; DR: Switching to longer dev cycles

[Sean McGinnis]

Hey everyone,

There was some discussion about this in the operator community, so I just
wanted to make sure folks were aware of this recap that Thierry did. I think it
nicely captures and summarizes some of the issues brought up in the long thread
in openstack-dev.

Thanks,
Sean

- Forwarded message from Thierry Carrez  -

Date: Mon, 18 Dec 2017 15:08:45 +0100
From: Thierry Carrez 
To: OpenStack Development Mailing List 
Subject: [openstack-dev] [all] TL;DR: Switching to longer dev cycles ?
Reply-To: "OpenStack Development Mailing List (not for usage questions)"

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 
Thunderbird/52.5.0

A lot of people chose to stay away from the 118-message long thread and
wish there was a summary of it and the discussions on IRC around it.

Mike posted one on the -dev digest:

https://www.openstack.org/blog/2017/12/developer-mailing-list-digest-december-9-15th/

Here is my try at summarizing, in hope it will be helpful.

First, you should read the initial proposal at:

http://lists.openstack.org/pipermail/openstack-dev/2017-December/125473.html


Most of the +1s appear to come from people that would welcome less of a
"taxing treadmill" (in jgriffith's words). A significant share of them
are in the packaging space, where the quantity of work is more directly
correlated to the release cadence.


A lot of objections were posted on the ML or on IRC (and for the record,
I agree with most of them):

1/ Treating the symptom rather than underlying cause

While increasing overall cycle length would ease the pain resulting from
a number of issues, it does not directly treat the underlying causes,
and it has other consequences that may not be as beneficial. The most
obvious of those being that we'd get less done overall, due to having
less forced rhythm/deadlines.

2/ Nobody would consume intermediary releases, resulting in longer
feedback loops

The proposal encourages releasing intermediary releases more often, as a
way to keep releasing early and often. However, for most projects, users
would likely wait for the "real" releases (those with proper feature
freezes, stabilization period and stable branches for post-release
bugfixes) rather than use the intermediaries. This would likely result
in a deteriorated longer feedback loop, with devs waiting up to one year
to get real-world usage / bugs on a feature they just added. So 6-month
"real" releases might still be the right sweet spot between what we can
deliver and a reasonable feedback loop length.

3/ The change would not simplify the life of part-time contributors

The main driver for the proposal is to make OpenStack development more
doable by people who can only spend 20% of their work time on OpenStack
upstream development, as our current pace can be overwhelming. However,
most of the difficulty to keep up is linked to change velocity, and
change velocity is not directly driven by cycle length, so it may stay
mostly the same. If a change is too big to land in 6-month work by a
part-time contributor, it can be split. And if a change misses the boat,
it's a one-year wait rather than a 6-month one.

4/ The overhead of cycle events is not that overwhelming

One goal of the proposal is to reduce the amount of time spent in a year
around development cycle events and process (PTL election, PTG
preparation, feature freeze, release process...) in order to get some
extra time back. Some objected that these don't take that much time,
that feature freeze or PTG prep are actually good things to do, or that
boilerplate can be reconsidered without lengthening the cycle. We could,
for example, only run one PTL election per year without increasing the
dev cycle length.

5/ Most issues that the proposal aims to solve can be solved with better
project management

Getting more realistic at planning the work that can get done within a
cycle would go a long way to make people feel better about the available
time they have. Actively engaging with part-time contributors and making
them feel welcome and valuable would probably drive more direct results
than an extrinsic change like increasing the cycle length. This may
point to the need to spend more time together in a productive
environment like the PTG rather than less.

6/ Larger events are easier to justify and get to

For a number of people it is easier to justify traveling to PTGs than to
smaller midcycles team meetups. Removing one PTG might therefore limit
valuable face-to-face time for team members. It would definitely reduce
the amount of time we spend on cross-project collaboration.

7/ Focus on the real solutions

Fast-forward upgrades (and supporting selected branches past their end
of life) are the real solutions for helping users and vendors keep up
with the rate of releases, not making less of them. We should focus on
that, and the 

Re: [Openstack-operators] thierry's longer dev cycle proposal

[Sean McGinnis]

Would be great to get ops-side input. I didn't want to cross-post because I'm
sure this is going to be a big thread and go on for a while. But I would
encourage anyone with input to jump in on that thread. We could also discuss it
separately here and I can try to answer questions or feed that input back in to
the -dev side.

Sean

On Wed, Dec 13, 2017 at 11:48:01AM -0700, David Medberry wrote:
> Hi all,
> 
> Please read Thierry's email to the openstack-dev list this morning and
> follow the thread (getting long already just two hours in.)
> 
> This references some ideas and concerns that have come from the Ops
> community, but this is specifically a -dev thread (but I suspect a lot of
> ramifications for ops as well.)
> 
> http://lists.openstack.org/pipermail/openstack-dev/2017-December/125473.html
> 
> title is:
> Switching to longer development cycles
> (and if you are sub'd to openstack-dev you will find it in there.)
> 
> 
> The thread of emails is at least 10+ deep already with folks weighing in on
> all sides of the aisle.
> 
> -dave

> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Ops Meetups team meeting 2017-10-24

[Sean McGinnis]

Hey Chris,

Sorry, I know I had an action from last week, but I had a conflict so couldn't
attend this week's meeting.

Just to give a quick update, there is still ongoing discussion around what
should be the policy for a deployment project to be considered to be "following
stable policy". There is a patch up as one possibility here:

https://review.openstack.org/#/c/511968/

This is also going to be discussed at the Summit. I'm actually not sure if this
will be its own topic (there are still some open slots if it needs to be), but
it will probably at least be discussed a little here:

https://www.openstack.org/summit/sydney-2017/summit-schedule/events/20456/what-do-operators-want-from-the-stable-policy

Either way, I would really encourage any ops focused attendees to make it to
that session. I think the name says it all. ;)

Thanks,
Sean

On Tue, Oct 24, 2017 at 02:02:24PM -0400, Chris Morgan wrote:
> Meeting minutes here:
> 
> Meeting ended Tue Oct 24 15:00:00 2017 UTC. Information about MeetBot at
> http://wiki.debian.org/MeetBot . (v 0.1.4)
> 11:00 AM Minutes:
> http://eavesdrop.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-10-24-14.00.html
> 11:00 AM Minutes (text):
> http://eavesdrop.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-10-24-14.00.txt
> 11:00 AM Log:
> http://eavesdrop.openstack.org/meetings/ops_meetup_team/2017/ops_meetup_team.2017-10-24-14.00.log.html
> 
> next week, amongst other topics, we are planning to discuss whether future
> mid-cycle meetups should be co-located with PTG. Please join if you have an
> opinion you wish to express on this (either way!).
> 
> Chris
> -- 
> Chris Morgan 

> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [cinder] Cant create Volume from Image, says checksums dont match

[Sean McGinnis]

On Thu, Oct 19, 2017 at 06:55:38PM -0700, Christopher Hull wrote:
> The qcow2 checksums are correct.  They run via nova.
> 
> I seem to recall that the cinder checksum calculation when reading in an
> image is faulty.  I'm simply going to remove the offending code.
> -Chris
> 

I have not heard this before. If you remove that check and verify everything is
working without it, it would be useful to post those results here.

I didn't see an open bug for this. If you could file a bug and include any log
files and details, that would be appreciated:

https://bugs.launchpad.net/cinder

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] Cinder v1 API Removal

[Sean McGinnis]

Just a heads up for anyone consuming Cinder APIs. The v1 API was 
deprecated in the Juno release, but
we've kept it around for quite awhile because we knew there were client 
implementations out there that

were lagging in getting up to date.

Well, it's now been many releases, and we've had the v1 API disabled by 
default for awhile now. Patch
https://review.openstack.org/#/c/499342/ now removes that API. Queens 
onward, it will not be possible

to run with V1 anymore.

The v3 API is the currently supported version. All tools and 
integrations should now be targeting that API.
This will hopefully be the final API version going forward, with all 
updates and changes being implemented

as microversions under the v3.0 version.

Thanks,

Sean


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [cinder] OpenStack Pike for Ubuntu 16.04 LTS

[Sean McGinnis]

First - yay, awesome work. Glad to see this made available quickly.
Nice work.

But second - are you aware of deployment issues with Cinder API
with these packages? I've had a report from someone on IRC that
they deployed their environment using these and they are getting
an error because it is not finding api-paste.ini. From just looking at
the logs, looks like it is not getting the full path to the file. The file
itself is at the expected location.


On 09/04/2017 11:08 AM, James Page wrote:

(Apologies for the slight lag from Fridays actual release - end user email
failure...)

Hi All,

The Ubuntu OpenStack team at Canonical is pleased to announce the
general availability of OpenStack Pike for Ubuntu 16.04 LTS via the
Ubuntu Cloud Archive.

Details of the Pike release can be found at [0].

# Ubuntu 16.04 LTS

You can enable the Ubuntu Cloud Archive pocket for OpenStack Pike
on Ubuntu 16.04 LTS installations by running the following commands:

sudo add-apt-repository cloud-archive:pike
sudo apt update

The Ubuntu Cloud Archive for Pike includes updates for:

aodh, barbican, ceilometer, ceph (12.2.0 Luminous), cinder, congress,
designate, designate-dashboard, dpdk (17.05.1), glance, gnocchi, heat,
horizon, ironic, libvirt (3.6.0), keystone, magnum, manila, manila-ui,
mistral, murano, murano-dashboard, networking-ovn, networking-sfc,
neutron, neutron-dynamic-routing, neutron-fwaas, neutron-lbaas,
neutron-lbaas-dashboard, nova, nova-lxd, openstack-trove,
openvswitch (2.8.0 pre-release), panko, qemu (2.10), sahara,
sahara-dashboard, senlin, swift, trove-dashboard, watcher and zaqar

Open vSwitch will be updated to the 2.8.0 release as soon as it’s
available.

For a full list of packages and versions, please refer to [1].

# Branch Package Builds

If you would like to try out the latest updates to git branches, we
deliver continuously integrated packages on each upstream
commit via the following PPA’s:

sudo add-apt-repository ppa:openstack-ubuntu-testing/newton
sudo add-apt-repository ppa:openstack-ubuntu-testing/ocata
sudo add-apt-repository ppa:openstack-ubuntu-testing/pike

# Reporting Bugs

If you have any issues please report bugs using the 'ubuntu-bug'
tool to ensure that bugs get logged in the right place in Launchpad:

sudo ubuntu-bug nova-conductor
Thanks to everyone who has contributed to OpenStack Pike, both
upstream and downstream!

Have fun and see you all for Queens!

Regards,

James

(on behalf of the Ubuntu OpenStack team)

[0] https://releases.openstack.org/pike/
[1] 
http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/pike_versions.html




___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] cinder/nova issues

[Sean McGinnis]

Hey Adam,

There have been some updates since Liberty to improve handling in the os-brick
library that handles the local device management. But with this showing the
paths down, I wonder if there's something else going on there between the
NetApp box and the Nova compute host.

Could you file a bug to track this? I think you could just copy and paste the
content of your original email since it captures a lot of great info.

https://bugs.launchpad.net/cinder/+filebug

We can tag it with netapp so maybe it will get some attention there.

Thanks,
Sean

On Wed, Aug 23, 2017 at 01:01:24PM -0400, Adam Dibiase wrote:
> Greetings,
> 
> I am having an issue with nova starting an instance that is using a root
> volume that cinder has extended. More specifically, a volume that has been
> extended past the max resize limit of our Netapp filer. I am running
> Liberty and upgraded cinder packages to 7.0.3 from 7.0.0 to take advantage
> of this functionality. From what I can gather, it uses sub-lun cloning to
> get past the hard limit set by Netapp when cloning past 64G (starting from
> a 4G volume).
> 
> *Environment*:
> 
>- Release: Liberty
>- Filer:   Netapp
>- Protocol: Fiberchannel
>- Multipath: yes
> 
> 
> 
> *Steps to reproduce: *
> 
>- Create new instance
>- stop instance
>- extend the volume by running the following commands:
>   - cinder reset-state --state available (volume-ID or name)
>   - cinder extend (volume-ID or name) 100
>   - cinder reset-state --state in-use (volume-ID or name)
>- start instance with either nova start or nova reboot --hard  --same
>result
> 
> 
> I can see that the instance's multipath status is good before the resize...
> 
> *360a98000417643556a2b496d58665473 dm-17 NETAPP  ,LUN *
> 
> size=20G features='1 queue_if_no_path' hwhandler='0' wp=rw
> 
> |-+- policy='round-robin 0' prio=-1 status=active
> 
> | |- 6:0:1:5 sdy   65:128 active undef  running
> 
> | `- 7:0:0:5 sdz   65:144 active undef  running
> 
> `-+- policy='round-robin 0' prio=-1 status=enabled
> 
>   |- 6:0:0:5 sdx   65:112 active undef  running
> 
>   `- 7:0:1:5 sdaa  65:160 active undef  running
> 
> 
> Once the volume is resized, the lun goes to a failed state and it does not
> show the new size:
> 
> 
> *360a98000417643556a2b496d58665473 dm-17 NETAPP  ,LUN *
> 
> size=20G features='1 queue_if_no_path' hwhandler='0' wp=rw
> 
> |-+- policy='round-robin 0' prio=-1 status=enabled
> 
> | |- 6:0:1:5 sdy   65:128 failed undef  running
> 
> | `- 7:0:0:5 sdz   65:144 failed undef  running
> 
> `-+- policy='round-robin 0' prio=-1 status=enabled
> 
>   |- 6:0:0:5 sdx   65:112 failed undef  running
> 
>   `- 7:0:1:5 sdaa  65:160 failed undef  running
> 
> 
> Like I said, this only happens on volumes that have been extended past 64G.
> Smaller sizes to not have this issue. I can only assume that the original
> lun is getting destroyed after the clone process and that is cause of the
> failed state. Why is it not picking up the new one and attaching it to the
> compute node?  Is there something I am missing?
> 
> Thanks in advance,
> 
> Adam

> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Experience with Cinder volumes as root disks?

[Sean McGinnis]

> 
> >·What has been your experience with this; any advice?
> 
> It works fine.  With Horizon you can do it in one step (select the image but
> tell it to boot from volume) but with the CLI I think you need two steps
> (make the volume from the image, then boot from the volume).  The extra
> steps are a moot point if you are booting programmatically (from a custom
> script or something like heat).
> 

One thing to keep in mind when using Horizon for this - there's currently
no way in Horizon to specify the volume type you would like to use for
creating this boot volume. So it will always only use the default volume
type.

That may be fine if you only have one, but if you have multiple backends,
or multiple settings controlled by volume types, then you will probably
want to use the CLI method for creating your boot volumes.

There has been some discussion about creating a Nova driver to just use
Cinder for ephemeral storage. There are some design challenges with how
to best implement that, but if operators are interested, it would be
great to hear that at the Forum and elsewhere so we can help raise the
priority of that between teams.

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Experience with Cinder volumes as root disks?

[Sean McGinnis]

One other thing to think about - I think at least starting with the Mitaka
release, we added a feature called image volume cache. So if you create a
boot volume, the first time you do so it takes some time as the image is
pulled down and written to the backend volume.

With image volume cache enabled, that still happens on the first volume
creation of the image. But then any subsequent volume creations on that
backend for that image will be much, much faster.

This is something that needs to be configured. Details can be found here:

https://docs.openstack.org/cinder/latest/admin/blockstorage-image-volume-cache.html

Sean

On Tue, Aug 01, 2017 at 10:47:26AM -0500, Sean McGinnis wrote:
> On Tue, Aug 01, 2017 at 11:14:03AM -0400, John Petrini wrote:
> > 
> > On the plus side for ephemeral storage; resizing the root disk of images
> > works better. As long as your image is configured properly it's just a
> > matter of initiating a resize and letting the instance reboot to grow the
> > root disk. When using volumes as your root disk you instead have to
> > shutdown the instance, grow the volume and boot.
> > 
> 
> Some sort of good news there. Starting with the Pike release, you will now
> be able to extend an attached volume. As long as both Cinder and Nova are
> at Pike or later, this should now be allowed.
> 
> Sean
> 
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Experience with Cinder volumes as root disks?

[Sean McGinnis]

On Tue, Aug 01, 2017 at 11:14:03AM -0400, John Petrini wrote:
> 
> On the plus side for ephemeral storage; resizing the root disk of images
> works better. As long as your image is configured properly it's just a
> matter of initiating a resize and letting the instance reboot to grow the
> root disk. When using volumes as your root disk you instead have to
> shutdown the instance, grow the volume and boot.
> 

Some sort of good news there. Starting with the Pike release, you will now
be able to extend an attached volume. As long as both Cinder and Nova are
at Pike or later, this should now be allowed.

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] UTC 14:00 henceforth for Ops Meet Up planning

[Sean McGinnis]

On Tue, May 23, 2017 at 09:16:13AM -0600, David Medberry wrote:
> I have picked "Tuesday, May 30, 2017 8:00 AM (Time zone: Mountain Time)" as
> final option(s) for the Doodle poll "Ops Meetup Preferred Time."

Hey David,

Sorry, I'm sure this was stated elsewhere, but where is this meeting held?

Thanks!
Sean


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] DB deadlocks due to connection string

[Sean McGinnis]

Just wanted to put this out there to hopefully spread awareness and
prevent it from happening more.

We had a bug reported in Cinder of hitting a deadlock when attempting
to deelte multiple volumes simultaneously:

https://bugs.launchpad.net/cinder/+bug/1685818

Some were seeing it, but others were not able to reproduce the error
in their environments.

What it came down to is the use of "mysql://" vs "mysql+pymysql://"
for the database connection string. Big thanks to Gerhard Muntingh
for noticing this difference.

Basically, when using "mysql://" for the connection string, that uses
blocking calls that prevent other "threads" from running at the same
time, causing these deadlocks.

This doesn't just impact Cinder, so I wanted to get the word out that
it may be worth checking your configurations and make sure you are
using "mysql+pymysql://" for your connections.

Sean


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Milan Ops Midcycle - Cinder session

[Sean McGinnis]

Thank you to everyone that attended the Cinder session at the ops
midcycle. I found it very helpful getting some feedback and hearing
concerns, and I hope everyone else was able to take away something
useful from the session and the event.

There wasn't much, but a few questions in the etherpad (link below) were
expanded on or answered.

Feel free to add more notes in the etherpad, and definitely feel free to
reach out to me directly if there are any Cinder related issues you have
questions or concerns about.

Thanks again to all attendees, and to everyone involved in hosting and
organizing the event. I found it well worth the trip.

Sean

On Mon, Mar 13, 2017 at 01:35:22PM -0500, Sean McGinnis wrote:
> The start of the Cinder session etherpad is available here:
> 
> https://etherpad.openstack.org/p/MIL-ops-cinder-rolling-upgrade
> 
> Please add whatever info you would like to it.
> 
> I think the main interest was in rolling upgrades, but feel free
> to add any other general Cinder topics you would like to discuss
> to the end and we can see how far we can get.
> 
> Thanks!
> 
> Sean
> 
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] Milan Ops Midcycle - Cinder session

[Sean McGinnis]

The start of the Cinder session etherpad is available here:

https://etherpad.openstack.org/p/MIL-ops-cinder-rolling-upgrade

Please add whatever info you would like to it.

I think the main interest was in rolling upgrades, but feel free
to add any other general Cinder topics you would like to discuss
to the end and we can see how far we can get.

Thanks!

Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] DRBD Cinder Driver - Potential Removal

[Sean McGinnis]

This is just a heads up for any operators that are interested. No
immediate action is being taken (at the moment), but that could change.

There was a discussion started on the openstack-dev mailing list
pointing out that the library "drbdmanage" has had its license changed.
It is no longer considered a free and open source library under the
OpenStack guidelines.

http://lists.openstack.org/pipermail/openstack-dev/2016-December/108738.html

Due to this, it no longer meets the requirements for having continuous
integration (CI) testing run against it by OpenStack infrastructure.

One of the requirements for having a driver included in Cinder is
keeping a stable CI system running to validate all patches against an
actual backend setup. If a new CI is not able to be set up to validate
this driver, we will need to mark it as not supported and remove it in
the next cycle.

Again, nothing is being done immediately, and the discussion is
continuing on that mailing list thread. This is just a notification for
anyone interested to make sure no one is caught off guard.

Thanks,
Sean

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [nova] How to expose the compute node local disks to instances

[Sean McGinnis]

Hi Belmiro,

In Cinder there is the "raw disk device" driver that has been used by some
for Hadoop and similar applications. That may be something to look at, but
with a big warning.

That being that it is deprecated in the Ocata release and will be removed
in Pike.

The reason it is going to be removed has a couple reasons though. One big
one is that it is unable to meet the "minimum feature set" that we require
for a Cinder driver. There are just too many restrictions on what can be
done when you're using raw disk.

The other reason was that after running a set of tests, it we really didn't
see that much of a difference in uing the raw device driver vs. configuring
the LVM driver appropriately. I don't have the full details of those tests
handy at the moment, but I'm sure we can track down some specifics if needed.

So I would recommend connecting your JBODs to some compute hosts and using
that disk space to back LVM, then configure Cinder with the LVM driver to
use that pool of space to present volumes to your compute instances.

Thanks,
Sean

On Thu, Dec 08, 2016 at 07:46:35PM +0100, Belmiro Moreira wrote:
> Hi,
> 
> we have a set of disk servers (JBOD) that we would like to integrate into
> our cloud to run   applications like Hadoop and Spark.
> 
> Using file disks for storage and a huge "/var/lib/nova" is not an option
> for these use cases so we would like to expose the local drives directly to
> the VMs as ephemeral disks.
> 
> 
> 
> Is there already something in Nova that allows this?
> 
> 
> 
> thanks,
> 
> Belmiro
> 
> CERN

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators