Re: [Openstack-operators] Way to check compute - rabbitmq connectivity

2015-01-15 Thread Sam Morrison
We’ve had a lot of issues with Icehouse related to rabbitMQ. Basically the change from openstack.rpc to oslo.messaging broke things. These things are now fixed in oslo.messaging version 1.5.1, there is still an issue with heartbeats and that patch is making it’s way through review process now.

[Openstack-operators] Specifying multiple tenants for aggregate_multitenancy_isolation_filter

2015-01-27 Thread Sam Morrison
Hi operators, I have a review up to fix this filter to allow multiple tenants, there are 2 proposed ways in which this can be specified. 1. using a comma e.g., tenantid1,tenantid2 2. Using a json list eg. [“tenantid1”, “tenantid2”] Which one do you think is better?

Re: [Openstack-operators] FYI: Rabbit Heartbeat Patch Landed

2015-03-20 Thread Sam Morrison
I’ve been trying to build a ubuntu deb of this in a juno environment. It’s a bit of a nightmare as they have changed all the module names from oslo.XXX to oslo_XXX Have fixed those up with a few sed replaces and had to remove support for aioeventlet as the dependencies aren’t in the ubuntu

Re: [Openstack-operators] max_age and until_refresh for fixing Nova quotas

2015-03-20 Thread Sam Morrison
We’ve had the following for a year or so but doesn’t help much, we still see it occurring every 10 mins or so. max_age = 10 until_refresh = 5 reservation_expire=600 We have a cron job that runs every 10 mins that figures out what projects are out of sync and corrects them. We’ve always been

Re: [Openstack-operators] max_age and until_refresh for fixing Nova quotas

2015-03-20 Thread Sam Morrison
single reservation for most users, right? Mike On 3/20/15, 4:18 AM, Sam Morrison sorri...@gmail.com wrote: We’ve had the following for a year or so but doesn’t help much, we still see it occurring every 10 mins or so. max_age = 10 until_refresh = 5 reservation_expire=600 We

Re: [Openstack-operators] max_age and until_refresh for fixing Nova quotas

2015-03-20 Thread Sam Morrison
, at 12:35 am, Kris G. Lindgren klindg...@godaddy.com wrote: Can you post of you cronjob/script that you use to correct the quotas? Kris Lindgren Senior Linux Systems Engineer GoDaddy, LLC. On 3/20/15, 4:18 AM, Sam Morrison sorri

Re: [Openstack-operators] FYI: Rabbit Heartbeat Patch Landed

2015-05-03 Thread Sam Morrison
am, Sam Morrison sorri...@gmail.com wrote: We’re running: kombu: 3.0.7 amqp: 1.4.5 rabbitmq, 3.3.5 erlang: R14B04 On 2 May 2015, at 1:51 am, Kris G. Lindgren klindg...@godaddy.com wrote: We are running: kombu 3.0.24 amqp 1.4.6 rabbitmq 3.4.0 erlang R16B-03.10

Re: [Openstack-operators] FYI: Rabbit Heartbeat Patch Landed

2015-05-03 Thread Sam Morrison
juno - kilo) On 4 May 2015, at 9:08 am, Sam Morrison sorri...@gmail.com wrote: We’re running: kombu: 3.0.7 amqp: 1.4.5 rabbitmq, 3.3.5 erlang: R14B04 On 2 May 2015, at 1:51 am, Kris G. Lindgren klindg...@godaddy.com wrote: We are running: kombu 3.0.24 amqp 1.4.6 rabbitmq

Re: [Openstack-operators] FYI: Rabbit Heartbeat Patch Landed

2015-04-30 Thread Sam Morrison
going to reach out to the Ubuntu Server (aka Cloud) team and so if they can throw up a PPA with this for Juno quickly (which they will likely NOT do but it doesn't hurt to ask.) We need to get the stable/juno team on board with this backport/regression. On Fri, Mar 20, 2015 at 4:14 AM, Sam

Re: [Openstack-operators] Flat network with linux bridge plugin

2015-04-08 Thread Sam Morrison
From: daniel.com...@gmail.com mailto:daniel.com...@gmail.com [mailto:daniel.com...@gmail.com mailto:daniel.com...@gmail.com] On Behalf Of Daniel Comnea Sent: Wednesday 08 April 2015 11:29 To: Daniele Venzano Cc: Sam Morrison; openstack-operators@lists.openstack.org mailto:openstack

Re: [Openstack-operators] Help improve Neutron!

2015-06-22 Thread Sam Morrison
Hi Piet, Is this a UX focused interview? Sam On 23 Jun 2015, at 6:04 am, Kruithof, Piet pieter.c.kruithof...@hp.com wrote: Hi Folks, The OpenStack UX team is looking for six people that would be willing to participate in a one hour interview. We’re specifically looking for folks

Re: [Openstack-operators] Help improve Neutron!

2015-06-22 Thread Sam Morrison
On 6/22/15, 5:54 PM, Sam Morrison sorri...@gmail.com wrote: Hi Piet, Is this a UX focused interview? Sam On 23 Jun 2015, at 6:04 am, Kruithof, Piet pieter.c.kruithof...@hp.com wrote: Hi Folks, The OpenStack UX team is looking for six people that would be willing to participate

Re: [Openstack-operators] [openstack-dev] [nova] [neutron] Re: How do your end users use networking?

2015-06-17 Thread Sam Morrison
On 17 Jun 2015, at 8:35 pm, Neil Jerram neil.jer...@metaswitch.com wrote: Hi Sam, On 17/06/15 01:31, Sam Morrison wrote: We at NeCTAR are starting the transition to neutron from nova-net and neutron almost does what we want. We have 10 “public networks and 10 “service networks

Re: [Openstack-operators] [openstack-dev] [nova] [neutron] Re: How do your end users use networking?

2015-06-16 Thread Sam Morrison
On 17 Jun 2015, at 10:56 am, Armando M. arma...@gmail.com wrote: On 16 June 2015 at 17:31, Sam Morrison sorri...@gmail.com mailto:sorri...@gmail.com wrote: We at NeCTAR are starting the transition to neutron from nova-net and neutron almost does what we want. We have 10 “public

Re: [Openstack-operators] [nova] [neutron] Re: How do your end users use networking?

2015-06-16 Thread Sam Morrison
We at NeCTAR are starting the transition to neutron from nova-net and neutron almost does what we want. We have 10 “public networks and 10 “service networks and depending on which compute node you land on you get attached to one of them. In neutron speak we have multiple shared externally

Re: [Openstack-operators] [keystone][all] Deprecating slash ('/') in project names

2015-07-06 Thread Sam Morrison
Do you mean project names or project IDs? Sam On 3 Jul 2015, at 12:12 am, Henrique Truta henriquecostatr...@gmail.com wrote: Hi everyone, In Kilo, keystone introduced the concept of Hierarchical Multitenancy[1], which allows cloud operators to organize projects in hierarchies. This

[Openstack-operators] [puppet] module dependencies and different openstack versions

2015-07-27 Thread Sam Morrison
We currently use our own custom puppet modules to deploy openstack, I have been looking into the official openstack modules and have a few barriers to switching. We are looking at doing this at a project at a time but the modules have a lot of dependencies. Eg. they all depend on the keystone

Re: [Openstack-operators] Informal Ops Meetup?

2015-10-29 Thread Sam Morrison
I’ll be there, talked to Tom too and he said there may be a room we can use else there is plenty of space around the dev lounge to use. See you tomorrow. Sam > On 29 Oct 2015, at 6:02 PM, Xav Paice wrote: > > Suits me :) > > On 29 October 2015 at 16:39, Kris G. Lindgren

Re: [Openstack-operators] Juno and Kilo Interoperability

2015-08-26 Thread Sam Morrison
Yeah we run a multitude of versions at the same time. We usually run N and N-1 in the same env but have also done N-2 (eg. Havana, Icehouse and Juno) Currently we are mainly Juno (keystone, heat, ceilometer, cinder) with a couple of icehouse things lying around. We are in the progress of

Re: [Openstack-operators] kilo - neutron - ipset problems?

2015-09-03 Thread Sam Morrison
Hi Kris, We are moving to neutron in a month and are hitting this in our pre prod environment too. Sam > On 2 Sep 2015, at 6:32 am, Kris G. Lindgren wrote: > > Hello, > > We ran into this again today. > > I created bug:

Re: [Openstack-operators] Adding v1 LIKE support to python-glanceclient releases 1.x.x

2015-09-09 Thread Sam Morrison
I’d just like things like image-list to work under the new client :-) https://bugs.launchpad.net/python-glanceclient/+bug/1492887 > On 10 Sep 2015, at 8:41 am, Nikhil Komawar wrote: > > Hi all, > > We recently release python-glanceclient 1.0.0 and it has the default >

[Openstack-operators] Neutron DHCP failover bug

2015-09-29 Thread Sam Morrison
Hi All, We are running Kilo and have come across this bug https://bugs.launchpad.net/neutron/+bug/1410067 Pretty easy to replicate, have 2 network nodes, shutdown 1 of them and DHCP etc. moves over to the new host fine. Except doing a

Re: [Openstack-operators] Neutron DHCP failover bug

2015-09-30 Thread Sam Morrison
> On 30 Sep 2015, at 3:48 pm, John Dewey wrote: > > Why not run neutron dhcp agents on both nodes? Yeah have tried this too. We don’t do this due to metadata, DHCP adds a static route to the metadata agent (which is the same as the network node running DHCP). If the network

Re: [Openstack-operators] [openstack-dev] [cinder] [all] The future of Cinder API v1

2015-09-28 Thread Sam Morrison
Yeah we’re still using v1 as the clients that are packaged with most distros don’t support v2 easily. Eg. with Ubuntu Trusty they have version 1.1.1, I just updated our “volume” endpoint to point to v2 (we have a volumev2 endpoint too) and the client breaks. $ cinder list ERROR: OpenStack

Re: [Openstack-operators] Potential deprecation of cinder.cross_az_attach option in nova

2015-09-23 Thread Sam Morrison
We very much rely on this and I see this is already merged! Great another patch I have to manage locally. I don’t understand what the confusion is. We have multiple availability zones in nova and each zone has a corresponding cinder-volume service(s) in the same availability zone. We don’t

Re: [Openstack-operators] Nova-network -> Neutron Migration

2015-12-08 Thread Sam Morrison
> On 9 Dec 2015, at 2:16 PM, Tom Fifield wrote: > > NeCTAR used this script (https://github.com/NeCTAR-RC/novanet2neutron ) with > success to do a live nova-net to neutron using Juno. That’s correct except we were on Kilo. I’m not sure I would try to do this on Icehouse

[Openstack-operators] [cinder] upgrade juno -> kilo (near live)

2015-12-01 Thread Sam Morrison
Hi All, We have got some patches that allows a near live upgrade of Cinder from Juno -> Kilo. It allows juno cinder-volumes to live with kilo API and schedulers. You basically need to ensure your juno hosts all have the patch Run the kilo DB migrations then upgrade api and scheduler to kilo.

Re: [Openstack-operators] How do I install specific versions of openstack/puppet-keystone

2015-11-25 Thread Sam Morrison
Can you get R10k to NOT install dependencies listed in metadata etc.? We use puppet-librarian and it can’t and so we have to change every puppet module eg. [1] as some of the dependencies break things for us. Sam [1]

[Openstack-operators] oslo_messaging, rabbit, ssl and mitaka and xenial

2016-06-02 Thread Sam Morrison
Hi all, We’ve been trying out some mitaka packages as well as some Xenial hosts and have been having some issues with rabbit and SSL. If using rabbitMQ 3.6.x on Trusty I can’t get a mitaka host (oslo_messaging 4.6.1, python-amqp 1.4.9) to connect to rabbit over SSL. If I use rabbitMQ 3.6.x

Re: [Openstack-operators] [craton] Versions of Python to support for Craton

2016-05-24 Thread Sam Morrison
I’m in favour of using 3.5. We are in the process of moving things to ubuntu xenial and 3.5 is native there. BTW when is Craton planning on getting into openstack gerrit etc? Sam > On 25 May 2016, at 6:20 AM, Jim Baker wrote: > > tl;dr - any reason why Craton should

Re: [Openstack-operators] [openstack-dev] [glance] Proposal for a mid-cycle virtual sync on operator issues

2016-05-25 Thread Sam Morrison
I’m hoping some people from the Large Deployment Team can come along. It’s not a good time for me in Australia but hoping someone else can join in. Sam > On 26 May 2016, at 2:16 AM, Nikhil Komawar wrote: > > Hello, > > > Firstly, I would like to thank Fei Long for

Re: [Openstack-operators] Ops Meetup event sizes

2016-06-01 Thread Sam Morrison
As an operator what should I prioritise now the main summit is changing, the thing formally known as the summit or the ops mid cycle? Will there be operator sessions at the summit still? Sorry if this has already been mentioned but still not 100% sure how operators fit into the new model.

Re: [Openstack-operators] [Openstack-Operators] Keystone cache strategies

2016-06-21 Thread Sam Morrison
> On 22 Jun 2016, at 9:42 AM, Matt Fischer <m...@mattfischer.com> wrote: > > On Tue, Jun 21, 2016 at 4:21 PM, Sam Morrison <sorri...@gmail.com > <mailto:sorri...@gmail.com>> wrote: >> >> On 22 Jun 2016, at 1:45 AM, Matt Fischer <m...@mattfischer

Re: [Openstack-operators] [Openstack-Operators] Keystone cache strategies

2016-06-21 Thread Sam Morrison
though the default cache time is 10 seconds for revocation lists. I might just set that to some large number to limit this traffic a bit. Sam > On Jun 21, 2016 5:55 PM, "Sam Morrison" <sorri...@gmail.com > <mailto:sorri...@gmail.com>> wrote: > >> O

Re: [Openstack-operators] [Openstack-Operators] Keystone cache strategies

2016-06-21 Thread Sam Morrison
> > On 22 Jun 2016, at 1:45 AM, Matt Fischer wrote: > > I don't have a solution for you, but I will concur that adding revocations > kills performance especially as that tree grows. I'm curious what you guys > are doing revocations on, anything other than logging out of

Re: [Openstack-operators] Keystone's DB_SYNC from Kilo to Liberty

2016-06-26 Thread Sam Morrison
That usually means your DB is at version 86 (you can check the DB table to see, the table is called migration_version or something) BUT your keystone version is older and doesn’t know about version 86. Is it possible the keystone version your running is older and doesn’t know about version 86?

Re: [Openstack-operators] Kilo ->-> Mitaka anyone have notes :)

2016-06-26 Thread Sam Morrison
I’ve done kilo -> mitaka with Keystone and all worked fine. Nothing special I needed to do. If you’re wanting to do live upgrades with nova you can’t skip a version from my understanding. Sam > On 25 Jun 2016, at 4:16 AM, Jonathan Proulx wrote: > > Hi All, > > I about

[Openstack-operators] Duplicates and confusion in nova policy.json files

2016-06-15 Thread Sam Morrison
Now that policy files in nova Liberty apparently work I’m going through the stock example one and see that there are duplicate entries in the policy.json like compute:create:forced_host os_compute_api:servers:create:forced_host Which one do I use to change who can do forced_host? both or a

Re: [Openstack-operators] Nova cells patches against Liberty

2016-05-19 Thread Sam Morrison
y uuid and I > think vm_state are set at that point. The name accessor method references > self.id, which (for some reason) is not lazy loadable. > > So I tried moving the _ensure_cells_system_metadata later in the save method, > after the object is loaded from the database (here:

Re: [Openstack-operators] [openstack-dev] [glance] glance-registry deprecation: Request for feedback

2016-05-15 Thread Sam Morrison
> On 14 May 2016, at 5:36 AM, Flavio Percoco <fla...@redhat.com> wrote: > >> On 5/12/16 9:20 PM, Sam Morrison wrote: >> >> We find glance registry quite useful. Have a central glance-registry api >> is useful when you have multiple datacenters all w

Re: [Openstack-operators] [glance] glance-registry deprecation: Request for feedback

2016-05-12 Thread Sam Morrison
We find glance registry quite useful. Have a central glance-registry api is useful when you have multiple datacenters all with glance-apis and talking back to a central registry service. I guess they could all talk back to the central DB server but currently that would be over the public

Re: [Openstack-operators] Nova cells patches against Liberty

2016-05-04 Thread Sam Morrison
Hi Mike, I’ve also been working on these and have some updated patches at: https://github.com/NeCTAR-RC/nova/commits/stable/liberty-cellsv1 There are a couple of patches that you have in your tree that need updating for Liberty. Mainly around supporting the v2.1 API and more things moved to

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-25 Thread Sam Morrison
> I don't think we can make case with sigkill any better. Other than that, the > issue could be investigated on Neutron side, maybe number of messages could > be reduced there. > > Thanks, > > Dmitry > > 2016-07-25 9:27 GMT+03:00 Sam Morrison <sorri...@gmail.com

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-26 Thread Sam Morrison
> On 27 Jul 2016, at 4:05 AM, Dmitry Mescheryakov <dmescherya...@mirantis.com> > wrote: > > > > 2016-07-26 2:15 GMT+03:00 Sam Morrison <sorri...@gmail.com > <mailto:sorri...@gmail.com>>: > The queue TTL happens on reply queues and fanout queues. I

Re: [Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-28 Thread Sam Morrison
> On 28 Jul 2016, at 10:17 PM, Dmitry Mescheryakov <dmescherya...@mirantis.com> > wrote: > > > > 2016-07-27 2:20 GMT+03:00 Sam Morrison <sorri...@gmail.com > <mailto:sorri...@gmail.com>>: > >> On 27 Jul 2016, at 4:05 AM, Dmitry Mescheryakov &l

[Openstack-operators] [oslo] RabbitMQ queue TTL issues moving to Liberty

2016-07-25 Thread Sam Morrison
We recently upgraded to Liberty and have come across some issues with queue build ups. This is due to changes in rabbit to set queue expiries as opposed to queue auto delete. See https://bugs.launchpad.net/oslo.messaging/+bug/1515278 for more information. The fix for this bug is in liberty

Re: [Openstack-operators] [gnocchi] monitoring storage use case inquiry

2016-07-31 Thread Sam Morrison
Hi Gordon, We are using the influxDB backend and we have our retention policies set to: Every minute for an hour Every 10 minutes for a day Every hour for a year Currently we hover around 8,000 instances. We understand the influxDB driver was taken out of gnocchi, bit annoyed as it wasn’t

Re: [Openstack-operators] Blazar? (Reservations and/or scheduled termination)

2016-08-03 Thread Sam Morrison
> On 4 Aug 2016, at 3:12 AM, Kris G. Lindgren wrote: > > We do something similar. We give everyone in the company an account on the > internal cloud. By default they have a user- project. We have a > Jenkins job that adds metadata to all vm’s that are in user-

Re: [Openstack-operators] OpenStack mitaka using swift as backend for glance

2016-07-17 Thread Sam Morrison
Hi Michael, This would indicate that glance can’t find the swift endpoint in the keystone catalog. You can either add it to the catalog or specify the swift url in the config. Cheers, Sam > On 15 Jul 2016, at 9:07 PM, Michael Stang > wrote: > > Hi everyone,

Re: [Openstack-operators] Trove Shadow Tenant

2017-02-05 Thread Sam Morrison
Hi Sergio, I’m very interested in this feature too, it might be worth asking in the openstack-trove IRC channel or on the openstack-dev mailing list (adding a [trove] in the subject should get their attention) to get some answers to this question. Cheers, Sam > On 4 Feb 2017, at 1:34 am,

Re: [Openstack-operators] What would you like in Pike?

2017-01-18 Thread Sam Morrison
I would love it if all the projects policy.json was actually usable. Too many times the policy.json isn’t the only place where authN happens with lots of hard coded is_admin etc. Just the ability to to have a certain role to a certain thing would be amazing. It makes it really hard to have

Re: [Openstack-operators] Murano in Production

2016-09-18 Thread Sam Morrison
We run completely separate clusters. I’m sure vhosts give you acceptable security but it means also sharing disk and ram which means if something went awry and generated lots of messages etc. it could take your whole rabbit cluster down. Sam > On 17 Sep 2016, at 3:34 PM, Joe Topjian

Re: [Openstack-operators] Designate service

2016-08-28 Thread Sam Morrison
Hi Alexandra, I don’t think that is supported in designate yet, I think there is/was a blueprint floating around somewhere for this but not sure if anyone has looked at implementing. If you wanted to code this up yourself you could also do this with a notification handler that looks for the

Re: [Openstack-operators] need feedback about Glance image 'visibility' migration in Ocata

2016-11-16 Thread Sam Morrison
> On 17 Nov. 2016, at 3:49 pm, Brian Rosmaita > wrote: > > Ocata workflow: (1) create an image with default visibility, (2) change > its visibility to 'shared', (3) add image members Unsure why this can’t be done in 2 steps, when someone adds an image member to

Re: [Openstack-operators] Audit Logging - Interested? What's missing?

2016-11-16 Thread Sam Morrison
Anybody using http://docs.openstack.org/developer/keystonemiddleware/audit.html ?? > On 17 Nov. 2016, at 11:51 am, Kris G. Lindgren wrote: > > I need to do a deeper dive on audit logging. > >

Re: [Openstack-operators] need feedback about Glance image 'visibility' migration in Ocata

2016-11-17 Thread Sam Morrison
can see the image, but then you're also now making it determine if the image can be shared or not. Cheers, Sam On Fri, Nov 18, 2016 at 12:27 AM, Brian Rosmaita < brian.rosma...@rackspace.com> wrote: > On 11/17/16, 1:39 AM, "Sam Morrison" <sorri...@gmail.com> wrote: >

Re: [Openstack-operators] Ceilometer/oslo.messaging connect to multiple RMQ endpoints

2016-11-03 Thread Sam Morrison
That was me! and yes you can do it when consuming notifications with ceilometer-agent-notification Eg in our ceilometer.conf we have [notification] workers=12 disable_non_metric_meters=true store_events = true batch_size = 50 batch_timeout = 5 messaging_urls =

Re: [Openstack-operators] [puppet][fuel][packstack][tripleo] puppet 3 end of life

2016-11-03 Thread Sam Morrison
Wow I didn’t realise puppet3 was being deprecated, is anyone actually using puppet4? I would hope that the openstack puppet modules would support puppet3 for a while still, at lest until the next ubuntu LTS is out else we would get to the stage where the openstack release supports Xenial but

Re: [Openstack-operators] [puppet][fuel][packstack][tripleo] puppet 3 end of life

2016-11-03 Thread Sam Morrison
> On 4 Nov. 2016, at 1:33 pm, Emilien Macchi <emil...@redhat.com> wrote: > > On Thu, Nov 3, 2016 at 9:10 PM, Sam Morrison <sorri...@gmail.com > <mailto:sorri...@gmail.com>> wrote: >> Wow I didn’t realise puppet3 was being deprecated, is anyone actually us

Re: [Openstack-operators] RabbitMQ 3.6.x experience?

2017-01-10 Thread Sam Morrison
sable, 3.6.5 hasn’t had any problems for us. Sam > -Original Message- > From: Mike Dorman [mailto:mdor...@godaddy.com] > Sent: Monday, January 09, 2017 6:00 PM > To: Ricardo Rocha; Sam Morrison > Cc: OpenStack Operators > Subject: Re: [Openstack-operators] RabbitMQ 3.6.

Re: [Openstack-operators] RabbitMQ 3.6.x experience?

2017-01-08 Thread Sam Morrison
We’ve been running 3.6.5 for sometime now and it’s working well. 3.6.1 - 3.6.3 are unusable, we had lots of issues with stats DB and other weirdness. Our setup is a 3 physical node cluster with around 9k connections, average around the 300 messages/sec delivery. We have the stats sample rate

Re: [Openstack-operators] Successful nova-network to Neutron Migration

2017-05-28 Thread Sam Morrison
Awesome! Glad to hear it all went well. Cheers, Sam > On 21 May 2017, at 4:51 am, Joe Topjian wrote: > > Hi all, > > There probably aren't a lot of people in this situation nowadays, but for > those that are, I wanted to report a successful nova-network to Neutron >

Re: [Openstack-operators] [nova] Cinder cross_az_attach=False changes/fixes

2017-06-06 Thread Sam Morrison
Hi Matt, Just looking into this, > On 1 Jun 2017, at 9:08 am, Matt Riedemann wrote: > > This is a request for any operators out there that configure nova to set: > > [cinder] > cross_az_attach=False > > To check out these two bug fixes: > > 1.

Re: [Openstack-operators] [openstack-dev] [nova][glance] Who needs multiple api_servers?

2017-05-01 Thread Sam Morrison
> On 1 May 2017, at 4:24 pm, Sean McGinnis wrote: > > On Mon, May 01, 2017 at 10:17:43AM -0400, Matthew Treinish wrote: >>> >> >> I thought it was just nova too, but it turns out cinder has the same exact >> option as nova: (I hit this in my devstack patch trying to

[Openstack-operators] Ubuntu Kernel with Meltdown mitigation SSL issues

2018-01-17 Thread Sam Morrison
Hi All, We updated our control infrastructure to the latest Ubuntu Xenial Kernel (4.4.0-109) which includes the meltdown fixes. We have found this kernel to have issues with SSL connections with python and have since downgraded. We get errors like: SSLError: SSL exception connecting to

Re: [Openstack-operators] Mixed service version CI testing

2018-01-01 Thread Sam Morrison
We usually upgrade nova last so would be helpful. Nectar has been running a mix of versions for a couple of years now and we treat each project as it’s own thing and upgrade everything separately. You can see what versions we run currently at

Re: [Openstack-operators] neutron-server memcached connections

2018-07-31 Thread Sam Morrison
Great, yeah we have also seen these issues with nova-api with keystonemiddle in newton and ocata. Thanks for the heads up as I was going to start digging deeper. Cheers, Sam > On 31 Jul 2018, at 10:09 am, iain MacDonnell > wrote: > > > Following up on my own question, in case it's

Re: [Openstack-operators] [nova][cinder][neutron] Cross-cell cold migration

2018-08-22 Thread Sam Morrison
I think in our case we’d only migrate between cells if we know the network and storage is accessible and would never do it if not. Thinking moving from old to new hardware at a cell level. If storage and network isn’t available ideally it would fail at the api request. There is also ceph

Re: [Openstack-operators] Openstack Kilo - Danger: There was an error submitting the form. Please try again.

2018-08-28 Thread Sam Morrison
Hi Anwar, The log message you posted below is not an error (it is at level DEBUG and can be ignored). I would look in your horizon logs and also. The API logs of nova/neutron/glance/cinder to see what’s going on. Good luck! Sam > On 28 Aug 2018, at 9:22 pm, Anwar Durrani wrote: > > Hi

Re: [Openstack-operators] Ubuntu Kernel with Meltdown mitigation SSL issues

2018-01-18 Thread Sam Morrison
to your specific setup. > > How is Keystone API endpoint TLS terminated in your setup? > > AFAIK in our OpenStack labs we haven't observed anything like this > although > > we terminate TLS on Nginx or HAProxy. > > > > > > On Thu, Jan 18, 2018 at 4:36 AM

[Openstack-operators] RabbitMQ and SSL

2018-10-31 Thread Sam Morrison
Hi all, We’ve been battling an issue after an upgrade to pike which essentially makes using rabbit with ssl impossible https://bugs.launchpad.net/oslo.messaging/+bug/1800957 We use ubuntu cloud archives so it might no exactly be olso

Re: [Openstack-operators] RabbitMQ and SSL

2018-11-12 Thread Sam Morrison
957> Sam > On 1 Nov 2018, at 11:04 am, Sam Morrison wrote: > > Hi all, > > We’ve been battling an issue after an upgrade to pike which essentially makes > using rabbit with ssl impossible > > https://bugs.launchpad.net/oslo.messaging/+bug/1800957 > <https

Re: [Openstack-operators] [neutron][lbaas][neutron-lbaas][octavia] Update on the previously announced deprecation of neutron-lbaas and neutron-lbaas-dashboard

2018-09-30 Thread Sam Morrison
Hi Michael, Are all the backends that are supported by lbaas supported by octavia? I can’t see a page that lists the supported backends. Eg. We use lbaas with the midonet driver and I can’t tell if this will still work when switching over? Thanks, Sam > On 29 Sep 2018, at 8:07 am, Michael