Re: [Openstack-operators] Newton LBaaS v2 settings

2017-12-15 Thread Grant Morley
-b0b2a6348959 10.1.4.14:80 weight 1 check inter 5s fall 3 rise 3 so, if you guess it's a problem with backend servers, you can play with HealthMonitor parameters in order to set appropriate timeouts for backend servers in this pool. On 12/15/17 12:11 PM, Grant Morley wrote: Hi Al

[Openstack-operators] Newton LBaaS v2 settings

2017-12-15 Thread Grant Morley
timeout settings for the "Health Monitor" Any help will be much appreciated. Regards, -- Grant Morley Senior Cloud Engineer Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io <http://www.absolutedevops.io/> gr...@abso

Re: [Openstack-operators] OSA upgrading Ubuntu14.04 Newton to Ubuntu16.04 Ocata

2017-10-04 Thread Grant Morley
hanks, Amy (spotz) On Wed, Oct 4, 2017 at 9:42 AM, Grant Morley <mailto:gr...@absolutedevops.io>> wrote: Hi All, Just have a quick question regarding upgrading from Newton to Ocata using OSA. We have a small installation ( 4 compute. 3 management, 2 network and some Ceph

[Openstack-operators] OSA upgrading Ubuntu14.04 Newton to Ubuntu16.04 Ocata

2017-10-04 Thread Grant Morley
things. We are going to test this on our dev environment and see what breaks, but just wondered if anyone here has come across this already and suffered the pain :) Any suggestions would be much appreciated. Many thanks, -- Grant Morley Senior Cloud Engineer Absolute DevOps Ltd Units H, J

[Openstack-operators] OSA Newton ssl endpoint issue

2017-07-21 Thread Grant Morley
the endpoints to the HAProxy VIP address and the SSL certs are not valid for an IP address. Do you need to add the IP address to the cert? or disable it until you have deployed ( which is what we have done to get around it ) Or is there a better method? Many thanks, -- Grant Morley Senior

Re: [Openstack-operators] Neutron not clearing up deleted routers

2017-06-08 Thread Grant Morley
Ignore that now all, Managed to fix it by restarting the l3-agent. Looks like it must have been cached in memory. Thanks, On 08/06/17 18:07, Grant Morley wrote: Hi All, We have noticed in our neutron-l3-agent logs that there are a number of routers that neutron seems to think exist

[Openstack-operators] Neutron not clearing up deleted routers

2017-06-08 Thread Grant Morley
l3.agent raise RuntimeError(msg) 2017-06-08 16:50:22.340 677 ERROR neutron.agent.l3.agent RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-c4080b5c-c9" Has anyone come across this before? We don't seem to have an entry for them anywhere from what

Re: [Openstack-operators] Ceph recovery going unusually slow

2017-06-02 Thread Grant Morley
et > those flags do all your osd’s stay up and in? > > On Jun 2, 2017, at 8:00 AM, Grant Morley > wrote: > > HEALTH_ERR 210 pgs are stuck inactive for more than 300 seconds; 296 pgs > backfill_wait; 3 pgs backfilling; 1 pgs degraded; 202 pgs peering; 1 pgs > recovery_wai

Re: [Openstack-operators] Ceph recovery going unusually slow

2017-06-02 Thread Grant Morley
o, do you have big enough limits ? >> >> Check on any host the content of: /proc/`pid_of_the_osd`/limits >> >> >> Saverio >> >> 2017-06-02 14:00 GMT+02:00 Grant Morley : >> > HEALTH_ERR 210 pgs are stuck inactive for more than 300 seconds; 296 pgs >

Re: [Openstack-operators] Ceph recovery going unusually slow

2017-06-02 Thread Grant Morley
58.230601, current state > remapped+peering, > > last acting [51,12,13] > > > > 41 osds have slow requests > > recovery 2/11091408 objects degraded (0.000%) > > recovery 1778127/11091408 objects misplaced (16.032%) > > nodown,noout,noscrub,nodeep-scrub flag(s) set &g

Re: [Openstack-operators] Ceph recovery going unusually slow

2017-06-02 Thread Grant Morley
not sure how to get around that. Thanks, On Fri, Jun 2, 2017 at 12:55 PM, Saverio Proto wrote: > Usually 'ceph health detail' gives better info on what is making > everything stuck. > > Saverio > > 2017-06-02 13:51 GMT+02:00 Grant Morley : > > Hi All, &g

[Openstack-operators] Ceph recovery going unusually slow

2017-06-02 Thread Grant Morley
Hi All, I wonder if anyone could help at all. We were doing some routine maintenance on our ceph cluster and after running a "service ceph-all restart" on one of our nodes we noticed that something wasn't quite right. The cluster has gone into an error mode and we have multiple stuck PGs and the

Re: [Openstack-operators] Ceph upgrade causing health warnings

2017-05-17 Thread Grant Morley
client iupgraded) before you set it though. We ran into some issues with that. Thanks, Kevin *From:* Grant Morley [gr...@absolutedevops.io] *Sent:* Wednesday, May 17, 2017 3:47 AM *To:* OpenStack Operators *Subject

[Openstack-operators] Ceph upgrade causing health warnings

2017-05-17 Thread Grant Morley
to be running fine. Just more of an annoyance that we are getting a lot of alerts from our monitoring systems. Regards, Grant -- Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io <http://www.a

Re: [Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
gards, On 02/03/17 16:08, Saverio Proto wrote: select * from images where name=''Ubuntu-16.04"; -- Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io <http://www.absolutedevops.io/> gr..

Re: [Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
03/17 16:08, Saverio Proto wrote: select * from images where name=''Ubuntu-16.04"; -- Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io <http://www.absolutedevops.io/> gr...@absolutedevop

Re: [Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
| public | +--+--+ On 02/03/17 15:56, Saverio Proto wrote: Can you share with us the output of: openstack image show for that image ? Saverio 2017-03-02 13:54 GMT+01:00 Grant Morley

Re: [Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
Unfortunately not, I still get the same error. Grant On 02/03/17 12:54, Saverio Proto wrote: If you pass the uuid of the image does it work ? Saverio 2017-03-02 13:49 GMT+01:00 Grant Morley <mailto:gr...@absolutedevops.io>>: Hi Saverio, We are running Mitaka - sorry

Re: [Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
Hi Saverio, We are running Mitaka - sorry forgot to mention that. Grant On 02/03/17 12:45, Saverio Proto wrote: What version of Openstack are we talking about ? Saverio 2017-03-02 12:11 GMT+01:00 Grant Morley <mailto:gr...@absolutedevops.io>>: Hi All, Not sure if anyone

[Openstack-operators] Unable to launch instances because of glance errors

2017-03-02 Thread Grant Morley
;: [{"href": "http://10.6.0.3:9191/v1/";, "rel": "self"}]}, {"status": "SUPPORTED", "id": "v1.0", "links": [{"href": "http://10.6.0.3:9191/v1/";, "rel": "self"}]

Re: [Openstack-operators] osa Mitaka api SSL end points

2017-02-28 Thread Grant Morley
28/02/17 11:42, Andy McCrae wrote: On 28 February 2017 at 09:59, Grant Morley <mailto:gr...@absolutedevops.io>> wrote: Hi All, We have an OSA Mitaka deployment and for some reason all of the end points ( keystone, neutron, glance etc.. ) are all reporting as HTTP ra

[Openstack-operators] osa Mitaka api SSL end points

2017-02-28 Thread Grant Morley
ternal | http://10.6.0.3:5000/v3 | Is there something else I have missed or do I need to put our SSL certs in a different directory for OSA to setup the endpoints with HTTPS on haproxy? Grateful for any help. Regards, Grant -- Grant Morley Cloud Lead Absolute DevOps Ltd

Re: [Openstack-operators] Glance with ceph backend stated to give errors

2017-01-31 Thread Grant Morley
re's a corresponding bug report on Launchpad for the Ubuntu Ceph package that has some more detail: https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1625489 Cheers. -- -Nick On 31 January 2017 at 10:49, Grant Morley <mailto:gr...@absolutedevops.io>> wrote: Hi All,

[Openstack-operators] Glance with ceph backend stated to give errors

2017-01-31 Thread Grant Morley
re working absolutely fine, it is just Glance and ceph has all of a sudden stopped working. Just wondered if anyone had any ideas? Regards, -- Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io <http://www

Re: [Openstack-operators] Mitaka to Newton networking issues

2016-12-08 Thread Grant Morley
space from the same router when it was removed from the L3 agent. For each L3 agent, can you shutdown the L3 agent, run the netns cleanup script, ensure all keepalived processes are dead, and then start the agent again? On Tue, Dec 6, 2016 at 4:59 AM, Grant Morley <mailto:gr...@absoluted

Re: [Openstack-operators] Mitaka to Newton networking issues

2016-12-06 Thread Grant Morley
ote: Can you do a 'neutron port-show' for both of those HA ports to check their status field? On Tue, Dec 6, 2016 at 2:29 AM, Grant Morley <mailto:gr...@absolutedevops.io>> wrote: Hi Kevin & Neil, Many thanks for the reply. I have attached a screen shot showing

[Openstack-operators] Mitaka to Newton networking issues

2016-12-05 Thread Grant Morley
terface driver. Has anyone else come across this at all or have any pointers? This was working fine in Mitaka it just seems since the upgrade to Newton, we have these issues. I am able to provide more logs if they are needed. Regards, -- Grant Morley Cloud Lead Absolute DevOps Ltd Units H,

Re: [Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)

2016-10-21 Thread Grant Morley
dgren Senior Linux Systems Engineer GoDaddy *From: *Grant Morley *Date: *Friday, October 21, 2016 at 6:14 AM *To: *OpenStack Operators *Cc: *"ian.ba...@serverchoice.com" *Subject: *[Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup) Hi

Re: [Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)

2016-10-21 Thread Grant Morley
7 2016-10-21 12:08:06.242 70661 ERROR nova.compute.manager [instance: 5633d98e-5f79-4c13-8d45-7544069f0e6f] File "*/openstack/venvs/*nova-12.0.16/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 117, in __init__ Chris On Fri, 21 Oct 2016 at 13:19 Gra

[Openstack-operators] Instances failing to launch when rbd backed (ansible Liberty setup)

2016-10-21 Thread Grant Morley
bd1 10.2.3-1trustyamd64RADOS block device client library ii python-rbd 10.2.3-1trustyamd64Python libraries for the Ceph librbd library Has anyone come across this before? Ceph is working fine for Glance, it just seems to be with th

Re: [Openstack-operators] VXLAN / Tenant Network Issue

2016-09-08 Thread Grant Morley
2_conf.ini , dhcp_agent.ini and l3_agent.ini files ? Regards VM From: Grant Morley <mailto:gr...@absolutedevops.io>> Date: Thursday 8 September 2016 at 15:12 To: OpenStack Operators <mailto:openstack-operators@lists.openstack.org>> Cc: mailto:ian.ba...@serverchoice.com>> Su

[Openstack-operators] VXLAN / Tenant Network Issue

2016-09-08 Thread Grant Morley
on any other interface on the compute host. No DHCP packet is observed on the network agent container running the DHCP namespace. output of the instance booting: Starting network... udhcpc (v1.20.1) started Sending discover... Sending discover... Sending discover... Usage: /sbin/cirros-dhc

Re: [Openstack-operators] Issue when trying to snapshot an instance

2016-06-03 Thread Grant Morley
Funny you should say that. We are using an external url using https via a load balancer. I switched over to the internal endpoint and it has made a very big improvement. Grant On 03/06/16 13:52, David Medberry wrote: On Fri, Jun 3, 2016 at 6:22 AM, Grant Morley <mailto

Re: [Openstack-operators] Issue when trying to snapshot an instance

2016-06-03 Thread Grant Morley
d2a-406e-969e-c22762db3455 bb07f987fbae485c9e05f06fb0d422c2 a22e503869c34a92bceb66b0c1da7231 - - -] Exception during message handling: Not authorized for imag e f9844dd5-5a92-4cd4-956d-8ad04cfc5e84. Any help will be appreciated. Regards, -- Grant Morley Cloud Lea