Re: [Openstack-operators] Neutron fails to notify nova on events network-vif-plugged

[Ahmed Mostafa]

Actually it is not authenticating it self against keystone, it is
communicating directly to nova by using a ketstone client

in neutron.conf you have to options which are set to True by default, they
are

notify_nova_on_port_status_changes

notify_nova_on_port_data_changes

If you set both of them to False, you wont have any errors any more, but if
for some reason you require using neutron nova notification, then you must
configure nova authentication and api url in neutron.conf

You will find in neutron.conf a section named nova, under it you will see
all condiguration option you can use to configure the notification driver
to notify nova on status or data changes on ports


On Tuesday, 1 November 2016, Davíð Örn Jóhannsson  wrote:

> I’m working on setting up a OpenStack Liberty development env on Ubuntu
> 14.04. At the present I have 3 nodes, Controller, Network and Compute. I am
> up to the place where I’m trying to spin up an instance where
> neutron-server seems to fail to notify nova because of an authentication
> error against keystone , I’ve been struggling for some time to figure out
> the cause of this and was hoping that some could lend me more experienced
> eyes
>
> Controller node /etc/neutron/neutron.conf http://paste.openstack.org/show/
> 587547/
> Openstack endpoint list http://paste.openstack.org/show/587548/
>
> 2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.session [-] RESP: [300]
> Content-Length: 635 Vary: X-Auth-Token Connection: keep-alive Date: Tue, 01
> Nov 2016 12:42:04 GMT Content-Type: application/json X-Distribution: Ubuntu
> RESP BODY: {"versions": {"values": [{"status": "stable", "updated":
> "2015-03-30T00:00:00Z", "media-types": [{"base": "application/json",
> "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4",
> "links": [{"href": "http://controller-01:35357/v3/;, "rel": "self"}]},
> {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types":
> [{"base": "application/json", "type": 
> "application/vnd.openstack.identity-v2.0+json"}],
> "id": "v2.0", "links": [{"href": "http://controller-01:35357/v2.0/;,
> "rel": "self"}, {"href": "http://docs.openstack.org/;, "type":
> "text/html", "rel": "describedby"}]}]}}
>  _http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/
> session.py:215
> 2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.auth.identity.v3.base
> [-] Making authentication request to http://controller-01:35357/v3/
> auth/tokens get_auth_ref /usr/lib/python2.7/dist-
> packages/keystoneclient/auth/identity/v3/base.py:188
> 2016-11-01 12:42:04.091 15888 DEBUG keystoneclient.session [-] Request
> returned failure status: 401 request /usr/lib/python2.7/dist-
> packages/keystoneclient/session.py:400
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova [-] Failed to
> notify nova on events: [{'status': 'completed', 'tag':
> u'bf092fd0-51ba-4fbf-8d3d-9c3004b3811f', 'name': 'network-vif-plugged',
> 'server_uuid': u'24616ae2-a6e4-4843-ade6-357a9ce80bc0'}]
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova Traceback (most
> recent call last):
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 248,
> in send_events
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova
> batched_events)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py",
> line 39, in create
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova
> return_raw=True)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in
> _create
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova _resp, body
> = self.api.client.post(url, body=body)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176,
> in post
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return
> self.request(url, 'POST', **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 91, in
> request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206,
> in request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova resp =
> super(LegacyJsonAdapter, self).request(*args, **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in
> request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return
> self.session.request(url, method, **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> 

Re: [Openstack-operators] Neutron fails to notify nova on events network-vif-plugged

[Davíð Örn Jóhannsson]

I’m trying to use the puppet modules for liberty, I’ve had some issues there 
where the modules do not seem to exactly represent the configuration intended 
for liberty

From: Matt Kassawara
Date: Tuesday 1 November 2016 at 15:40
To: David Orn Johannsson
Cc: 
"openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>"
Subject: Re: [Openstack-operators] Neutron fails to notify nova on events 
network-vif-plugged

You are mixing a lot of current and defunct configuration options. I recommend 
removing the defunct options and then comparing your configuration files with 
the installation guide [1].

[1] http://docs.openstack.org/liberty/install-guide-ubuntu/

On Tue, Nov 1, 2016 at 7:05 AM, Davíð Örn Jóhannsson 
<davi...@siminn.is<mailto:davi...@siminn.is>> wrote:
I’m working on setting up a OpenStack Liberty development env on Ubuntu 14.04. 
At the present I have 3 nodes, Controller, Network and Compute. I am up to the 
place where I’m trying to spin up an instance where neutron-server seems to 
fail to notify nova because of an authentication error against keystone , I’ve 
been struggling for some time to figure out the cause of this and was hoping 
that some could lend me more experienced eyes

Controller node /etc/neutron/neutron.conf 
http://paste.openstack.org/show/587547/
Openstack endpoint list http://paste.openstack.org/show/587548/

2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.session [-] RESP: [300] 
Content-Length: 635 Vary: X-Auth-Token Connection: keep-alive Date: Tue, 01 Nov 
2016 12:42:04 GMT Content-Type: application/json X-Distribution: Ubuntu
RESP BODY: {"versions": {"values": [{"status": "stable", "updated": 
"2015-03-30T00:00:00Z", "media-types": [{"base": "application/json", "type": 
"application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": 
[{"href": "http://controller-01:35357/v3/;, "rel": "self"}]}, {"status": 
"stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": 
"application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], 
"id": "v2.0", "links": [{"href": "http://controller-01:35357/v2.0/;, "rel": 
"self"}, {"href": "http://docs.openstack.org/;, "type": "text/html", "rel": 
"describedby"}]}]}}
 _http_log_response 
/usr/lib/python2.7/dist-packages/keystoneclient/session.py:215
2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.auth.identity.v3.base [-] 
Making authentication request to http://controller-01:35357/v3/auth/tokens 
get_auth_ref 
/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:188
2016-11-01 12:42:04.091 15888 DEBUG keystoneclient.session [-] Request returned 
failure status: 401 request 
/usr/lib/python2.7/dist-packages/keystoneclient/session.py:400
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova [-] Failed to notify 
nova on events: [{'status': 'completed', 'tag': 
u'bf092fd0-51ba-4fbf-8d3d-9c3004b3811f', 'name': 'network-vif-plugged', 
'server_uuid': u'24616ae2-a6e4-4843-ade6-357a9ce80bc0'}]
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova Traceback (most 
recent call last):
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 248, in 
send_events
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova batched_events)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py",
 line 39, in create
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return_raw=True)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova _resp, body = 
self.api.client.post(url, body=body)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return 
self.request(url, 'POST', **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 91, in request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in 
request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova r

Re: [Openstack-operators] Neutron fails to notify nova on events network-vif-plugged

[Matt Kassawara]

You are mixing a lot of current and defunct configuration options. I
recommend removing the defunct options and then comparing your
configuration files with the installation guide [1].

[1] http://docs.openstack.org/liberty/install-guide-ubuntu/

On Tue, Nov 1, 2016 at 7:05 AM, Davíð Örn Jóhannsson 
wrote:

> I’m working on setting up a OpenStack Liberty development env on Ubuntu
> 14.04. At the present I have 3 nodes, Controller, Network and Compute. I am
> up to the place where I’m trying to spin up an instance where
> neutron-server seems to fail to notify nova because of an authentication
> error against keystone , I’ve been struggling for some time to figure out
> the cause of this and was hoping that some could lend me more experienced
> eyes
>
> Controller node /etc/neutron/neutron.conf http://paste.openstack.org/show/
> 587547/
> Openstack endpoint list http://paste.openstack.org/show/587548/
>
> 2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.session [-] RESP: [300]
> Content-Length: 635 Vary: X-Auth-Token Connection: keep-alive Date: Tue, 01
> Nov 2016 12:42:04 GMT Content-Type: application/json X-Distribution: Ubuntu
> RESP BODY: {"versions": {"values": [{"status": "stable", "updated":
> "2015-03-30T00:00:00Z", "media-types": [{"base": "application/json",
> "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4",
> "links": [{"href": "http://controller-01:35357/v3/;, "rel": "self"}]},
> {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types":
> [{"base": "application/json", "type": 
> "application/vnd.openstack.identity-v2.0+json"}],
> "id": "v2.0", "links": [{"href": "http://controller-01:35357/v2.0/;,
> "rel": "self"}, {"href": "http://docs.openstack.org/;, "type":
> "text/html", "rel": "describedby"}]}]}}
>  _http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/
> session.py:215
> 2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.auth.identity.v3.base
> [-] Making authentication request to http://controller-01:35357/v3/
> auth/tokens get_auth_ref /usr/lib/python2.7/dist-
> packages/keystoneclient/auth/identity/v3/base.py:188
> 2016-11-01 12:42:04.091 15888 DEBUG keystoneclient.session [-] Request
> returned failure status: 401 request /usr/lib/python2.7/dist-
> packages/keystoneclient/session.py:400
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova [-] Failed to
> notify nova on events: [{'status': 'completed', 'tag':
> u'bf092fd0-51ba-4fbf-8d3d-9c3004b3811f', 'name': 'network-vif-plugged',
> 'server_uuid': u'24616ae2-a6e4-4843-ade6-357a9ce80bc0'}]
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova Traceback (most
> recent call last):
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 248,
> in send_events
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova
> batched_events)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py",
> line 39, in create
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova
> return_raw=True)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in
> _create
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova _resp, body
> = self.api.client.post(url, body=body)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176,
> in post
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return
> self.request(url, 'POST', **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 91, in
> request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206,
> in request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova resp =
> super(LegacyJsonAdapter, self).request(*args, **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in
> request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return
> self.session.request(url, method, **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return
> func(*args, **kwargs)
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 304,
> in request
> 2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova
> auth_headers = self.get_auth_headers(auth)
> 

[Openstack-operators] Neutron fails to notify nova on events network-vif-plugged

[Davíð Örn Jóhannsson]

I’m working on setting up a OpenStack Liberty development env on Ubuntu 14.04. 
At the present I have 3 nodes, Controller, Network and Compute. I am up to the 
place where I’m trying to spin up an instance where neutron-server seems to 
fail to notify nova because of an authentication error against keystone , I’ve 
been struggling for some time to figure out the cause of this and was hoping 
that some could lend me more experienced eyes

Controller node /etc/neutron/neutron.conf 
http://paste.openstack.org/show/587547/
Openstack endpoint list http://paste.openstack.org/show/587548/

2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.session [-] RESP: [300] 
Content-Length: 635 Vary: X-Auth-Token Connection: keep-alive Date: Tue, 01 Nov 
2016 12:42:04 GMT Content-Type: application/json X-Distribution: Ubuntu
RESP BODY: {"versions": {"values": [{"status": "stable", "updated": 
"2015-03-30T00:00:00Z", "media-types": [{"base": "application/json", "type": 
"application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": 
[{"href": "http://controller-01:35357/v3/;, "rel": "self"}]}, {"status": 
"stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": 
"application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], 
"id": "v2.0", "links": [{"href": "http://controller-01:35357/v2.0/;, "rel": 
"self"}, {"href": "http://docs.openstack.org/;, "type": "text/html", "rel": 
"describedby"}]}]}}
 _http_log_response 
/usr/lib/python2.7/dist-packages/keystoneclient/session.py:215
2016-11-01 12:42:04.067 15888 DEBUG keystoneclient.auth.identity.v3.base [-] 
Making authentication request to http://controller-01:35357/v3/auth/tokens 
get_auth_ref 
/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:188
2016-11-01 12:42:04.091 15888 DEBUG keystoneclient.session [-] Request returned 
failure status: 401 request 
/usr/lib/python2.7/dist-packages/keystoneclient/session.py:400
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova [-] Failed to notify 
nova on events: [{'status': 'completed', 'tag': 
u'bf092fd0-51ba-4fbf-8d3d-9c3004b3811f', 'name': 'network-vif-plugged', 
'server_uuid': u'24616ae2-a6e4-4843-ade6-357a9ce80bc0'}]
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova Traceback (most 
recent call last):
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 248, in 
send_events
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova batched_events)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py",
 line 39, in create
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return_raw=True)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova _resp, body = 
self.api.client.post(url, body=body)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return 
self.request(url, 'POST', **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 91, in request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in 
request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova resp = 
super(LegacyJsonAdapter, self).request(*args, **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in 
request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return 
self.session.request(url, method, **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return 
func(*args, **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 304, in 
request
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova auth_headers = 
self.get_auth_headers(auth)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 617, in 
get_auth_headers
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova return 
auth.get_headers(self, **kwargs)
2016-11-01 12:42:04.091 15888 ERROR neutron.notifiers.nova   File 
"/usr/lib/python2.7/dist-packages/keystoneclient/auth/base.py", line 142, in 
get_headers
2016-11-01 12:42:04.091 15888