Re: [Openstack-operators] OpenStack-operators Digest, Vol 76, Issue 30

[Y.Rahulan]

Hi,

 I am trying to integrate ODL as external OVS manager and network
controller but I could not mange to connect br-ex as ODL_BRIDGE_MAPPING
with public interface eth2,

ovs vsctl show:- Manager "tcp:192.67.27.27:6640" is_connected: true Manager
"ptcp:6641:127.0.0.1" is_connected: true Bridge br-int Controller "tcp:
192.67.27.27:6653" is_connected: true fail_mode: secure Port br-int
Interface br-int type: internal Port "tap95af7f98-35" Interface
"tap95af7f98-35" type: internal Bridge br-ex Port br-ex Interface br-ex
type: internal Port "eth2" Interface "eth2" ovs_version: "2.5.0"

also it is creating two managers when I create network shown below:-

. openrc admin admin openstack network create admin-external --external
--provider-network-type flat --provider-physical-network public
--provider-physical-network physnet1 openstack subnet create ext-subnet
--network admin-external --allocation-pool
start=192.67.20.29,end=192.67.20.69 --no-dhcp --gateway 192.67.10.10
--subnet-range 192.67.27.0/24 openstack router create ext-rtr openstack
router set ext-rtr --external-gateway admin-external openstack network
create admin-internal --internal openstack subnet create vx-subnet
--network admin-internal --subnet-range 192.0.0.0/24 --dns-nameserver
192.67.10.10 neutron router-interface-add ext-rtr vx-subnet

Is there any one face the same issues and would like to get some answers
please!

Kind Regards, Rahul

On Tue, Feb 21, 2017 at 12:00 PM, <
openstack-operators-requ...@lists.openstack.org> wrote:

> Send OpenStack-operators mailing list submissions to
> openstack-operators@lists.openstack.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack-operators
>
> or, via email, send a message with subject or body 'help' to
> openstack-operators-requ...@lists.openstack.org
>
> You can reach the person managing the list at
> openstack-operators-ow...@lists.openstack.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of OpenStack-operators digest..."
>
>
> Today's Topics:
>
>1. Re: Instances are not creating after adding 3 additional nova
>   nodes (Kostyantyn Volenbovskyi)
>2. Re: Instances are not creating after adding 3 additional nova
>   nodes (Kevin Benton)
>3. [osops][osops-tools-monitoring] Updates for   monitoring
>   plugins (Major Hayden)
>4. [nova] Metadata service over virtio-vsock (Artom Lifshitz)
>5. Re: [nova] Metadata service over virtio-vsock (Clint Byrum)
>6. Re: [nova] Metadata service over virtio-vsock (Jeremy Stanley)
>7. Re: [nova] Metadata service over virtio-vsock (Clint Byrum)
>8. [publiccloud-wg]Atlanta Virtual PTG agenda (Zhipeng Huang)
>9. Re: [nova] Metadata service over virtio-vsock (Daniel P. Berrange)
>   10. Re: [nova] Metadata service over virtio-vsock (Daniel P. Berrange)
>   11. Re: [nova] Metadata service over virtio-vsock (Clint Byrum)
>
>
> --
>
> Message: 1
> Date: Mon, 20 Feb 2017 14:14:11 +0100
> From: Kostyantyn Volenbovskyi 
> To: Anwar Durrani 
> Cc: openstack-operators 
> Subject: Re: [Openstack-operators] Instances are not creating after
> adding 3 additional nova nodes
> Message-ID: <04649dc8-ca9a-4eed-bc64-26bd02948...@yandex.ru>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
> this 'Unexpected vif_type=binding_failed? is as well fairly-generic, but
> you can change focus from Nova to Neutron+virtual switch.
>
> So check:
> -Neutron server logs
> -Logs of Neutron agent on target Compute Host(s)
> -OVS logs and possibly things like /var/log/messages for things related to
> virtual networking.
>
> The root cause is typically:
> -misconfiguration of mechanism driver/type driver.
> -misconfiguration of virtual switching (typically OVS)
> Go through installation documents in docs.openstack.org, that provides a
> guide for values
> parameters related to that.
>
>
> BR,
> Konstantin
>
> > On Feb 20, 2017, at 8:16 AM, Anwar Durrani 
> wrote:
> >
> > Further when i tried and attempt to launch new instance, i can see the
> following
> >
> > tail -f /var/log/nova/nova-compute.log
> >
> >
> >
> > 2017-02-20 12:45:26.596 5365 TRACE nova.compute.manager [instance:
> 1840ac2e-5a54-4941-a96f-a431b2a2c236] flavor, virt_type)
> >
> > 2017-02-20 12:45:26.596 5365 TRACE nova.compute.manager [instance:
> 1840ac2e-5a54-4941-a96f-a431b2a2c236]   File "/usr/lib/python2.7/site-
> packages/nova/virt/libvirt/vif.py", line 374, in get_config
> >
> > 2017-02-20 12:45:26.596 5365 TRACE nova.compute.manager [instance:
> 1840ac2e-5a54-4941-a96f-a431b2a2c236] _("Unexpected vif_type=%s") %
> vif_type)
> >
> > 2017-02-20 12:45:26.596 5365 TRACE nova.compute.manager [instance:
> 

[Openstack-operators] unsubscribe

[Rajkumar Shivage]

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Large deployments] Neutron issues in Openstack Large deployment using DVR

[Satyanarayana Patibandla]

Hi Saverio,

Thanks for your inputs. Will test with statable/ocata branch code and will
share the result.

Thanks,
Satya.P

On Wed, Feb 22, 2017 at 1:54 AM, Saverio Proto  wrote:

> Hello,
>
> I would use at least the stable/ocata branch. If you just use master
> that is not supposed to be stable, and also I am not sure if you can
> fill a bug against a specific commit in master.
>
> Saverio
>
> 2017-02-21 21:12 GMT+01:00 Satyanarayana Patibandla
> :
> > Hi Saverio,
> >
> > We have tried to create 20 VMs each time using heat template. There is 1
> sec
> > time gap between each VM creation request. When we reached 114 VMs we got
> > the error mentioned in the below mail.Heat template will boot instance
> from
> > volume and it assigns floating IP to the instance.
> >
> > Except neutron-server container we restarted all the neutron agent
> > containers which are present on all network and compute nodes. We are
> using
> > kolla to deploy openstack services.
> >
> > We are using 1 month old master branch openstack code to deploy our
> > services.
> >
> > Please find the error logs in the below link.
> > http://paste.openstack.org/show/599892/
> >
> > Thanks,
> > Satya.P
> >
> > On Wed, Feb 22, 2017 at 12:21 AM, Saverio Proto 
> wrote:
> >>
> >> Hello Satya,
> >>
> >> I would fill a bug on launchpad for this issue.
> >> 114 VMs is not much. Can you identify how to trigger the issue to
> >> reproduce it ? or it just happens randomly ?
> >>
> >> When you say rebooting the network node, do you mean the server
> >> running the neutron-server process ?
> >>
> >> what version and distribution of openstack are you using ?
> >>
> >> thank you
> >>
> >> Saverio
> >>
> >>
> >> 2017-02-21 13:54 GMT+01:00 Satyanarayana Patibandla
> >> :
> >> > Hi All,
> >> >
> >> > We are trying to deploy Openstack in our production environment. For
> >> > networking we are using DVR with out L3 HA. We are able to create 114
> >> > VMs
> >> > with out any issue. After creating 114 VMs we are getting the below
> >> > error.
> >> >
> >> > Error: 504 Gateway Time-out The server didn't
> >> > respond
> >> > in time. 
> >> >
> >> > Neutron services are getting freezed up due to a persistent lock on
> the
> >> > agents table. it seems one of the network node is holding the lock on
> >> > the
> >> > table. After rebooting the network node, the Neutron CLI was
> responsive
> >> > again.
> >> >
> >> > Neutron agent and neutron server is throwing below errors.
> >> >
> >> > Neutron-server errors:
> >> > ERROR oslo_db.sqlalchemy.exc_filters "Can't reconnect until
> invalid
> >> > "
> >> > ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't
> >> > reconnect
> >> > until invalid transaction is rolled back
> >> > ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-
> 59db203e72c6
> >> > 3eb776587c9c40569731ebe5c3557bc7 f43e8699cd5a46e89ffe39e3cac75341 -
> - -]
> >> > index failed: No details.
> >> > ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid
> >> > transaction is rolled back
> >> >
> >> >
> >> > Neutron agents errors:
> >> > MessagingTimeout: Timed out waiting for a reply to message ID
> >> > 40638b6bf12c44cd9a404ecaa14a9909
> >> >
> >> > Could you please provide us your valuable inputs or suggestions for
> >> > above
> >> > errors.
> >> >
> >> > Thanks,
> >> > Satya.P
> >> >
> >> > ___
> >> > OpenStack-operators mailing list
> >> > OpenStack-operators@lists.openstack.org
> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/
> openstack-operators
> >> >
> >
> >
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] HTTP/S Termination with Haproxy + Keystone

[Mike Lowe]

Do you have this in your haproxy front end config?

reqadd X-Forwarded-Proto:\ https

And this in your keystone.conf ?

secure_proxy_ssl_header=HTTP_X_FORWARDED_PROTO

I think that’s what I had to do to tell haproxy to add a headder that keystone 
then matched to know when to return https.

> On Feb 21, 2017, at 8:56 PM, Chris Apsey  wrote:
> 
> I'm having a strange issue with keystone after migrating all public endpoints 
> to https (haproxy terminates the SSL connection for each service):
> 
> openstack endpoint list
> 
> +--+---+--++-+---+-+
> | ID   | Region| Service Name | Service Type  
>  | Enabled | Interface | URL |
> +--+---+--++-+---+-+
> ...
> | 99d302d00ab3461cb9362236c865a430 | RegionOne | keystone | identity  
>  | True| public| https://some.domain.place:5000/v3 |
> ...
> 
> I have also updated my rc files appropriately.  Whenever I try and use the 
> CLI against the public endpoints in debug mode, everything starts out looking 
> good:
> 
> REQ: curl -g -i -X GET https://some.domain.place:5000/v3 -H "Accept: 
> application/json" -H "User-Agent: osc-lib keystoneauth1/2.12.1 
> python-requests/2.11.1 CPython/2.7.9"
> 
> But then, the response body gives a non-https URL:
> 
> RESP BODY: {"version": {"status": "stable", "updated": 
> "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": 
> "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", "links": 
> [{"href": "http://some.domain.place:5000/v3/;, "rel": "self"}]}}
> 
> and then the attempt to authenticate fails:
> 
> Making authentication request to http://some.domain.place:5000/v3/auth/tokens
> Starting new HTTP connection (1): some.domain.place
> Unable to establish connection to http://some.domain.place:5000/v3/auth/tokens
> 
> I've restarted apache2 on my keystone hosts and I have scoured the database 
> for any reference to a non-https public endpoint for keystone; I cannot find 
> one.
> 
> Does anyone know why my response body is giving the wrong URL?  Horizon works 
> perfectly fine with the https endpoints; it's just the command line clients 
> that are having issues.
> 
> Thanks in advance,
> 
> -- 
> v/r
> 
> Chris Apsey
> bitskr...@bitskrieg.net
> https://www.bitskrieg.net
> 
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] HTTP/S Termination with Haproxy + Keystone

[Chris Apsey]

I'm having a strange issue with keystone after migrating all public 
endpoints to https (haproxy terminates the SSL connection for each 
service):


openstack endpoint list

+--+---+--++-+---+-+
| ID   | Region| Service Name | Service 
Type   | Enabled | Interface | URL   
  |

+--+---+--++-+---+-+
...
| 99d302d00ab3461cb9362236c865a430 | RegionOne | keystone | identity 
  | True| public| https://some.domain.place:5000/v3  
   |

...

I have also updated my rc files appropriately.  Whenever I try and use 
the CLI against the public endpoints in debug mode, everything starts 
out looking good:


REQ: curl -g -i -X GET https://some.domain.place:5000/v3 -H "Accept: 
application/json" -H "User-Agent: osc-lib keystoneauth1/2.12.1 
python-requests/2.11.1 CPython/2.7.9"


But then, the response body gives a non-https URL:

RESP BODY: {"version": {"status": "stable", "updated": 
"2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", 
"type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", 
"links": [{"href": "http://some.domain.place:5000/v3/;, "rel": 
"self"}]}}


and then the attempt to authenticate fails:

Making authentication request to 
http://some.domain.place:5000/v3/auth/tokens

Starting new HTTP connection (1): some.domain.place
Unable to establish connection to 
http://some.domain.place:5000/v3/auth/tokens


I've restarted apache2 on my keystone hosts and I have scoured the 
database for any reference to a non-https public endpoint for keystone; 
I cannot find one.


Does anyone know why my response body is giving the wrong URL?  Horizon 
works perfectly fine with the https endpoints; it's just the command 
line clients that are having issues.


Thanks in advance,

--
v/r

Chris Apsey
bitskr...@bitskrieg.net
https://www.bitskrieg.net

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [telecom-nfv] Meeting #19 tomorrow

[Curtis]

Hi All,

We will have our bi-weekly meeting tomorrow.

If you can attend, please do, and feel free to add to the agenda as it
is quite light right now [1]. Might be a short meeting without some
more items. :)

Thanks,
Curtis.

[1]: https://etherpad.openstack.org/p/ops-telco-nfv-meeting-agenda

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Large deployments] Neutron issues in Openstack Large deployment using DVR

[Satyanarayana Patibandla]

Hi Saverio,

We have tried to create 20 VMs each time using heat template. There is 1
sec time gap between each VM creation request. When we reached 114 VMs we
got the error mentioned in the below mail.Heat template will boot instance
from volume and it assigns floating IP to the instance.

Except neutron-server container we restarted all the neutron agent
containers which are present on all network and compute nodes. We are using
kolla to deploy openstack services.

We are using 1 month old master branch openstack code to deploy our
services.

Please find the error logs in the below link.
http://paste.openstack.org/show/599892/

Thanks,
Satya.P

On Wed, Feb 22, 2017 at 12:21 AM, Saverio Proto  wrote:

> Hello Satya,
>
> I would fill a bug on launchpad for this issue.
> 114 VMs is not much. Can you identify how to trigger the issue to
> reproduce it ? or it just happens randomly ?
>
> When you say rebooting the network node, do you mean the server
> running the neutron-server process ?
>
> what version and distribution of openstack are you using ?
>
> thank you
>
> Saverio
>
>
> 2017-02-21 13:54 GMT+01:00 Satyanarayana Patibandla
> :
> > Hi All,
> >
> > We are trying to deploy Openstack in our production environment. For
> > networking we are using DVR with out L3 HA. We are able to create 114 VMs
> > with out any issue. After creating 114 VMs we are getting the below
> error.
> >
> > Error: 504 Gateway Time-out The server didn't
> respond
> > in time. 
> >
> > Neutron services are getting freezed up due to a persistent lock on the
> > agents table. it seems one of the network node is holding the lock on the
> > table. After rebooting the network node, the Neutron CLI was responsive
> > again.
> >
> > Neutron agent and neutron server is throwing below errors.
> >
> > Neutron-server errors:
> > ERROR oslo_db.sqlalchemy.exc_filters "Can't reconnect until invalid "
> > ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't reconnect
> > until invalid transaction is rolled back
> > ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-59db203e72c6
> > 3eb776587c9c40569731ebe5c3557bc7 f43e8699cd5a46e89ffe39e3cac75341 - - -]
> > index failed: No details.
> > ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid
> > transaction is rolled back
> >
> >
> > Neutron agents errors:
> > MessagingTimeout: Timed out waiting for a reply to message ID
> > 40638b6bf12c44cd9a404ecaa14a9909
> >
> > Could you please provide us your valuable inputs or suggestions for above
> > errors.
> >
> > Thanks,
> > Satya.P
> >
> > ___
> > OpenStack-operators mailing list
> > OpenStack-operators@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [scientific][scientific-wg] Reminder: IRC meeting Tuesday 2100

[Stig Telfer]

Greetings -

We have a Scientific WG IRC meeting shortly, at 2100 UTC in channel 
#openstack-meeting

The agenda is available here[1] and full IRC meeting details are here[2].

This week we’ll survey some new work in the HPC monitoring landscape, look at 
the study coming together for identity federation and look at some options for 
an evening social at the Boston summit.

Best wishes,
Stig

[1] 
https://wiki.openstack.org/wiki/Scientific_working_group#IRC_Meeting_February_21st_2017
 

[2] http://eavesdrop.openstack.org/#Scientific_Working_Group 


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [Large deployments] Neutron issues in Openstack Large deployment using DVR

[Saverio Proto]

Hello Satya,

I would fill a bug on launchpad for this issue.
114 VMs is not much. Can you identify how to trigger the issue to
reproduce it ? or it just happens randomly ?

When you say rebooting the network node, do you mean the server
running the neutron-server process ?

what version and distribution of openstack are you using ?

thank you

Saverio


2017-02-21 13:54 GMT+01:00 Satyanarayana Patibandla
:
> Hi All,
>
> We are trying to deploy Openstack in our production environment. For
> networking we are using DVR with out L3 HA. We are able to create 114 VMs
> with out any issue. After creating 114 VMs we are getting the below error.
>
> Error: 504 Gateway Time-out The server didn't respond
> in time. 
>
> Neutron services are getting freezed up due to a persistent lock on the
> agents table. it seems one of the network node is holding the lock on the
> table. After rebooting the network node, the Neutron CLI was responsive
> again.
>
> Neutron agent and neutron server is throwing below errors.
>
> Neutron-server errors:
> ERROR oslo_db.sqlalchemy.exc_filters "Can't reconnect until invalid "
> ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't reconnect
> until invalid transaction is rolled back
> ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-59db203e72c6
> 3eb776587c9c40569731ebe5c3557bc7 f43e8699cd5a46e89ffe39e3cac75341 - - -]
> index failed: No details.
> ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid
> transaction is rolled back
>
>
> Neutron agents errors:
> MessagingTimeout: Timed out waiting for a reply to message ID
> 40638b6bf12c44cd9a404ecaa14a9909
>
> Could you please provide us your valuable inputs or suggestions for above
> errors.
>
> Thanks,
> Satya.P
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Daniel P. Berrange]

On Tue, Feb 21, 2017 at 10:08:54AM -0500, David Medberry wrote:
> Doesn't the virtio solution assume/require a libvirt or more exactly a
> QEMU/KVM based hypervisor?
> 
> What about the N-1 other hypervisors?

vsock is an equivalent of UNIX domain sockets, for host<->guest communication
that was invented by VMWare.  KVM adopted the same socket protocol and layers
it over virtio. So from an application developer POV, whether you use VMWare
or KVM, vsock works the same way.  From a technology POV it wouldbe possible
to use vsock over virtio with Xen, since Xen supports the virtio bus, but I
doubt XenAPI will support it any time soon. Likewise I doubt hyperv will
support it, but they may well provide an equivalent alternatie, since this
is a commonly needed concept.

> I think the idea of a "hot remove, hot add" of the configdrive has some
> merit (but remember it is not always ISO-9660 but could be VFAT as well to
> aid in some migrations.)

NB current config drive is not well suited to hot remove. If using CDROM
with ISO-9660 you the guest to unlock the CDROM tray before the host is
able to eject & change media in it.  If using virtio disk with vfat, you
need the guest to honour ACPI request to unplug the PCI device. If the
disk is mounted, this unplug will not complete.

So realistically you'd need to change to use USB disks, which the host is
able to rip out from under the guest unconditionally, even if it is mountd
by the guest. Albeit with the caveat that the guest now needs to clean
up dangling mount points.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[David Medberry]

Doesn't the virtio solution assume/require a libvirt or more exactly a
QEMU/KVM based hypervisor?

What about the N-1 other hypervisors?

I think the idea of a "hot remove, hot add" of the configdrive has some
merit (but remember it is not always ISO-9660 but could be VFAT as well to
aid in some migrations.)

On the plus side, we do actually run the md service.

On Mon, Feb 20, 2017 at 1:22 PM, Artom Lifshitz  wrote:

> We've been having a discussion [1] in openstack-dev about how to best
> expose dynamic metadata that changes over a server's lifetime to the
> server. The specific use case is device role tagging with hotplugged
> devices, where a network interface or volume is attached with a role
> tag, and the guest would like to know what that role tag is right
> away.
>
> The metadata API currently fulfills this function, but my
> understanding is that it's not hugely popular amongst operators and is
> therefore not universally deployed.
>
> Dan Berrange came up with an idea [2] to add virtio-vsock support to
> Nova. To quote his explanation, " think of this as UNIX domain sockets
> between the host and guest. [...] It'd likely address at least some
> people's security concerns wrt metadata service. It would also fix the
> ability to use the metadata service in IPv6-only environments, as we
> would not be using IP at all."
>
> So to those operators who are not deploying the metadata service -
> what are your reasons for doing so, and would those concerns be
> addressed by Dan's idea?
>
> Cheers!
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2017-
> February/112490.html
> [2] http://lists.openstack.org/pipermail/openstack-dev/2017-
> February/112602.html
>
> --
> Artom Lifshitz
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Daniel P. Berrange]

On Tue, Feb 21, 2017 at 01:09:43PM +, Jeremy Stanley wrote:
> On 2017-02-21 06:24:20 -0500 (-0500), Clint Byrum wrote:
> [...]
> > Why not just make it a virtual USB drive that ejects and
> > re-attaches on changes?
> [...]
> 
> Is there a graceful way to trigger that from the host so that the
> guest knows to unmount it prior to ejection? Or is that not actually
> necessary?

No, there's no way to coordinate USB removal with the guest - you would
just be ripping the drive out from under the guest and expecting the
guest to clean up its mounts if it had it mounted. Not too pleasant,
but also not an entirely unusual scenario for an OS to deal with. After
all, users of physical hardware sometimes pull out USB sticks without
going through any unmount / safe remove process.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Jeremy Stanley]

On 2017-02-21 06:24:20 -0500 (-0500), Clint Byrum wrote:
[...]
> Why not just make it a virtual USB drive that ejects and
> re-attaches on changes?
[...]

Is there a graceful way to trigger that from the host so that the
guest knows to unmount it prior to ejection? Or is that not actually
necessary?
-- 
Jeremy Stanley

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] [Large deployments] Neutron issues in Openstack Large deployment using DVR

[Satyanarayana Patibandla]

Hi All,

We are trying to deploy Openstack in our production environment. For
networking we are using DVR with out L3 HA. We are able to create 114 VMs
with out any issue. After creating 114 VMs we are getting the below error.

Error: 504 Gateway Time-out The server didn't respond
in time. 

Neutron services are getting freezed up due to a persistent lock on the
agents table. it seems one of the network node is holding the lock on the
table. After rebooting the network node, the Neutron CLI was responsive
again.

Neutron agent and neutron server is throwing below errors.

Neutron-server errors:
ERROR oslo_db.sqlalchemy.exc_filters "Can't reconnect until invalid "
ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't reconnect
until invalid transaction is rolled back
ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-59db203e72c6
3eb776587c9c40569731ebe5c3557bc7 f43e8699cd5a46e89ffe39e3cac75341 - - -]
index failed: No details.
ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid
transaction is rolled back


Neutron agents errors:
MessagingTimeout: Timed out waiting for a reply to message ID
40638b6bf12c44cd9a404ecaa14a9909

Could you please provide us your valuable inputs or suggestions for above
errors.

Thanks,
Satya.P
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Clint Byrum]

Excerpts from Daniel P. Berrange's message of 2017-02-21 10:40:02 +:
> On Mon, Feb 20, 2017 at 02:36:15PM -0500, Clint Byrum wrote:
> > What exactly is the security concern of the metadata service? Perhaps
> > those concerns can be addressed directly?
> > 
> > I ask because anything that requires special software on the guest is
> > a non-starter IMO. virtio is a Linux thing, so what does this do for
> > users of Windows?  FreeBSD? etc.
> 
> Red Hat is investing in creating virtio vsock drivers for Windows
> but I don't have an ETA for that yet. There's no work in *BSD in
> this area that I know of, but BSD does have support for virtio
> in general, so if virtio-vsock becomes used in any important
> places I would not be suprised if some BSD developers implemented
> vsock too.
> 

> In any case, I don't think it neccessarily needs to be supported
> in every single possible scenario. The config drive provides the
> same data in a highly portable manner, albeit with the caveat
> about it being read-only. The use of metadata service (whether
> TCP or vsock based) is useful for cases needing the info from
> config drive to be dynamically updated - eg the role device
> tagging metadata. Only a very small subset of guests running on
> openstack actually use that data today. So it would not be the
> end of the world if some guests don't support vsock in the short
> to medium term - if the facility proves to be critically important
> to a wider range of guests that'll motivate developers of those
> OS to support it.
> 

Cool, so there's a chance it gets to near ubiquitous usability.

However, I wonder, there's no need for performance here. Why not just
make it a virtual USB drive that ejects and re-attaches on changes? That
way you don't need Windows/BSD drivers.

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Daniel P. Berrange]

On Mon, Feb 20, 2017 at 02:36:15PM -0500, Clint Byrum wrote:
> What exactly is the security concern of the metadata service? Perhaps
> those concerns can be addressed directly?
> 
> I ask because anything that requires special software on the guest is
> a non-starter IMO. virtio is a Linux thing, so what does this do for
> users of Windows?  FreeBSD? etc.

Red Hat is investing in creating virtio vsock drivers for Windows
but I don't have an ETA for that yet. There's no work in *BSD in
this area that I know of, but BSD does have support for virtio
in general, so if virtio-vsock becomes used in any important
places I would not be suprised if some BSD developers implemented
vsock too.

In any case, I don't think it neccessarily needs to be supported
in every single possible scenario. The config drive provides the
same data in a highly portable manner, albeit with the caveat
about it being read-only. The use of metadata service (whether
TCP or vsock based) is useful for cases needing the info from
config drive to be dynamically updated - eg the role device
tagging metadata. Only a very small subset of guests running on
openstack actually use that data today. So it would not be the
end of the world if some guests don't support vsock in the short
to medium term - if the facility proves to be critically important
to a wider range of guests that'll motivate developers of those
OS to support it.


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [nova] Metadata service over virtio-vsock

[Daniel P. Berrange]

On Mon, Feb 20, 2017 at 08:08:00PM +, Jeremy Stanley wrote:
> On 2017-02-20 14:36:15 -0500 (-0500), Clint Byrum wrote:
> > What exactly is the security concern of the metadata service? Perhaps
> > those concerns can be addressed directly?
> [...]
> 
> A few I'm aware of:
> 
> 1. It's something that runs in the control plane but needs to be
> reachable from untrusted server instances (which may themselves even
> want to be on completely non-routed networks).

That is the key problem that virtio-vsock solves, by separating
traffic out from the network stack there's no way for a guest
to use vsock to access anything except services on the local
compute host listening on vsock

> 2. If you put a Web proxy between your server instances and the
> metadata service and also make it reachable without going through
> that proxy then instances may be able to spoof one another
> (OSSN-0074).

FYI, with virtio-vsock it is impossible for the guest to spoof
the sending address of another guest. So the process on the host
can use the socket peer address to reliably identify which guest
it is communicating with. With the IP based metadata service
you need to setup firewall rules on the host to drop traffic
with spoofed source mac/ip address.

> 3. Lots of things, for example facter, like to beat on it heavily
> which makes for a fun DDoS and so is a bit of a scaling challenge in
> large deployments.

FYI, with virtio-vsock, you would need to either run the metdata
service on every compute host, or have some kind of vhost<->tcp
proxy on every compute host that forwards requests to the real
metadata service off-host.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators