[Yahoo-eng-team] [Bug 1585510] Re: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer
Looks like this was fixed in neutron. Let me know if there's anything left to be done on the oslo side. Thanks. ** Changed in: oslo.rootwrap Status: New => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1585510 Title: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer Status in neutron: Fix Released Status in oslo.rootwrap: Won't Fix Bug description: As titled, when XenServer is hypervisor we want to implement rootwrap daemon mode in neutron-openvswitch-agent which runs in compute node. neutron-openvswitch-agent which runs in compute node(DomU) cannot support rootwrap daemon mode. This is because XenServer has the seperation of Dom0(privileged domain) and DomU(user domain), br-int bridge of neutron-openvswitch-agent(in compute node) resides in Dom0, so all the ovs-vsctl/ovs-ofctl/iptables/ipset commands executed by neutron-openvswitch-agent(in compute node) need to be executed in Dom0 not DomU which is different with other hypervisors. https://github.com/openstack/neutron/blob/master/bin/neutron-rootwrap- xen-dom0 is current implementation but cannot support rootwrap daemon. We noticed rootwrap produces significant performance overhead and We want to implement the rootwrap daemon mode when XenServer is hypervisor to improve the performance. Also, we discoverde that calls to netwrap (and creation of lots of sessions) are causing huge logging in dom0. Logrotate can handle those logs, but it will make diagnosis of issues very difficult indeed due to the very regular rotations. Also, it seems that perhaps the excessive logging is causing the host to be **very** slow downloading an image from glance due to contention on the disk (looking at iostat, %iowait is up over 60% the majority of the time, sometimes up to 90%) So, it's not stable and strong enough for a production OpenStack environment. Proposal: subclass and override some class/functions from oslo.rootwrap to achive the goal. Actually I have did the POC which can work well. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1585510/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1585510] Re: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer
Reviewed: https://review.openstack.org/390931 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8047da17db2d4d50f797d99880f3b76d0eba2084 Submitter: Jenkins Branch:master commit 8047da17db2d4d50f797d99880f3b76d0eba2084 Author: Jianghua Wang Date: Thu Oct 27 00:43:11 2016 +0800 XenAPI: Support daemon mode for rootwrap For Neutron's compute agent in a XenServer's compute node, the commands actually need run in Dom0. Currently XenServer only supports rootwrap for that purpose by invoking a script which invokes XenAPI to execute commands in dom0. There are much performance overhead due to it requires parsing on the script and the configuration file every time running commands. This change is to support daemon mode with which each agent service will call XenAPI directly to execute commands in dom0. And it will keep the single XenAPI session. DocImpact: Need update the following configuration. file: /etc/neutron/plugins/ml2/openvswitch_agent.ini [agent] root_helper_daemon = xenapi_root_helper [xenapi] connection_url = http://169.254.0.1 connection_username = root connection_password = xenroot Closes-Bug: #1585510 Change-Id: I684034359fe0571bc92dbcf342a9821553b1da35 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1585510 Title: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer Status in neutron: Fix Released Status in oslo.rootwrap: New Bug description: As titled, when XenServer is hypervisor we want to implement rootwrap daemon mode in neutron-openvswitch-agent which runs in compute node. neutron-openvswitch-agent which runs in compute node(DomU) cannot support rootwrap daemon mode. This is because XenServer has the seperation of Dom0(privileged domain) and DomU(user domain), br-int bridge of neutron-openvswitch-agent(in compute node) resides in Dom0, so all the ovs-vsctl/ovs-ofctl/iptables/ipset commands executed by neutron-openvswitch-agent(in compute node) need to be executed in Dom0 not DomU which is different with other hypervisors. https://github.com/openstack/neutron/blob/master/bin/neutron-rootwrap- xen-dom0 is current implementation but cannot support rootwrap daemon. We noticed rootwrap produces significant performance overhead and We want to implement the rootwrap daemon mode when XenServer is hypervisor to improve the performance. Also, we discoverde that calls to netwrap (and creation of lots of sessions) are causing huge logging in dom0. Logrotate can handle those logs, but it will make diagnosis of issues very difficult indeed due to the very regular rotations. Also, it seems that perhaps the excessive logging is causing the host to be **very** slow downloading an image from glance due to contention on the disk (looking at iostat, %iowait is up over 60% the majority of the time, sometimes up to 90%) So, it's not stable and strong enough for a production OpenStack environment. Proposal: subclass and override some class/functions from oslo.rootwrap to achive the goal. Actually I have did the POC which can work well. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1585510/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1585510] Re: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer
** Changed in: neutron Status: Opinion => In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1585510 Title: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer Status in neutron: In Progress Status in oslo.rootwrap: New Bug description: As titled, when XenServer is hypervisor we want to implement rootwrap daemon mode in neutron-openvswitch-agent which runs in compute node. neutron-openvswitch-agent which runs in compute node(DomU) cannot support rootwrap daemon mode. This is because XenServer has the seperation of Dom0(privileged domain) and DomU(user domain), br-int bridge of neutron-openvswitch-agent(in compute node) resides in Dom0, so all the ovs-vsctl/ovs-ofctl/iptables/ipset commands executed by neutron-openvswitch-agent(in compute node) need to be executed in Dom0 not DomU which is different with other hypervisors. https://github.com/openstack/neutron/blob/master/bin/neutron-rootwrap- xen-dom0 is current implementation but cannot support rootwrap daemon. We noticed rootwrap produces significant performance overhead and We want to implement the rootwrap daemon mode when XenServer is hypervisor to improve the performance. Also, we discoverde that calls to netwrap (and creation of lots of sessions) are causing huge logging in dom0. Logrotate can handle those logs, but it will make diagnosis of issues very difficult indeed due to the very regular rotations. Also, it seems that perhaps the excessive logging is causing the host to be **very** slow downloading an image from glance due to contention on the disk (looking at iostat, %iowait is up over 60% the majority of the time, sometimes up to 90%) So, it's not stable and strong enough for a production OpenStack environment. Proposal: subclass and override some class/functions from oslo.rootwrap to achive the goal. Actually I have did the POC which can work well. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1585510/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1585510] Re: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer
We may get back to the bug in neutron scope if and when the solution implemented in oslo scope will require some small integration bits in neutron, like we have for neutron-rootwrap wrapper in setup.cfg. ** Changed in: neutron Status: Incomplete => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1585510 Title: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer Status in neutron: Opinion Status in oslo.rootwrap: New Bug description: As titled, when XenServer is hypervisor we want to implement rootwrap daemon mode in neutron-openvswitch-agent which runs in compute node. neutron-openvswitch-agent which runs in compute node(DomU) cannot support rootwrap daemon mode. This is because XenServer has the seperation of Dom0(privileged domain) and DomU(user domain), br-int bridge of neutron-openvswitch-agent(in compute node) resides in Dom0, so all the ovs-vsctl/ovs-ofctl/iptables/ipset commands executed by neutron-openvswitch-agent(in compute node) need to be executed in Dom0 not DomU which is different with other hypervisors. https://github.com/openstack/neutron/blob/master/bin/neutron-rootwrap- xen-dom0 is current implementation but cannot support rootwrap daemon. We noticed rootwrap produces significant performance overhead and We want to implement the rootwrap daemon mode when XenServer is hypervisor to improve the performance. Also, we discoverde that calls to netwrap (and creation of lots of sessions) are causing huge logging in dom0. Logrotate can handle those logs, but it will make diagnosis of issues very difficult indeed due to the very regular rotations. Also, it seems that perhaps the excessive logging is causing the host to be **very** slow downloading an image from glance due to contention on the disk (looking at iostat, %iowait is up over 60% the majority of the time, sometimes up to 90%) So, it's not stable and strong enough for a production OpenStack environment. Proposal: subclass and override some class/functions from oslo.rootwrap to achive the goal. Actually I have did the POC which can work well. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1585510/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1585510] Re: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer
As I stated in the PoC patch, I don't believe neutron is the right place to maintain the code. Instead, oslo.rootwrap should get support for dom0 hypervisors. ** Also affects: oslo.rootwrap Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1585510 Title: [RFE] openvswitch-agent support rootwrap daemon when hypervisor is XenServer Status in neutron: Opinion Status in oslo.rootwrap: New Bug description: As titled, when XenServer is hypervisor we want to implement rootwrap daemon mode in neutron-openvswitch-agent which runs in compute node. neutron-openvswitch-agent which runs in compute node(DomU) cannot support rootwrap daemon mode. This is because XenServer has the seperation of Dom0(privileged domain) and DomU(user domain), br-int bridge of neutron-openvswitch-agent(in compute node) resides in Dom0, so all the ovs-vsctl/ovs-ofctl/iptables/ipset commands executed by neutron-openvswitch-agent(in compute node) need to be executed in Dom0 not DomU which is different with other hypervisors. https://github.com/openstack/neutron/blob/master/bin/neutron-rootwrap- xen-dom0 is current implementation but cannot support rootwrap daemon. We noticed rootwrap produces significant performance overhead and We want to implement the rootwrap daemon mode when XenServer is hypervisor to improve the performance. Also, we discoverde that calls to netwrap (and creation of lots of sessions) are causing huge logging in dom0. Logrotate can handle those logs, but it will make diagnosis of issues very difficult indeed due to the very regular rotations. Also, it seems that perhaps the excessive logging is causing the host to be **very** slow downloading an image from glance due to contention on the disk (looking at iostat, %iowait is up over 60% the majority of the time, sometimes up to 90%) So, it's not stable and strong enough for a production OpenStack environment. Proposal: subclass and override some class/functions from oslo.rootwrap to achive the goal. Actually I have did the POC which can work well. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1585510/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp