** Also affects: neutron
Importance: Undecided
Status: New
** No longer affects: openvswitch (Ubuntu)
** Summary changed:
- ovsdb-client processes not getting cleaned up
+ neutron-rootwrap processes not getting cleaned up
** Description changed:
- This can be recreated with the openstack charms using xenial-newton-staging.
On newton deploys, neutron-gateway and nova-compute units will exhaust memory
due to compounding ovsdb-client processes:
+ neutron-rootwrap processes aren't getting cleaned up on Newton. I'm
+ testing with Newton rc3.
+
+ I was noticing memory exhaustion on my neutron gateway units, which turned
out to be due to compounding neutron-rootwrap processes:
sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client
monitor Interface name,ofport,external_ids --format=json
$ top -n1 -b -o VIRT
http://paste.ubuntu.com/23252407/
$ ps aux|grep ovsdb-client
http://paste.ubuntu.com/23252658/
Restarting openvswitch cleans up the processes but they just start piling
again up soon after:
sudo systemctl restart openvswitch-switch
+
+ At first I thought this was an openvswitch issue, however I reverted the
+ code in get_root_helper_child_pid() and neutron-rootwrap processes
+ started getting cleaned up. See corresponding commit at [1].
+
+ This can be recreated with the openstack charms using xenial-newton-
+ staging. On newton deploys, neutron-gateway and nova-compute units will
+ exhaust memory due to compounding ovsdb-client processes.
+
+ [1]
+ commit fd93e19f2a415b3803700fc491749daba01a4390
+ Author: Assaf Muller <amul...@redhat.com>
+ Date: Fri Mar 18 16:29:26 2016 -0400
+
+ Change get_root_helper_child_pid to stop when it finds cmd
+
+ get_root_helper_child_pid recursively finds the child of pid,
+ until it can no longer find a child. However, the intention is
+ not to find the deepest child, but to strip away root helpers.
+ For example 'sudo neutron-rootwrap x' is supposed to find the
+ pid of x. However, in cases 'x' spawned quick lived children of
+ its own (For example: ip / brctl / ovs invocations),
+ get_root_helper_child_pid returned those pids if called in
+ the wrong time.
+
+ Change-Id: I582aa5c931c8bfe57f49df6899445698270bb33e
+ Closes-Bug: #1558819
** Description changed:
neutron-rootwrap processes aren't getting cleaned up on Newton. I'm
testing with Newton rc3.
I was noticing memory exhaustion on my neutron gateway units, which turned
out to be due to compounding neutron-rootwrap processes:
sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client
monitor Interface name,ofport,external_ids --format=json
$ top -n1 -b -o VIRT
http://paste.ubuntu.com/23252407/
$ ps aux|grep ovsdb-client
http://paste.ubuntu.com/23252658/
Restarting openvswitch cleans up the processes but they just start piling
again up soon after:
sudo systemctl restart openvswitch-switch
At first I thought this was an openvswitch issue, however I reverted the
code in get_root_helper_child_pid() and neutron-rootwrap processes
- started getting cleaned up. See corresponding commit at [1].
+ started getting cleaned up. See corresponding commit for code that
+ possibly introduced this at [1].
This can be recreated with the openstack charms using xenial-newton-
staging. On newton deploys, neutron-gateway and nova-compute units will
exhaust memory due to compounding ovsdb-client processes.
[1]
commit fd93e19f2a415b3803700fc491749daba01a4390
Author: Assaf Muller <amul...@redhat.com>
Date: Fri Mar 18 16:29:26 2016 -0400
- Change get_root_helper_child_pid to stop when it finds cmd
-
- get_root_helper_child_pid recursively finds the child of pid,
- until it can no longer find a child. However, the intention is
- not to find the deepest child, but to strip away root helpers.
- For example 'sudo neutron-rootwrap x' is supposed to find the
- pid of x. However, in cases 'x' spawned quick lived children of
- its own (For example: ip / brctl / ovs invocations),
- get_root_helper_child_pid returned those pids if called in
- the wrong time.
-
- Change-Id: I582aa5c931c8bfe57f49df6899445698270bb33e
- Closes-Bug: #1558819
+ Change get_root_helper_child_pid to stop when it finds cmd
+
+ get_root_helper_child_pid recursively finds the child of pid,
+ until it can no longer find a child. However, the intention is
+ not to find the deepest child, but to strip away root helpers.
+ For example 'sudo neutron-rootwrap x' is supposed to find the
+ pid of x. However, in cases 'x' spawned quick lived children of
+ its own (For example: ip / brctl / ovs invocations),
+ get_root_helper_child_pid returned those pids if called in
+ the wrong time.
+
+ Change-Id: I582aa5c931c8bfe57f49df6899445698270bb33e
+ Closes-Bug: #1558819
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1629097
Title:
neutron-rootwrap processes not getting cleaned up
Status in neutron:
New
Bug description:
neutron-rootwrap processes aren't getting cleaned up on Newton. I'm
testing with Newton rc3.
I was noticing memory exhaustion on my neutron gateway units, which turned
out to be due to compounding neutron-rootwrap processes:
sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client
monitor Interface name,ofport,external_ids --format=json
$ top -n1 -b -o VIRT
http://paste.ubuntu.com/23252407/
$ ps aux|grep ovsdb-client
http://paste.ubuntu.com/23252658/
Restarting openvswitch cleans up the processes but they just start piling
again up soon after:
sudo systemctl restart openvswitch-switch
At first I thought this was an openvswitch issue, however I reverted
the code in get_root_helper_child_pid() and neutron-rootwrap processes
started getting cleaned up. See corresponding commit for code that
possibly introduced this at [1].
This can be recreated with the openstack charms using xenial-newton-
staging. On newton deploys, neutron-gateway and nova-compute units
will exhaust memory due to compounding ovsdb-client processes.
[1]
commit fd93e19f2a415b3803700fc491749daba01a4390
Author: Assaf Muller <amul...@redhat.com>
Date: Fri Mar 18 16:29:26 2016 -0400
Change get_root_helper_child_pid to stop when it finds cmd
get_root_helper_child_pid recursively finds the child of pid,
until it can no longer find a child. However, the intention is
not to find the deepest child, but to strip away root helpers.
For example 'sudo neutron-rootwrap x' is supposed to find the
pid of x. However, in cases 'x' spawned quick lived children of
its own (For example: ip / brctl / ovs invocations),
get_root_helper_child_pid returned those pids if called in
the wrong time.
Change-Id: I582aa5c931c8bfe57f49df6899445698270bb33e
Closes-Bug: #1558819
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1629097/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
