Public bug reported:
For no particular reason nova-scheduler cpu utilization can jump to
100%. I was unable to find any pattern and reason why this is happening.
We've small cluster 1 cloud controller and 7 node controllers. Except
high cpu usage nothing bad happens, we're able to create/delete
instances, after nova-scheduler restart everything goes back to normal
state.
I was able to strace 2 processes while nova-scheduler was using 100%
cpu.
1st process is in loop and it prints:
<12>2014-09-16 00:02:21.501 5668 WARNING nova.openstack.common.loopingcall [-]
task run outlasted interval by 12.322771 sec\0"
2nd processes is in loop as well and it's repeating:
epoll_ctl(6, EPOLL_CTL_ADD, 3, {EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=3,
u64=40095890530107395}}) = 0
epoll_wait(6, {{EPOLLOUT, {u32=3, u64=40095890530107395}}}, 1023, 878) = 1
epoll_ctl(6, EPOLL_CTL_DEL, 3,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRNORM|EPOLLMSG|EPOLLHUP|0x2485d020, {u32=32708,
u64=23083065509183428}}) = 0
sendto(3, "<14>2014-09-16 00:44:19.272 5673 INFO
oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on
10.3.128.254:5672\0", 125, 0, NULL, 0) = -1 ENOTCONN (Transport endpoint is not
connected)
Other processes doesn't have any issues with AMQP server just only nova-
scheduler.
We're using Debian, kernel 3.14-1-amd64, nova-scheduler 2014.1.2-1.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1371084
Title:
nova-scheduler high cpu usage
Status in OpenStack Compute (Nova):
New
Bug description:
For no particular reason nova-scheduler cpu utilization can jump to
100%. I was unable to find any pattern and reason why this is
happening. We've small cluster 1 cloud controller and 7 node
controllers. Except high cpu usage nothing bad happens, we're able to
create/delete instances, after nova-scheduler restart everything goes
back to normal state.
I was able to strace 2 processes while nova-scheduler was using 100%
cpu.
1st process is in loop and it prints:
<12>2014-09-16 00:02:21.501 5668 WARNING nova.openstack.common.loopingcall
[-] task run outlasted interval by 12.322771 sec\0"
2nd processes is in loop as well and it's repeating:
epoll_ctl(6, EPOLL_CTL_ADD, 3, {EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=3,
u64=40095890530107395}}) = 0
epoll_wait(6, {{EPOLLOUT, {u32=3, u64=40095890530107395}}}, 1023, 878) = 1
epoll_ctl(6, EPOLL_CTL_DEL, 3,
{EPOLLRDNORM|EPOLLRDBAND|EPOLLWRNORM|EPOLLMSG|EPOLLHUP|0x2485d020, {u32=32708,
u64=23083065509183428}}) = 0
sendto(3, "<14>2014-09-16 00:44:19.272 5673 INFO
oslo.messaging._drivers.impl_rabbit [-] Connecting to AMQP server on
10.3.128.254:5672\0", 125, 0, NULL, 0) = -1 ENOTCONN (Transport endpoint is not
connected)
Other processes doesn't have any issues with AMQP server just only
nova-scheduler.
We're using Debian, kernel 3.14-1-amd64, nova-scheduler 2014.1.2-1.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1371084/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
