** Changed in: nova
Status: Fix Committed => Fix Released
** Changed in: nova
Milestone: None => liberty-2
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1450438
Title:
loopingcall: if a time drift to the future occurs, all timers will be
blocked
Status in OpenStack Compute (nova):
Fix Released
Status in oslo-incubator:
Fix Committed
Status in oslo.service:
Fix Released
Bug description:
Due to the fact that loopingcall.py uses time.time for recording wall-
clock time which is not guaranteed to be monotonic, if a time drift to
the future occurs, and then gets corrected, all the timers will get
blocked until the actual time reaches the moment of the original
drift.
This can be pretty bad if the interval is not insignificant - in
Nova's case - all services uses FixedIntervalLoopingCall for it's
heartbeat periodic tasks - if a drift is on the order of magnitude of
several hours, no heartbeats will happen.
DynamicLoopingCall is affected by this as well but because it relies
on eventlet which would also use a non-monotonic time.time function
for it's internal timers.
Solving this will require looping calls to start using a monotonic
timer (for python 2.7 there is a monotonic package).
Also all services that want to use timers and avoid this issue should
doe something like
import monotonic
hub = eventlet.get_hub()
hub.clock = monotonic.monotonic
immediately after calling eventlet.monkey_patch()
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1450438/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
