I'm floating this in case the idea has come up before.

I've spent most of my day so far digging through logs and code for gate race bug 1353962 [1] where an admin tenant is going over-quota when trying to allocate fixed IPs for a test instance (the test setup creates 2 instances and more test instances are created in some of the test cases).

I'm going to push up some patches to nova to make the logs more useful when we hit this, i.e. log the quota limit and current usage for the given resource (fixed_ips in this case).

But the idea that keeps coming up when I'm digging into stuff like this is it'd be nice to know when we're nearing over-quota so we can walk that back and figure out what was going on prior to us falling over the cliff. This isn't a novel idea, but I haven't heard it discussed before.

What I'm thinking is basically add a config option for setting a threshold value which would dump warnings when you're over that threshold. This could be configured per resource, e.g. I only care about fixed_ips, instances, etc. It could be configured per tenant (maybe we only care about this for the admin tenant?).

My use case would be specifically related to the bug in question, so I want to set the threshold at let's say 80%. With a default fixed_ips quota of 10, when I've gotten over 8 reserved for the admin tenant, I start logging warnings that I'm nearing over-quota. When we go over-quota on a request, we could then see what operations/requests were going on before this that tipped us over.

Thoughts?

[1] https://bugs.launchpad.net/nova/+bug/1353962

--

Thanks,

Matt Riedemann


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to