Re: [Openstack] Configure Rate limits on OS API
On Fri, Dec 30, 2011 at 11:34 AM, Jay Pipes jaypi...@gmail.com wrote: I struggled to figure out how to change this ratelimiting stuff for ages while working on Tempest. Eventually, I just removed the ratelimit middleware entirely from the pipeline since I could find no documentation whatsoever on how to change the limits. It would be awesome if this was documented in the Nova docs. I've added a bug to remind about it: https://bugs.launchpad.net/nova/+bug/910193 Cheers, There's a merge prop for the compute admin documentation at https://review.openstack.org/#change,3653 Comments appreciated. mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
On Tue, Dec 27, 2011 at 2:33 PM, Nirmal Ranganathan rnir...@gmail.comwrote: You can configure those values thru the paste conf. [filter:ratelimit] paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory limits =(POST, *, .*, 10, MINUTE);(POST, */servers, ^/servers, 50, DAY);(PUT, *, .*, 10, MINUTE);(GET, *changes-since*, .*changes-since.*, 3, MINUTE);(DELETE, *, .*, 100, MINUTE) Am I correct in assuming that this will only work with setting the global limits? Is there anyway to specify different limits for different accounts or groups of accounts? -Blake On Mon, Dec 19, 2011 at 1:28 PM, Day, Phil philip@hp.com wrote: Hi Folks, ** ** Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? ** ** I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? ** ** Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nirmal http://rnirmal.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
On Tue, 2012-01-10 at 16:06 -0600, Blake Yeager wrote: Am I correct in assuming that this will only work with setting the global limits? Is there anyway to specify different limits for different accounts or groups of accounts? You are correct that the 'limits = […]' syntax sets global limits. However, if you use 'user:username = […]', that should allow you to set specific limits for a given user. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
Hi Blake, Repose is capable of rate limiting based on group. It also supports querying limits and maintaining the limits consistent even as nodes are scaled horizontally. You can find the code on git hub: https://github.com/rackspace/repose Here's the presentation I gave on the subject on Essex: https://github.com/rackspace/repose/raw/master/documentation/presentations/OpenStack_Essex_2011/ReposePresentation.pdf Our mailing lists if you have further questions here: http://lists.openrepose.org/mailman/listinfo -jOrGe W. On Jan 10, 2012, at 4:06 PM, Blake Yeager wrote: On Tue, Dec 27, 2011 at 2:33 PM, Nirmal Ranganathan rnir...@gmail.commailto:rnir...@gmail.com wrote: You can configure those values thru the paste conf. [filter:ratelimit] paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory limits =(POST, *, .*, 10, MINUTE);(POST, */servers, ^/servers, 50, DAY);(PUT, *, .*, 10, MINUTE);(GET, *changes-since*, .*changes-since.*, 3, MINUTE);(DELETE, *, .*, 100, MINUTE) Am I correct in assuming that this will only work with setting the global limits? Is there anyway to specify different limits for different accounts or groups of accounts? -Blake On Mon, Dec 19, 2011 at 1:28 PM, Day, Phil philip@hp.commailto:philip@hp.com wrote: Hi Folks, Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nirmal http://rnirmal.comhttp://rnirmal.com/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
2011/12/29 Vishvananda Ishaya vishvana...@gmail.com: In addition to the rate limiting changes, you can also run multiple copies of nova-api on different ports and put a load balancer (like ha-proxy) in front. You may have to set up your proxy to add an X-Forwarded-For header and specify the --use_forwarded_for flag to get things like metadata to work. I can't think of any reason we can't just fork() N times (where N defaults to number of cores, but could be overridden by a flag) after socket.listen(), but before we go into the socket.accept() loop? -- Soren Hansen | http://linux2go.dk/ Ubuntu Developer | http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
On Mon, Jan 2, 2012 at 9:00 AM, Soren Hansen so...@linux2go.dk wrote: 2012/1/2 Jay Pipes jaypi...@gmail.com: On Mon, Jan 2, 2012 at 6:27 AM, Soren Hansen so...@linux2go.dk wrote: 2011/12/29 Vishvananda Ishaya vishvana...@gmail.com: I can't think of any reason we can't just fork() N times (where N defaults to number of cores, but could be overridden by a flag) after socket.listen(), but before we go into the socket.accept() loop? This is precisely what Swift does. And, very shortly, Glance: https://review.openstack.org/#change,2486 Ah, neat! Now, if only we could get stuff like this moved into openstack-common :) I'm all for that, as you know :) -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
I struggled to figure out how to change this ratelimiting stuff for ages while working on Tempest. Eventually, I just removed the ratelimit middleware entirely from the pipeline since I could find no documentation whatsoever on how to change the limits. It would be awesome if this was documented in the Nova docs. I've added a bug to remind about it: https://bugs.launchpad.net/nova/+bug/910193 Cheers, -jay On Tue, Dec 27, 2011 at 3:33 PM, Nirmal Ranganathan rnir...@gmail.com wrote: You can configure those values thru the paste conf. [filter:ratelimit] paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory limits =(POST, *, .*, 10, MINUTE);(POST, */servers, ^/servers, 50, DAY);(PUT, *, .*, 10, MINUTE);(GET, *changes-since*, .*changes-since.*, 3, MINUTE);(DELETE, *, .*, 100, MINUTE) On Mon, Dec 19, 2011 at 1:28 PM, Day, Phil philip@hp.com wrote: Hi Folks, Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nirmal http://rnirmal.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
Wow! Didn't know about it. Really valuable information. I have been looking for that information for ages. Happy New Year folks, Diego -- Diego Parrilla http://www.stackops.com/*CEO* *www.stackops.com | * diego.parri...@stackops.com** | +34 649 94 43 29 | skype:diegoparrilla* * http://www.stackops.com/ * * ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law. On Tue, Dec 27, 2011 at 9:33 PM, Nirmal Ranganathan rnir...@gmail.comwrote: You can configure those values thru the paste conf. [filter:ratelimit] paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory limits =(POST, *, .*, 10, MINUTE);(POST, */servers, ^/servers, 50, DAY);(PUT, *, .*, 10, MINUTE);(GET, *changes-since*, .*changes-since.*, 3, MINUTE);(DELETE, *, .*, 100, MINUTE) On Mon, Dec 19, 2011 at 1:28 PM, Day, Phil philip@hp.com wrote: Hi Folks, ** ** Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? ** ** I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? ** ** Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nirmal http://rnirmal.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
You can configure those values thru the paste conf. [filter:ratelimit] paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory limits =(POST, *, .*, 10, MINUTE);(POST, */servers, ^/servers, 50, DAY);(PUT, *, .*, 10, MINUTE);(GET, *changes-since*, .*changes-since.*, 3, MINUTE);(DELETE, *, .*, 100, MINUTE) On Mon, Dec 19, 2011 at 1:28 PM, Day, Phil philip@hp.com wrote: Hi Folks, ** ** Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? ** ** I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? ** ** Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Nirmal http://rnirmal.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
I have a relevant question about API throughput. I'm using Diablo release version and I found out that there's only one process of nova-api to handle all API requests from even different users. That results very low (10) API throughput (definition: API requests handled per second). Similar symptom was observed back in Cactus release. Is there anyway to configure API service to run in multi-threaded mode so that it can utilize multi-core hardware? I'd really appreciate if anyone can also shed some light on why Nova API was designed to work in this way. Thank you in advance. On Tue, Dec 20, 2011 at 3:28 AM, Day, Phil philip@hp.com wrote: Hi Folks, ** ** Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? ** ** I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? ** ** Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Configure Rate limits on OS API
Hi Folks, Is there a file that can be used to configure the API rate limits for the OS API on a per user basis ? I can see where the default values are set in the code, but it looks as if there should be a less brutal configuration mechanism to go along with this ? Thanks Phil ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
