Just FYI- swift's ratelimiting doesn't limit on bandwidth. Just requests per second to specific resources.
David On Mar 30, 2012, at 1:40 PM, Caitlin Bestler wrote: > <Caitlin> Replies inline </Caitlin> > > -----Original Message----- > From: andi abes [mailto:[email protected]] > Sent: Friday, March 30, 2012 11:32 AM > To: Jay Pipes; Caitlin Bestler > Cc: [email protected] > Subject: Re: [Openstack] Question on wsgi rate limiting > > Caitlin, I'm curious what were the use cases and concerns in DCB? > If my memory serves me right (from rate limiting at L2 level) the main issues > are guaranteeing QoS, effective bandwidth usage, fair allocation of memory > buffer space. All of those goals damaged pretty badly if congestion occurs > and any of the resources are exhausted (leading to indiscriminate packet > loss, for lack of any other recourse). > <Caitlin> > Correct. The fundamental goal was to allow storage-oriented classes of > service which could be effectively guaranteed to be drop-free within a > Datacenter. > FCoE as a specific application needed this to be a very strong guarantee, but > congestion drops triggering TCP back-offs is generally not a good thing for > storage traffic. > > And once you cut through the match, this is ultimately about good algorithms > that can be implemented in a distributed fashion that allocate the buffering > capacity > Of the network elements somewhat intelligently and robustly. > </Caitlin> > > The use cases, at least the way as I conceive them for OS are very different. > They're not intended to resolve resource constraints as a primary goal > (though, that's definitely a secondary goal). As an example - issuing lots > of Nova API calls is by itself not the problem > - executing the downstream effects of what those requests trigger is whats > being protected (access to DB, spawning VM's, Rabbit MQ message rates etc). > > For Swift, where pure bandwidth is a primary concern, and the primary > resource being consumed - I imagine you're right. Some L2/L3 traffic shaping > (and monitoring) would be advisable - but that's not to say that's the only > resource. e.g. creating and deleting containers repeatedly will consume > relatively little bandwidth, but will exert quite a lot of resource > consumption on the back end. Rate limiting these API calls is probably > prudent at the API layer. > <Caitlin> > Yes, Nova APIs calls are very unlikely to cause network congestion. > Bulk payload transfers (whether Swift or Nova Volumes) is an issue. > My concern is that a truly effective solution will have to be at the Quantum > level, not wsgi. > </Caitlin> > > > > > > On Fri, Mar 30, 2012 at 1:56 PM, Jay Pipes <[email protected]> wrote: >> You make some good points about what is the appropriate level in the >> stack to do rate shaping, but I would just like to have a >> configurable, manageable and monitorable ratelimit/quota solution that >> doesn't seem like a giant hack >> :) >> >> Baby steps. >> >> -jay >> >> >> On 03/30/2012 01:23 PM, Caitlin Bestler wrote: >>> >>> Throughout the discussion on distributed rate limiting I've had the >>> annoying feeling that I've heard this joke before. >>> >>> Basically, are we looking for our keys under the street lamp because >>> the light is good rather than looking for them where they were lost? >>> >>> Has anyone studied the effectiveness of rate limitations implemented >>> at this layer. From my experience with rate >>> >>> shaping discussions in IEEE 802.1 Data Center Bridging group I am >>> concerned that the response time working this >>> >>> far up the stack will preclude effective rate shaping. >>> >>> Of course, if someone has studied this and shown it to be effective >>> then this would be great news. The light is >>> >>> Better under the street lamp. >>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack Post to : >>> [email protected] Unsubscribe : >>> https://launchpad.net/~openstack More help : >>> https://help.launchpad.net/ListHelp >> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack Post to : >> [email protected] Unsubscribe : >> https://launchpad.net/~openstack More help : >> https://help.launchpad.net/ListHelp > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : [email protected] > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
