> On Jun 24, 2019, at 8:33 PM, Dk Jack <dnj0...@gmail.com> wrote:
>
> I suspect connection pools would be using the http pipeline feature to
> multiplex multiple client connections on to the pool connections towards
> ATS.
Right. But, you can turn off the pipelining support on ATS, if there isn’t a
clear benefit (particularly, true if your client is a CDN which can already
leverage connection pools).
> In my case, rate-limiting was implemented as a security feature i.e
> rate-limiting malicious IPs etc. Moreover, we don't have control over it at
> the entry point... since most of the time we have to look at the requests
> at the http level (sometimes even into the bodies) to figure if we should
> apply rate-limiting or not...
>
> On Mon, Jun 24, 2019 at 7:28 PM Sudheer Vinukonda
> <sudheervinuko...@yahoo.com.invalid> wrote:
>
>> Interesting setup - But, wouldn’t the CDN have connection pools to ATS
>> already? Curious to know if enabling HTTP pipelining provides benefits
>> beyond what connection pools can give.
>>
>> That said, I don’t think it makes a lot of sense to implement rate
>> limiting for _end clients_ in this sort of a setup. Typically, you’d want
>> to rate limit at the entry point of the user traffic, which means the rate
>> limiting for end users would be done at the CDN in this setup. You could
>> still add rate limiting for the CDN itself but yes in that case it makes
>> sense to just reject the requests and let the CDN retry.
>>
>> @Weixi Li,
>>
>> The more I read about your use case, it seems like you could probably
>> directly leverage the rate limiting that ATS core already supports (unless
>> you’ve other custom requirements). You can configure the below settings per
>> origin service in your remap config, and you should get the behavior your
>> use case described.
>>
>>
>> proxy.config.http.per_server.connection.max
>> Scope
>> CONFIG
>> Type
>> INT
>> Default
>> 0
>> Reloadable
>> Yes
>> Overridable
>> Yes
>> Set a limit for the number of concurrent connections to an upstream server
>> group. A value of 0 disables checking. If a transaction attempts to connect
>> to a group which already has the maximum number of concurrent connections
>> the transaction either rechecks after a delay or a 503
>> (HTTP_STATUS_SERVICE_UNAVAILABLE) error response is sent to the user agent.
>> To configure
>>
>> Number of transactions that can be delayed concurrently
>> See proxy.config.http.per_server.connection.queue_size.
>>
>> How long to delay before rechecking
>> See proxy.config.http.per_server.connection.queue_delay.
>>
>> Upstream server group definition
>>
>>
>>
>>> On Jun 24, 2019, at 6:58 PM, Dk Jack <dnj0...@gmail.com> wrote:
>>>
>>> Leif,
>>> I was talking about the case where ATS is deployed after CDN.
>>>
>>> client --> CDN Network -------> ATS --> Origin-Server
>>>
>>> In most such deployments, the connection between the CDN and ATS is HTTP
>>> 1.1. and it is pipelined. In that case, you cannot stall the clients
>>> because you could run into head-of-line blocking situation.
>>>
>>>
>>>> On Mon, Jun 24, 2019 at 4:07 PM Leif Hedstrom <zw...@apache.org> wrote:
>>>>
>>>> You definitely would not want to stall the client after you’ve made an
>>>> origin connection. I.e. you only want to stall the UA.
>>>>
>>>> — Leif
>>>>
>>>>> On Jun 24, 2019, at 14:42, Dk Jack <dnj0...@gmail.com> wrote:
>>>>>
>>>>> Also, stalling will not be good, if you have CDN upstream from you that
>>>> is pipelining requests on the same connection for multiple clients. In
>> our
>>>> plugin we tried delay request, but we couldn’t solve http 1.1 pipeline
>>>> issue. Hence ended up going with the simple approach of deny.
>>>>>
>>>>> Dk.
>>>>>
>>>>>> On Jun 24, 2019, at 1:12 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Jun 24, 2019, at 13:12, Weixi Li (BLOOMBERG/ PRINCETON) <
>>>> wli...@bloomberg.net> wrote:
>>>>>>>
>>>>>>> Hi Dk,
>>>>>>>
>>>>>>> Thanks a lot for the example. It is really inspiring. I have a
>>>> question though: in case of "bucket->consume()" returning false, the
>> code
>>>> would return a 429 response. In our use-case, we don't want to deny the
>>>> client with a 429 (but want to queue the request instead). How to handle
>>>> the false case then? Should addPlugin() add the plugin itself? (That's
>> why
>>>> I was asking about queue and async timer).
>>>>>>
>>>>>> Oh do you want to stall the client, not deny? Depending on the plugin
>>>> and the hook, you can probably just reschedule the HttpSM to come back
>> on
>>>> say 500ms and try again, rather than let it continue like you normally
>>>> would. Have to make sure they don’t stall fore ever though, and perhaps
>>>> have a priority such that older clients are given tokens before new
>> ones.
>>>>>>
>>>>>> Should be doable, but definitely more complex.
>>>>>>
>>>>>> — Leif
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Weixi
>>>>>>>
>>>>>>> From: dev@trafficserver.apache.org At: 06/24/19 01:28:40To: Weixi
>> Li
>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>
>>>>>>> Not sure you need queues. Here's a sample implementation based on
>>>>>>> TokenBucket which should do what you want...
>>>>>>>
>>>>>>>
>>>>>>> void
>>>>>>> handleReadRequestHeadersPreRemap(Transaction& transaction)
>>>>>>> {
>>>>>>> std::string host;
>>>>>>> Headers& headers = transaction.getClientRequest().getHeaders();
>>>>>>> Headers::iterator ii = headers.find("Host");
>>>>>>>
>>>>>>> if (ii != headers.end()) {
>>>>>>> host = (*ii).values();
>>>>>>> }
>>>>>>>
>>>>>>> if (!host.empty()) {
>>>>>>> auto bucket = map.find(host); // assumes u created a token bucket
>>>> per
>>>>>>> site.
>>>>>>> // std::map<std::string,
>>>>>>> std::shared_ptr<TokenBucket>>
>>>>>>> // map.emplace("www.cnn.com",
>>>>>>> // std::make_shred<
>>>>>>> TokenBucket>(rate, burst));
>>>>>>> // rate is number of req.s/min.
>>>>>>> // burst can be same as rate.
>>>>>>>
>>>>>>> if (bucket->consume(1)) { // you are requesting 1 token.
>>>>>>> transaction.resume();
>>>>>>> } else {
>>>>>>> // see CustomResponseTransactionPlugin in atscppapi examples.
>>>>>>> transaction.addPlugin(new
>>>>>>> CustomResponseTransactionPlugin(transaction, 429,
>>>>>>> "Too Many Requests", "Too Many Requests"));
>>>>>>> return;
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> transaction.resume();
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Jun 23, 2019 at 8:04 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>
>>>>>>>> Hi Dk/Eric/Leif,
>>>>>>>>
>>>>>>>> Thanks for the advice. Our use-case simply requires sending out the
>>>> queued
>>>>>>>> requests at a fixed rate (although variable by site). My plan was
>>>> simply
>>>>>>>> put all incoming transactions in separate queues by site, and then
>>>> invoke
>>>>>>>> "transaction.resume()" at fixed interval via a timer for each queue.
>>>> Would
>>>>>>>> the delay (calling "transaction.resume()" asynchronously) disrupt
>> the
>>>> ATS
>>>>>>>> event processing? If yes, what's correct way to avoid that? If no,
>> is
>>>> there
>>>>>>>> any example plugin implementing such a timer (was thinking about
>>>> AsyncTimer
>>>>>>>> in atscppapi)?
>>>>>>>>
>>>>>>>> The TokenBucket is a very interesting data structure, but I've not
>>>> figured
>>>>>>>> out how to apply it yet. Will look into it a bit more.
>>>>>>>>
>>>>>>>> Thanks to Eric's point, TS_HTTP_SEND_REQUEST_HDR_HOOK will indeed be
>>>> too
>>>>>>>> late. But for this use-case the "rate-limit" only applies to
>>>> cache-misses
>>>>>>>> and revalidations (fresh cache-hits should be served as fast as
>>>> possible).
>>>>>>>> Therefore the hook probably should be
>>>> TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK.
>>>>>>>> The plugin then lets the transaction resume immediately if the cache
>>>> hit is
>>>>>>>> fresh, or queue the transaction otherwise.
>>>>>>>>
>>>>>>>> Still learning ATS, let me know if I'm wrong. Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 23:11:03To: Weixi
>> Li
>>>>>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>
>>>>>>>> Uhm! why async timers? You'd want to implement a leaky/token bucket
>>>> per
>>>>>>>> site. Check out...
>>>>>>>>
>>>>>>>> https://github.com/rigtorp/TokenBucket
>>>>>>>>
>>>>>>>> It's single header file lock free implementation for token bucket
>> and
>>>> it
>>>>>>>> works very well...
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jun 21, 2019 at 7:38 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>
>>>>>>>>> What a great community! So many good tips in such a short time!
>>>>>>>>>
>>>>>>>>> Especially the atscppai, I would've never noticed it. the async
>>>> examples
>>>>>>>>> look very promising.
>>>>>>>>>
>>>>>>>>> It looks like the following might be necessary (let me know if I'm
>>>>>>>> wrong):
>>>>>>>>> * A hook to TS_HTTP_SEND_REQUEST_HDR_HOOK
>>>>>>>>> * A map of queues (one queue per rate-limited site)
>>>>>>>>> * A map of async timers (one timer per queue)
>>>>>>>>>
>>>>>>>>> I will study the ATS code more to understand the event and
>> threading
>>>>>>>> model
>>>>>>>>> better.
>>>>>>>>>
>>>>>>>>> Thank you all.
>>>>>>>>>
>>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 19:52:44To:
>>>>>>>>> dev@trafficserver.apache.org
>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>
>>>>>>>>> I have implemented rate-limit in my plugin using atscppapi. We are
>>>> using
>>>>>>>>> ats in
>>>>>>>>> security context for mitigation. If the request matches certain
>>>> criteria
>>>>>>>>> (ip,
>>>>>>>>> method, host, uri and header values) then we apply rate-limit to
>>>> that ip.
>>>>>>>>>
>>>>>>>>> Dk.
>>>>>>>>>
>>>>>>>>>> On Jun 21, 2019, at 3:15 PM, Leif Hedstrom <zw...@apache.org>
>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Jun 21, 2019, at 16:09, Weixi Li (BLOOMBERG/ PRINCETON)
>>>>>>>>> <wli...@bloomberg.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi team,
>>>>>>>>>>>
>>>>>>>>>>> We are experimenting with ATS in *forward* proxy mode. Our
>> use-case
>>>>>>>>> requires
>>>>>>>>> a rate-limiting component that enforces rules based on the
>>>> destination.
>>>>>>>>>>>
>>>>>>>>>>> For example:
>>>>>>>>>>>
>>>>>>>>>>> For all incoming requests targeting "www.cnn.com", we want to
>>>> limit
>>>>>>>>> the
>>>>>>>>> outgoing rate to be 10 requests per minute; for "www.reddit.com",
>> we
>>>>>>>> want
>>>>>>>>> the
>>>>>>>>> rate to be 20 requests per minute; and so on. If there were more
>>>> requests
>>>>>>>>> than
>>>>>>>>> the limit specified, the requests must be queued before they could
>> go
>>>>>>>> out.
>>>>>>>>>>
>>>>>>>>>> Seems very straight forward to implement as a plugin. For example
>>>> the
>>>>>>>>> geo_acl
>>>>>>>>> plugin might be a good start, since it limits access based on
>> source
>>>> IP.
>>>>>>>>>>
>>>>>>>>>> Would be interesting to hear more about your use case too, it’s
>>>> always
>>>>>>>>> exciting to hear about different solutions that people use ATS for.
>>>> Maybe
>>>>>>>>> at
>>>>>>>>> the next ATS summit? :-)
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> — Leif
>>>>>>>>>>>
>>>>>>>>>>> Is it possible to implement this requirement using a plugin?
>>>>>>>>>>>
>>>>>>>>>>> If not, we wouldn't mind forking the code and modifying whichever
>>>>>>>> parts
>>>>>>>>> that
>>>>>>>>> would be necessary. But which are the potentially relevant
>>>> components?
>>>>>>>>>>>
>>>>>>>>>>> If any experts could give us some pointers on the design, that
>>>> would
>>>>>>>> be
>>>>>>>>> really appreciated.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Weixi
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>
