> On Jun 24, 2019, at 22:01, Sudheer Vinukonda
> <sudheervinuko...@yahoo.com.invalid> wrote:
>
>
>
>> On Jun 24, 2019, at 8:33 PM, Dk Jack <dnj0...@gmail.com> wrote:
>>
>> I suspect connection pools would be using the http pipeline feature to
>> multiplex multiple client connections on to the pool connections towards
>> ATS.
>
> Right. But, you can turn off the pipelining support on ATS, if there isn’t a
> clear benefit (particularly, true if your client is a CDN which can already
> leverage connection pools).
How? There used to be settings for this, but they didn’t do anything, so were
removed.
— Leif
>
>> In my case, rate-limiting was implemented as a security feature i.e
>> rate-limiting malicious IPs etc. Moreover, we don't have control over it at
>> the entry point... since most of the time we have to look at the requests
>> at the http level (sometimes even into the bodies) to figure if we should
>> apply rate-limiting or not...
>>
>> On Mon, Jun 24, 2019 at 7:28 PM Sudheer Vinukonda
>> <sudheervinuko...@yahoo.com.invalid> wrote:
>>
>>> Interesting setup - But, wouldn’t the CDN have connection pools to ATS
>>> already? Curious to know if enabling HTTP pipelining provides benefits
>>> beyond what connection pools can give.
>>>
>>> That said, I don’t think it makes a lot of sense to implement rate
>>> limiting for _end clients_ in this sort of a setup. Typically, you’d want
>>> to rate limit at the entry point of the user traffic, which means the rate
>>> limiting for end users would be done at the CDN in this setup. You could
>>> still add rate limiting for the CDN itself but yes in that case it makes
>>> sense to just reject the requests and let the CDN retry.
>>>
>>> @Weixi Li,
>>>
>>> The more I read about your use case, it seems like you could probably
>>> directly leverage the rate limiting that ATS core already supports (unless
>>> you’ve other custom requirements). You can configure the below settings per
>>> origin service in your remap config, and you should get the behavior your
>>> use case described.
>>>
>>>
>>> proxy.config.http.per_server.connection.max
>>> Scope
>>> CONFIG
>>> Type
>>> INT
>>> Default
>>> 0
>>> Reloadable
>>> Yes
>>> Overridable
>>> Yes
>>> Set a limit for the number of concurrent connections to an upstream server
>>> group. A value of 0 disables checking. If a transaction attempts to connect
>>> to a group which already has the maximum number of concurrent connections
>>> the transaction either rechecks after a delay or a 503
>>> (HTTP_STATUS_SERVICE_UNAVAILABLE) error response is sent to the user agent.
>>> To configure
>>>
>>> Number of transactions that can be delayed concurrently
>>> See proxy.config.http.per_server.connection.queue_size.
>>>
>>> How long to delay before rechecking
>>> See proxy.config.http.per_server.connection.queue_delay.
>>>
>>> Upstream server group definition
>>>
>>>
>>>
>>>> On Jun 24, 2019, at 6:58 PM, Dk Jack <dnj0...@gmail.com> wrote:
>>>>
>>>> Leif,
>>>> I was talking about the case where ATS is deployed after CDN.
>>>>
>>>> client --> CDN Network -------> ATS --> Origin-Server
>>>>
>>>> In most such deployments, the connection between the CDN and ATS is HTTP
>>>> 1.1. and it is pipelined. In that case, you cannot stall the clients
>>>> because you could run into head-of-line blocking situation.
>>>>
>>>>
>>>>> On Mon, Jun 24, 2019 at 4:07 PM Leif Hedstrom <zw...@apache.org> wrote:
>>>>>
>>>>> You definitely would not want to stall the client after you’ve made an
>>>>> origin connection. I.e. you only want to stall the UA.
>>>>>
>>>>> — Leif
>>>>>
>>>>>> On Jun 24, 2019, at 14:42, Dk Jack <dnj0...@gmail.com> wrote:
>>>>>>
>>>>>> Also, stalling will not be good, if you have CDN upstream from you that
>>>>> is pipelining requests on the same connection for multiple clients. In
>>> our
>>>>> plugin we tried delay request, but we couldn’t solve http 1.1 pipeline
>>>>> issue. Hence ended up going with the simple approach of deny.
>>>>>>
>>>>>> Dk.
>>>>>>
>>>>>>> On Jun 24, 2019, at 1:12 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Jun 24, 2019, at 13:12, Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>
>>>>>>>> Hi Dk,
>>>>>>>>
>>>>>>>> Thanks a lot for the example. It is really inspiring. I have a
>>>>> question though: in case of "bucket->consume()" returning false, the
>>> code
>>>>> would return a 429 response. In our use-case, we don't want to deny the
>>>>> client with a 429 (but want to queue the request instead). How to handle
>>>>> the false case then? Should addPlugin() add the plugin itself? (That's
>>> why
>>>>> I was asking about queue and async timer).
>>>>>>>
>>>>>>> Oh do you want to stall the client, not deny? Depending on the plugin
>>>>> and the hook, you can probably just reschedule the HttpSM to come back
>>> on
>>>>> say 500ms and try again, rather than let it continue like you normally
>>>>> would. Have to make sure they don’t stall fore ever though, and perhaps
>>>>> have a priority such that older clients are given tokens before new
>>> ones.
>>>>>>>
>>>>>>> Should be doable, but definitely more complex.
>>>>>>>
>>>>>>> — Leif
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Weixi
>>>>>>>>
>>>>>>>> From: dev@trafficserver.apache.org At: 06/24/19 01:28:40To: Weixi
>>> Li
>>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>
>>>>>>>> Not sure you need queues. Here's a sample implementation based on
>>>>>>>> TokenBucket which should do what you want...
>>>>>>>>
>>>>>>>>
>>>>>>>> void
>>>>>>>> handleReadRequestHeadersPreRemap(Transaction& transaction)
>>>>>>>> {
>>>>>>>> std::string host;
>>>>>>>> Headers& headers = transaction.getClientRequest().getHeaders();
>>>>>>>> Headers::iterator ii = headers.find("Host");
>>>>>>>>
>>>>>>>> if (ii != headers.end()) {
>>>>>>>> host = (*ii).values();
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (!host.empty()) {
>>>>>>>> auto bucket = map.find(host); // assumes u created a token bucket
>>>>> per
>>>>>>>> site.
>>>>>>>> // std::map<std::string,
>>>>>>>> std::shared_ptr<TokenBucket>>
>>>>>>>> // map.emplace("www.cnn.com",
>>>>>>>> // std::make_shred<
>>>>>>>> TokenBucket>(rate, burst));
>>>>>>>> // rate is number of req.s/min.
>>>>>>>> // burst can be same as rate.
>>>>>>>>
>>>>>>>> if (bucket->consume(1)) { // you are requesting 1 token.
>>>>>>>> transaction.resume();
>>>>>>>> } else {
>>>>>>>> // see CustomResponseTransactionPlugin in atscppapi examples.
>>>>>>>> transaction.addPlugin(new
>>>>>>>> CustomResponseTransactionPlugin(transaction, 429,
>>>>>>>> "Too Many Requests", "Too Many Requests"));
>>>>>>>> return;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> transaction.resume();
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Jun 23, 2019 at 8:04 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>
>>>>>>>>> Hi Dk/Eric/Leif,
>>>>>>>>>
>>>>>>>>> Thanks for the advice. Our use-case simply requires sending out the
>>>>> queued
>>>>>>>>> requests at a fixed rate (although variable by site). My plan was
>>>>> simply
>>>>>>>>> put all incoming transactions in separate queues by site, and then
>>>>> invoke
>>>>>>>>> "transaction.resume()" at fixed interval via a timer for each queue.
>>>>> Would
>>>>>>>>> the delay (calling "transaction.resume()" asynchronously) disrupt
>>> the
>>>>> ATS
>>>>>>>>> event processing? If yes, what's correct way to avoid that? If no,
>>> is
>>>>> there
>>>>>>>>> any example plugin implementing such a timer (was thinking about
>>>>> AsyncTimer
>>>>>>>>> in atscppapi)?
>>>>>>>>>
>>>>>>>>> The TokenBucket is a very interesting data structure, but I've not
>>>>> figured
>>>>>>>>> out how to apply it yet. Will look into it a bit more.
>>>>>>>>>
>>>>>>>>> Thanks to Eric's point, TS_HTTP_SEND_REQUEST_HDR_HOOK will indeed be
>>>>> too
>>>>>>>>> late. But for this use-case the "rate-limit" only applies to
>>>>> cache-misses
>>>>>>>>> and revalidations (fresh cache-hits should be served as fast as
>>>>> possible).
>>>>>>>>> Therefore the hook probably should be
>>>>> TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK.
>>>>>>>>> The plugin then lets the transaction resume immediately if the cache
>>>>> hit is
>>>>>>>>> fresh, or queue the transaction otherwise.
>>>>>>>>>
>>>>>>>>> Still learning ATS, let me know if I'm wrong. Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 23:11:03To: Weixi
>>> Li
>>>>>>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>
>>>>>>>>> Uhm! why async timers? You'd want to implement a leaky/token bucket
>>>>> per
>>>>>>>>> site. Check out...
>>>>>>>>>
>>>>>>>>> https://github.com/rigtorp/TokenBucket
>>>>>>>>>
>>>>>>>>> It's single header file lock free implementation for token bucket
>>> and
>>>>> it
>>>>>>>>> works very well...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 21, 2019 at 7:38 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>>
>>>>>>>>>> What a great community! So many good tips in such a short time!
>>>>>>>>>>
>>>>>>>>>> Especially the atscppai, I would've never noticed it. the async
>>>>> examples
>>>>>>>>>> look very promising.
>>>>>>>>>>
>>>>>>>>>> It looks like the following might be necessary (let me know if I'm
>>>>>>>>> wrong):
>>>>>>>>>> * A hook to TS_HTTP_SEND_REQUEST_HDR_HOOK
>>>>>>>>>> * A map of queues (one queue per rate-limited site)
>>>>>>>>>> * A map of async timers (one timer per queue)
>>>>>>>>>>
>>>>>>>>>> I will study the ATS code more to understand the event and
>>> threading
>>>>>>>>> model
>>>>>>>>>> better.
>>>>>>>>>>
>>>>>>>>>> Thank you all.
>>>>>>>>>>
>>>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 19:52:44To:
>>>>>>>>>> dev@trafficserver.apache.org
>>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>>
>>>>>>>>>> I have implemented rate-limit in my plugin using atscppapi. We are
>>>>> using
>>>>>>>>>> ats in
>>>>>>>>>> security context for mitigation. If the request matches certain
>>>>> criteria
>>>>>>>>>> (ip,
>>>>>>>>>> method, host, uri and header values) then we apply rate-limit to
>>>>> that ip.
>>>>>>>>>>
>>>>>>>>>> Dk.
>>>>>>>>>>
>>>>>>>>>>> On Jun 21, 2019, at 3:15 PM, Leif Hedstrom <zw...@apache.org>
>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Jun 21, 2019, at 16:09, Weixi Li (BLOOMBERG/ PRINCETON)
>>>>>>>>>> <wli...@bloomberg.net> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi team,
>>>>>>>>>>>>
>>>>>>>>>>>> We are experimenting with ATS in *forward* proxy mode. Our
>>> use-case
>>>>>>>>>> requires
>>>>>>>>>> a rate-limiting component that enforces rules based on the
>>>>> destination.
>>>>>>>>>>>>
>>>>>>>>>>>> For example:
>>>>>>>>>>>>
>>>>>>>>>>>> For all incoming requests targeting "www.cnn.com", we want to
>>>>> limit
>>>>>>>>>> the
>>>>>>>>>> outgoing rate to be 10 requests per minute; for "www.reddit.com",
>>> we
>>>>>>>>> want
>>>>>>>>>> the
>>>>>>>>>> rate to be 20 requests per minute; and so on. If there were more
>>>>> requests
>>>>>>>>>> than
>>>>>>>>>> the limit specified, the requests must be queued before they could
>>> go
>>>>>>>>> out.
>>>>>>>>>>>
>>>>>>>>>>> Seems very straight forward to implement as a plugin. For example
>>>>> the
>>>>>>>>>> geo_acl
>>>>>>>>>> plugin might be a good start, since it limits access based on
>>> source
>>>>> IP.
>>>>>>>>>>>
>>>>>>>>>>> Would be interesting to hear more about your use case too, it’s
>>>>> always
>>>>>>>>>> exciting to hear about different solutions that people use ATS for.
>>>>> Maybe
>>>>>>>>>> at
>>>>>>>>>> the next ATS summit? :-)
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> — Leif
>>>>>>>>>>>>
>>>>>>>>>>>> Is it possible to implement this requirement using a plugin?
>>>>>>>>>>>>
>>>>>>>>>>>> If not, we wouldn't mind forking the code and modifying whichever
>>>>>>>>> parts
>>>>>>>>>> that
>>>>>>>>>> would be necessary. But which are the potentially relevant
>>>>> components?
>>>>>>>>>>>>
>>>>>>>>>>>> If any experts could give us some pointers on the design, that
>>>>> would
>>>>>>>>> be
>>>>>>>>>> really appreciated.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Weixi
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>
