> On Jun 24, 2019, at 22:42, Dk Jack <dnj0...@gmail.com> wrote:
>
> Perhaps. I think we are talking two different things. I was trying to say,
> it’s a bad idea to stall a client connection when multiple clients are
> getting multiplexed on the same connection (between cdn and ats). From that
> perspective, the result is the same on the client side of ats. Stalling one
> client will cause problems for other client on that connection
Hmm, does that happen? When a client makes a request, an origin connection is
allocated from the pools. I don’t think two client connections can share a
server connection at the same time; They can be reused between multiple client
connections though.
— Leif
>
> Dk.
>
>> On Jun 24, 2019, at 9:01 PM, Sudheer Vinukonda
>> <sudheervinuko...@yahoo.com.INVALID> wrote:
>>
>>
>>
>>> On Jun 24, 2019, at 8:33 PM, Dk Jack <dnj0...@gmail.com> wrote:
>>>
>>> I suspect connection pools would be using the http pipeline feature to
>>> multiplex multiple client connections on to the pool connections towards
>>> ATS.
>>
>> Right. But, you can turn off the pipelining support on ATS, if there isn’t a
>> clear benefit (particularly, true if your client is a CDN which can already
>> leverage connection pools).
>>
>>> In my case, rate-limiting was implemented as a security feature i.e
>>> rate-limiting malicious IPs etc. Moreover, we don't have control over it at
>>> the entry point... since most of the time we have to look at the requests
>>> at the http level (sometimes even into the bodies) to figure if we should
>>> apply rate-limiting or not...
>>>
>>> On Mon, Jun 24, 2019 at 7:28 PM Sudheer Vinukonda
>>> <sudheervinuko...@yahoo.com.invalid> wrote:
>>>
>>>> Interesting setup - But, wouldn’t the CDN have connection pools to ATS
>>>> already? Curious to know if enabling HTTP pipelining provides benefits
>>>> beyond what connection pools can give.
>>>>
>>>> That said, I don’t think it makes a lot of sense to implement rate
>>>> limiting for _end clients_ in this sort of a setup. Typically, you’d want
>>>> to rate limit at the entry point of the user traffic, which means the rate
>>>> limiting for end users would be done at the CDN in this setup. You could
>>>> still add rate limiting for the CDN itself but yes in that case it makes
>>>> sense to just reject the requests and let the CDN retry.
>>>>
>>>> @Weixi Li,
>>>>
>>>> The more I read about your use case, it seems like you could probably
>>>> directly leverage the rate limiting that ATS core already supports (unless
>>>> you’ve other custom requirements). You can configure the below settings per
>>>> origin service in your remap config, and you should get the behavior your
>>>> use case described.
>>>>
>>>>
>>>> proxy.config.http.per_server.connection.max
>>>> Scope
>>>> CONFIG
>>>> Type
>>>> INT
>>>> Default
>>>> 0
>>>> Reloadable
>>>> Yes
>>>> Overridable
>>>> Yes
>>>> Set a limit for the number of concurrent connections to an upstream server
>>>> group. A value of 0 disables checking. If a transaction attempts to connect
>>>> to a group which already has the maximum number of concurrent connections
>>>> the transaction either rechecks after a delay or a 503
>>>> (HTTP_STATUS_SERVICE_UNAVAILABLE) error response is sent to the user agent.
>>>> To configure
>>>>
>>>> Number of transactions that can be delayed concurrently
>>>> See proxy.config.http.per_server.connection.queue_size.
>>>>
>>>> How long to delay before rechecking
>>>> See proxy.config.http.per_server.connection.queue_delay.
>>>>
>>>> Upstream server group definition
>>>>
>>>>
>>>>
>>>>> On Jun 24, 2019, at 6:58 PM, Dk Jack <dnj0...@gmail.com> wrote:
>>>>>
>>>>> Leif,
>>>>> I was talking about the case where ATS is deployed after CDN.
>>>>>
>>>>> client --> CDN Network -------> ATS --> Origin-Server
>>>>>
>>>>> In most such deployments, the connection between the CDN and ATS is HTTP
>>>>> 1.1. and it is pipelined. In that case, you cannot stall the clients
>>>>> because you could run into head-of-line blocking situation.
>>>>>
>>>>>
>>>>>> On Mon, Jun 24, 2019 at 4:07 PM Leif Hedstrom <zw...@apache.org> wrote:
>>>>>>
>>>>>> You definitely would not want to stall the client after you’ve made an
>>>>>> origin connection. I.e. you only want to stall the UA.
>>>>>>
>>>>>> — Leif
>>>>>>
>>>>>>> On Jun 24, 2019, at 14:42, Dk Jack <dnj0...@gmail.com> wrote:
>>>>>>>
>>>>>>> Also, stalling will not be good, if you have CDN upstream from you that
>>>>>> is pipelining requests on the same connection for multiple clients. In
>>>> our
>>>>>> plugin we tried delay request, but we couldn’t solve http 1.1 pipeline
>>>>>> issue. Hence ended up going with the simple approach of deny.
>>>>>>>
>>>>>>> Dk.
>>>>>>>
>>>>>>>> On Jun 24, 2019, at 1:12 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Jun 24, 2019, at 13:12, Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>>
>>>>>>>>> Hi Dk,
>>>>>>>>>
>>>>>>>>> Thanks a lot for the example. It is really inspiring. I have a
>>>>>> question though: in case of "bucket->consume()" returning false, the
>>>> code
>>>>>> would return a 429 response. In our use-case, we don't want to deny the
>>>>>> client with a 429 (but want to queue the request instead). How to handle
>>>>>> the false case then? Should addPlugin() add the plugin itself? (That's
>>>> why
>>>>>> I was asking about queue and async timer).
>>>>>>>>
>>>>>>>> Oh do you want to stall the client, not deny? Depending on the plugin
>>>>>> and the hook, you can probably just reschedule the HttpSM to come back
>>>> on
>>>>>> say 500ms and try again, rather than let it continue like you normally
>>>>>> would. Have to make sure they don’t stall fore ever though, and perhaps
>>>>>> have a priority such that older clients are given tokens before new
>>>> ones.
>>>>>>>>
>>>>>>>> Should be doable, but definitely more complex.
>>>>>>>>
>>>>>>>> — Leif
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Weixi
>>>>>>>>>
>>>>>>>>> From: dev@trafficserver.apache.org At: 06/24/19 01:28:40To: Weixi
>>>> Li
>>>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>
>>>>>>>>> Not sure you need queues. Here's a sample implementation based on
>>>>>>>>> TokenBucket which should do what you want...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> void
>>>>>>>>> handleReadRequestHeadersPreRemap(Transaction& transaction)
>>>>>>>>> {
>>>>>>>>> std::string host;
>>>>>>>>> Headers& headers = transaction.getClientRequest().getHeaders();
>>>>>>>>> Headers::iterator ii = headers.find("Host");
>>>>>>>>>
>>>>>>>>> if (ii != headers.end()) {
>>>>>>>>> host = (*ii).values();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> if (!host.empty()) {
>>>>>>>>> auto bucket = map.find(host); // assumes u created a token bucket
>>>>>> per
>>>>>>>>> site.
>>>>>>>>> // std::map<std::string,
>>>>>>>>> std::shared_ptr<TokenBucket>>
>>>>>>>>> // map.emplace("www.cnn.com",
>>>>>>>>> // std::make_shred<
>>>>>>>>> TokenBucket>(rate, burst));
>>>>>>>>> // rate is number of req.s/min.
>>>>>>>>> // burst can be same as rate.
>>>>>>>>>
>>>>>>>>> if (bucket->consume(1)) { // you are requesting 1 token.
>>>>>>>>> transaction.resume();
>>>>>>>>> } else {
>>>>>>>>> // see CustomResponseTransactionPlugin in atscppapi examples.
>>>>>>>>> transaction.addPlugin(new
>>>>>>>>> CustomResponseTransactionPlugin(transaction, 429,
>>>>>>>>> "Too Many Requests", "Too Many Requests"));
>>>>>>>>> return;
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> transaction.resume();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Jun 23, 2019 at 8:04 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Dk/Eric/Leif,
>>>>>>>>>>
>>>>>>>>>> Thanks for the advice. Our use-case simply requires sending out the
>>>>>> queued
>>>>>>>>>> requests at a fixed rate (although variable by site). My plan was
>>>>>> simply
>>>>>>>>>> put all incoming transactions in separate queues by site, and then
>>>>>> invoke
>>>>>>>>>> "transaction.resume()" at fixed interval via a timer for each queue.
>>>>>> Would
>>>>>>>>>> the delay (calling "transaction.resume()" asynchronously) disrupt
>>>> the
>>>>>> ATS
>>>>>>>>>> event processing? If yes, what's correct way to avoid that? If no,
>>>> is
>>>>>> there
>>>>>>>>>> any example plugin implementing such a timer (was thinking about
>>>>>> AsyncTimer
>>>>>>>>>> in atscppapi)?
>>>>>>>>>>
>>>>>>>>>> The TokenBucket is a very interesting data structure, but I've not
>>>>>> figured
>>>>>>>>>> out how to apply it yet. Will look into it a bit more.
>>>>>>>>>>
>>>>>>>>>> Thanks to Eric's point, TS_HTTP_SEND_REQUEST_HDR_HOOK will indeed be
>>>>>> too
>>>>>>>>>> late. But for this use-case the "rate-limit" only applies to
>>>>>> cache-misses
>>>>>>>>>> and revalidations (fresh cache-hits should be served as fast as
>>>>>> possible).
>>>>>>>>>> Therefore the hook probably should be
>>>>>> TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK.
>>>>>>>>>> The plugin then lets the transaction resume immediately if the cache
>>>>>> hit is
>>>>>>>>>> fresh, or queue the transaction otherwise.
>>>>>>>>>>
>>>>>>>>>> Still learning ATS, let me know if I'm wrong. Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 23:11:03To: Weixi
>>>> Li
>>>>>>>>>> (BLOOMBERG/ PRINCETON ) , dev@trafficserver.apache.org
>>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>>
>>>>>>>>>> Uhm! why async timers? You'd want to implement a leaky/token bucket
>>>>>> per
>>>>>>>>>> site. Check out...
>>>>>>>>>>
>>>>>>>>>> https://github.com/rigtorp/TokenBucket
>>>>>>>>>>
>>>>>>>>>> It's single header file lock free implementation for token bucket
>>>> and
>>>>>> it
>>>>>>>>>> works very well...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 21, 2019 at 7:38 PM Weixi Li (BLOOMBERG/ PRINCETON) <
>>>>>>>>>> wli...@bloomberg.net> wrote:
>>>>>>>>>>
>>>>>>>>>>> What a great community! So many good tips in such a short time!
>>>>>>>>>>>
>>>>>>>>>>> Especially the atscppai, I would've never noticed it. the async
>>>>>> examples
>>>>>>>>>>> look very promising.
>>>>>>>>>>>
>>>>>>>>>>> It looks like the following might be necessary (let me know if I'm
>>>>>>>>>> wrong):
>>>>>>>>>>> * A hook to TS_HTTP_SEND_REQUEST_HDR_HOOK
>>>>>>>>>>> * A map of queues (one queue per rate-limited site)
>>>>>>>>>>> * A map of async timers (one timer per queue)
>>>>>>>>>>>
>>>>>>>>>>> I will study the ATS code more to understand the event and
>>>> threading
>>>>>>>>>> model
>>>>>>>>>>> better.
>>>>>>>>>>>
>>>>>>>>>>> Thank you all.
>>>>>>>>>>>
>>>>>>>>>>> From: dev@trafficserver.apache.org At: 06/21/19 19:52:44To:
>>>>>>>>>>> dev@trafficserver.apache.org
>>>>>>>>>>> Subject: Re: Implementing Rate-limiting in forward proxy mode
>>>>>>>>>>>
>>>>>>>>>>> I have implemented rate-limit in my plugin using atscppapi. We are
>>>>>> using
>>>>>>>>>>> ats in
>>>>>>>>>>> security context for mitigation. If the request matches certain
>>>>>> criteria
>>>>>>>>>>> (ip,
>>>>>>>>>>> method, host, uri and header values) then we apply rate-limit to
>>>>>> that ip.
>>>>>>>>>>>
>>>>>>>>>>> Dk.
>>>>>>>>>>>
>>>>>>>>>>>> On Jun 21, 2019, at 3:15 PM, Leif Hedstrom <zw...@apache.org>
>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Jun 21, 2019, at 16:09, Weixi Li (BLOOMBERG/ PRINCETON)
>>>>>>>>>>> <wli...@bloomberg.net> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi team,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are experimenting with ATS in *forward* proxy mode. Our
>>>> use-case
>>>>>>>>>>> requires
>>>>>>>>>>> a rate-limiting component that enforces rules based on the
>>>>>> destination.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For example:
>>>>>>>>>>>>>
>>>>>>>>>>>>> For all incoming requests targeting "www.cnn.com", we want to
>>>>>> limit
>>>>>>>>>>> the
>>>>>>>>>>> outgoing rate to be 10 requests per minute; for "www.reddit.com",
>>>> we
>>>>>>>>>> want
>>>>>>>>>>> the
>>>>>>>>>>> rate to be 20 requests per minute; and so on. If there were more
>>>>>> requests
>>>>>>>>>>> than
>>>>>>>>>>> the limit specified, the requests must be queued before they could
>>>> go
>>>>>>>>>> out.
>>>>>>>>>>>>
>>>>>>>>>>>> Seems very straight forward to implement as a plugin. For example
>>>>>> the
>>>>>>>>>>> geo_acl
>>>>>>>>>>> plugin might be a good start, since it limits access based on
>>>> source
>>>>>> IP.
>>>>>>>>>>>>
>>>>>>>>>>>> Would be interesting to hear more about your use case too, it’s
>>>>>> always
>>>>>>>>>>> exciting to hear about different solutions that people use ATS for.
>>>>>> Maybe
>>>>>>>>>>> at
>>>>>>>>>>> the next ATS summit? :-)
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> — Leif
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is it possible to implement this requirement using a plugin?
>>>>>>>>>>>>>
>>>>>>>>>>>>> If not, we wouldn't mind forking the code and modifying whichever
>>>>>>>>>> parts
>>>>>>>>>>> that
>>>>>>>>>>> would be necessary. But which are the potentially relevant
>>>>>> components?
>>>>>>>>>>>>>
>>>>>>>>>>>>> If any experts could give us some pointers on the design, that
>>>>>> would
>>>>>>>>>> be
>>>>>>>>>>> really appreciated.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Weixi
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>
