Re: [ANN] Throttler: a library for rate limiting
I'm glad you're liking Throttler, Don!
As per having a random frequency between an interval, could you clarify a
bit? Do you want the time between requests to be drawn randomly from some
distribution each time? If that's the case, the best way I can think of
would require a change in the implementation of the bucket filling go
thread. Instead of doing:
(timeout some-constant)
it would have to do
(timeout (sample distribution))
Or maybe even more generally just
(timeout (gen-timeout))
where gen-timeout is just a function that returns a timeout value for the
next function call. For the common case it would have to be `(constantly
some-number)`.
If however all you want is to have an upper and a lower bound in the
throughput,
you can play with the burstiness value. For the range 1/15 to 1/5 req/s (4
req/minute to 12 req/minute), your request rate is '(4 :minute) and your
token number is (12 req/minute - 4 req/minute) * 1 minute = 8. So calling
(throttle-{fn,chan} 4 :minute 8)
would result in your desired average rate and burst rate (with a granularity
of 1 minute).
I'm curious, what drives this use case of a random throughput?
brunov
On Friday, May 16, 2014 8:31:42 PM UTC-3, dcj wrote:
On May 14, 2014, at 9:43 AM, Bruno Vecchi vecc...@gmail.com javascript:
wrote:
Throttler[1] is a little library I wrote out of need for one of my
personal projects. It lets you control the maximum rate of function calls
or message transmissions through core.async channels.
This is way cool, thanks for sharing it!
A while back I needed something like this to throttle http requests, and I
cobbled something together that used Thread/sleep,
it worked, but vastly inferior to your library, AFAICT. I intent to throw
out my code out and use your library.
In my application, I would like to have a throttle-fn that was guaranteed
not to occur faster than some number, but also want to add a (bounded)
random number to that limit.
For example, let’s say I wanted my http requests to be no more frequent
than every 5 seconds, but some random number of seconds between 5 seconds
and 15 seconds.
Is there any way to add/modify throttler to handle a case like that?
In my application, I use clj-http to make http requests, and what I did
was to create a wrap-throttle middleware and added that to the middleware
stack for the
(customized) http client I used for rate-limited requests. I am thinking
that approach would work well with your throttle-fn….
Best regards,
Don
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
On May 16, 2014, at 11:11 PM, Bruno Vecchi vecch...@gmail.com wrote: As per having a random frequency between an interval, could you clarify a bit? Do you want the time between requests to be drawn randomly from some distribution each time? If that's the case, the best way I can think of would require a change in the implementation of the bucket filling go thread. Instead of doing: (timeout some-constant) it would have to do (timeout (sample distribution)) Or maybe even more generally just (timeout (gen-timeout)) where gen-timeout is just a function that returns a timeout value for the next function call. For the common case it would have to be `(constantly some-number)`. I like gen-timeout the best. It is the most general. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
On May 14, 2014, at 9:43 AM, Bruno Vecchi vecch...@gmail.com wrote: Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. This is way cool, thanks for sharing it! A while back I needed something like this to throttle http requests, and I cobbled something together that used Thread/sleep, it worked, but vastly inferior to your library, AFAICT. I intent to throw out my code out and use your library. In my application, I would like to have a throttle-fn that was guaranteed not to occur faster than some number, but also want to add a (bounded) random number to that limit. For example, let's say I wanted my http requests to be no more frequent than every 5 seconds, but some random number of seconds between 5 seconds and 15 seconds. Is there any way to add/modify throttler to handle a case like that? In my application, I use clj-http to make http requests, and what I did was to create a wrap-throttle middleware and added that to the middleware stack for the (customized) http client I used for rate-limited requests. I am thinking that approach would work well with your throttle-fn Best regards, Don -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ANN] Throttler: a library for rate limiting
Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. It supports bursty traffic shaping via the Token-Bucket algorithm and is fully asynchronous. I just published a [blog post][1] on it, expanding a little bit on the project's README[1]. This is my first Clojure library! I would greatly appreciate any feedback. Thanks, Bruno Vecchi -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
Hm, apparently I forgot to post the links to the actual things I wanted to share. Here they go: Github: https://github.com/brunoV/throttle Blog Post: http://brunov.org/clojure/2014/05/14/throttler/ Bruno On Wednesday, May 14, 2014 1:43:34 PM UTC-3, Bruno Vecchi wrote: Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. It supports bursty traffic shaping via the Token-Bucket algorithm and is fully asynchronous. I just published a [blog post][1] on it, expanding a little bit on the project's README[1]. This is my first Clojure library! I would greatly appreciate any feedback. Thanks, Bruno Vecchi -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
Hm, apparently I forgot to post the links to the actual things I wanted to share. Here they go: Github: https://github.com/brunoV/throttlehttps://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FbrunoV%2Fthrottlesa=Dsntz=1usg=AFQjCNHgMi136k1NCSTxXzBleF297MVOUw r Blog Post: http://brunov.org/clojure/2014/05/14/throttler/ Bruno On Wednesday, May 14, 2014 1:43:34 PM UTC-3, Bruno Vecchi wrote: Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. It supports bursty traffic shaping via the Token-Bucket algorithm and is fully asynchronous. I just published a [blog post][1] on it, expanding a little bit on the project's README[1]. This is my first Clojure library! I would greatly appreciate any feedback. Thanks, Bruno Vecchi -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
Hm, apparently I forgot to post the links to the actual things I wanted to share. Here they go: Github: https://github.com/brunoV/throttler Blog Post: http://brunov.org/clojure/2014/05/14/throttler/ Bruno On Wednesday, May 14, 2014 1:43:34 PM UTC-3, Bruno Vecchi wrote: Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. It supports bursty traffic shaping via the Token-Bucket algorithm and is fully asynchronous. I just published a [blog post][1] on it, expanding a little bit on the project's README[1]. This is my first Clojure library! I would greatly appreciate any feedback. Thanks, Bruno Vecchi -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Throttler: a library for rate limiting
Awesome! Thanks. On Thu, May 15, 2014 at 1:58 AM, Bruno Vecchi vecch...@gmail.com wrote: Hm, apparently I forgot to post the links to the actual things I wanted to share. Here they go: Github: https://github.com/brunoV/throttler Blog Post: http://brunov.org/clojure/2014/05/14/throttler/ Bruno On Wednesday, May 14, 2014 1:43:34 PM UTC-3, Bruno Vecchi wrote: Throttler[1] is a little library I wrote out of need for one of my personal projects. It lets you control the maximum rate of function calls or message transmissions through core.async channels. It supports bursty traffic shaping via the Token-Bucket algorithm and is fully asynchronous. I just published a [blog post][1] on it, expanding a little bit on the project's README[1]. This is my first Clojure library! I would greatly appreciate any feedback. Thanks, Bruno Vecchi -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Kind Regards, Atamert Ölçgen -+- --+ +++ www.muhuk.com -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
