Re: [Review] Slow server, not maxing out CPU

[Вук Мировић]

On Thursday, July 21, 2016 at 6:15:46 AM UTC+2, James Reeves wrote:
>
> On 21 July 2016 at 05:05, Ashish Negi  > wrote:
>
>> with core async `go` you can not do blocking IO or any time consuming 
>> work.
>> `go` uses fixed threadpool (no of cpus + 2 or something).
>> and in your async-handler you are using `> https://clojure.github.io/core.async/
>>
>> Try with `future`. https://clojuredocs.org/clojure.core/future
>>
>
> While futures will work, it's worth pointing out you'd get the same result 
> just by increasing the thread count on the adapter. Either way you're using 
> up a thread per request.
>
> Since the database functions are not asynchronous, core.async isn't really 
> useful in this context. My suggestion would be to drop it altogether.
>
> - James
>

http-kit threadpool was definitely bottleneck, and probably hakiri-cp 
connection pool.
I reverted to initial version without core.async and bumped both pools to 
128 and now CPU is fully saturated, and performance is much better. 
Comparable to Go version I wrote, maybe even better.
Thanks you both for your time :)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Review] Slow server, not maxing out CPU

[James Reeves]

On 21 July 2016 at 05:05, Ashish Negi  wrote:

> with core async `go` you can not do blocking IO or any time consuming work.
> `go` uses fixed threadpool (no of cpus + 2 or something).
> and in your async-handler you are using ` https://clojure.github.io/core.async/
>
> Try with `future`. https://clojuredocs.org/clojure.core/future
>

While futures will work, it's worth pointing out you'd get the same result
just by increasing the thread count on the adapter. Either way you're using
up a thread per request.

Since the database functions are not asynchronous, core.async isn't really
useful in this context. My suggestion would be to drop it altogether.

- James

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Review] Slow server, not maxing out CPU

[Ashish Negi]

with core async `go` you can not do blocking IO or any time consuming work.
`go` uses fixed threadpool (no of cpus + 2 or something).
and in your async-handler you are using `https://clojure.github.io/core.async/

Try with `future`. https://clojuredocs.org/clojure.core/future

Simplest would be to pass your `channel` in with-channel to the future 
which would put the data in it after it gets from `work`.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Review] Slow server, not maxing out CPU

[James Reeves]

If you're going to be working with blocking I/O, then you're probably going
to get constrained by the default threadpool size of HTTP Kit, which is
only 4 threads. Up it to around 1000 and see if that improves matters.

You can get around this by writing asynchronous code, but your
async-handler isn't asynchronous.

- James

On 20 July 2016 at 22:02, Вук Мировић  wrote:

> I'm fooling around with Clojure and I implemented simple server that will
> randomly query/insert/update/delete random data from DB on every request.
> When I benchmark (wrk) server is slow (compared to similar solution) and I
> noticed that CPU is not maxed out, it's about ~30-40%.
> I used http-kit for HTTP server, Hakiri-CP as connection pool. Added some
> core.async hoping to improve the speed but results are the same.
> There is clearly something wrong with my code but I can't figure it out.
> I'm hoping if someone could throw a glimpse on my code to give a sugestion.
> Here is the code:
> https://github.com/wooque/bench/blob/master/clojure/src/bench/core.clj
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Review] Slow server, not maxing out CPU

[Вук Мировић]

I'm fooling around with Clojure and I implemented simple server that will 
randomly query/insert/update/delete random data from DB on every request.
When I benchmark (wrk) server is slow (compared to similar solution) and I 
noticed that CPU is not maxed out, it's about ~30-40%.
I used http-kit for HTTP server, Hakiri-CP as connection pool. Added some 
core.async hoping to improve the speed but results are the same.
There is clearly something wrong with my code but I can't figure it out. 
I'm hoping if someone could throw a glimpse on my code to give a sugestion.
Here is the 
code: https://github.com/wooque/bench/blob/master/clojure/src/bench/core.clj

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.