Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread George Neuner



On 6/9/2020 8:11 AM, Jon Zeppieri wrote:

On Tue, Jun 9, 2020 at 7:59 AM Bogdan Popa  wrote:
>
> I think we'd need to also set `SO_REUSEPORT', which is not available on
> all platforms, to support multiple processes listening on the same port
> without reusing file descriptors.

And even where it is available, it doesn't work the same way. The
Linux version can be used to load balance accept()s across processes,
but the BSD version (also in OS X) cannot. (FreeBSD apparently has a
variant, SO_REUSEPORT_LB, that behaves like the Linux version of
SO_REUSEPORT.)


And Windows has 2 options that mean essentially the same thing: 
SO_REUSE_UNICASTPORT  and  SO_PORT_SCALABILITY.


SO_PORT_SCALABILITY  was introduced in Windows 7 / Server 2008.
SO_REUSE_UNICASTPORT  was introduced in Windows 10.

Whichever port reuse option is supported by the platform, it normally is 
set automagically whenever  SO_REUSEADDR  is specified. Only functions 
that require an explicit bind need it to be set manually.




It's clear that enabling the functionality would have to be studied 
carefully.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/908a290c-75a3-30f0-515f-b93ade3ac103%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread George Neuner



On 6/9/2020 7:59 AM, Bogdan Popa wrote:

George Neuner writes:

> Multiple Racket applications *should* all be able to listen on the
> same port without having been spawned from the same ancestor process.
> If that isn't working now, something has gotten hosed.

I don't know whether this used to work in the past or not, but currently
only `SO_REUSEADDR' is set on TCP sockets:

https://github.com/racket/racket/blob/60bf8f970e97caae391bfe919b78c370b2d01bdd/racket/src/rktio/rktio_network.c#L1427

I think we'd need to also set `SO_REUSEPORT', which is not available on
all platforms, to support multiple processes listening on the same port
without reusing file descriptors.


It has been discussed before, but perhaps now, with RacketCS imminent, 
the socket functions should be changed / expanded to allow changing 
options without resorting to FFI.


AFAICS, the functionality of SO_REUSEPORT (if not the verbatim option) 
is available on all supported platforms.  Granted it is (relatively) a 
recent addition to both Linux and Windows.


Android doesn't support it, but AFAIK, Android is not considered a 
supported system.




> I'm not sure what you mean by "sharing" a listener,  but using a
> single listener with a pool of processing places actually is possible
> (though tricky).

I mean passing a `tcp-listener?' around between places so that each
place can call `tcp-accept' on it.  Something like this:

 #lang racket/base

 (require racket/place
  racket/tcp)

 (define ch
   (place ch
 (define listener (place-channel-get ch))
 (tcp-accept listener)
 (place-channel-put 'ok)))

 (module+ main
   (define listener
 (tcp-listen  512 #t))
   (place-channel-put ch listener)
   (place-channel-get ch))

Currently, this fails with:

 place-channel-put: contract violation
   expected: place-message-allowed?
   given: #
   context...:
 ...

My understanding is that there's nothing preventing this from working
apart from the fact that no one's yet added support for this in the
rktio layer.  As you mentioned, though, even if this was supported we
might run into other limitations when running many places at once.


Only serialized[1] functions can be passed between place ... and the 
receiving place still has to have imported the libraries (if any) 
necessary to understand it.  I don't know if a listener even can be 
serialized, though ... AFAIK a serializable function has to be compiled 
that way.


George

[1]  https://docs.racket-lang.org/web-server-internal/closure.html

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/ea9627a2-34d7-0c09-8246-32fd2d955c25%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread Jon Zeppieri
On Tue, Jun 9, 2020 at 7:59 AM Bogdan Popa  wrote:
>
> I think we'd need to also set `SO_REUSEPORT', which is not available on
> all platforms, to support multiple processes listening on the same port
> without reusing file descriptors.

And even where it is available, it doesn't work the same way. The
Linux version can be used to load balance accept()s across processes,
but the BSD version (also in OS X) cannot. (FreeBSD apparently has a
variant, SO_REUSEPORT_LB, that behaves like the Linux version of
SO_REUSEPORT.)

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/CAKfDxxxg_CanKg1Kgj0XP4%2B8aaGNB64ysmmGPCuK4PXq0nmQhg%40mail.gmail.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread Bogdan Popa


George Neuner writes:

> Multiple Racket applications *should* all be able to listen on the
> same port without having been spawned from the same ancestor process.
> If that isn't working now, something has gotten hosed.

I don't know whether this used to work in the past or not, but currently
only `SO_REUSEADDR' is set on TCP sockets:

https://github.com/racket/racket/blob/60bf8f970e97caae391bfe919b78c370b2d01bdd/racket/src/rktio/rktio_network.c#L1427

I think we'd need to also set `SO_REUSEPORT', which is not available on
all platforms, to support multiple processes listening on the same port
without reusing file descriptors.  Running a second instance of this
program:

#lang racket/base

(require racket/tcp)

(tcp-listen  512 #t)
(sync/enable-break never-evt)

Currently fails with:

tcp-listen: listen failed
  port number: 
  system error: Address already in use; errno=48
  context...:
...

Greg Hendershott pointed out to me on Slack a while back that the
`ffi/usafe/port' module could be used to bind sockets with custom
options using the FFI and then convert them to ports, but I have yet to
try that.

> I'm not sure what you mean by "sharing" a listener,  but using a
> single listener with a pool of processing places actually is possible
> (though tricky).

I mean passing a `tcp-listener?' around between places so that each
place can call `tcp-accept' on it.  Something like this:

#lang racket/base

(require racket/place
 racket/tcp)

(define ch
  (place ch
(define listener (place-channel-get ch))
(tcp-accept listener)
(place-channel-put 'ok)))

(module+ main
  (define listener
(tcp-listen  512 #t))
  (place-channel-put ch listener)
  (place-channel-get ch))

Currently, this fails with:

place-channel-put: contract violation
  expected: place-message-allowed?
  given: #
  context...:
...

My understanding is that there's nothing preventing this from working
apart from the fact that no one's yet added support for this in the
rktio layer.  As you mentioned, though, even if this was supported we
might run into other limitations when running many places at once.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m24krkih6h.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread George Neuner



On 6/9/2020 3:02 AM, Bogdan Popa wrote:

Alex Harsanyi writes:

> Question 1: Based on this benchmark, is there any reason to chose anything
> else but "drogon"?  Even if one chooses the second best on that list, which
> is "actix", they already loose about 6% performance and things degrade
> quickly afterwards.  The framework at position 10 is already half the speed
> of the top one.

My take on these benchmarks is all that matters is that the framework
doesn't get in your way once you add business logic.  The vast majority
of "real" web applications out there don't (and most likely can't) do
50k rps.


Or anywhere close to that given more realistic database queries.



You can see that in the "multiple queries" and "data updates" tests
where the results are packed closer together because the logic is closer
to what a lot of database-backed web applications do and the database
ends up being a bottleneck.  A description of the requirements of each
test can be found here:

https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview

If you know and are willing to deal with writing and maintaining C++
then drogon looks like it might be a great choice.

> Question 2:  Based on Bogdans message in this thread, it seems that most of
> the performance improvement for the Racket benchmark comes from the nginx
> configuration (which has nothing to do with Racket) and the next
> improvement has to do with how the user program is written (by supplying a
> "Content-Length" header).  So, is this benchmark really testing the Racket
> web server performance, or is it testing a very specific deployment?

The largest improvement comes from making the Racket application take
advantage of all the hardware threads on the machine.  Because Racket
doesn't currently have a way to share TCP listeners across places and
because fork isn't natively supported (I mentioned that it works via the
FFI earlier in the thread, but I believe it needs some support from the
runtime (handling of `EAGAIN') to work efficiently and not cause a lot
of churn) I did the next best thing: I made the benchmark run one Racket
process for each thread[1] and added nginx as a load balancer in front.

The nginx process listens on port 8080, forks one subprocess per core
(which lets the subprocesses reuse the same port) and then proxies any
incoming requests on that port to one of the Racket processes so every
single request is ultimately served by the Racket app.  What this means
in terms of this benchmark is that, compared to others, we're actually
paying a toll for using nginx here because its own workers are consuming
resources on the machine, but, to my knowledge, we don't have a better
alternative at the moment.


Hmm.  SSL which can be tricky to set up right for Racket, but that's the 
only reason I can see for using a separate HTTP server. Multiple Racket 
applications *should* all be able to listen on the same port without 
having been spawned from the same ancestor process.  If that isn't 
working now, something has gotten hosed.


I'm not sure what you mean by "sharing" a listener,  but using a single 
listener with a pool of processing places actually is possible (though 
tricky).  TCP ports can be passed among places, so a single listener 
instance can direct multiple processing instances.  I don't know how 
passing off the port would interact with web-server response handling (I 
would think the listener place could just shut down / abandon the port 
and leave response to the process place but I have never actually tried 
that).


Dynamic (thread in process) places seem to have issues when there are 
many instances, so for lots of cores it is better to use distributed 
(parallel process) places.  Paulo Matos's  Loci  package makes using 
process places much easier [and it also works on Windows if that matters].

https://pkgs.racket-lang.org/package/loci



[1]: 
https://github.com/TechEmpower/FrameworkBenchmarks/blob/988f052c8170da661c49dd51d1f33d500a871031/frameworks/Racket/racket/scripts/run#L15-L19


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/b763ea64-6319-b162-747b-825cf1cabcb9%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-09 Thread Bogdan Popa


Alex Harsanyi writes:

> Question 1: Based on this benchmark, is there any reason to chose anything
> else but "drogon"?  Even if one chooses the second best on that list, which
> is "actix", they already loose about 6% performance and things degrade
> quickly afterwards.  The framework at position 10 is already half the speed
> of the top one.

My take on these benchmarks is all that matters is that the framework
doesn't get in your way once you add business logic.  The vast majority
of "real" web applications out there don't (and most likely can't) do
50k rps.

You can see that in the "multiple queries" and "data updates" tests
where the results are packed closer together because the logic is closer
to what a lot of database-backed web applications do and the database
ends up being a bottleneck.  A description of the requirements of each
test can be found here:

https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview

If you know and are willing to deal with writing and maintaining C++
then drogon looks like it might be a great choice.

> Question 2:  Based on Bogdans message in this thread, it seems that most of
> the performance improvement for the Racket benchmark comes from the nginx
> configuration (which has nothing to do with Racket) and the next
> improvement has to do with how the user program is written (by supplying a
> "Content-Length" header).  So, is this benchmark really testing the Racket
> web server performance, or is it testing a very specific deployment?

The largest improvement comes from making the Racket application take
advantage of all the hardware threads on the machine.  Because Racket
doesn't currently have a way to share TCP listeners across places and
because fork isn't natively supported (I mentioned that it works via the
FFI earlier in the thread, but I believe it needs some support from the
runtime (handling of `EAGAIN') to work efficiently and not cause a lot
of churn) I did the next best thing: I made the benchmark run one Racket
process for each thread[1] and added nginx as a load balancer in front.

The nginx process listens on port 8080, forks one subprocess per core
(which lets the subprocesses reuse the same port) and then proxies any
incoming requests on that port to one of the Racket processes so every
single request is ultimately served by the Racket app.  What this means
in terms of this benchmark is that, compared to others, we're actually
paying a toll for using nginx here because its own workers are consuming
resources on the machine, but, to my knowledge, we don't have a better
alternative at the moment.

[1]: 
https://github.com/TechEmpower/FrameworkBenchmarks/blob/988f052c8170da661c49dd51d1f33d500a871031/frameworks/Racket/racket/scripts/run#L15-L19

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m27dwgiuwo.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-08 Thread Alex Harsanyi


On Monday, June 8, 2020 at 6:11:37 PM UTC+8, Yury Bulka wrote:
>
> Wow, from 695 requests per second to 49,516 is a huge improvement! 
>
> Since we were comparing to django previously, it's now much closer with 
> django (which does 78,132 rps.) 
>

I know very little about web development, so these may be some stupid 
questions, but I am very curious.

I am looking at 
http://www.techempower.com/benchmarks/#section=data-r19=ph=composite, 
and even with 78k requests/second, Django is at the bottom of that list 
(position 98 out of 104).  In fact, most of the web frameworks that I can 
recognize are at the bottom of this list, and their score is also very 
modest, compared to the frameworks at the top.  The highest framework that 
I can recognize in that list is "flask" and its performance is a mere 4.8% 
of the top one.

Question 1: Based on this benchmark, is there any reason to chose anything 
else but "drogon"?  Even if one chooses the second best on that list, which 
is "actix", they already loose about 6% performance and things degrade 
quickly afterwards.  The framework at position 10 is already half the speed 
of the top one.

Question 2:  Based on Bogdans message in this thread, it seems that most of 
the performance improvement for the Racket benchmark comes from the nginx 
configuration (which has nothing to do with Racket) and the next 
improvement has to do with how the user program is written (by supplying a 
"Content-Length" header).  So, is this benchmark really testing the Racket 
web server performance, or is it testing a very specific deployment?

Alex.

 

>
> Django also does run multiple worker processes (3 per cpu): 
>
> https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Python/django/gunicorn_conf.py#L8
>  
>
> The other tests are also doing much better. 
>
> So do I undestand correctly that the factors that contribute to the 
> improvement are: 
>
> 1. running multiple worker processes behind nginx 
> 2. adding content-length to all responses 
> 3. using CS variant of Racket 
> 4. using Racket 7.7 
> 5. tuning nginx config (enabling http 1.1 especially) 
>
> I'm curious what was the individual contribution of these factors. In 
> the PR regarding #5 you already stated that it gives a 4-5x improvement. 
>
> #1 and #5 are the things one would normally do anyway in a production 
> setup I guess. As is #4 most likely. 
>
> #2 is something that seems to require manual work in the client code, 
> but maybe that can be made easier on web-server-lib side somehow. 
>
> Curious how much do #3 contribute. 
>
> Big thanks again for everyone investing time in investigating this. 
>
> -- 
> Yury Bulka 
> https://mamot.fr/@setthemfree 
> #NotOnFacebook  
>
>
> Bogdan Popa > writes: 
>
> > Small update on this: I've updated the benchmarks to run multiple Racket 
> > processes with an Nginx load balancer in front.  After some tuning[1], 
> here 
> > is what the results look like on my 12 core AMD Ryzen 9 3900 server: 
> > 
> > 
> https://www.techempower.com/benchmarks/#section=test=669bfab7-9242-4c26-8921-a4fe9ccd8530=ph=composite=2
>  
> > 
> > 50k/s is a respectable number for the plaintext benchmark IMO and we 
> > could get it to go higher if we could ditch Nginx or spend more time 
> > improving the server's internals, as Sam suggested. 
> > 
> > The `racket-perf' benchmark is for a branch[2] that I have where I've 
> made 
> > some small improvements to the server's internals. 
> > 
> > [0]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5727 
> > [1]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5737 
> > [2]: https://github.com/racket/web-server/pull/94 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/6a31e2c4-c698-4841-93ba-274cb4f24581o%40googlegroups.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-08 Thread Bogdan Popa


Yury Bulka writes:

> Wow, from 695 requests per second to 49,516 is a huge improvement!
>
> Since we were comparing to django previously, it's now much closer with
> django (which does 78,132 rps.)

I expect the Racket benchmark will do even better on TechEmpower's hw
than it did on mine because their machines are more powerful and they
run the benchmarking code, the database and the servers on different
machines so the gap should end up being even smaller.

Also worth keeping in mind that the server Django uses for these
benchmarks, meinheld, is completely written in C:

* 
https://github.com/mopemope/meinheld/blob/311acbc4e7bd38fa3f3d0e158b35cde9ef73f8e5/meinheld/gmeinheld.py#L11
* 
https://github.com/mopemope/meinheld/tree/311acbc4e7bd38fa3f3d0e158b35cde9ef73f8e5/meinheld/server

So I think it's really impressive that we're able to get this close with
just pure Racket, with the overhead of running one nginx process per
core in front of the app and with the overhead of connecting those nginx
processes to the app via TCP.  Speaking of which, another thing we could
do to improve performance in this benchmark is define a custom `tcp^'
unit based on `unix-socket-lib' and have nginx connect to the backends
through unix sockets rather than TCP.

> So do I undestand correctly that the factors that contribute to the
> improvement are:
>
> 1. running multiple worker processes behind nginx
> 2. adding content-length to all responses
> 3. using CS variant of Racket
> 4. using Racket 7.7
> 5. tuning nginx config (enabling http 1.1 especially)

I'm afraid I haven't kept notes on how much improvement each of these
improvements yielded.  From what I remember: I don't think #4 was much
of a factor, #1 and #5 were the biggest factors followed by #2 and #3,
which I changed at the same time so I couldn't say which made more of a
difference.

> #2 is something that seems to require manual work in the client code,
> but maybe that can be made easier on web-server-lib side somehow.

The work one has to do is pretty minimal[1] and I personally like that
the default is to stream data so I think the only thing to improve here
is awareness: the docs for `response' should be updated to mention that
unless a `Content-Length' is provided, the responses will use chunked
transfer encoding.

[1]: 
https://github.com/TechEmpower/FrameworkBenchmarks/pull/5727/files#diff-b21f7e3ecfa09726dac9ce079f612719R47-R70

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m2a71dismf.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-08 Thread Stephen De Gabrielle
Thank you Bogdan!

On Sun, Jun 7, 2020 at 6:39 PM Bogdan Popa  wrote:

> Small update on this: I've updated the benchmarks to run multiple Racket
> processes with an Nginx load balancer in front.  After some tuning[1], here
> is what the results look like on my 12 core AMD Ryzen 9 3900 server:
>
>
> https://www.techempower.com/benchmarks/#section=test=669bfab7-9242-4c26-8921-a4fe9ccd8530=ph=composite=2
>
> 50k/s is a respectable number for the plaintext benchmark IMO and we
> could get it to go higher if we could ditch Nginx or spend more time
> improving the server's internals, as Sam suggested.
>
> The `racket-perf' benchmark is for a branch[2] that I have where I've made
> some small improvements to the server's internals.
>
> [0]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5727
> [1]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5737
> [2]: https://github.com/racket/web-server/pull/94
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/m2o8pupyge.fsf%40192.168.0.142
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/CAGHj7-KCyyspVnGpHZYfZdheVq9WMV0gGUYx_sTrzKwHKkxBQg%40mail.gmail.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-08 Thread Yury Bulka
Wow, from 695 requests per second to 49,516 is a huge improvement!

Since we were comparing to django previously, it's now much closer with
django (which does 78,132 rps.)

Django also does run multiple worker processes (3 per cpu):
https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Python/django/gunicorn_conf.py#L8

The other tests are also doing much better.

So do I undestand correctly that the factors that contribute to the
improvement are:

1. running multiple worker processes behind nginx
2. adding content-length to all responses
3. using CS variant of Racket
4. using Racket 7.7
5. tuning nginx config (enabling http 1.1 especially)

I'm curious what was the individual contribution of these factors. In
the PR regarding #5 you already stated that it gives a 4-5x improvement.

#1 and #5 are the things one would normally do anyway in a production
setup I guess. As is #4 most likely.

#2 is something that seems to require manual work in the client code,
but maybe that can be made easier on web-server-lib side somehow.

Curious how much do #3 contribute.

Big thanks again for everyone investing time in investigating this.

--
Yury Bulka
https://mamot.fr/@setthemfree
#NotOnFacebook


Bogdan Popa  writes:

> Small update on this: I've updated the benchmarks to run multiple Racket
> processes with an Nginx load balancer in front.  After some tuning[1], here
> is what the results look like on my 12 core AMD Ryzen 9 3900 server:
>
> https://www.techempower.com/benchmarks/#section=test=669bfab7-9242-4c26-8921-a4fe9ccd8530=ph=composite=2
>
> 50k/s is a respectable number for the plaintext benchmark IMO and we
> could get it to go higher if we could ditch Nginx or spend more time
> improving the server's internals, as Sam suggested.
>
> The `racket-perf' benchmark is for a branch[2] that I have where I've made
> some small improvements to the server's internals.
>
> [0]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5727
> [1]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5737
> [2]: https://github.com/racket/web-server/pull/94

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/87y2oxq33h.fsf%40privacyrequired.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-07 Thread Bogdan Popa
Small update on this: I've updated the benchmarks to run multiple Racket
processes with an Nginx load balancer in front.  After some tuning[1], here
is what the results look like on my 12 core AMD Ryzen 9 3900 server:

https://www.techempower.com/benchmarks/#section=test=669bfab7-9242-4c26-8921-a4fe9ccd8530=ph=composite=2

50k/s is a respectable number for the plaintext benchmark IMO and we
could get it to go higher if we could ditch Nginx or spend more time
improving the server's internals, as Sam suggested.

The `racket-perf' benchmark is for a branch[2] that I have where I've made
some small improvements to the server's internals.

[0]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5727
[1]: https://github.com/TechEmpower/FrameworkBenchmarks/pull/5737
[2]: https://github.com/racket/web-server/pull/94

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m2o8pupyge.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-02 Thread Bogdan Popa
The reason for the poor performance relative to the other langs/frameworks is 
that there is currently no easy way to take advantage of multiple cores using 
the web framework so what’s being benchmarked is single core perf. This is 
mainly a problem for benchmarks such as this, but not really an issue in the 
real world where you’d just run multiple processes w/ a load balancer in front.

> On Jun 1, 2020, at 1:43 PM, hashim muqtadir  wrote:
> 
> 
> A new version of these benchmarks was just published ("Round 19"): 
> https://www.techempower.com/benchmarks/#section=data-r19
> 
> Racket scores rather poorly (0 errors is nice, though). Anybody has any 
> guesses as to why this is? Racket performs well enough as far as I'm 
> concerned, and while I don't use the web server much, sounds unlikely that it 
> should be so far below django, for example. Maybe they run it in a 
> handicapped configuration or something.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/2a1062a4-e4f1-484c-a8a2-f63c161959e8%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/FD7C8C62-8AE2-411E-A32D-3AC180DFB3D6%40defn.io.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-02 Thread Bogdan Popa


Bogdan Popa writes:

> Only dispatchers get direct access to the output port for the socket.
> If you use `dispatch/servlet', then it takes care of taking your
> `response' value and calling `output-response' on it.  Unless the server
> knows the connection should be closed and unless the request was a HEAD
> request, then it outputs the response by chunking it:
>
> https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/http/response.rkt#L133-L181

I was wrong about this.  The web-server does the right thing if the
`Content-Length' header is present in the response value:

https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/http/response.rkt#L120

When I change the benchmark to take advantage of that, throughput doubles:

https://www.techempower.com/benchmarks/#section=test=ab930604-6b19-4ab4-a2a5-93e674a81804=ph=plaintext=2

I'll make a PR against the main repo to make sure all of the responses
have content lengths.

Out of curiosity, I made the app fork through FFI calls[1]:

https://www.techempower.com/benchmarks/#section=test=0197c25f-d544-4357-b3be-a40d4a8760ef=ph=plaintext=2

This worked although multiple processes wake up when a connection can be
accepted and the IO layer doesn't seem to handle `EAGAIN':

racket: Connection error: tcp-accept: accept from listener failed
racket:   system error: Resource temporarily unavailable; errno=11
racket:   context...:
racket:.../more-scheme.rkt:261:28
racket:/root/.racket/7.7/pkgs/compatibility-lib/mzlib/thread.rkt:75:14: loop


[1]: 
https://gist.github.com/Bogdanp/b7b72ff7845f7f2c51e64bde553128d0#file-fork-app-rkt-L232-L256

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m28sh5rc87.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-02 Thread Bogdan Popa
George Neuner writes:

> What I did say is that Python's threads are core limited - and *that*
> is true.   As a technical matter, Python *may* in fact start threads
> on different cores, but the continual need to take the GIL quickly
> forces every running thread in the process onto the same core.

I was pointing out that the GIL is irrelevant in this case.  All the
Python implementations in this benchmark use either green threads by
monkeypatching the standard threading and IO modules (gevent, meinheld)
or coroutines (asyncio) for concurrency and they fork subprocesses for
parallelism.


> That one actually is expected:  Racket's JSON (de)serializer is
> relatively slow.

It's the same as the plaintext test, except JSON is written to the
client instead of plain text:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/bfd7c1442f33e620524b6aa0751c11576b412e72/frameworks/Racket/racket/servlet.rkt#L140-L143


> What wasn't expected was Sam's results from the "plain text" test
> which also showed Racket much slower than Python.  That does hint at a
> lot of overhead in the Racket framework.

Here's single-core Python vs Racket in the plaintext benchmark on my
machine:

https://www.techempower.com/benchmarks/#section=test=464938de-3ec0-4931-bc68-41566c22=ph=plaintext=2

It is surprising that Racket does worse on this benchmark that it does
on the JSON one, despite the fact that `response/json' uses
`response/output' under the hood.

I see these errors from Racket when I run the plaintext benchmark, but
they don't occur in any of the others:

racket: tcp-addresses: could not get peer address
racket:   system error: Transport endpoint is not connected; errno=107
racket:   context...:
racket:
/usr/share/racket/collects/racket/contract/private/arrow-higher-order.rkt:375:33
racket:.../more-scheme.rkt:261:28
racket:
/usr/share/racket/collects/racket/contract/private/arrow-higher-order.rkt:375:33
racket:
/root/.racket/7.6/pkgs/web-server-lib/web-server/private/dispatch-server-with-connect-unit.rkt:144:4:
 connection-loop

I'll try to figure out what's causing these.


> To my knowledge, continuations will not be a factor unless either 1)
> the application is written in the #web-server language (which converts
> everything to CPS), or 2) the code invokes one of the send/suspend/*
> functions.

Whether you use the web interaction functions or not, servlets have to
do some bookkeeping (create new "instances", insert continuation
prompts) to support continuations:

* 
https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/servlet-dispatch.rkt#L78-L92
* 
https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/servlet/setup.rkt#L52-L72
* 
https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/dispatchers/dispatch-servlets.rkt#L63-L100

Bypassing all of this is what I considered cheating, because most people
probably won't.  At the same time, though, I haven't measured what the
overhead of all this stuff is.  It could be minimal.


> My understanding is that the port passed to  response/output  is the
> actual socket ... so you can front-end it and write directly.  But
> that might be "cheating" under your definition.

That's what I did in the benchmark:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/bfd7c1442f33e620524b6aa0751c11576b412e72/frameworks/Racket/racket/servlet.rkt#L135-L138

Only dispatchers get direct access to the output port for the socket.
If you use `dispatch/servlet', then it takes care of taking your
`response' value and calling `output-response' on it.  Unless the server
knows the connection should be closed and unless the request was a HEAD
request, then it outputs the response by chunking it:

https://github.com/racket/web-server/blob/547b3fd736684651e94ebd78902633374be6bcae/web-server-lib/web-server/http/response.rkt#L133-L181

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m2h7vtrjv9.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread George Neuner



On 6/1/2020 4:21 PM, Bogdan Popa wrote:

George Neuner writes:

> But Python's DB pool is threaded, and Python's threads are core
> limited by the GIL in all the major implementations (excepting
> Jython).

Python's Postgres pooling does not[1] use POSIX threads under the hood
to manage the connections if that's what you mean, nor is the
concurrency of the Python applications based on system threads.  All of
the Python examples use either asyncio + fork(2) or green threads +
fork(2).  This includes the django example[3].


I said nothing whatsoever about POSIX.  And FYI, POSIX does not imply 
any particular implementation - POSIX specifies only the API, and a 
compliant implementation may provide kernel threads, user space threads, 
or a combination of both.


What I did say is that Python's threads are core limited - and *that* is 
true.   As a technical matter, Python *may* in fact start threads on 
different cores, but the continual need to take the GIL quickly forces 
every running thread in the process onto the same core.




> There are a few things Python can do faster than Racket, but the VAST
> difference in performance shown in the techempower tests isn't
> explained by them.

Here's a benchmark that doesn't touch the DB at all, showing an even
bigger difference in throughput between the two:

https://www.techempower.com/benchmarks/#section=data-r19=ph=json


That one actually is expected:  Racket's JSON (de)serializer is 
relatively slow.



What wasn't expected was Sam's results from the "plain text" test which 
also showed Racket much slower than Python.  That does hint at a lot of 
overhead in the Racket framework.




I wrote the latest implementation of the Racket code for that benchmark
and I considered doing things like bypassing the "standard"
`dispatch/servlet' implementation to avoid the overhead of all the
continuation machinery in the web server, but that felt like cheating.


To my knowledge, continuations will not be a factor unless either 1) the 
application is written in the #web-server language (which converts 
everything to CPS), or 2) the code invokes one of the send/suspend/*  
functions.


FWIW: I try to avoid using client facing continuations in my own web 
applications - for my money there are too many uncertainties connected 
with them.


Also the stuffer does a fair amount of work to deal with long 
continuation URLs.




Another area where the web server does more work than it should is in
generating responses: the web server uses chunked transfer encoding for
all responses; whereas all the Python web servers simply write the
response directly to the socket when the length of the content is known
ahead of time.


My understanding is that the port passed to  response/output  is the 
actual socket ... so you can front-end it and write directly.  But that 
might be "cheating" under your definition.



George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/2fd131be-ca0e-eaa1-6595-d31a410d43d4%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread Brian Adkins
I may look into this in more detail later, but I ran a simple benchmark 
comparison on my modest AWS EC2 server (ApacheBench can behave poorly on 
MacOS).

1) I ran ApacheBench w/ 6 processes to fetch a simple "hello world" static 
html file using only nginx. I got roughly 650 requests per second.

2) I then ran ApacheBench w/ 6 processes against one of my Racket web apps 
using a monitoring endpoint that does a simple db query to determine the 
health of the server, so this went from nginx (acting as a load balancer 
and https handler) to the Racket processes via proxy_pass (only 2 running 
in parallel) which exercises my custom dispatching and a simple postgres 
query. No continuations and completely stateless. I got roughly 350 
requests per second.

At first glance, that doesn't appear to be that much overhead when 
comparing the two. In fact, I would've expected the very small static html 
request to be significantly more than double the req/sec of the Racket app 
db request.

I developed Rails apps for over a decade, and I know my Racket web apps are 
significantly faster in similar "database backed web apps" context.

I believe there is something wrong with those benchmarks at the moment.

Brian

On Monday, June 1, 2020 at 3:40:52 PM UTC-4, Sam Tobin-Hochstadt wrote:
>
> I'm skeptical both of the DB explanation and the multi-core 
> explanation. As you say, the difference between something like Django 
> and Racket is much too large to be explained by that. For example, on 
> the "plaintext" benchmark, Racket serves about 700 req/sec (I get 
> similar results on my machine). Many of the benchmarks in languages 
> like Python and Ruby do more than 1000x better, which means that even 
> if we had perfect speedup on 32 cores, we'd be nowhere close. 
> Additionally, the "plaintext" benchmark doesn't touch the DB at all. I 
> tried commenting out all of the DB code entirely, and it did not 
> change the results. 
>
> My guess is that the web server is just doing a lot of per-response 
> work that would need to be optimized. 
>
> Sam 
>
> On Mon, Jun 1, 2020 at 2:12 PM George Neuner  > wrote: 
> > 
> > 
> > On 6/1/2020 1:40 PM, Bogdan Popa wrote: 
> > > I replied earlier today off of my Phone, but, for whatever reason 
> > > (caught in the moderation queue?), it's not showing up in this thread. 
> > > 
> > > Here's what it said: 
> > > 
> > > The reason for poor performance relative to the other 
> > > langs/frameworks is that there is currently no easy way to take 
> > > advantage of multiple cores using the web framework so that's 
> being 
> > > benchmarked is single-core performance. 
> > > 
> > > This is mainly a problem for benchmarks such as this, but not 
> really 
> > > an issue in the real world where you'd just run multiple processes 
> > > with a load balancer in front. 
> > 
> > Single core [by itself] doesn't explain the enormous performance 
> > difference between Racket and Django. 
> > 
> > I haven't looked at the Django submission - Python's (in)comprehensions 
> > give me a headache.  But Python's DB pool is threaded, and Python's 
> > threads are core limited by the GIL in all the major implementations 
> > (excepting Jython). 
> > 
> > There are a few things Python can do faster than Racket, but the VAST 
> > difference in performance shown in the techempower tests isn't explained 
> > by them.  My suspicion is that the Racket application is making too many 
> > database connections and not relying enough on its open connection 
> > pool.  Hundreds of trivial requests can be served in the time it takes 
> > to spin up a new backend process. 
> > 
> > YMMV, 
> > George 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Racket Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to racket...@googlegroups.com . 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/9236dcff-81df-1db8-c2ef-06b20e4690ec%40comcast.net.
>  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/ce26a93b-e1f2-425a-abc9-be991ff0e4e4%40googlegroups.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread Bogdan Popa


George Neuner writes:

> But Python's DB pool is threaded, and Python's threads are core
> limited by the GIL in all the major implementations (excepting
> Jython).

Python's Postgres pooling does not[1] use POSIX threads under the hood
to manage the connections if that's what you mean, nor is the
concurrency of the Python applications based on system threads.  All of
the Python examples use either asyncio + fork(2) or green threads +
fork(2).  This includes the django example[3].

> There are a few things Python can do faster than Racket, but the VAST
> difference in performance shown in the techempower tests isn't
> explained by them.

Here's a benchmark that doesn't touch the DB at all, showing an even
bigger difference in throughput between the two:

https://www.techempower.com/benchmarks/#section=data-r19=ph=json

Here's the same benchmark running on my local machine where I
intentionally limited the Django app to a single CPU and I made it use
the `gevent' library for its workers so it is more comparable to the
Racket implementation:

https://www.techempower.com/benchmarks/#section=test=14ecbf16-cdb3-4501-8b7d-a2b8a549f73c=ph=json=2

And here's what happens when I let it use as much parallelism as it can:

https://www.techempower.com/benchmarks/#section=test=d3ad4d79-c7a7-4ca0-b297-ffda549947c8=ph=json=2

I do agree that improving the parallelism part wouldn't be enough to
catch up (clearly, there's a 2x difference even on a single core), but
it is a large factor here.

I wrote the latest implementation of the Racket code for that benchmark
and I considered doing things like bypassing the "standard"
`dispatch/servlet' implementation to avoid the overhead of all the
continuation machinery in the web server, but that felt like cheating.

Another area where the web server does more work than it should is in
generating responses: the web server uses chunked transfer encoding for
all responses; whereas all the Python web servers simply write the
response directly to the socket when the length of the content is known
ahead of time.

Another thing of note about the django implementation is that it uses
ujson, written in C with the express intent of being as fast as
possible, to generate the JSON data.


[1]: They call the default implementation a `ThreadedConnectionPool'[2],
but that's just because it uses the mutexes that the `threading' module
provides.

[2]: 
https://github.com/psycopg/psycopg2/blob/779a1370ceeac130de07edc0510f2c55846be1bd/lib/pool.py#L155

[3]: 
https://github.com/TechEmpower/FrameworkBenchmarks/blob/c49524762379a2cdf82627b0032c654f3a9eafb6/frameworks/Python/django/gunicorn_conf.py#L8-L21

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m2mu5mr0zy.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread George Neuner



On 6/1/2020 3:40 PM, Sam Tobin-Hochstadt wrote:

I'm skeptical both of the DB explanation and the multi-core
explanation. As you say, the difference between something like Django
and Racket is much too large to be explained by that. For example, on
the "plaintext" benchmark, Racket serves about 700 req/sec (I get
similar results on my machine). Many of the benchmarks in languages
like Python and Ruby do more than 1000x better, which means that even
if we had perfect speedup on 32 cores, we'd be nowhere close.
Additionally, the "plaintext" benchmark doesn't touch the DB at all. I
tried commenting out all of the DB code entirely, and it did not
change the results.

My guess is that the web server is just doing a lot of per-response
work that would need to be optimized.

Sam


Possibly ... I admit that I did not look at the plain text results: I 
was drawn to the fact that the DB results impose a lot of non-Racket 
overhead.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/cfc6c027-180d-d822-6d36-9f1cfb1849fc%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread Sam Tobin-Hochstadt
I'm skeptical both of the DB explanation and the multi-core
explanation. As you say, the difference between something like Django
and Racket is much too large to be explained by that. For example, on
the "plaintext" benchmark, Racket serves about 700 req/sec (I get
similar results on my machine). Many of the benchmarks in languages
like Python and Ruby do more than 1000x better, which means that even
if we had perfect speedup on 32 cores, we'd be nowhere close.
Additionally, the "plaintext" benchmark doesn't touch the DB at all. I
tried commenting out all of the DB code entirely, and it did not
change the results.

My guess is that the web server is just doing a lot of per-response
work that would need to be optimized.

Sam

On Mon, Jun 1, 2020 at 2:12 PM George Neuner  wrote:
>
>
> On 6/1/2020 1:40 PM, Bogdan Popa wrote:
> > I replied earlier today off of my Phone, but, for whatever reason
> > (caught in the moderation queue?), it's not showing up in this thread.
> >
> > Here's what it said:
> >
> > The reason for poor performance relative to the other
> > langs/frameworks is that there is currently no easy way to take
> > advantage of multiple cores using the web framework so that's being
> > benchmarked is single-core performance.
> >
> > This is mainly a problem for benchmarks such as this, but not really
> > an issue in the real world where you'd just run multiple processes
> > with a load balancer in front.
>
> Single core [by itself] doesn't explain the enormous performance
> difference between Racket and Django.
>
> I haven't looked at the Django submission - Python's (in)comprehensions
> give me a headache.  But Python's DB pool is threaded, and Python's
> threads are core limited by the GIL in all the major implementations
> (excepting Jython).
>
> There are a few things Python can do faster than Racket, but the VAST
> difference in performance shown in the techempower tests isn't explained
> by them.  My suspicion is that the Racket application is making too many
> database connections and not relying enough on its open connection
> pool.  Hundreds of trivial requests can be served in the time it takes
> to spin up a new backend process.
>
> YMMV,
> George
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/9236dcff-81df-1db8-c2ef-06b20e4690ec%40comcast.net.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/CAK%3DHD%2BauHeC6veH-867PuiihSbYmZB2hKyx5ZnBH%2B-1sXUKwxA%40mail.gmail.com.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread George Neuner



On 6/1/2020 1:40 PM, Bogdan Popa wrote:

I replied earlier today off of my Phone, but, for whatever reason
(caught in the moderation queue?), it's not showing up in this thread.

Here's what it said:

The reason for poor performance relative to the other
langs/frameworks is that there is currently no easy way to take
advantage of multiple cores using the web framework so that's being
benchmarked is single-core performance.

This is mainly a problem for benchmarks such as this, but not really
an issue in the real world where you'd just run multiple processes
with a load balancer in front.


Single core [by itself] doesn't explain the enormous performance 
difference between Racket and Django.


I haven't looked at the Django submission - Python's (in)comprehensions 
give me a headache.  But Python's DB pool is threaded, and Python's 
threads are core limited by the GIL in all the major implementations 
(excepting Jython).


There are a few things Python can do faster than Racket, but the VAST 
difference in performance shown in the techempower tests isn't explained 
by them.  My suspicion is that the Racket application is making too many 
database connections and not relying enough on its open connection 
pool.  Hundreds of trivial requests can be served in the time it takes 
to spin up a new backend process.


YMMV,
George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/9236dcff-81df-1db8-c2ef-06b20e4690ec%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread Bogdan Popa
I replied earlier today off of my Phone, but, for whatever reason
(caught in the moderation queue?), it's not showing up in this thread.

Here's what it said:

   The reason for poor performance relative to the other
   langs/frameworks is that there is currently no easy way to take
   advantage of multiple cores using the web framework so that's being
   benchmarked is single-core performance.

   This is mainly a problem for benchmarks such as this, but not really
   an issue in the real world where you'd just run multiple processes
   with a load balancer in front.


hashim muqtadir writes:

> A new version of these benchmarks was just published ("Round 19"):
> https://www.techempower.com/benchmarks/#section=data-r19
>
> Racket scores rather poorly (0 errors is nice, though). Anybody has any
> guesses as to why this is? Racket performs well enough as far as I'm
> concerned, and while I don't use the web server much, sounds unlikely that
> it should be so far below django, for example. Maybe they run it in a
> handicapped configuration or something.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/m2tuzusn0u.fsf%40192.168.0.142.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread George Neuner



On 6/1/2020 11:12 AM, Sam Tobin-Hochstadt wrote:

I think the biggest thing is that no one has looked at optimizing
these benchmarks in Racket. If you tried out running one of these
benchmarks and ran the profiler it would probably show something
interesting.

Sam
The code[1] itself isn't bad.  There are a couple of minor things I 
personally would tweak, but in my opinion the main problem is with the 
database access.


Postgresql uses process parallelism - it forks a new backend process to 
handle each connection.  Assuming (???) there are enough concurrent 
requests to open the maximum number of DB pool connections, then 1024 is 
WAY too many.  PG experts recommend no more than 5..10 backend processes 
per core, but according to the environment details[2], the test machines 
have only 14 HT cores (28 threads).


Mind you I'm only guessing, but it looks to me like this application 
could be losing an enormous amount of time in the starting of new 
backend database processes.  And since it is running inside a Docker 
container, there also is some per connection overhead there.


The requests made by the tests[3] are so trivial (needing little 
additional Racket processing) that I would consider reducing the number 
of PG pool connections to just a few per core and see how that goes.  My 
expectation is that the time saved by spinning up many fewer PG 
processes will vastly outweigh any loss in absolute request 
concurrency.  [Particularly if they also are running PG itself inside 
Docker.]


YMMV,
George

[1] 
https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Racket/racket/servlet.rkt
[2] 
https://www.techempower.com/benchmarks/#section=environment=ph=fortune
[3] 
https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview


--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/eafb41e8-734c-7666-e684-edade3731e79%40comcast.net.


Re: [racket-users] Re: [racket] Web Framework Benchmarks

2020-06-01 Thread Sam Tobin-Hochstadt
I think the biggest thing is that no one has looked at optimizing
these benchmarks in Racket. If you tried out running one of these
benchmarks and ran the profiler it would probably show something
interesting.

Sam

On Mon, Jun 1, 2020 at 6:43 AM hashim muqtadir
 wrote:
>
> A new version of these benchmarks was just published ("Round 19"): 
> https://www.techempower.com/benchmarks/#section=data-r19
>
> Racket scores rather poorly (0 errors is nice, though). Anybody has any 
> guesses as to why this is? Racket performs well enough as far as I'm 
> concerned, and while I don't use the web server much, sounds unlikely that it 
> should be so far below django, for example. Maybe they run it in a 
> handicapped configuration or something.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/2a1062a4-e4f1-484c-a8a2-f63c161959e8%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/CAK%3DHD%2BYJh96XAabvFRrW2HTFW9ECjKdnFOKomK0mdKeoA_4GNQ%40mail.gmail.com.