Re: TCP server app performance
On 08/14/2018 09:30 PM, Navdeep Parhar wrote: On 8/12/18 9:50 AM, Honda Michio wrote: Hi, I'm measuring TCP server app performance using my toy web server. It just accept TCP connections and responds back HTTP OK to the clients. It monitors sockets using kqueue, and processes each ready descriptor using a pair of read() and write(). (in more detail, it's https://github.com/micchie/netmap/tree/paste/apps/phttpd) Using 100 persistent TCP connections (the client sends 44 B HTTP GET and the server responds with 151 B of HTTP OK) and a single CPU core, I only get 152K requests per second, which is 2.5x slower than Linux that runs the same app (except that it uses epoll instead of kqueue). I cannot justify this by myself. Does anybody has some intuition about how much FreeBSD would get with such workloads? I tried disabling TCP delayed ack and changing interrupt rates, but no significant difference was observed. I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b). For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate only a single queue as I test with a single CPU core). All the offloadings are disabled. I hope hw L3/L4 checksumming is still on? They are off but exchanged messages are too small to benefit from these offloads anyways. Are your results similar to what you get with 100 (same number as your test clients) netperf's doing TCP_RR on this setup, or wildly different? I cannot find any netperf option that sends multiple connections with TCP_RR. I also don't find any equivalent in iperf. But single-connection performance is similar to what I get with wrk and my HTTP server. Running 100 netperf processes does not show aggregate throughput, and that also means running 100 kevent() loops, which is quite different behaviour from what I want. Cheers, - Michio Regards, Navdeep ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TCP server app performance
On 8/12/18 9:50 AM, Honda Michio wrote: > Hi, > > I'm measuring TCP server app performance using my toy web server. > It just accept TCP connections and responds back HTTP OK to the clients. > It monitors sockets using kqueue, and processes each ready descriptor using > a pair of read() and write(). (in more detail, it's > https://github.com/micchie/netmap/tree/paste/apps/phttpd) > > Using 100 persistent TCP connections (the client sends 44 B HTTP GET and > the server responds with 151 B of HTTP OK) and a single CPU core, I only > get 152K requests per second, which is 2.5x slower than Linux that runs the > same app (except that it uses epoll instead of kqueue). > I cannot justify this by myself. Does anybody has some intuition about how > much FreeBSD would get with such workloads? > I tried disabling TCP delayed ack and changing interrupt rates, but no > significant difference was observed. > > I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b). > For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate > only a single queue as I test with a single CPU core). All the offloadings > are disabled. I hope hw L3/L4 checksumming is still on? Are your results similar to what you get with 100 (same number as your test clients) netperf's doing TCP_RR on this setup, or wildly different? Regards, Navdeep ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TCP server app performance
El día domingo, agosto 12, 2018 a las 09:32:28p. m. +0200, Michio Honda escribió: > Hi Rozhuk, > > I only use a single CPU core and thread, so SO_REUSEPORT_LB has no > effect. Also, I already accept() in loop until getting error, but this > is not a point as I use persistent TCP connections... > > Cheers, > - Michio Hi Michio, Don't top post, please. matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub 13. August 2018: Ich würde gern einen AUSREISEANTRAG stellen. August 13, 2018: I'd like to ask for an exit permit of this country. signature.asc Description: PGP signature
Re: TCP server app performance
Hi Rozhuk, I only use a single CPU core and thread, so SO_REUSEPORT_LB has no effect. Also, I already accept() in loop until getting error, but this is not a point as I use persistent TCP connections... Cheers, - Michio On 08/12/2018 07:50 PM, Rozhuk Ivan wrote: On Sun, 12 Aug 2018 18:50:17 +0200 Honda Michio wrote: I'm measuring TCP server app performance using my toy web server. It just accept TCP connections and responds back HTTP OK to the clients. It monitors sockets using kqueue, and processes each ready descriptor using a pair of read() and write(). (in more detail, it's https://github.com/micchie/netmap/tree/paste/apps/phttpd) PS: 2. SO_REUSEPORT_LB ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TCP server app performance
On Sun, 12 Aug 2018 18:50:17 +0200 Honda Michio wrote: > I'm measuring TCP server app performance using my toy web server. > It just accept TCP connections and responds back HTTP OK to the > clients. It monitors sockets using kqueue, and processes each ready > descriptor using a pair of read() and write(). (in more detail, it's > https://github.com/micchie/netmap/tree/paste/apps/phttpd) > PS: 2. SO_REUSEPORT_LB ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: TCP server app performance
On Sun, 12 Aug 2018 18:50:17 +0200 Honda Michio wrote: > I'm measuring TCP server app performance using my toy web server. > It just accept TCP connections and responds back HTTP OK to the > clients. It monitors sockets using kqueue, and processes each ready > descriptor using a pair of read() and write(). (in more detail, it's > https://github.com/micchie/netmap/tree/paste/apps/phttpd) 1. Call accept() in loop until get error. 2. Use reuseport option for load balance. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
TCP server app performance
Hi, I'm measuring TCP server app performance using my toy web server. It just accept TCP connections and responds back HTTP OK to the clients. It monitors sockets using kqueue, and processes each ready descriptor using a pair of read() and write(). (in more detail, it's https://github.com/micchie/netmap/tree/paste/apps/phttpd) Using 100 persistent TCP connections (the client sends 44 B HTTP GET and the server responds with 151 B of HTTP OK) and a single CPU core, I only get 152K requests per second, which is 2.5x slower than Linux that runs the same app (except that it uses epoll instead of kqueue). I cannot justify this by myself. Does anybody has some intuition about how much FreeBSD would get with such workloads? I tried disabling TCP delayed ack and changing interrupt rates, but no significant difference was observed. I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b). For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate only a single queue as I test with a single CPU core). All the offloadings are disabled. Cheers, - Michio ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
