In my example in the previous email, I accidentally used a very old version
of echo. If you use the latest ("github.com/labstack/echo/v4"), then it's a
lot faster than using simple string splitting, resulting in about 22,000
requests per second.
Concurrency Level: 100
Time taken for tests: 0.443 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 2160000 bytes
HTML transferred: 430000 bytes
Requests per second: 22565.77 [#/sec] (mean)
Time per request: 4.431 [ms] (mean)
Time per request: 0.044 [ms] (mean, across all concurrent requests)
Transfer rate: 4759.97 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 0.5 2 9
Processing: 1 2 0.6 2 9
Waiting: 0 2 0.6 2 9
Total: 3 4 0.8 4 11
Percentage of the requests served within a certain time (ms)
50% 4
66% 4
75% 5
80% 5
90% 5
95% 5
98% 6
99% 6
100% 11 (longest request)
On Mon, Jun 10, 2019 at 4:12 PM Marcin Romaszewicz <marc...@gmail.com>
wrote:
> One more followup.
>
> Here's an example using an HTTP router named Echo, which I use in
> production. With proper HTTP parsing and validation, and no regular
> expressions involved in routing, it still does about 14,000 requests per
> second. I stubbed some of your stuff which doesn't affect the result. This
> is a much better implementation than speaking HTTP yourself over sockets
> and it's stupendously fast.
>
> Concurrency Level: 100
> Time taken for tests: 0.717 seconds
> Complete requests: 10000
> Failed requests: 0
> Total transferred: 2160000 bytes
> HTML transferred: 430000 bytes
> Requests per second: 13940.29 [#/sec] (mean)
> Time per request: 7.173 [ms] (mean)
> Time per request: 0.072 [ms] (mean, across all concurrent requests)
> Transfer rate: 2940.53 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 3 1.2 3 8
> Processing: 1 4 1.2 4 9
> Waiting: 0 3 1.2 3 8
> Total: 2 7 1.3 7 13
>
> Percentage of the requests served within a certain time (ms)
> 50% 7
> 66% 7
> 75% 8
> 80% 8
> 90% 9
> 95% 10
> 98% 11
> 99% 11
> 100% 13 (longest request)
>
>
> package main
>
> import (
> "flag"
> "fmt"
> "net/http"
> "strconv"
>
> "github.com/labstack/echo"
> )
>
> const pixel =
> "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\xFF\xFF\xFF\x00\x00\x00\x21\xF9\x04\x01\x00\x00\x00\x00\x2C\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3B"
>
> func main() {
> var port = flag.Int("port", 8080, "service port")
> flag.Parse()
>
> autoProxy := fmt.Sprintf(
> "function FindProxyForURL(url, host) { return \"PROXY %s:3128;
> DIRECT\"; }",
> "stubbed.host")
> autoProxyBuf := []byte(autoProxy)
>
> e := echo.New()
>
> serveAutoProxy := func(c echo.Context) error {
> response := c.Response()
> response.Header().Add("Connection", "close")
> return c.Blob(http.StatusOK, "application/octet-stream", autoProxyBuf)
> }
> e.GET("/proxy.pac", serveAutoProxy)
> e.GET("/wpad.dat", serveAutoProxy)
>
> pixelBuf := []byte(pixel)
> servePixel := func(c echo.Context) error {
> response := c.Response()
> response.Header().Add("Connection", "close")
> response.Header().Add("ETag", "dbab")
> response.Header().Add("Cache-Control", "public, max-age=31536000")
> response.Header().Add("Content-length", strconv.Itoa(len(pixelBuf)))
> return c.Blob(http.StatusOK, "image/gif", pixelBuf)
> }
> e.GET("*", servePixel)
>
> // Start server
> e.Logger.Fatal(e.Start(fmt.Sprintf("0.0.0.0:%d", *port)))
> }
>
>
>
> On Mon, Jun 10, 2019 at 4:07 PM Marcin Romaszewicz <marc...@gmail.com>
> wrote:
>
>> I think the others were correct in pointing the finger at the RegEx
>> engine in Go. It is quite slow. I hacked your inside loop which checks the
>> request not to use regular expressions, and it's tons faster. You can't say
>> that something can't be responsible for too much slowdown because it "1
>> line", since that one line has a lot of weight behind it. Using regular
>> expressions, a benchmark showed 1782 requests per second. Using my simple
>> hack, it's 18827 per second. Let's call it 10x faster.
>>
>> if strings.Contains(line, "HTTP") {
>> parts := strings.Split(line, " ")
>> req.Method = strings.ToUpper(strings.TrimSpace(parts[0]))
>> req.URL = strings.ToUpper(strings.TrimSpace(parts[1]))
>> req.Version = strings.ToUpper(strings.TrimSpace(parts[2]))
>> continue
>> }
>>
>>
>> For this benchmark it behaves correctly, I realize this is fragile. I ran
>> `ab -n 10000 -c 100...` to run the tests.
>>
>> Benchmarks using regular expressions:
>> Concurrency Level: 100
>> Time taken for tests: 5.610 seconds
>> Complete requests: 10000
>> Failed requests: 0
>> Total transferred: 1790000 bytes
>> HTML transferred: 430000 bytes
>> Requests per second: 1782.41 [#/sec] (mean)
>> Time per request: 56.104 [ms] (mean)
>> Time per request: 0.561 [ms] (mean, across all concurrent requests)
>> Transfer rate: 311.57 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 46 352.7 8 3557
>> Processing: 0 10 10.6 8 158
>> Waiting: 0 10 10.6 8 158
>> Total: 0 56 352.5 21 3565
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 21
>> 66% 24
>> 75% 24
>> 80% 24
>> 90% 25
>> 95% 26
>> 98% 164
>> 99% 3549
>> 100% 3565 (longest request)
>>
>> Benchmarks using my hack:
>>
>> Concurrency Level: 100
>> Time taken for tests: 0.531 seconds
>> Complete requests: 10000
>> Failed requests: 0
>> Total transferred: 1790000 bytes
>> HTML transferred: 430000 bytes
>> Requests per second: 18827.39 [#/sec] (mean)
>> Time per request: 5.311 [ms] (mean)
>> Time per request: 0.053 [ms] (mean, across all concurrent requests)
>> Transfer rate: 3291.12 [Kbytes/sec] received
>>
>> Connection Times (ms)
>> min mean[+/-sd] median max
>> Connect: 0 3 0.3 3 4
>> Processing: 1 3 0.4 3 5
>> Waiting: 0 3 0.4 3 5
>> Total: 3 5 0.5 5 8
>>
>> Percentage of the requests served within a certain time (ms)
>> 50% 5
>> 66% 5
>> 75% 5
>> 80% 5
>> 90% 6
>> 95% 6
>> 98% 7
>> 99% 7
>> 100% 8 (longest request)
>>
>>
>> On Mon, Jun 10, 2019 at 2:28 PM Tong Sun <suntong...@gmail.com> wrote:
>>
>>> Just to clarify some facts.
>>>
>>> On Sun, Jun 9, 2019 at 11:09 AM 'Axel Wagner' wrote:
>>> >
>>> > As I've also mentioned: I don't think this test is meaningful.
>>> >
>>> > First, as it has been pointed out, your Perl program isn't actually a
>>> web server. It only understands ridiculously simple requests and as such
>>> violates the spec left and right. It's also super naive in how it treats
>>> malformed input or actively malicious clients - all of which are handled by
>>> the Go http package, so of course it's going to have some overhead.
>>>
>>> There is a second test well before your this post, which is a direct
>>> translation of Perl code, that is now reading and writing directly to
>>> a socket. Hanging on to the first test method and not referring to the
>>> second test is not a very constructive way of discussion, let alone
>>> using words like "ridiculously ...".
>>>
>>> > In its most generous form, you translate the programs faithfully to do
>>> the same syscalls and implement the same logic - and at that point, you are
>>> essentially benchmarking the Perl regular expression engine against the Go
>>> regular expression engine, as that's the only thing the program really is
>>> doing: Branching on a regexp-match. And the Perl RE-engine is famously
>>> optimized and the Go RE-engine famously is not. In fact, you aren't even
>>> benchmarking Perl against Go, you are benchmarking *C* against Go, as Perls
>>> RE-engine is written in C, AFAIK.
>>>
>>> Over 90% of the code are NOT doing regular expression matching.
>>> Focusing *only* on regular expression, not >90% of the rest, is not
>>> very convincing but miss the elephant in the room, at least seems to
>>> me. Many people have given valid inputs as where things might get
>>> improved, including the one you are quoting.
>>>
>>> > Lastly, as I read the results I get with either net/http *or* the
>>> naive regexp-parsing, I just… disagree with your conclusions. All the
>>> numbers I've seen seem to imply that Go responded *on average* a lot faster
>>> and *might* have higher variance and a slightly higher tail latency
>>> (though, FTR, the measurements I got also suggest that the data is mostly
>>> noise). And I'm struggling to find an application, where that would matter
>>> a lot. Like, yeah, tail latency matters, but so does average latency. In
>>> basically all applications I can think of, you a) want tail latencies to
>>> not be catastrophic, especially when considering fan-out, but b) lower
>>> averages are just as important anyway. So, I think you should, for a decent
>>> test, formulate your criteria beforehand. Currently, there seems to be a
>>> lot of reading in tea-leafs involved.
>>>
>>> Please don't get personal and emotional over technical discussions,
>>> and please refrain from using terms like "ridiculously simple" or
>>> "reading in tea-leafs" in future discussions. It is inappropriately
>>> condescending, thus not the correct attitude of communication. It
>>> does not align with the code of conduct of this mlist, and neither of
>>> google's, I believe.
>>>
>>> If you have test results that contradict with mine, please show me
>>> yours -- Let's talk the data, and not let the emotion get in the way
>>> of technical discussion, and fact the fact, whatever it is.
>>>
>>> > Anyway, all of that being said: While I don't think the test you
>>> devised allows the broad generalizations you are making, ultimately I don't
>>> have any stakes in this (which is why I haven't responded further on the
>>> blog post). If you like Perl and think it performs good enough or better
>>> for your use case - then go ahead and use Perl. No one here will begrudge
>>> you for it.
>>>
>>> As I've comment in the blog many times, I'm not trying to prove Perl
>>> performs better than Go. On the contrary, I was trying to improve Perl
>>> performs with Go, that's why the whole thing get started, as I have
>>> replied to your comment in the blog:"that's why I was rewriting the
>>> Perl code to Go".
>>>
>>> It is still the case, and I want to try out everyone's suggestion to
>>> see how things can improve.
>>>
>>> Moreover, if you have taken a look at my second test, you will
>>> understand my other goal is to make it clear "who and what to trust".
>>> Because if you have taken a look at the httperf test result I posted
>>> to this mlist before you reply, you may have realized that, of all the
>>> performance testing tools I've used so far, all are suggesting Perl
>>> performs better than Go, except for httperf. So maybe the performance
>>> testing tools are biased toward Go, and I made it clear in my second
>>> blog that I want to get to the bottom of it.
>>>
>>> > All of that being said
>>> >
>>> > IMO this is just not a meaningful test. Or its results are totally
>>> unsurprising.
>>>
>>> Again, I was trying to improve Perl performs with Go. That
>>> "ridiculously simple" Perl code is the foundation of the Debian dbab
>>> package, and I was trying to improve it.
>>>
>>> It might be meaningless to you but it is perfectly meaningful to me.
>>> Please don't be so judgmental.
>>>
>>> > On Sun, Jun 9, 2019 at 4:54 AM Justin Israel <justinisr...@gmail.com>
>>> wrote:
>>> >>
>>> >> I'm wondering about a couple factors in this comparison that seem to
>>> make a difference in my local test:
>>> >>
>>> >> I think perl sockets are write buffered. So would the equivalent be
>>> to wrap the net.Conn in bufio.NewWriter(c) and flush before the Close?
>>> >> Since this is a straigh-line test where both servers are not using
>>> concurrent handling of connections (uncommon for a Go server even on 1
>>> core), would it not make sense to run the Go server with GOMAXPROCS=1?
>>> >>
>>> >> - Justin
>>> >>
>>> >> On Saturday, June 8, 2019 at 1:36:49 AM UTC+12, Tong Sun wrote:
>>> >>>
>>> >>> I had always believed that the web projects build with Go should be
>>> much faster than Perl, since Go is a compiled language.
>>> >>>
>>> >>> However that belief was crushed brutally last night, when I did a
>>> comparison -- the Go implementation is 8 times worse than the Perl! -- the
>>> mean response time jumped from 6ms to 48ms.
>>> >>>
>>> >>> I know this is the simplest possible web server, but still, when it
>>> comes to simple web servers like this, I have to say that Perl performs
>>> much better than Go.
>>> >>>
>>> >>> I don't think there is much I can twist on the Go side, since it
>>> can't be more simpler than that. However, I also believe it won't hurt to
>>> ask and confirm. So,
>>> >>>
>>> >>> Have I missed anything? Is it possible for me to make my Go
>>> implementation anywhere near the Perl's performance?
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> >> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to golang-nuts+unsubscr...@googlegroups.com.
>>> >> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/37a1ac7e-85ee-4775-b348-5673c41a162c%40googlegroups.com
>>> .
>>> >> For more options, visit https://groups.google.com/d/optout.
>>> >
>>> > --
>>> > You received this message because you are subscribed to a topic in the
>>> Google Groups "golang-nuts" group.
>>> > To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/golang-nuts/iH2Ck_hpCpI/unsubscribe.
>>> > To unsubscribe from this group and all its topics, send an email to
>>> golang-nuts+unsubscr...@googlegroups.com.
>>> > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfF62D5v%2BRiGtAtzuH0wAHzCLqcwNUidC1Oe2KN9DfRv6Q%40mail.gmail.com
>>> .
>>> > For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/CAMmz1OcefkrZN3_5R7%2BbjuwHjRx%2BLzttaiXmer0ytwmXUjxN_A%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CA%2Bv29LtxoqO6UJW-vntc9vDJFv%3DO%3D_izYD68nJKFN7u-oTx_2g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
