Have you tried a tcpdump of the packets between the Go program and nginx? Is it using HTTP/1.1 or HTTP/2? If it's HTTP/1.1, does tcpdump show that it actually starts all 50 HTTP client requests simultaneously?
You are making all these concurrent requests to the same host. The default of MaxConnsPerHost I believe is 0 (unlimited), but DefaultMaxIdleConnsPerHost is 2. It could be worth cranking that up. I wonder if it's being forced to close down 48 of those connections immediately because it can't return them to the pool. I also wonder whether there's tuning required at the Nginx side, e.g. for the backlog queue. On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote: > I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In > general, though, using pprof is the best way to start to break down a > question like this. Would the actual workload involve more structured JSON, > or more computation with decoded values? > > On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org wrote: > >> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder <diogo...@gmail.com> wrote: >> >>> Hi guys, >>> >>> I've been working on some experiments with different web application >>> stacks to check their performances under a specific scenario: one in which >>> I have to make several concurrent requests and then gather the results >>> together (in order) and throw them out as JSON in the response body. (This >>> project is only an experiment, but it's informing me for decisions that >>> have to be made for a real-world project where we have a similar scenario.) >>> >>> However, probably due to my ignorance in Go, I cannot make it perform as >>> well as I expected - actually the best I'm getting are results that are >>> even slower than Python, which was a surprise to me. Here they are: >>> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin >>> >>> So, looking at the code here: >>> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go >>> >>> - does anybody see any problem in the implementation that could be hurting >>> performance? I tried using a WaitGroup, tried sharing memory (nasty, I >>> know, but just for the sake of experimentation), tried multiple JSON >>> codecs, different web frameworks, and nothing worked so far. I have a >>> feeling that I'm doing something fundamentally wrong and stupid, and that >>> somehow I can make a small change to make the experiment much faster. >>> >> >> Have you measured how much time is spent on the http.Get calls? It is >> likely that the 50 concurrent http.Get calls is the bottleneck. >> >> Also note that you don't need a channel there. You can simply use a >> waitgroup and set the results from inside the goroutine, because each >> goroutine knows the index. But that is unlikely to change anything >> measurable when compared to the Get calls. >> >> >> >>> >>> Thanks in advance, I'm sure this will help me learning more about the >>> language! :-) >>> >>> Cheers! >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0894c03c-ff5b-403f-97c8-74193821c10en%40googlegroups.com.
