In short, because of GIL (https://wiki.python.org/moin/GlobalInterpreterLock ).
The `max_workers` flag is meant for IO intensive tasks. If you are reading from a database or calling an API in your handler, you should observe performance increase as you increase `max_workers`. Unfortunately, for CPU-bound tasks. The more thread workers you spawned in gRPC will result in more GIL contentions, and slow the whole process down. Also, due to some legacy issue, the synchronization overhead between threads is quite significant. If you DO CARE about Python server performance, please help me by posting issue in GitHub! https://github.com/grpc/grpc/issues So the priority of improving Python server performance can raise. On Saturday, July 20, 2019 at 3:02:56 AM UTC-7, alireza hoseini wrote: > > I have used the below sample code to run the server: > > self.server = grpc.server(futures.ThreadPoolExecutor(max_workers=1)), > maximum_concurrent_rpcs=3000) > > Some unnecessary codes are removed for better readability. I have used ghz > <https://ghz.sh/docs/usage> to load test my gRPC server: > > ghz --insecure --total=1000 --proto my_proto.proto --call my_proto.XServer > .GetUserById -d '{"user_id": "5d32d36bec68e90844d88ae7"}' 0.0.0.0:9000 > > Summary: > Count: 1000 > Total: 1.05 s > Slowest: 66.93 ms > Fastest: 5.59 ms > Average: 51.43 ms > * Requests/sec: 948.35* > > Response time histogram: > 5.586 [1] | > 11.721 [3] | > 17.856 [4] | > 23.990 [4] | > 30.125 [4] | > 36.260 [5] | > 42.395 [6] | > 48.530 [11] |∎ > 54.665 [870] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ > 60.799 [84] |∎∎∎∎ > 66.934 [8] | > > Latency distribution: > 10% in 49.43 ms > 25% in 50.32 ms > 50% in 51.82 ms > 75% in 53.61 ms > 90% in 54.61 ms > 95% in 55.38 ms > 99% in 59.97 ms > > Status code distribution: > [OK] 1000 responses > > > But when I increase *max_workers *to 2 I get RPS of around 700 and if I > increase *max_workers* I reach to 600 RPS and lower. Why this behavior > happens? > > My understanding is that max_workers increase workers that need to get the > job done I have one worker then it will be served far more slowly. Why this > happens? Shouldn't max_workers increase RPS? > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/14f4585d-c8e2-4d97-87b3-9a5d588845b6%40googlegroups.com.
