lidavidm commented on issue #40360: URL: https://github.com/apache/arrow/issues/40360#issuecomment-1978730694
> Can I configure the number of concurrent requests my server handles at a time via Arrow Flight RPC? No. gRPC doesn't expose meaningful control over this. Handle it at the application layer (e.g. a semaphore or a mutex+condition variable+counter). > Is it possible to split the server to multiple workers as if I'd have different instances of the server (including the model) running in parallel? Not really. If you host on say Kubernetes or otherwise have control over DNS, if you have your client connect to a host that resolves to multiple IP addresses, gRPC can distribute requests over those hosts. https://github.com/grpc/grpc/blob/master/doc/load-balancing.md#pick_first (The wording is extremely confusing because it's overly abstracted and they also tried to invent their own load balancing API standard that everything is written in terms of, despite being deprecated and never really used) Otherwise, use a proxy/load balancer in the middle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
