Hey Eric Thanks for the reply. A few inline responses:
On Tue, Jul 21, 2026 at 4:29 PM Eric Anderson <[email protected]> wrote: > Heyo! > > On Tue, Jul 21, 2026 at 3:31 PM 'Carl Mastrangelo' via grpc.io < > [email protected]> wrote: > >> 1. What thread do ServerInterceptors run on? I would assume it's the >> ServerBuilder executor, but that presents a problem. To limit the number >> of threads, we need to limit the number of calls. The ServerInterceptor is >> the natural place to do this, but if the executor is used to enforce the >> call limit, it's too late. We already spent a thread on it. >> > > Yes, the ServerBuilder executor. You can track the concurrency present and > fail the RPC within the interceptor. Yes, you used a thread, but for a > short period of time. This is pretty easy and reasonably effective. > > If you limit the number of threads, then you've moved the problem to the > queue that forms. Ordinarily you'd configure the executor to throw > RejectExecutionException, but that isn't supported > <https://github.com/grpc/grpc-java/issues/636>. *Somewhere* I remember > mentioning the idea of having *yet another* executor, and only use it for > cancelling RPCs. It could be a single thread. But that's not implemented, > although it wouldn't be that hard. > Yes, I was hoping that other executor would basically run on the network thread, with cancellation being a cheap thing. Basically a concurrency limiter. A hacky solution I have is a custom executor that checks for concurrency limits, and then funnels the callbacks to a secondary, thread pool executor. Workable but less than ideal. Minor request: I think having Javadocs on https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerBuilder.html#executor(java.util.concurrent.Executor). would help clarify the usage. I couldn't remember if this covered ServerCall methods, or the Interceptor methods too. Actually it's kinda vauge what specifically the executors are intended to be used for. > > We added ServerBuilder.callExecutor() > <https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerBuilder.html#callExecutor(io.grpc.ServerCallExecutorSupplier)> > which > lets you change the executor for an RPC. This is great if you want to > segment your services to separate pools. ServerCallExecutorSupplier is > called on the ServerBuilder executor as well, because we have to look up > the MethodDescriptor before we can create the ServerCall to provide > context. ServerBuilder executor being used is fine from a resource usage > perspective. But it can increase latency. If you don't have a > ServerBuilder.fallbackHandlerRegistry() or you know it executes quickly, > then you can use callExecutor(directExecutor()) and always return an > executor from the ServerCallExecutorSupplier (to avoid using > directExecutor() for application processing), without paying two thread > hops. > > For FIFO (based on RPC concurrently limit, not queue length), > ServerCallExecutorSupplier could increment for each RPC and use an > interceptor to decrement when complete. If this new RPC would exceed a > threshold, fail the RPC, and either return null or a "cleanup" executor for > the fast-running cancellation processing. > > Part of the problem with RejectedExecutionException in Java is you can't > implement LIFO to reduce average latencies. For LIFO, you have to get > "fancy." > > For LIFO (based on "queue length"), ServerCallExecutorSupplier can store > call information in the returned executor to annotate the Runnables *for > that specific RPC*. If it integrates with the executor's queue stack and > wants to then kill RPCs at the bottom of the stack, it can, because it can > identify where those Runnables came from and it knows the application is > currently not receiving callbacks. It still has to run the Runnables to > release resources, and because grpc-java doesn't support cancellation on > server-side, I think you still need to pair this with an interceptor. It is > probably much easier for unary RPCs since there is no need for > potentially-slow application cleanup. I suspect nobody has actually tried > this approach. > > 2. What happens when there is both a Context deadline and an AbstractStub >> deadline? I couldn't find in the documentation which one takes precedence, >> or how they are combined. I assume the Context deadline fires anyways if >> present, but that means that using AbstractStub.withDeadlineAfter() may be >> misleading. The deadline would not be able to be increased. >> > > They don't impact each other; both apply. So the shortest "wins." This is > no different than Context.withDeadlineAfter() on a Context that already has > a deadline, which is also not documented, but due to cancellation cascading > that's the only possible implementation. The child context can't outlive > the parent. > That's kinda what I remembered too, but then it made me think that there wasn't a way to increase it. Another request: [Javadoc] guidance on when to use the Stub, CallOption, or Context for setting deadlines. One other idea: Is it possible to fork a context with the current key-values, but drop the cancellation propagation? Something like Context.ROOT.withAllKeys(Context.current()).attach(). -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/grpc-io/CAL3uUy7PUhsMWBPrevF8quT6nFyGD8zGGSdgJJxmdzs_bC%2BBiw%40mail.gmail.com.
