Hi Yi, Thank you for the great work on FLIP-582. As GPU resources become increasingly valuable, RpcOperator can significantly improve resource utilization and strengthening Flink’s competitiveness for AI workloads.I am particularly interested in the failover and flexible scaling aspects of this FLIP, and I have two questions that I hope you can help clarify.
1. Relationship between RpcOperator and JobGraph recovery semantics The current FLIP models RpcOperator as an independent JobVertex in the JobGraph. This means that although it can be isolated from the data plane in terms of resources and regions, it still belongs to the same job graph in terms of scheduling, recovery, and rescaling semantics. For example, in AdaptiveScheduler, rescaling or failure recovery may enter the Restarting -> CreatingExecutionGraph path and rebuild the ExecutionGraph. An external autoscaler can also adjust the parallelism requirements of job vertices through the JobResourceRequirements REST API; with AdaptiveScheduler, this may further trigger rescale / restart. In such cases, RpcOperator may also be restarted together with the data-plane tasks, since it is part of the same JobGraph / ExecutionGraph. This seems to leave some gap with the goal of RpcOperator being an independent service that is not affected by the data plane. Therefore, I would like to confirm whether the FLIP plans to introduce a more fine-grained recovery and restart mechanism, so that RpcOperator can restart, fail over, or rescale independently from data processing vertices. 2. Client support for future flexible scaling The FLIP mentions that RpcOperator instances are independent service instances, and that an instance going online or offline should not affect other instances or the data processing flow. I understand that the current FLIP may not need to fully support dynamic scaling in the first phase. However, if flexible scaling of RpcOperator is expected in the future, the client may need to be aware of changes in RpcOperator parallelism and the instance list. For example, during a future scale-out, the system may start a new RpcOperator instance without restarting existing ones. After the new instance becomes ready, the client needs to discover it in time and include it in request routing. During scale-in, the client also needs to detect instance removal in time and avoid sending new requests to instances that are about to exit. Therefore, I would like to confirm whether the current ROSClient design can support push-based discovery of RpcOperator instance additions and removals for future no-restart dynamic scaling. Best, Lei Yi Zhang <[email protected]> 于2026年5月27日周三 14:12写道： > Hi everyone, > > > > I would like to start a discussion on FLIP-582: Support RpcOperator > Service [1]. > > > AI-oriented workloads like multimodal data processing and model inference > are > growing rapidly in recent years. These workloads are characterized by > expensive > resources (GPUs) and high initialization costs (seconds to minutes for > model > loading). In today's Flink, embedding them in the data plane couples their > parallelism and failover with surrounding operators; deploying them as > external > services disconnects their lifecycle from the job and doubles operational > overhead. > > > This FLIP introduces RpcOperator Service — a framework-level primitive > that runs > user-defined compute as RPC services in an independent Pipelined Region > within > the Flink job. Because the service is isolated at the scheduling level, it > can achieve > fault isolation, independent scaling, and dedicated resource allocation. > As a native > Flink primitive, it also lays the foundation for automatic flow control, > flexible load > balancing, and coordinated auto-scaling — all without introducing external > infrastructure or additional operational burden. > > > > > Looking forward to your feedback and suggestions! > > > > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-582%3A+Support+RpcOperator+Service > > > > > > Best Regards, > Yi Zhang
