jerqi commented on PR #308: URL: https://github.com/apache/incubator-uniffle/pull/308#issuecomment-1338643310
> Thanks for your thoughts about this feature. @jerqi > > > 1. If we use Netty, the feature may not take effect. > > So I think we should we could support this feature when we use GRPC. If we have Netty, Netty won't support this feature first. You can accomplish this feature based on Netty version. > > Yes. The client need to be compatible with concrete RPC implementation. I don't know the netty could implement a retry policy to tolerant the rpc-server temporary out-of-service. @leixm Could you give some advice about netty retry? > > > If we use multi replicas, it may cause some errors. > > If we retry a recovery replica too many times. I think this feature can't work with multi relicas. > > Yes. And this feature will effect the fast fail mechanism, especially when encountering network failure. > > > Recovery is a complicated operation. It's hard to review every condition, error will occur sometimes. If blockBitMap is persist, do you need crc? How to check whether we lose some state? > > The validation mechanism of crc looks unnecessary. We should respect the correctness of serialization. > > > This feature is hard to maintain. If you add other feature, the feature may involve other meta. how to reduce impact? Our project is developed quickly. Do this feature slow down our speed? > > Yes. Nice catch. The new critical meta data should be involved in this feature. I don't think this feature will slow down. As I know, if the new meta structure is introduced, it needs to do 2 things. > > 1. To serialize > 2. To deserialize > > If you think this process is difficult and unnecessary, I think maybe we could introduce the annotation to indicate the stateful data structure. > > I think the client retry mechanism needs to be discuss more. Do we really need this? Or introduce a new shuffle-server state of `stateful-upgrading` to let client know this, instead of retrying simply? How to inform the client the server state? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
