pegasas commented on PR #1136: URL: https://github.com/apache/incubator-uniffle/pull/1136#issuecomment-1676876933
> for one shuffleId, it only will use one type of ess/rss Hi, @zuston , So the solution is In `registerShuffle`, If RSS can be accessed and we initial RSS Shuffle Manager in the lazy mode for getting ShuffleHandle by ShuffleId. Then ShuffleWriter/ShuffleReader binds to RSS then. Right? Also, HybridShuffleManager save (ShuffleId, ESS/RSS) mapping in memory for reader/writer to know which shufflehandle should be processed by ESS/RSS? If failover happens and mapping not exists, will trigger registerShuffle again to reassign shuffleId to ESS/RSS? Or we should persist (ShuffleId, ESS/RSS) mapping in DFS? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
