xianjingfeng commented on code in PR #276:
URL: https://github.com/apache/incubator-uniffle/pull/276#discussion_r1031961832
##########
storage/src/main/java/org/apache/uniffle/storage/factory/ShuffleHandlerFactory.java:
##########
@@ -53,80 +54,84 @@ public static synchronized ShuffleHandlerFactory
getInstance() {
return INSTANCE;
}
+
public ClientReadHandler
createShuffleReadHandler(CreateShuffleReadHandlerRequest request) {
+ if (CollectionUtils.isEmpty(request.getShuffleServerInfoList())) {
+ throw new RuntimeException("Shuffle servers should not be empty!");
+ }
+ if (request.getShuffleServerInfoList().size() > 1) {
Review Comment:
> I prefer to add a global data structure in composed handler, may be called
"progress".
> It stores the information of consumed replicas and servers.
> We could add the fallback in composed handler, and the each layer of
handler can restart from the the last by reading the progress.
This logic has the same problem as the previous version of this pr. If we
read fail from the memory handler and read successful from the localfile
handler. And then, the memory data flush to localfile and we read from memory
again, some data maybe lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
