frankliee commented on code in PR #276:
URL: https://github.com/apache/incubator-uniffle/pull/276#discussion_r1031627360
##########
common/src/main/java/org/apache/uniffle/common/ShuffleServerInfo.java:
##########
@@ -27,6 +27,13 @@ public class ShuffleServerInfo implements Serializable {
private int port;
+ // Only for test
+ public ShuffleServerInfo(String host, int port) {
+ this.id = host + "-" + port;
+ this.host = host;
Review Comment:
It is a little strange to add a constructor just for test, we can just use
`new ShuffleServerInfo(host + "_" + String.valueOf(port), host, port)`
##########
storage/src/main/java/org/apache/uniffle/storage/factory/ShuffleHandlerFactory.java:
##########
@@ -53,80 +54,84 @@ public static synchronized ShuffleHandlerFactory
getInstance() {
return INSTANCE;
}
+
public ClientReadHandler
createShuffleReadHandler(CreateShuffleReadHandlerRequest request) {
+ if (CollectionUtils.isEmpty(request.getShuffleServerInfoList())) {
+ throw new RuntimeException("Shuffle servers should not be empty!");
+ }
+ if (request.getShuffleServerInfoList().size() > 1) {
Review Comment:
I agree with @jerqi, the current logic is too complicated.
It is better to use an unified code path (by the way, one server is a
special case of multiple servers)
I prefer to add a global data structure in composed handler, may be called
"progress".
It stores the information of consumed replicas and servers.
We could add the fallback in composed handler, and the each layer of handler
can restart from the the position of last retry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
