smengcl commented on a change in pull request #2903:
URL: https://github.com/apache/ozone/pull/2903#discussion_r766427920
##########
File path:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SimpleContainerDownloader.java
##########
@@ -74,38 +74,28 @@ public SimpleContainerDownloader(
public CompletableFuture<Path> getContainerDataFromReplicas(
long containerId, List<DatanodeDetails> sourceDatanodes) {
- CompletableFuture<Path> result = null;
-
final List<DatanodeDetails> shuffledDatanodes =
shuffleDatanodes(sourceDatanodes);
for (DatanodeDetails datanode : shuffledDatanodes) {
try {
- if (result == null) {
- result = downloadContainer(containerId, datanode);
- } else {
-
- result = result.exceptionally(t -> {
- LOG.error("Error on replicating container: " + containerId, t);
- try {
- return downloadContainer(containerId, datanode).get();
- } catch (ExecutionException | IOException e) {
- LOG.error("Error on replicating container: " + containerId,
- e);
- } catch (InterruptedException e) {
- Thread.currentThread().interrupt();
- }
- return null;
- });
- }
+ CompletableFuture<Path> result =
+ downloadContainer(containerId, datanode);
+ result.get();
Review comment:
The goal of this `getContainerDataFromReplicas` I think is to find a
working source datanode, so it makes sense to call `get()` inside to wait for a
successful retrieval. `get()` here will trigger a full download of the
container and return the `Path` to the local replica.
It looks like the only caller of this method
`DownloadAndImportReplicator#replicate` is also handling the same set of
exceptions when calling `CompletableFuture#get()`.
Is the container retrieved twice if `get()` is called twice (or is it smart
enough to use cache/checksum/skip same file name)? If this is the case we
probably should just straight return `result.get` (`Path`) for
`getContainerDataFromReplicas` after the simplification? Could probably save a
round trip here
CC @avijayanhwx for more insights.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
