smengcl commented on code in PR #10595:
URL: https://github.com/apache/ozone/pull/10595#discussion_r3472603342
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeStateMachine.java:
##########
@@ -205,20 +211,36 @@ public DatanodeStateMachine(HddsDatanodeService
hddsDatanodeService,
importer,
new SimpleContainerDownloader(conf, certClient));
ContainerReplicator pushReplicator = new PushReplicator(conf,
- new OnDemandContainerReplicationSource(container.getController()),
+ new OnDemandContainerReplicationSource(container.getController(),
+ replicationConfig),
new GrpcContainerUploader(conf, certClient, container.getController())
);
pullReplicatorWithMetrics = new MeasuredReplicator(pullReplicator, "pull");
pushReplicatorWithMetrics = new MeasuredReplicator(pushReplicator, "push");
- ReplicationConfig replicationConfig =
- conf.getObject(ReplicationConfig.class);
supervisor = ReplicationSupervisor.newBuilder()
.stateContext(context)
.datanodeConfig(dnConf)
.replicationConfig(replicationConfig)
.clock(clock)
+ .volumeChooser(task -> {
+ if (task instanceof ReplicationTask) {
+ ReplicationTask repTask = (ReplicationTask) task;
+ if (repTask.getTarget() == null) {
+ try {
+ return
importer.chooseNextVolume(importer.getDefaultReplicationSpace());
Review Comment:
This volume selection has a reservation side effect: chooseNextVolume()
increments the target volume's committed bytes. Doing it here makes the
reservation lifetime hard to pair with execution.
If the task is later dropped before DownloadAndImportReplicator runs, the
reservation is leaked. Conversely, if a pull attempt hits
REPLICATION_LIMIT_REACHED, DownloadAndImportReplicator releases the
reservation, marks the same task QUEUED, and the retry reuses task.getVolume()
without reserving again, so committed bytes can be decremented repeatedly.
Can we move volume selection/reservation into each actual pull attempt, or
explicitly track whether the task currently owns a reservation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
