sodonnel commented on a change in pull request #2973:
URL: https://github.com/apache/ozone/pull/2973#discussion_r784116849
##########
File path:
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ExcludeList.java
##########
@@ -34,31 +37,54 @@
*/
public class ExcludeList {
- private final Set<DatanodeDetails> datanodes;
+ private final Map<DatanodeDetails, Long> datanodes;
private final Set<ContainerID> containerIds;
private final Set<PipelineID> pipelineIds;
+ private long expiryTime = 0;
+ private java.time.Clock clock;
public ExcludeList() {
- datanodes = new HashSet<>();
+ datanodes = new HashMap<>();
containerIds = new HashSet<>();
pipelineIds = new HashSet<>();
}
+ public ExcludeList(long autoExpiryTime, java.time.Clock clock) {
+ this();
+ this.expiryTime = autoExpiryTime;
+ this.clock = clock;
+ }
+
public Set<ContainerID> getContainerIds() {
return containerIds;
}
public Set<DatanodeDetails> getDatanodes() {
- return datanodes;
+ if (expiryTime > 0) {
+ cleanExpiredNodes();
+ }
+ return new HashSet<>(datanodes.keySet());
+ }
+
+ private void cleanExpiredNodes() {
+ Iterator<Map.Entry<DatanodeDetails, Long>> iterator =
+ datanodes.entrySet().iterator();
+ while (iterator.hasNext()) {
+ Map.Entry<DatanodeDetails, Long> entry = iterator.next();
+ Long value = entry.getValue();
+ if (clock.millis() > value + expiryTime) {
+ iterator.remove(); // removing
+ }
+ }
}
public void addDatanodes(Collection<DatanodeDetails> dns) {
dns.forEach(dn -> addDatanode(dn));
}
public void addDatanode(DatanodeDetails dn) {
- datanodes.add(dn);
+ datanodes.put(dn, clock.millis());
Review comment:
I think Pifta suggested something like this on a different PR recently.
Rather than storing the current time here, store current_time + expiry_Time.
Then when checking if it needs to be expired, we can do:
```
if (clock.millis() > storedTime) {
```
which saves an addition operation for every key in the map each time we
check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
