ashishkumar50 commented on code in PR #10928:
URL: https://github.com/apache/ozone/pull/10928#discussion_r3897175232
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/PendingContainerTracker.java:
##########
@@ -143,14 +157,98 @@ synchronized boolean remove(ContainerID containerID) {
synchronized int getCount() {
return currentWindow.size() + previousWindow.size();
}
+ }
+
+ /**
+ * Pending container allocations for one datanode, grouped by storage type.
+ */
+ public static class PendingContainerAllocations {
+ private final Map<StorageType, TwoWindowBucket> typedBuckets =
+ new EnumMap<>(StorageType.class);
+ private final TwoWindowBucket unknownBucket;
+ private final long rollIntervalMs;
+ private final DatanodeID datanodeID;
+
+ PendingContainerAllocations(DatanodeID id, long rollIntervalMs) {
+ this.datanodeID = id;
+ this.rollIntervalMs = rollIntervalMs;
+ this.unknownBucket = new TwoWindowBucket(id, rollIntervalMs);
+ }
+
+ synchronized void rollIfNeeded() {
+ unknownBucket.rollIfNeeded();
+ typedBuckets.values().forEach(TwoWindowBucket::rollIfNeeded);
+ }
+
+ synchronized boolean contains(ContainerID containerID) {
+ return unknownBucket.contains(containerID)
+ || typedBuckets.values().stream()
+ .anyMatch(bucket -> bucket.contains(containerID));
+ }
+
+ /**
+ * Count pending containers of the given storage type.
+ * Unknown storage type entries are counted for typed checks because they
+ * may occupy the requested storage type.
+ */
+ synchronized int getCount(StorageType storageType) {
+ if (checksAllStorageTypes(storageType)) {
+ return getCount();
+ }
+ TwoWindowBucket bucket = typedBuckets.get(storageType);
+ return unknownBucket.getCount() + (bucket != null ? bucket.getCount() :
0);
Review Comment:
Why do we need unknownbucket count for all storage tier?
Example:
SSD physical available = 100 GB
SSD pending = 20 GB
UNKNOWN physical available = 2 TB
UNKNOWN pending = 200 GB
In this case allocation in SSD will fail, even though SSD has space.
Why do we need to consider unknown?
If we do like this, and if UNKNOWN exist in system, then known storage tier
will be always under-utilized to its capacity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
