Hexiaoqiao commented on a change in pull request #2288:
URL: https://github.com/apache/hadoop/pull/2288#discussion_r490219801
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -134,6 +134,9 @@
private final FileIoProvider fileIoProvider;
private final DataNodeVolumeMetrics metrics;
private URI baseURI;
+ private boolean enableSameDiskArchival;
+ private final String device;
Review comment:
what about using `storageID` replace `device`? IMO both of them are in
order to index single volume, right?
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -412,16 +435,31 @@ long getBlockPoolUsed(String bpid) throws IOException {
*/
@VisibleForTesting
public long getCapacity() {
+ long capacity;
if (configuredCapacity < 0L) {
long remaining;
if (cachedCapacity > 0L) {
remaining = cachedCapacity - getReserved();
} else {
remaining = usage.getCapacity() - getReserved();
}
- return Math.max(remaining, 0L);
+ capacity = Math.max(remaining, 0L);
+ } else {
+ capacity = configuredCapacity;
+ }
+
+ if (enableSameDiskArchival) {
+ double reservedForArchival = conf.getDouble(
Review comment:
`reservedForArchival` here is same as `this.reservedForArchival` when
init?
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -190,6 +193,26 @@
}
this.conf = conf;
this.fileIoProvider = fileIoProvider;
+ this.enableSameDiskArchival =
+ conf.getBoolean(DFSConfigKeys.DFS_DATANODE_ALLOW_SAME_DISK_TIERING,
+ DFSConfigKeys.DFS_DATANODE_ALLOW_SAME_DISK_TIERING_DEFAULT);
+ if (enableSameDiskArchival) {
+ this.device = usage.getMount();
+ reservedForArchive = conf.getDouble(
+ DFSConfigKeys.DFS_DATANODE_RESERVE_FOR_ARCHIVE_PERCENTAGE,
+ DFSConfigKeys.DFS_DATANODE_RESERVE_FOR_ARCHIVE_PERCENTAGE_DEFAULT);
+ if (reservedForArchive >= 1) {
+ FsDatasetImpl.LOG.warn("Value of reserve-for-archival is >= 100% for "
+ + currentDir + ". Setting it to 99%.");
+ reservedForArchive = 0.99;
Review comment:
Why `reservedForArchive` has to less than 1 here, IIUC it means that
this is ARCHIVE device when `reservedForArchive` set to 1. Right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
