Hexiaoqiao commented on a change in pull request #2288:
URL: https://github.com/apache/hadoop/pull/2288#discussion_r494085423
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -412,16 +435,28 @@ long getBlockPoolUsed(String bpid) throws IOException {
*/
@VisibleForTesting
public long getCapacity() {
+ long capacity;
if (configuredCapacity < 0L) {
long remaining;
if (cachedCapacity > 0L) {
remaining = cachedCapacity - getReserved();
} else {
remaining = usage.getCapacity() - getReserved();
}
- return Math.max(remaining, 0L);
+ capacity = Math.max(remaining, 0L);
+ } else {
+ capacity = configuredCapacity;
+ }
+
+ if (enableSameDiskArchival) {
Review comment:
The return value seems not expected as annotation says if enable this
feature.
> the capacity of the file system excluding space reserved for non-HDFS.
IMO, the part for ARCHIVE should also be calculated. It seems be not
differentiated by NameNode for DISK or ARCHIVE per storage of DataNode. Please
correct if something wrong.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -452,7 +487,33 @@ public long getAvailable() throws IOException {
}
long getActualNonDfsUsed() throws IOException {
- return usage.getUsed() - getDfsUsed();
+ // DISK and ARCHIVAL on same disk
Review comment:
same confused as the last comment.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
##########
@@ -190,6 +193,26 @@
}
this.conf = conf;
this.fileIoProvider = fileIoProvider;
+ this.enableSameDiskArchival =
+ conf.getBoolean(DFSConfigKeys.DFS_DATANODE_ALLOW_SAME_DISK_TIERING,
+ DFSConfigKeys.DFS_DATANODE_ALLOW_SAME_DISK_TIERING_DEFAULT);
+ if (enableSameDiskArchival) {
+ this.mount = usage.getMount();
+ reservedForArchive = conf.getDouble(
Review comment:
`reservedForArchive` try to define reserve for archive percentage. If
there are heterogeneous disks located one node, do we need config them separate？
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
##########
@@ -62,9 +64,14 @@
private final VolumeChoosingPolicy<FsVolumeImpl> blockChooser;
private final BlockScanner blockScanner;
+ private boolean enableSameDiskTiering;
Review comment:
`enableSameDiskTiering` here vs `enableSameDiskArchival` at
FsVolumeImpl, we should unified variable name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
