[jira] [Commented] (HDFS-8392) DataNode support for multiple datasets

[Arpit Agarwal (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HDFS-8392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568109#comment-14568109
 ] 

Arpit Agarwal commented on HDFS-8392:
-------------------------------------

Thanks for the review Jitendra. I fixed the first point and committed to 
branch-7240 since the delta is trivial.

bq. Ideally we should change the tests to use the datasetsMap instead of legacy 
field 'data'. That can be taken in a separate jira.
I'll file a Jira to make sure we fix it in trunk post merge. Deferring to avoid 
frequent merging pain.

Delta for reference:
{code}
--- 
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
+++ 
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
@@ -2598,7 +2598,7 @@ public void scheduleAllBlockReport(long delay) {
    *
    * @return the fsdataset that stores the blocks
    */
-  @VisibleForTesting
+  @Deprecated
   public FsDatasetSpi<?> getFSDataset() {
     Preconditions.checkState(datasets.size() <= 1,
         "Did not expect more than one Dataset here.");
@@ -2621,7 +2621,7 @@ public BlockScanner getBlockScanner() {
    *
    * @return
    */
-  @VisibleForTesting
+  @Deprecated
   DirectoryScanner getDirectoryScanner() {
     return directoryScannersMap.get(getFSDataset());
   }
{code}


> DataNode support for multiple datasets
> --------------------------------------
>
>                 Key: HDFS-8392
>                 URL: https://issues.apache.org/jira/browse/HDFS-8392
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-8392-HDFS-7240.01.patch, 
> HDFS-8392-HDFS-7240.02.patch, HDFS-8392-HDFS-7240.03.patch
>
>
> For HDFS-7240 we would like to share available DataNode storage across HDFS 
> blocks and Ozone objects.
> The DataNode already supports sharing available storage across multiple block 
> pool IDs for the federation feature. However all federated block pools use 
> the same dataset implementation i.e. {{FsDatasetImpl}}.
> We can extend the DataNode to support multiple dataset implementations so the 
> same storage space can be shared across one or more HDFS block pools and one 
> or more Ozone block pools.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)