[
https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213639#comment-16213639
]
Virajith Jalaparti commented on HDFS-12665:
-------------------------------------------
Thanks for posting this [~ehiggs]. A few comments/questions:
# Can you please add javadocs for all the new classes added?
# Is there a reason to refactor {{FileRegion}} and introduce
{{ProvidedStorageLocation}}? Also, I think the name {{ProvidedStorageLocation}}
is confusing given there is also a {{StorageLocation}}, which is something very
different. May be rename to {{ProvidedLocation}}.
# The new {{AliasMap}} class has a confusing name. It is supposed to be an
implementation of the {{AliasMapProtocol}} but the name is a prefix of the
latter.
# Renaming {{LevelDBAliasMapClient}} to something along the lines
{{InMemoryLevelDBAliasMap}} will make it a more descriptive name for the class.
In general, adding a similar prefix to {{AliasMapProtocol}},
{{LevelDBAliasMapServer}} will improve the readability of the code.
# Can we move {{LevelDBAliasMapClient}} to the
{{org.apache.hadoop.hdfs.server.common.BlockAliasMapImpl}} package. On a
related note, we should rename this to
{{org.apache.hadoop.hdfs.server.common.blockaliasmap.impl}} in HDFS-11902. I
can fix this when I post the next version of the patch for HDFS-11902.
# {{ITAliasMap}} only contains unit tests. I believe the convention is to start
the name of the class with Test.
# Why was the block pool id removed from {{FileRegion}}? It was used as a check
in the DN so that only blocks belonging to the correct block pool id were
reported to the NN.
# Why rename {{getVolumeMap}} to {{fetchVolumeMap}} in
{{ProvidedBlockPoolSlice}}?
# In {{startAliasMapServerIfNecessary}}, I think the aliasmap should be started
only if provided is configured. i.e., check if
{{DFSConfigKeys.DFS_NAMENODE_PROVIDED_ENABLED}} is set to true.
# Some of the changes have lead to lines crossing the 80 character limit. Can
you please fix them?
> [AliasMap] Create a version of the AliasMap that runs in memory in the
> Namenode (leveldb)
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-12665
> URL: https://issues.apache.org/jira/browse/HDFS-12665
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ewan Higgs
> Assignee: Ewan Higgs
> Attachments: HDFS-12665-HDFS-9806.001.patch,
> HDFS-12665-HDFS-9806.002.patch
>
>
> The design of Provided Storage requires the use of an AliasMap to manage the
> mapping between blocks of files on the local HDFS and ranges of files on a
> remote storage system. To reduce load from the Namenode, this can be done
> using a pluggable external service (e.g. AzureTable, Cassandra, Ratis).
> However, to aide adoption and ease of deployment, we propose an in memory
> version.
> This AliasMap will be a wrapper around LevelDB (already a dependency from the
> Timeline Service) and use protobuf for the key (blockpool, blockid, and
> genstamp) and the value (url, offset, length, nonce). The in memory service
> will also have a configurable port on which it will listen for updates from
> Storage Policy Satisfier (SPS) Coordinating Datanodes (C-DN).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
