[
https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewan Higgs updated HDFS-12665:
------------------------------
Attachment: HDFS-12665-HDFS-9806.006.patch
[~virajith], thanks for updating the patch.
I'm attaching a patch that addresses some of your comments:
{quote}InMemoryAliasMap uses
LoggerFactory.getLogger(InMemoryLevelDBAliasMapServer.class). Was this
intentional?{quote}
Fixed.
{quote}Should we rename InMemoryAliasMap to InMemoryLevelDBAliasMap, similar to
the other classes?{quote}
I did not address this. I personally think the name is fine, but if you feel
strongly we can change it.
{quote}Why add a new iterator() to BlockAliasMap and not use the
BlockAliasMap.Reader?{quote}
+1 and fixed.
{quote}TestNameNodeProvidedImplementation#testInMemoryAliasMap fails. {quote}
This would involve changing the protocol and a lot of shotgun edits that I will
just have to put back. No thanks. Instead I've marked the test as {{
@Test(expected=BlockMissingException.class) }}
This will be addressed in HDFS-12713.
{quote}InMemoryLevelDBAliasMapClient#LevelDbWriter doesn't work without a
running server. {quote} This is correct but you can use
{{LevelDBFileRegionFormat}} in HDFS-12591 (which certainly needs a rebase onto
the merged HDFS-11902 work).
> [AliasMap] Create a version of the AliasMap that runs in memory in the
> Namenode (leveldb)
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-12665
> URL: https://issues.apache.org/jira/browse/HDFS-12665
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ewan Higgs
> Assignee: Ewan Higgs
> Attachments: HDFS-12665-HDFS-9806.001.patch,
> HDFS-12665-HDFS-9806.002.patch, HDFS-12665-HDFS-9806.003.patch,
> HDFS-12665-HDFS-9806.004.patch, HDFS-12665-HDFS-9806.005.patch,
> HDFS-12665-HDFS-9806.006.patch
>
>
> The design of Provided Storage requires the use of an AliasMap to manage the
> mapping between blocks of files on the local HDFS and ranges of files on a
> remote storage system. To reduce load from the Namenode, this can be done
> using a pluggable external service (e.g. AzureTable, Cassandra, Ratis).
> However, to aide adoption and ease of deployment, we propose an in memory
> version.
> This AliasMap will be a wrapper around LevelDB (already a dependency from the
> Timeline Service) and use protobuf for the key (blockpool, blockid, and
> genstamp) and the value (url, offset, length, nonce). The in memory service
> will also have a configurable port on which it will listen for updates from
> Storage Policy Satisfier (SPS) Coordinating Datanodes (C-DN).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
