[
https://issues.apache.org/jira/browse/HDFS-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-8210:
--------------------------------
Attachment: HDFS-8210-HDFS-7240.006.patch
I'm attaching patch v006. This refreshes the patch after the rebase of the
feature branch and also updates a few things based on ongoing testing:
# {{StorageContainerManager}} now provides a stub implementation of
{{getStorageContainerLocations}}. The stub implementation simply reports all
registered DataNodes as viable locations. This will evolve of course, but the
stub is sufficient to enable some end-to-end testing of basic service
interactions.
# The partitioning code from previous revisions of this patch is not needed
immediately and thus not included. As per above, partitioning will become
important at a later phase of development when {{getStorageContainerLocations}}
is no longer a stub.
# I did a lot of internal refactoring for code reuse within
{{StorageContainerManager}}. In particular, the constructor is much shorter
now.
# I created {{MiniOzoneCluster}}, which starts a StorageContainerManager and
multiple DataNodes, suitable for writing tests. Unlike {{MiniDFSCluster}}, it
does not start a {{NameNode}}. I needed to make a small change in
{{MiniDFSCluster}} to accommodate this.
# {{TestStorageContainerManager}} is a new test suite that covers basic service
interactions with DataNodes registering to the StorageContainerManager and
clients then calling StorageContainerManager to find container locations.
In addition to the new test suite, I tested manually by building a distro and
running a StorageContainerManager, DataNode, and client.
[~anu] or [~jnp], would you like to review?
> Ozone: Implement storage container manager
> -------------------------------------------
>
> Key: HDFS-8210
> URL: https://issues.apache.org/jira/browse/HDFS-8210
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Jitendra Nath Pandey
> Assignee: Chris Nauroth
> Attachments: HDFS-8210-HDFS-7240.006.patch,
> HDFS-8210-HDFS-7240.1.patch, HDFS-8210-HDFS-7240.2.patch,
> HDFS-8210-HDFS-7240.3.patch, HDFS-8210-HDFS-7240.4.patch,
> HDFS-8210-HDFS-7240.5.patch
>
>
> The storage container manager collects datanode heartbeats, manages
> replication and exposes API to lookup containers. This jira implements
> storage container manager by re-using the block manager implementation in
> namenode. This jira provides initial implementation that works with
> datanodes. The additional protocols will be added in subsequent jiras.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
