[
https://issues.apache.org/jira/browse/HDFS-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-7159:
--------------------------------
Attachment: HDFS-7159.04.patch
Thank you for the review Jing.
bq. Nit: the instanciation of lpPolicy can be moved into the "if" block.
Fixed.
bq. This can addressed in a separate jira: looks like the changes in
BlockStoragePolicy#chooseStorageTypes depends on the assumption that the
transient storage types are listed before others. Can we have a more general
implementation here?
Good catch, fixed it.
bq. In DFSOutputStream, instead of creating a default policy suite locally, it
may be better to use the RPC call getStoragePolicies so that the policy's
read-only semantic can be hidden on the server side.
Would it be okay if we fixed it separately? We would be adding a new RPC call
for every create/append call. We could do the RPC call in a static context but
that might not be a good idea either since a client could keep holding stale
policy information long after it is update on NN.
The updated patch also fixes TestDFSShell and TestHDFSCLI, and removes two
unnecessary classes - FsTransientVolumeImpl and FsVolumeImplAllocator.
> Use block storage policy to set lazy persist preference
> -------------------------------------------------------
>
> Key: HDFS-7159
> URL: https://issues.apache.org/jira/browse/HDFS-7159
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-7159.01.patch, HDFS-7159.02.patch,
> HDFS-7159.03.patch, HDFS-7159.04.patch
>
>
> Now that HDFS-6584 feature is ready and supports block storage policies on
> both files and directories, we can make use of Storage Policies to store the
> LAZY_PERSIST preference.
> This only affects how the preference is persisted in the FsImage/Edit logs.
> There is no change to the client API or to NN-DN interaction.
> An additional advantage is that can support setting the lazy persist
> preference on a directory and it will take effect for all new files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
