[
https://issues.apache.org/jira/browse/YARN-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728967#comment-14728967
]
Tsuyoshi Ozawa commented on YARN-3874:
--------------------------------------
[~djp], thanks for your reply.
{quote}
Do you have any solid scenarios and cases that need this feature to work with
optimization?
{quote}
Yes, HDFS backing storage is useful for following scenarios(Quoting description
of YARN-3841).
1. For Hadoop clusters which don't run HBase. LevelDB implementation in this
case is not enough since its durability is sometimes not enough for our
customers.
2. For fallback from HBase when HBase cluster is temporary unavailable. Quoting
ATS design document of YARN-2928:
In the case the HBase
{quote}
storage is not available, the plugin should buffer the writes temporarily (e.g.
HDFS), and flush
them once the storage comes back online. Reading and writing to hdfs as the the
backup storage
could potentially use the HDFS writer plugin unless the complexity of
generalizing the HDFS
writer plugin for this purpose exceeds the benefits of reusing it here.
{quote}
If current priority of this feature is low, I'd like to lead and support its
development :-)
> Optimize and synchronize FS Reader and Writer Implementations
> -------------------------------------------------------------
>
> Key: YARN-3874
> URL: https://issues.apache.org/jira/browse/YARN-3874
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-3874-YARN-2928.01.patch,
> YARN-3874-YARN-2928.02.patch, YARN-3874-YARN-2928.03.patch
>
>
> Combine FS Reader and Writer Implementations and make them consistent with
> each other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
