[
https://issues.apache.org/jira/browse/YARN-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-2765:
-----------------------------
Attachment: YARN-2765v3.patch
Thanks for the review, Jian!
bq. Patch needs updated on top of YARN-2404.
Done.
{quote}
DB_SCHEMA_VERSION_KEY -> VERSION_NODE
EPOCH_KEY -> EPOCH_NODE
{quote}
Done.
bq. RM_DT_SEQUENCE_NUMBER_KEY: use DELEGATION_TOKEN_SEQUENCE_NUMBER_PREFIX for
“RMDTSequentialNumber”
I did not make this change. Unlike the filesystem state store we are not using
the key to encode the data but rather just as a static key that is atomically
updated with new values. Therefore it's not appropriate to use a key that ends
in a prefix, IMHO. This store is closer to what the ZK store is doing which
also uses a separate key string for the sequence number.
bq. this check is always true and so unnecessary, as attemptState is earlier
initialized with attemptId
Left this check since after the refactoring for YARN-2404 the check will verify
the attempt ID parsed from the key matches the attempt ID parsed from the value.
{quote}
Some naming suggestions:
loadRMAppAttemptState-> createAttemptState
loadRMAppState(String appIdStr, byte[] data) -> createApplicationState
String itemName = key.substring(keyPrefix.length()); rename itemName to
attemptId, and keyPrefix to attemptNodePrefix
{quote}
Done.
bq. Question: does the levelDB client API handle retry ?
No, there are no retries in leveldb itself AFAIK. The retries are handled by
the OS/driver/device layers, as we are hitting the OS filesystem rather than
some network layer like HDFS or ZK does. Retries would be highly unlikely to
be successful at the leveldb or app level.
> Add leveldb-based implementation for RMStateStore
> -------------------------------------------------
>
> Key: YARN-2765
> URL: https://issues.apache.org/jira/browse/YARN-2765
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-2765.patch, YARN-2765v2.patch, YARN-2765v3.patch
>
>
> It would be nice to have a leveldb option to the resourcemanager recovery
> store. Leveldb would provide some benefits over the existing filesystem store
> such as better support for atomic operations, fewer I/O ops per state update,
> and far fewer total files on the filesystem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
