[
https://issues.apache.org/jira/browse/YARN-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-2765:
-----------------------------
Attachment: YARN-2765v2.patch
Thanks for the review, Tsuyoshi!
bq. How about adding helper methods like getKeyPrefix/getNodePath for getting
key prefix and node path?
Sure, added some helper methods to compute leveldb keys for various things.
bq. I found that the patch includes lots hard-coded "/". I think it's better to
have private field SEPARATOR = "/".
IMHO this makes the code less readable, similar to a code style like {{final
int ONE = 1}}. But I don't care too strongly about it and changed all
occurrences to SEPARATOR.
For Zhijie's comments:
bq. One drawback I can think of is that while LeveldbRMStateStore is
lightweight for single RM restarting, multiple RMs of HA are not able to share
this single-host database.
This should work if the leveldb database is on a network store like a filer.
Leveldb uses locks to prevent multiple processes from trying to access the
database simultaneously, so there's a little bit of help for the fencing
scenarios. However the fencing script actions would have to do some extra work
to force a poorly-behaving resourcemanager to let go of the locks so a standby
RM can open the store and become active.
bq. Did you have a chance to think of an enhanced k/v db: rocksdb?
I briefly considered using rocksdb for this but decided against it for a couple
of reasons:
* leveldb is already used by the timeline server and nodemanager, and I would
rather avoid adding yet another new dependency for this
* leveldb supports win32/win64, but it doesn't appear that the standard
rocksdbjni distribution has support for Windows.
> Add leveldb-based implementation for RMStateStore
> -------------------------------------------------
>
> Key: YARN-2765
> URL: https://issues.apache.org/jira/browse/YARN-2765
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: YARN-2765.patch, YARN-2765v2.patch
>
>
> It would be nice to have a leveldb option to the resourcemanager recovery
> store. Leveldb would provide some benefits over the existing filesystem store
> such as better support for atomic operations, fewer I/O ops per state update,
> and far fewer total files on the filesystem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
