Jason Lowe updated YARN-2765:
    Attachment: YARN-2765v3.patch

Thanks for the review, Jian!

bq. Patch needs updated on top of YARN-2404.



I did not make this change.  Unlike the filesystem state store we are not using 
the key to encode the data but rather just as a static key that is atomically 
updated with new values.  Therefore it's not appropriate to use a key that ends 
in a prefix, IMHO.  This store is closer to what the ZK store is doing which 
also uses a separate key string for the sequence number.

bq. this check is always true and so unnecessary, as attemptState is earlier 
initialized with attemptId

Left this check since after the refactoring for YARN-2404 the check will verify 
the attempt ID parsed from the key matches the attempt ID parsed from the value.

    Some naming suggestions:
    loadRMAppAttemptState-> createAttemptState
    loadRMAppState(String appIdStr, byte[] data) -> createApplicationState
    String itemName = key.substring(keyPrefix.length()); rename itemName to 
attemptId, and keyPrefix to attemptNodePrefix

bq. Question: does the levelDB client API handle retry ?
No, there are no retries in leveldb itself AFAIK.  The retries are handled by 
the OS/driver/device layers, as we are hitting the OS filesystem rather than 
some network layer like HDFS or ZK does.  Retries would be highly unlikely to 
be successful at the leveldb or app level.

> Add leveldb-based implementation for RMStateStore
> -------------------------------------------------
>                 Key: YARN-2765
>                 URL: https://issues.apache.org/jira/browse/YARN-2765
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-2765.patch, YARN-2765v2.patch, YARN-2765v3.patch
> It would be nice to have a leveldb option to the resourcemanager recovery 
> store. Leveldb would provide some benefits over the existing filesystem store 
> such as better support for atomic operations, fewer I/O ops per state update, 
> and far fewer total files on the filesystem.

This message was sent by Atlassian JIRA

Reply via email to