[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11537: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks [~vagarychen] for the contribution. I've commit the patch to the feature branch. > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDFS-7240.004.patch, > HDFS-11537-HDFS-7240.005.patch, HDFS-11537-HDSF-7240.001.patch, > HDFS-11537-HDSF-7240.002.patch, HDFS-11537-HDSF-7240.003.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11537: -- Attachment: HDFS-11537-HDFS-7240.005.patch post v005 patch to resolve the findbug, checkstyle and javadoc warnnings. Some of the checkstyle warnings were still about the hidden fields of setting method, probably because the setting methods are for builder and thus not recognized as settings. These warnnings are ignored. > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDFS-7240.004.patch, > HDFS-11537-HDFS-7240.005.patch, HDFS-11537-HDSF-7240.001.patch, > HDFS-11537-HDSF-7240.002.patch, HDFS-11537-HDSF-7240.003.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11537: -- Attachment: HDFS-11537-HDFS-7240.004.patch Jenkins tried to apply to trunk, as I made a silly typo in patch name...resubmit v003 patch as v004 patch with the name fixed. > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDFS-7240.004.patch, > HDFS-11537-HDSF-7240.001.patch, HDFS-11537-HDSF-7240.002.patch, > HDFS-11537-HDSF-7240.003.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11537: -- Attachment: HDFS-11537-HDSF-7240.003.patch There is one overlapping config key as in HDFS-11361. Update v003 to remove it. Also removed a test that is covered. The rest are the same as in v002 patch. > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDSF-7240.001.patch, > HDFS-11537-HDSF-7240.002.patch, HDFS-11537-HDSF-7240.003.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11537: -- Status: Patch Available (was: Open) > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDSF-7240.001.patch, > HDFS-11537-HDSF-7240.002.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[
https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang updated HDFS-11537:
--
Attachment: HDFS-11537-HDSF-7240.002.patch
Thanks [~xyao] for the review and comments! All except the following are
addressed in v002 patch.
bq. CBlockConfigKeys.java
I realized that a number of keys in {{CBlockConfigKeys.java}} are currently
not, and will probably never be used. So I've removed all the currently unused
keys in this file, and will add keys later on when needed.
bq. Line 612: can we track the TODO with a Apache JIRA?
filed HDFS-11539
bq. Line 97: should we ensure shutdown of xceiverClientManager?
I think this should be fine, as there is nothing in particular to clean up in
this class, it does not have a shutdown method or anything similar anyway. And
existing unit tests already uses this class like this.
> Block Storage : add cache layer
> ---
>
> Key: HDFS-11537
> URL: https://issues.apache.org/jira/browse/HDFS-11537
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
>Reporter: Chen Liang
>Assignee: Chen Liang
> Attachments: HDFS-11537-HDSF-7240.001.patch,
> HDFS-11537-HDSF-7240.002.patch
>
>
> This JIRA adds the cache layer. Specifically, this JIRA implements the cache
> interface in HDFS-11361 and adds the code that actually talks to containers.
> The upper layer can simply view the storage as a cache with simple put and
> get interface, while in the backend the get and put are actually talking to
> containers. This is a critical part to the cblock performance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11537: -- Description: This JIRA adds the cache layer. Specifically, this JIRA implements the cache interface in HDFS-11361 and adds the code that actually talks to containers. The upper layer can simply view the storage as a cache with simple put and get interface, while in the backend the get and put are actually talking to containers. This is a critical part to the cblock performance. (was: This JIRA adds the cache layer. Specifically, this JIRA implements the cache interface in HDFS-11361 and adds the code that actually talks to containers. The upper layer can simply view the storage as a cache with simple put and get interface, while in the backend the get and put are actually talking to containers. This is a critical part to the cblock performance. [~anu] is actually the author who contributed to most of this part.) > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDSF-7240.001.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11537) Block Storage : add cache layer
[ https://issues.apache.org/jira/browse/HDFS-11537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11537: -- Attachment: HDFS-11537-HDSF-7240.001.patch > Block Storage : add cache layer > --- > > Key: HDFS-11537 > URL: https://issues.apache.org/jira/browse/HDFS-11537 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chen Liang >Assignee: Chen Liang > Attachments: HDFS-11537-HDSF-7240.001.patch > > > This JIRA adds the cache layer. Specifically, this JIRA implements the cache > interface in HDFS-11361 and adds the code that actually talks to containers. > The upper layer can simply view the storage as a cache with simple put and > get interface, while in the backend the get and put are actually talking to > containers. This is a critical part to the cblock performance. [~anu] is > actually the author who contributed to most of this part. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
