[
https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974472#comment-13974472
]
Arpit Agarwal commented on HDFS-5851:
-------------------------------------
Hi Andrew,
bq. There are also plans to move towards sub-block caching. Whole-block caching
is wasteful for columnar formats like ORC and Parquet. With sub-block caching,
automatic cache replacement looks a lot more attractive (another planned
feature). These are both things we can support with HDFS-4949's infrastructure.
I'm not sure about with HSM.
Does CCM support block-level caching or is it just _cache all blocks in a file_?
bq. Anyway, either pulling this towards HDFS-4949 or vice versa, we should
figure out these details before moving ahead. I'll echo Colin's desire for a
meeting to discuss this. We're willing to host at our Palo Alto office.
Thanks for the offer! We are discussing use cases and proposed API at high
level. I am not sure there is much overlap between CCM and our work and I
expect them to solve different use cases. However I am also open to discussing
how and whether we can align them more closely once we have shared our initial
proposal.
> Support memory as a storage medium
> ----------------------------------
>
> Key: HDFS-5851
> URL: https://issues.apache.org/jira/browse/HDFS-5851
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
>
> Memory can be used as a storage medium for smaller/transient files for fast
> write throughput.
> More information/design will be added later.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
