[
https://issues.apache.org/jira/browse/ACCUMULO-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978531#comment-13978531
]
Josh Elser commented on ACCUMULO-1015:
--------------------------------------
bq. How much of this would we get with a combination of HDFS caching and HDFS
short circuit reads?
Well, you'll still be incurring the overhead of the IPC to the datanode and
reading the data from there, so I imagine it is still "significant" as opposed
to completely in memory. I think it would likely depend on the level of
implementation of an in-memory table. If we just keep the HDFS blocks in the
tserver memory, that would be fairly "simple" and should still perform better,
but I don't have any idea of actual numbers. That would be a good first step
that someone could evaluate without implementing anything new
in-memory
1. Load some data in a table
2. Warm the caches for that table
3. Test performance
short-circuit reads
1. Turn off caches
2. Ingest data into a table
3. Compact to disk
4. Test performance
> Lock a table to memory
> ----------------------
>
> Key: ACCUMULO-1015
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1015
> Project: Accumulo
> Issue Type: New Feature
> Components: tserver
> Reporter: Josh Elser
> Priority: Trivial
>
> Index block and data block caching get you part of the way, but it would be
> nice to have the ability to lock an entire table to memory.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
