[
https://issues.apache.org/jira/browse/ACCUMULO-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271475#comment-13271475
]
Keith Turner commented on ACCUMULO-140:
---------------------------------------
I was looking into this a bit more yesterday. Its shocking how difficult it is
to tell the linux kernel to not cache data.
http://blog.mikemccandless.com/2010/06/lucene-and-fadvisemadvise.html
https://issues.apache.org/jira/browse/CASSANDRA-1470
There are options to fadvise that linux simply ignores. It honors fadvise
DONTNEED, but this only causes data already written to a file to be dropped
from cache. Looking at the HDFS changes it seems that they periodically call
fadvise dontneed as the file is written to. Besides using fadvise DONTNEED,
direct IO seems to be another option.
> Prevent linux from caching write ahead log files
> ------------------------------------------------
>
> Key: ACCUMULO-140
> URL: https://issues.apache.org/jira/browse/ACCUMULO-140
> Project: Accumulo
> Issue Type: Improvement
> Components: logger
> Reporter: Keith Turner
> Assignee: Eric Newton
> Fix For: 1.5.0
>
>
> While at ApacheCon I attended a talk about Cassandra. The developer
> mentioned using posix_fadvise to prevent linux from storing edit/write ahead
> log data in the page cache. This sounds great. Accumulo can do this for its
> write ahead logs.
> Using posix_fadvise was also mentioned for compactions, this is something
> Accumulo can not do because its compactions read and write to hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
