Re: Help on MMap of SSTables
This issue has to be looked from a micro and macro level. On the microlevel the best way is workload specific. On the macro level this mostly boils down to data and memory size. Companions are going to churn cache, this is unavoidable. Imho solid state makes the micro optimization meanless in the big picture. Not that we should not consider tweaking flags but just saying it is hard to believe anything like that is a game change. On Monday, December 10, 2012, Rob Coli rc...@palominodb.com wrote: On Thu, Dec 6, 2012 at 7:36 PM, aaron morton aa...@thelastpickle.com wrote: So for memory mapped files, compaction can do a madvise SEQUENTIAL instead of current DONTNEED flag after detecting appropriate OS versions. Will this help? AFAIK Compaction does use memory mapped file access. The history : https://issues.apache.org/jira/browse/CASSANDRA-1470 =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Re: Help on MMap of SSTables
On Thu, Dec 6, 2012 at 7:36 PM, aaron morton aa...@thelastpickle.com wrote: So for memory mapped files, compaction can do a madvise SEQUENTIAL instead of current DONTNEED flag after detecting appropriate OS versions. Will this help? AFAIK Compaction does use memory mapped file access. The history : https://issues.apache.org/jira/browse/CASSANDRA-1470 =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb
Re: Help on MMap of SSTables
Will MMapping data files be detrimental for reads, in this case?
No.
In general, when should we opt for MMap data files and what are the factors
that need special attention when enabling the same?
mmapping is the default, so I would say use it until you have a reason not to.
mmapping will map the entire file, but pages of data are read into memory on
demand and purged when space is needed.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 4/12/2012, at 11:59 PM, Ravikumar Govindarajan
ravikumar.govindara...@gmail.com wrote:
Our current SSTable sizes are far greater than RAM. {150 Gigs of data, 32GB
RAM}. Currently we run with mlockall and mmap_index_only options and don't
experience swapping at all.
We use wide rows and size-tiered-compaction, so a given key will definitely
be spread across multiple sstables. Will MMapping data files be detrimental
for reads, in this case?
In general, when should we opt for MMap data files and what are the factors
that need special attention when enabling the same?
--
Ravi
