[
https://issues.apache.org/jira/browse/CASSANDRA-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899326#action_12899326
]
Peter Schuller commented on CASSANDRA-1214:
-------------------------------------------
Well, posix_fadvise() is potentially a bit more problematic than mlockall(). It
again takes flags, whose values I suppose may be as practically standardized as
supposedly for mlockall() (though I have not yet checked). In addition it takes
an off_t which, being an abstract type, would have potential for portability
concerns but a quick Googling suggests
(http://markmail.org/message/qvf7hhq2mgmwwmw3) JNA has some particular support
for the off_t data type though I did not find it right now in the API docs
(will have to check more carefully).
The other thing is that posix_fadvise() will need a file descriptor in integer
form. java.io.FileDescriptor is decidedly abstract and does not expose this
information (which is understandable). I am not aware, off hand, of a good way
for us to obtain the relevant underlying file descriptor; anyone? Molesting
FileDescriptor with reflection should technically do the trick with openjdk/sun
derived VM:s (at least based on current openjdk7 FileDescriptor.java), but....
yuck.
If it weren't for the build problems implied by JNI I would strongly prefer it.
Under the circumstances I'm not sure. One observation is that given the kind of
ifs and buts one seems to have to resort to anyway, writing some simple
semi-portable build rules in Ant, specifically targetting certain platforms and
compilers, does not feel so bad. Even if one hard-codes each common platform to
avoid solving the native build problem generally, that does not feel worse to
me in practice than making the assumptions necessary with JNA and stuff like
using reflection to access private fields...
As long as the native building remain optional and does not hinder anyone
getting Cassandra to work with just Java, and as long as it is relatively easy
for someone on an unsupported/problematic platform to simply build the JNI
libraries themselves (doable by e.g. a simple Makefile with clear instructions
for pointing to JDK headers etc), JNI feels pretty reasonable to me.
Thoughts? Am I painting a bleaker picture than reality with respect to using
JNA?
> Force linux to not swap the JVM
> -------------------------------
>
> Key: CASSANDRA-1214
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1214
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: James Golick
> Fix For: 0.6.5
>
> Attachments: mlockall-jna.patch.txt, Read Throughput with mmap.jpg,
> trunk-1214.txt
>
>
> The way mmap()'d IO is handled in cassandra is dangerous. It allocates
> potentially massive buffers without any care for bounding the total size of
> the program's buffers. As the node's dataset grows, this *will* lead to
> swapping and instability.
> This is a dangerous and wrong default for a couple of reasons.
> 1) People are likely to test cassandra with the default settings. This issue
> is insidious because it only appears when you have sufficient data in a
> certain node, there is absolutely no way to control it, and it doesn't at all
> respect the memory limits that you give to the JVM.
> That can all be ascertained by reading the code, and people should certainly
> do their homework, but nevertheless, cassandra should ship with sane defaults
> that don't break down when you cross some magic unknown threshold.
> 2) It's deceptive. Unless you are extremely careful with capacity planning,
> you will get bit by this. Most people won't really be able to use this in
> production, so why get them excited about performance that they can't
> actually have?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
