[
https://issues.apache.org/jira/browse/CASSANDRA-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243924#comment-13243924
]
Vijay commented on CASSANDRA-3997:
----------------------------------
Hi Pavel,
Looks like howard malloc can work seamlessly with LD_PRELOAD but JEmalloc
doesn't work well with LD_PRELOAD.
{quote}
Jason says: The crash in free() is the only one that tells me anything at all,
and my only guesses are 1) mixed allocator usage or 2) application error, e.g.
double free(). I really don't know anything about how the JVM is structured
internally , how it interacts with malloc, how it uses/abuses dlopen(), etc.,
so I'm not going to be of much help without a lot more background information.
{quote}
In other hand the attached patch avoids the crashes.
Comparision of the mallocs:
{code}
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java -cp
/apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.MallocAllocator 50000 2000000
total used free shared buffers cached
Mem: 71688220 10569792 61118428 0 146360 1864972
-/+ buffers/cache: 8558460 63129760
Swap: 0 0 0
**** Starting Test! ****
Total bytes read: 101423216640
Time taken: 28587
total used free shared buffers cached
Mem: 71688220 15950408 55737812 0 146360 1865184
-/+ buffers/cache: 13938864 57749356
Swap: 0 0 0
**** ending Test!****
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ export
LD_LIBRARY_PATH=/home/vijay_tcasstest/howard/
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java
-Djava.library.path=/home/vijay_tcasstest/howard/ -cp
/apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.HowardMallocAllocator 50000 2000000
total used free shared buffers cached
Mem: 71688220 10573476 61114744 0 146320 1864972
-/+ buffers/cache: 8562184 63126036
Swap: 0 0 0
**** Starting Test! ****
Total bytes read: 101366196224
Time taken: 33959
total used free shared buffers cached
Mem: 71688220 16292664 55395556 0 146320 1865184
-/+ buffers/cache: 14281160 57407060
Swap: 0 0 0
**** ending Test!****
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ export
LD_LIBRARY_PATH=/home/vijay_tcasstest/jemalloc/lib/
[vijay_tcasstest@vijay_tcass-i-08e1f16c java]$ java
-Djava.library.path=/home/vijay_tcasstest/jemalloc/lib/ -cp
/apps/nfcassandra_server/lib/concurrentlinkedhashmap-lru-1.2.jar:/apps/nfcassandra_server/lib/jna-3.3.0.jar:/apps/nfcassandra_server/lib/apache-cassandra-1.1.0-beta2-SNAPSHOT.jar:.
com.sun.jna.JEMallocAllocator 50000 2000000
total used free shared buffers cached
Mem: 71688220 10572896 61115324 0 146332 1864972
-/+ buffers/cache: 8561592 63126628
Swap: 0 0 0
**** Starting Test! ****
Total bytes read: 101360272384
Time taken: 29310
total used free shared buffers cached
Mem: 71688220 13243604 58444616 0 146340 1865184
-/+ buffers/cache: 11232080 60456140
Swap: 0 0 0
**** ending Test!****
{code}
> Make SerializingCache Memory Pluggable
> --------------------------------------
>
> Key: CASSANDRA-3997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3997
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Labels: cache
> Fix For: 1.2
>
> Attachments: 0001-CASSANDRA-3997.patch, jna.zip
>
>
> Serializing cache uses native malloc and free by making FM pluggable, users
> will have a choice of gcc malloc, TCMalloc or JEMalloc as needed.
> Initial tests shows less fragmentation in JEMalloc but the only issue with it
> is that (both TCMalloc and JEMalloc) are kind of single threaded (at-least
> they crash in my test otherwise).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
