[
https://issues.apache.org/jira/browse/CASSANDRA-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968728#comment-13968728
]
Benedict commented on CASSANDRA-7030:
-------------------------------------
[[email protected]] I think there was something wrong with your benchmark.
Since it uses a different random seed each time it's run, the end state that
you check may vary a great deal. Or possibly reporting total system memory
isn't accurate in some way. As I find completely the opposite effect: ~2.3Gb
resident memory for the process working with unsafe, and ~2.7Gb resident for
JEMalloc.
{noformat}
UNSAFE:
Items: 1M
Elapsed: 4.373s
Allocated: 2046Mb
VM total:172
vsz: 3717
rsz: 2268
Items: 10M
Allocated: 2050Mb
VM total: 186
vsz: 3717
rsz: 2308
Items: 100M
Elapsed: 329.087s
Allocated: 2047Mb
VM total:186
vsz: 3717
rsz: 2308
JEMALLOC:
Items: 1M
Elapsed: 8.259s
Allocated: 2046Mb
VM total:161
vsz: 4128
rsz: 2651
Items: 10M
Allocated: 2050
VM total: 192
vsz: 4132
rsz: 2706
Items: 100M
Elapsed: 791.370s
Allocated: 2047Mb
VM total:192
vsz: 4136
rsz: 2710
{noformat}
{code}
public static void main(String[] args) throws InterruptedException,
IOException
{
String pid =
ManagementFactory.getRuntimeMXBean().getName().split("@")[0];
System.out.println(pid);
final IAllocator allocator = new JEMallocAllocator();
final Random rand = new Random(0);
final AtomicLong total = new AtomicLong();
EvictionListener<UUID, Memory> listener = new EvictionListener<UUID,
Memory>()
{
public void onEviction(UUID k, Memory mem)
{
total.addAndGet(-mem.size());
mem.free(allocator);
}
};
Map<UUID, Memory> map = new ConcurrentLinkedHashMap.Builder<UUID,
Memory>().weigher(Weighers.<Memory> singleton())
.initialCapacity(8 * 65536).maximumWeightedCapacity(4 * 65536)
.listener(listener).build();
long start = System.nanoTime();
byte[] keyBytes = new byte[16];
for (int i = 0 ; i < 1000000 ; i++)
{
int size = rand.nextInt(128);
if (size <= 0)
continue;
rand.nextBytes(keyBytes);
Memory mem = new Memory(allocator, size * 128);
mem.setMemory(0, mem.size(), (byte) 2);
map.put(UUID.nameUUIDFromBytes(keyBytes), mem);
total.addAndGet(size * 128);
}
long end = System.nanoTime();
System.out.println(String.format("Elapsed: %.3fs", (end - start) *
0.000000001d));
System.out.println(String.format("Allocated: %.0fMb", total.get() /
(double) (1 << 20)));
System.out.println(String.format("VM total:%.0f",
Runtime.getRuntime().totalMemory() / (double) (1 << 20)));
memuse("vsz", pid);
memuse("rsz", pid);
Thread.sleep(100);
}
private static void memuse(String type, String pid) throws IOException
{
Process p = new ProcessBuilder().command("ps", "-o", type,
pid).redirectErrorStream(true).start();
BufferedReader reader = new BufferedReader(new
InputStreamReader(p.getInputStream()));
reader.readLine();
System.out.println(String.format("%s: %.0f", type,
Integer.parseInt(reader.readLine()) / 1024d));
}
{code}
> Remove JEMallocAllocator
> ------------------------
>
> Key: CASSANDRA-7030
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7030
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Benedict
> Assignee: Benedict
> Priority: Minor
> Labels: performance
> Fix For: 2.1 beta2
>
> Attachments: 7030.txt
>
>
> JEMalloc, whilst having some nice performance properties by comparison to
> Doug Lea's standard malloc algorithm in principle, is pointless in practice
> because of the JNA cost. In general it is around 30x more expensive to call
> than unsafe.allocate(); malloc does not have a variability of response time
> as extreme as the JNA overhead, so using JEMalloc in Cassandra is never a
> sensible idea. I doubt if custom JNI would make it worthwhile either.
> I propose removing it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
