Hi all, I am trying to benchmark ignite with ycsb tool as follows:
Ignite Cluster: ---------------- 3 ignite-nodes each running in 1 VM (3 seperate VMs in cluster) VM details: -------------- CPU: 4 V-CPU (Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz) RAM: 16 Gb SSD: 40 GB Network: 1 Gb/s Apache Ignite Parameters ------------------------------ Version: Apache Ignite 2.6 Persistence: Disabled (pure in-memory key-value data store). Caches mode: Partitioned / ATOMIC. Backups: 1 Write synchronization mode: PRIMARY_SYNC. Max_Size/Initial_Size: 12 GB / 12 GB YCSB Workload Configuration: --------------------------------- recordcount=6000000 ----------------> 6 million records each of size 1kb (total size:; 5.7 GB) operationcount=6000000 workload=com.yahoo.ycsb.workloads.CoreWorkload readallfields=true insertproportion=1 ------------------> 100% only INSERT operation readproportion=0 updateproportion=0 scanproportion=0 requestdistribution=zipfian threadcount=100 -------------------> 100 threads target=75000 -------------------> 75k Operations Per Second (OPS) Benchmark Results: ---------------------- [OVERALL], RunTime(ms), 64877 [OVERALL], Throughput(ops/sec), 46241.34901428858 ---------------> 46k OPS******* [TOTAL_GCS_PS_Scavenge], Count, 87 [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 162 [TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.2497032846771583 [TOTAL_GCS_PS_MarkSweep], Count, 2 [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 66 [TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.10173096783143488 [TOTAL_GCs], Count, 89 [TOTAL_GC_TIME], Time(ms), 228 [TOTAL_GC_TIME_%], Time(%), 0.35143425250859317 [CLEANUP], Operations, 100 [CLEANUP], AverageLatency(us), 359.8 [CLEANUP], MinLatency(us), 0 [CLEANUP], MaxLatency(us), 35775 [CLEANUP], 95thPercentileLatency(us), 6 [CLEANUP], 99thPercentileLatency(us), 11 [INSERT], Operations, 3000000 [INSERT], AverageLatency(us), 2089.05358 -----------------> 2 MilliSeconds Latency [INSERT], MinLatency(us), 328 [INSERT], MaxLatency(us), 64927 [INSERT], 95thPercentileLatency(us), 3969 [INSERT], 99thPercentileLatency(us), 6835 As you can see from the benchmark results that: Expected OPS: 75k OPS Actual OPS result: only 46k OPS During this run, i have captured the system resource details as shown below: CPU (Ignite / YCSB ) : 40% / 35% Mem(Ignite / YCSB ) : 6.5 GB / 1.1 GB Network(Ignite/YCSB in Mbps): 700rx-350tx / 30rx-500tx My question is: 1. There is no system resource limit hit in both ignite and ycsb, So why is that the actual result is not going above 46k OPS ? what is stopping here? I even tried increasing threads in ycsb but not getting any improvement even with that. 2. On ignite machine, in the top command i can see below ignite app showing > 100%: ==CPU% MEM% VIRT RES PID USER TIME+ THR NI S R/s W/s Command ==141.8 37.1 16.6G 5.76G 19099 root 10:39 86 0 S 0 0 /usr/bin/java -Xms1g -Xmx1g But the overall cpu display(not individual process cpu as shown above) at the top command was never crossed 50% (Load factor is 2 during this time). Since its not crossed 50% i assume CPU still has juice to take more ignite ops. Is this correct? 3. Is the latency displayed in ycsb output above(2 millisecond) be still optimized? Again network/cpu is not exhausted. So it should be possible? Is there a ignite-latency measurement tool (similar to redis?) available? 4. Any other parameters in the system is blocking to this limit? like tcp open connections/tcp buffer size etc) ? Any help highly appreciated. Please clarify. Thanks summa -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
