Hello all,
Recently we added one of the table fields from as Map<text, text> in Cassandra
2.1.11. Currently we read every field from Map and overwrite map values. Map is
of size 3. We saw that writes are 30-40% slower while reads are 70-80% slower.
Please find below some metrics that can help.
My question is, Are there any known issues in Cassandra map performance? As I
understand it each of the CQL3 Map entry, maps to a column in cassandra, with
that assumption we are just creating 3 columns right? Any insight on this issue
would be helpful.
Datastax Java Driver 2.1.6.
Machine: Amazon C3 2x large
CPU – pretty much same as before (around 30%)
Memory – max around 4.8 GB
CFSTATS:
Keyspace: Keyspace
Read Count: 28359044
Read Latency: 2.847392469259542 ms.
Write Count: 1152765
Write Latency: 0.14778018590085576 ms.
Pending Flushes: 0
Table: table1
SSTable count: 1
SSTables in each level: [1, 0, 0, 0, 0, 0, 0, 0, 0]
Space used (live): 4119699
Space used (total): 4119699
Space used by snapshots (total): 90323640
Off heap memory used (total): 2278
SSTable Compression Ratio: 0.23172161124142604
Number of keys (estimate): 14
Memtable cell count: 6437
Memtable data size: 872912
Memtable off heap memory used: 0
Memtable switch count: 7626
Local read count: 27754634
Local read latency: 1.921 ms
Local write count: 1113668
Local write latency: 0.142 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 96
Bloom filter off heap memory used: 88
Index summary off heap memory used: 46
Compression metadata off heap memory used: 2144
Compacted partition minimum bytes: 315853
Compacted partition maximum bytes: 4055269
Compacted partition mean bytes: 2444011
Average live cells per slice (last five minutes): 17.536775249005437
Maximum live cells per slice (last five minutes): 1225.0
Average tombstones per slice (last five minutes): 34.99979575985972
Maximum tombstones per slice (last five minutes): 3430.0
Table: table2
SSTable count: 1
SSTables in each level: [1, 0, 0, 0, 0, 0, 0, 0, 0]
Space used (live): 869900
Space used (total): 869900
Space used by snapshots (total): 17279824
Off heap memory used (total): 387
SSTable Compression Ratio: 0.3999013540551859
Number of keys (estimate): 2
Memtable cell count: 1958
Memtable data size: 8
Memtable off heap memory used: 0
Memtable switch count: 7484
Local read count: 604412
Local read latency: 45.421 ms
Local write count: 39097
Local write latency: 0.337 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 96
Bloom filter off heap memory used: 88
Index summary off heap memory used: 35
Compression metadata off heap memory used: 264
Compacted partition minimum bytes: 1955667
Compacted partition maximum bytes: 2346799
Compacted partition mean bytes: 2346799
Average live cells per slice (last five minutes): 1963.0632242863855
Maximum live cells per slice (last five minutes): 5001.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
NETSTATS:
Mode: NORMAL
Not sending any streams.
Read Repair Statistics:
Attempted: 2853996
Mismatch (Blocking): 67386
Mismatch (Background): 9233
Pool Name Active Pending Completed
Commands n/a 0 33953165
Responses n/a 0 370301
IOSTAT
avg-cpu: %user %nice %system %iowait %steal %idle
15.20 0.83 0.56 0.10 0.04 83.27
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
xvda 2.79 0.47 69.86 553719 82619304
xvdb 14.49 3.39 775.56 4009600 917227536
xvdc 15.13 2.98 819.93 3522250 969708944
dm-0 49.67 6.36 1595.49 7525858 1886936320
TPSTAT:
Pool Name Active Pending Completed Blocked All
time blocked
MutationStage 0 0 1199683 0
0
ReadStage 0 0 28449207 0
0
RequestResponseStage 0 0 33983356 0
0
ReadRepairStage 0 0 2865749 0
0
CounterMutationStage 0 0 0 0
0
MiscStage 0 0 0 0
0
HintedHandoff 0 0 2 0
0
GossipStage 0 0 270364 0
0
CacheCleanupExecutor 0 0 0 0
0
InternalResponseStage 0 0 0 0
0
CommitLogArchiver 0 0 0 0
0
CompactionExecutor 0 0 111234 0
0
ValidationExecutor 0 0 0 0
0
MigrationStage 0 0 7 0
0
AntiEntropyStage 0 0 0 0
0
PendingRangeCalculator 0 0 3 0
0
Sampler 0 0 0 0
0
MemtableFlushWriter 0 0 45884 0
0
MemtablePostFlush 0 0 47461 0
0
MemtableReclaimMemory 0 0 45884 0
0
Message type Dropped
READ 0
RANGE_SLICE 0
_TRACE 0
MUTATION 0
COUNTER_MUTATION 0
BINARY 0
REQUEST_RESPONSE 0
PAGED_RANGE 0
READ_REPAIR 0
Thanks,
Pratik
