[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16215984#comment-16215984
]
Dan Kinder commented on CASSANDRA-13973:
----------------------------------------
[~jjirsa]
This happens on about 12 out of 16 nodes, much greater than {{RF}}. (No real
need to anonymize the names.)
{noformat}
$ nodetool tablehistograms walker links
walker/links histograms
Percentile SSTables Write Latency Read Latency Partition Size
Cell Count
(micros) (micros) (bytes)
50% 4.00 51.01 1155149.91 258
6
75% 5.00 73.46 3449259.15 1597
42
95% 5.00 152.32 7152383.77 73457
2299
98% 5.00 182.79 8582860.53 315852
9887
99% 5.00 219.34 53142810.15 943127
24601
Min 0.00 8.24 88.15 43
0
Max 5.00 1155149.91 53142810.15 394855527800
962624926
$ nodetool tablestats walker.links
Keyspace: walker
Read Count: 72932
Read Latency: 834.917616670323 ms.
Write Count: 15501219
Write Latency: 0.4392410533003888 ms.
Pending Flushes: 0
Table: links
SSTable count: 14269
SSTables in each level: [0, 9, 68, 349, 2843, 11000, 0, 0, 0]
Space used (live): 5310634842695
Space used (total): 5310634842695
Space used by snapshots (total): 12868175419329
Off heap memory used (total): 16339462
SSTable Compression Ratio: 0.0
Number of partitions (estimate): 20112302
Memtable cell count: 2216835
Memtable data size: 347672840
Memtable off heap memory used: 0
Memtable switch count: 4
Local read count: 80395
Local read latency: 747.603 ms
Local write count: 16891401
Local write latency: 0.118 ms
Pending flushes: 0
Bloom filter false positives: 178
Bloom filter false ratio: 0.00081
Bloom filter space used: 14912616
Bloom filter off heap memory used: 14798464
Index summary off heap memory used: 1540998
Compression metadata off heap memory used: 0
Compacted partition minimum bytes: 43
Compacted partition maximum bytes: 394855527800
Compacted partition mean bytes: 249068
Average live cells per slice (last five minutes):
11010.451219512195
Maximum live cells per slice (last five minutes): 11864
Average tombstones per slice (last five minutes): 1.0
Maximum tombstones per slice (last five minutes): 1
{noformat}
> IllegalArgumentException in upgradesstables compaction
> ------------------------------------------------------
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Dan Kinder
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
