[
https://issues.apache.org/jira/browse/CASSANDRA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235374#comment-16235374
]
Sylvain Lebresne commented on CASSANDRA-13973:
----------------------------------------------
bq. do you two feel this is safe?
I can't think of something this would hard break off the top of my head. That
said, and for what it's worth, my more complete initial reflexion is that:
* on 3.0/3.X, this feels a tad risky: we're adding new code to the file
indexing (granted, not excessively complex one) and it doesn't get a lot more
critical. It could also change the performance profile, and while it might be
on the good side in many cases, it may not always be (especially when the patch
rely on 2 new settings whose defaults may happen to not be the right ones for
someone's pratical workload). As few people will run into this problem in the
first place, asking those rare users to change {{column_index_size_in_kb}}
would probably be safer overall (tbc, I'm suggesting here to improve the error
message of the checked cast to point people to that work-around, not to leave
people on their own as done currently).
* on 4.0, we already have CASSANDRA-11206 (which is in fact in 3.11 as well) to
help work with large indexes, and things like CASSANDRA-9754 are supposed to
make that even better, so the benefit on memory of this aren't that clear.
CASSANDRA-11206 doesn't solve the {{AsssertionError}} of this ticket, but we
could move the index size from {{int}} to {{long}} (or varint) for that. Which
isn't necessarilly saying we shouldn't do this, but adding multiple ways to fix
the same problem, each with their own new config setting (CASSANDRA-11206 added
one, this patch adds 2) doesn't feel ideal, so that's to be taken into
consideration imo.
> IllegalArgumentException in upgradesstables compaction
> ------------------------------------------------------
>
> Key: CASSANDRA-13973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13973
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Dan Kinder
> Assignee: Jeff Jirsa
> Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> After an upgrade from 2.2.6 to 3.0.15 (sstable version la to mc), when I try
> to run upgradesstables, most of them upgrade fine but I see the exception
> below on several nodes, and it doesn't complete.
> CASSANDRA-12717 looks similar but the stack trace is not the same, so I
> assumed it is not identical. The various nodes this happens on all give the
> same trace.
> Might be notable that this is an analytics cluster with some large
> partitions, in the GB size.
> {noformat}
> error: Out of range: 7316844981
> -- StackTrace --
> java.lang.IllegalArgumentException: Out of range: 7316844981
> at com.google.common.primitives.Ints.checkedCast(Ints.java:91)
> at
> org.apache.cassandra.db.RowIndexEntry$IndexedEntry.promotedSize(RowIndexEntry.java:329)
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:133)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.append(BigTableWriter.java:409)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.afterAppend(BigTableWriter.java:120)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:157)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:88)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:424)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:311)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
