Stefania commented on CASSANDRA-12014:

The test results are clean now, the failures for testall on trunk happen on the 
unpatched branch as well. There were a couple of failing dtests for 3.0 that I 
could not trace to known failures, but they look unrelated and they passed 
locally and in the following builds on Jenkins. 

Therefore, this is ready for review.
This is a relatively large patch for what it is trying to achieve, and 
consideration should be given on whether we want the patch in 3.0, at least in 
its full form. The reason is that in order to estimate the key size, we need to 
modify the call chain to create sstable writers, which not only involves adding 
a new parameter to a very long chain but also changing the vast number of 
callers. We should have a valid estimated key size when opening sstables, 
downsampling them, flushing memtables or compacting existing sstables. For 
other cases, mostly offline tools that don't use the summary, we use a default 
constant value that can be changed with a system property.

There is also an extra commit in trunk for refactoring the large number of 
parameters in the call chain to create sstable writers. I tried to add the 
additional parameter introduced by CASSANDRA-10678 to the new class that I 
introduced to group the parameters that should be known by the callers and that 
are optional, the new class also allows setting the parameters via a builder 
pattern. It should arguably be the same as the writer factory, but at the 
moment the factory is static. I don't think the creation of a new instance of 
this class is a performance concern, since we do this per sstable so I don't 
understand why so far we've relied on a static factory with a large number of 
parameters in a very long call chain, which is good for performance but leaves 
very little flexibility.

I did not use the builder pattern for 3.0, I was trying to avoid refactoring 
3.0 code and, as mentioned already above, perhaps in 3.0 we should stick to 
using a default constant value for the key size.

> IndexSummary > 2G causes an assertion error
> -------------------------------------------
>                 Key: CASSANDRA-12014
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Brandon Williams
>            Assignee: Stefania
>            Priority: Minor
>             Fix For: 3.0.x, 3.x
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444  
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
>     at 
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) 
> ~[cassandra-all-]
>     at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[cassandra-all-]
>     at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[cassandra-all-]
>     at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>  ~[cassandra-all-]
>     at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
>     at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
>     at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
>     at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
>     at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should 
> have a better method of coping with this than throwing the AE.

This message was sent by Atlassian JIRA

Reply via email to