[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231400#comment-15231400
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
Ok, as a quick update, I think we know what is going on here, [~jrwest] is
working on the changes to TokenTree, and it's most definitely caused by changes
in CASSANDRA-11383. We still going to use your files to validate plus add
additional tests to prevent this in the future.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231348#comment-15231348
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
Sounds good, [~doanduyhai]! Meanwhile we are trying to reproduce based on what
we can figure out theoretically.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231340#comment-15231340
]
DOAN DuyHai commented on CASSANDRA-11525:
-
Ok I'm uploading to
https://drive.google.com/folderview?id=0B6wR2aj4Cb6wYWdfcl9Pb05CYW8=sharing
There are 9 SSTables and 9 index files
The SSTable size sums up to more than 20Gb
It will take 2h at least for the upload to finish
I will rebuild the index with 3.4 tomorrow and re-test.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231220#comment-15231220
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
[~doanduyhai] Alright, it's most likely is related to how the index is stitched
together again, we'll wait for you to upload files.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231212#comment-15231212
]
DOAN DuyHai commented on CASSANDRA-11525:
-
Ok your intuition was correct [~xedin], I did this code patch :
{noformat}
key = decorateKey(ByteBufferUtil.readWithShortLength(in));
// hint read path about key location if caching is enabled
// this saves index summary lookup and index file iteration which
whould be pretty costly
// especially in presence of promoted column indexes
try
{
if (isKeyCacheSetup())
cacheKey(key, rowIndexEntrySerializer.deserialize(in));
} catch (IndexOutOfBoundsException ex)
{
try {
final String keyValue =
keyValidator.getString(key.getKey().duplicate());
logger.error(String.format(
"Error when reading index entry for token '%s', partition
key '%s' at indexPosition %s ",
key.getToken().getTokenValue(), keyValue, indexPosition));
throw ex;
} catch (Exception ex2)
{
logger.error(String.format(
"Error when reading index entry for token '%s' at
indexPosition %s ",
key.getToken().getTokenValue(), indexPosition));
throw ex;
}
}
{noformat}
And it seems that we're falling into the second exception catch block, which
is a hint that the deserialization of the partition key ByteBuffer may have
thrown an exception as well so yes, very probable that it's just a random set
of 32k bytes
I'm going to fetch the SSTables and index structures to upload them.
By the way, I tested with trunk (3.6-SNAPSHOT) and same error too
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230742#comment-15230742
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
/cc [~jrwest]
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230714#comment-15230714
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
Also it looks like the actual key from the index file might be read
successfully, so maybe along side of printing token you can also print actual
data from the key that might be helpful to figure out if it's a real key or
just random set of 32k bytes.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230687#comment-15230687
]
Pavel Yaskevich commented on CASSANDRA-11525:
-
I think we just use incorrect serializer in some of the situations e.g. when
{{clustering order by}} is used, that's what the problem is because it can't
properly deserialize index entry.
[~doanduyhai] It would be great if you could share sstables again so i can
reproduce locally.
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
[jira] [Commented] (CASSANDRA-11525) SASI index corruption
[
https://issues.apache.org/jira/browse/CASSANDRA-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230453#comment-15230453
]
DOAN DuyHai commented on CASSANDRA-11525:
-
If necessary I can upload the base SSTables and corresponding SASI index files
> SASI index corruption
> -
>
> Key: CASSANDRA-11525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11525
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Environment: Cassandra 3.5-SNAPSHOT
>Reporter: DOAN DuyHai
>
> Bug reproduced in *Cassandra 3.5-SNAPSHOT* (after the fix of OOM)
> {noformat}
> create table if not exists test.resource_bench (
> dsr_id uuid,
> rel_seq bigint,
> seq bigint,
> dsp_code varchar,
> model_code varchar,
> media_code varchar,
> transfer_code varchar,
> commercial_offer_code varchar,
> territory_code varchar,
> period_end_month_int int,
> authorized_societies_txt text,
> rel_type text,
> status text,
> dsp_release_code text,
> title text,
> contributors_name list,
> unic_work text,
> paying_net_qty bigint,
> PRIMARY KEY ((dsr_id, rel_seq), seq)
> ) WITH CLUSTERING ORDER BY (seq ASC);
> CREATE CUSTOM INDEX resource_period_end_month_int_idx ON test.resource_bench
> (period_end_month_int) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH
> OPTIONS = {'mode': 'PREFIX'};
> {noformat}
> So the index is a {{DENSE}} numerical index.
> When doing the request {{SELECT dsp_code, unic_work, paying_net_qty FROM
> test.resource_bench WHERE period_end_month_int = 201401}} using server-side
> paging.
> I bumped into this stack trace:
> {noformat}
> WARN [SharedPool-Worker-1] 2016-04-06 00:00:30,825
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: -55
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserialize(ClusteringPrefix.java:268)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:128)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.Serializers$2.deserialize(Serializers.java:120)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:148)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:218)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.keyAt(SSTableReader.java:1823)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:168)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex$DecoratedKeyFetcher.apply(SSTableIndex.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:518)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.TokenTree$KeyIterator.computeNext(TokenTree.java:504)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.tryToComputeNext(AbstractIterator.java:116)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.utils.AbstractIterator.hasNext(AbstractIterator.java:110)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:106)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.plan.QueryPlan$ResultIterator.computeNext(QueryPlan.java:71)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.5-SNAPSHOT.jar:3.5-SNAPSHOT]
> at
>
