Can you file a Jira for this? Would be good to make sure 3.10 doesn't get
released with this bug.
On Fri, Nov 4, 2016 at 6:11 PM Voytek Jarnot <voytek.jar...@gmail.com>
wrote:
> Thought I'd follow-up to myself, in case anyone else comes across this
> problem. I found a reasonably easy test case to reproduce the problem:
>
> This works in 3.9, but doesn't work in 3.10-snapshot:
>
> CREATE KEYSPACE vjtest WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> use vjtest ;
> create table tester(id1 text, id2 text, id3 text, val1 text, primary
> key((id1, id2), id3));
> create custom index tester_idx_val1 on tester(val1) using
> 'org.apache.cassandra.index.sasi.SASIIndex';
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','1-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','2-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','3-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','4-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','5-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','6-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','7-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','8-3','asdf');
> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','9-3','asdf');
>
> That's it - when Cassandra tries to flush all hell breaks loose (well,
> maaybe not, but an unhandled error gets logged). Also, the index doesn't
> actually work subsequently.
>
> On Fri, Nov 4, 2016 at 3:58 PM, Voytek Jarnot <voytek.jar...@gmail.com>
> wrote:
>
> Wondering if anyone has encountered the same...
>
> Full story and stacktraces below, short version is that creating a SASI
> index fails for me when running a 3.10-SNAPSHOT build. One caveat: creating
> the index on an empty table doesn't fail; however, soon after I start
> pumping data into the table similar problems occur.
>
> I created CASSANDRA-12877 for this, but am beginning to suspect it might
> be related to CASSANDRA-11990. The thing that's throwing me is that I
> can't seem to duplicate this with a simple test table.
>
> Background:
>
> Ended up building/loading a 3.10-SNAPSHOT to try to get past
> CASSANDRA-11670, CASSANDRA-12223, and CASSANDRA-12689.
>
> 1) built/installed 3.10-SNAPSHOT from git branch cassandra-3.X
> 2) created keyspace (SimpleStrategy, RF 1)
> 3) created table: (simplified version below, many more valX columns
> present)
>
> CREATE TABLE test_table (
> id1 text,
> id2 text,
> id3 date,
> id4 timestamp,
> id5 text,
> val1 text,
> val2 text,
> val3 text,
> task_id text,
> val4 text,
> val5 text,
> PRIMARY KEY ((id1, id2), id3, id4, id5)
> ) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id5 ASC)
>
> 4) created materialized view:
>
> CREATE MATERIALIZED VIEW test_table_by_task_id AS
> SELECT *
> FROM test_table
> WHERE id1 IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4
> IS NOT NULL AND id5 IS NOT NULL AND task_id IS NOT NULL
> PRIMARY KEY (task_id, id3, id4, id1, id2, id5)
> WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5
> ASC)
>
> 5) inserted 27 million "rows" (i.e., unique values for id5)
> 6) create index attempt
>
> create custom index idx_test_table_val5 on test_table(val5) using
> 'org.apache.cassandra.index.sasi.SASIIndex';
>
> 7) no error in cqlsh, but system.log shows many of the following:
>
> INFO [SASI-General:1] 2016-11-04 13:46:47,578
> PerSSTableIndexWriter.java:277 - Flushed index segment
> /mydir/cassandra/apache-cassandra-3.10-SNAPSHOT/data/data/mykeyspace/test_table-133dd090a2b411e6b1bf6df2a1af06f0/mc-149-big-SI_idx_test_table_val5.db_0,
> took 869 ms.
> ERROR [SASI-General:1] 2016-11-04 13:46:47,584 CassandraDaemon.java:229 -
> Exception in thread Thread[SASI-General:1,5,main]
> java.lang.AssertionError: cannot have more than 8 overflow collisions per
> leaf, but had: 12
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createOverflowEntry(AbstractTokenTreeBuilder.java:357)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder.java:346)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$DynamicLeaf.serializeData(DynamicTokenTreeBuilder.java:180)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenTreeBuilder.java:306)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder.write(AbstractTokenTreeBuilder.java:90)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableDataBlock.flushAndClear(OnDiskIndexBuilder.java:629)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.flush(OnDiskIndexBuilder.java:446)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.add(OnDiskIndexBuilder.java:433)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.addTerm(OnDiskIndexBuilder.java:207)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:293)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:258)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:241)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:267)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>
> As well as some of these:
>
> ERROR [CompactionExecutor:3] 2016-11-04 13:49:13,142 DataTracker.java:168
> - Can't open index file at
> /mydir/cassandra/apache-cassandra-3.10-SNAPSHOT/data/data/mykeyspace/test_table-133dd090a2b411e6b1bf6df2a1af06f0/mc-300-big-SI_idx_test_table_val5.db,
> skipping.
> java.lang.IllegalArgumentException: position: 3472329188772431788, limit:
> 8180
> at
> org.apache.cassandra.index.sasi.utils.MappedBuffer.position(MappedBuffer.java:106)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.disk.OnDiskIndex.<init>(OnDiskIndex.java:147)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SSTableIndex.<init>(SSTableIndex.java:62)
> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.conf.DataTracker.getIndexes(DataTracker.java:150)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.conf.DataTracker.update(DataTracker.java:69)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.update(ColumnIndex.java:147)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SASIIndexBuilder.completeSSTable(SASIIndexBuilder.java:156)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.index.sasi.SASIIndexBuilder.build(SASIIndexBuilder.java:125)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> org.apache.cassandra.db.compaction.CompactionManager$14.run(CompactionManager.java:1583)
> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_101]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>
>
>
