[jira] [Commented] (CASSANDRA-11198) Materialized view inconsistency
[
https://issues.apache.org/jira/browse/CASSANDRA-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004334#comment-16004334
]
Keith Wansbrough commented on CASSANDRA-11198:
--
You can see in my comment above that I reproduced a similar-looking issue in
3.0.7.1159. Full details given above. However I'm no longer working on this so
I'm unable to attempt a further repro / reopen at this time.
> Materialized view inconsistency
> ---
>
> Key: CASSANDRA-11198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11198
> Project: Cassandra
> Issue Type: Bug
>Reporter: Gábor Auth
>Assignee: Carl Yeksigian
> Attachments: CASSANDRA-11198.trace
>
>
> Here is a materialized view:
> {code}
> > DESCRIBE MATERIALIZED VIEW unit_by_transport ;
> CREATE MATERIALIZED VIEW unit_by_transport AS
> SELECT *
> FROM unit
> WHERE transportid IS NOT NULL AND type IS NOT NULL
> PRIMARY KEY (transportid, id)
> WITH CLUSTERING ORDER BY (id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> I cannot reproduce this but sometimes and somehow happened the same issue
> (https://issues.apache.org/jira/browse/CASSANDRA-10910):
> {code}
> > SELECT transportid, id, type FROM unit_by_transport WHERE
> > transportid=24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 and
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id
> | type
> --+--+--
> 24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 | 99c05a70-d686-11e5-a169-97287061d5d1
> | null
> (1 rows)
> > SELECT transportid, id, type FROM unit WHERE
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id | type
> -++--
> (0 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12418) sstabledump JSON fails after row tombstone
[
https://issues.apache.org/jira/browse/CASSANDRA-12418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476734#comment-15476734
]
Keith Wansbrough commented on CASSANDRA-12418:
--
Thanks!
> sstabledump JSON fails after row tombstone
> --
>
> Key: CASSANDRA-12418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12418
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>Assignee: Dave Brosius
> Fix For: 3.0.x
>
>
> sstabledump fails in JSON generation on an sstable containing a row deletion,
> using Cassandra 3.10-SNAPSHOT accf7a4724e244d6f1ba921cb11d2554dbb54a76 from
> 2016-07-26.
> There are two exceptions displayed:
> * Fatal error parsing partition: aye
> org.codehaus.jackson.JsonGenerationException: Can not start an object,
> expecting field name
> * org.codehaus.jackson.JsonGenerationException: Current context not an ARRAY
> but OBJECT
> Steps to reproduce:
> {code}
> cqlsh> create KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> create TABLE foo.bar (id text, str text, primary key (id));
> cqlsh> insert into foo.bar (id, str) values ('aye', 'alpha');
> cqlsh> insert into foo.bar (id, str) values ('bee', 'beta');
> cqlsh> delete from foo.bar where id = 'bee';
> cqlsh> insert into foo.bar (id, str) values ('bee', 'beth');
> cqlsh> select * from foo.bar;
> id | str
> -+---
> bee | beth
> aye | alpha
> (2 rows)
> cqlsh>
> {code}
> Now find the sstable:
> {code}
> $ cassandra/bin/nodetool flush
> $ cassandra/bin/sstableutil foo bar
> [..]
> Listing files...
> [..]
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> [..]
> {code}
> Now check with sstabledump \-d. This works just fine.
> {code}
> $ cassandra/tools/bin/sstabledump -d
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> [bee]@0 deletedAt=1470737827008101, localDeletion=1470737827
> [bee]@0 Row[info=[ts=1470737832405510] ]: | [str=beth ts=1470737832405510]
> [aye]@31 Row[info=[ts=1470737784401778] ]: | [str=alpha ts=1470737784401778]
> {code}
> Now run sstabledump. This should work as well, but it fails as follows:
> {code}
> $ cassandra/tools/bin/sstabledump
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> ERROR 10:26:07 Fatal error parsing partition: aye
> org.codehaus.jackson.JsonGenerationException: Can not start an object,
> expecting field name
> at
> org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.codehaus.jackson.impl.WriterBasedGenerator._verifyValueWrite(WriterBasedGenerator.java:836)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.codehaus.jackson.impl.WriterBasedGenerator.writeStartObject(WriterBasedGenerator.java:273)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:181)
> ~[main/:na]
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> ~[na:1.8.0_77]
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> ~[na:1.8.0_77]
> at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_77]
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> ~[na:1.8.0_77]
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> ~[na:1.8.0_77]
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> ~[na:1.8.0_77]
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> ~[na:1.8.0_77]
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:99)
> ~[main/:na]
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:237)
> ~[main/:na]
> [
> {
> "partition" : {
> "key" : [ "bee" ],
> "position" : 0,
> "deletion_info" : { "marked_deleted" : "2016-08-09T10:17:07.008101Z",
> "local_delete_time" : "2016-08-09T10:17:07Z" }
> }
> }
> ]org.codehaus.jackson.JsonGenerationException: Current context not an ARRAY
> but OBJECT
> at
> org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
> at
>
[jira] [Commented] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432697#comment-15432697
]
Keith Wansbrough commented on CASSANDRA-12424:
--
I guess this was probably CASSANDRA-12247. I'll retest in 3.0.9 when it is
released.
> Assertion failure in ViewUpdateGenerator
>
>
> Key: CASSANDRA-12424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
> Project: Cassandra
> Issue Type: Bug
>Reporter: Keith Wansbrough
>Assignee: Carl Yeksigian
> Attachments: cassandra.log
>
>
> Using released apache-cassandra-3.7.0, we have managed to get a node into a
> state where it won't start up. The exception is {{java.lang.AssertionError:
> We shouldn't have got there is the base row had no associated entry}} and it
> appears in
> ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
> I still have the offending node; what diags/data would be useful for
> diagnosis? I've attached the full cassandra.log. In summary, cassandra.log
> contains multiple instances of the following when replaying the commit log on
> startup, leading ultimately to failure to start up.
> {code}
> ERROR 15:24:17 Unknown exception caught while attempting to update
> MaterializedView! edison.scs_subscriber
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN 15:24:17 Uncaught exception on thread
> Thread[SharedPool-Worker-4,5,main]: {}
> {code}
> and ultimately
> {code}
> ERROR 15:24:18 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12177) sstabledump fails if sstable path includes dot
[
https://issues.apache.org/jira/browse/CASSANDRA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432693#comment-15432693
]
Keith Wansbrough commented on CASSANDRA-12177:
--
Super, thanks!
> sstabledump fails if sstable path includes dot
> --
>
> Key: CASSANDRA-12177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12177
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>Assignee: Yuki Morishita
> Fix For: 3.0.9
>
>
> If there is a dot in the file path passed to sstabledump, it fails with an
> error {{partitioner org.apache.cassandra.dht.Murmur3Partitioner does not
> match system partitioner org.apache.cassandra.dht.LocalPartitioner.}}
> I can work around this by renaming the directory containing the file, but it
> seems like a bug. I expected the directory name to be irrelevant.
> Example (assumes you have a keyspace test containing a table called sport,
> but should repro with any keyspace/table):
> {code}
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> testdir
> $ sstabledump testdir/mb-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "2" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-07-11T10:15:22.766107Z" },
> "cells" : [
> { "name" : "score", "value" : "Golf" },
> { "name" : "sport_type", "value" : "5" }
> ]
> }
> ]
> }
> ]
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> test.dir
> $ sstabledump test.dir/mb-1-big-Data.db
> ERROR 15:02:52 Cannot open /home/centos/test.dir/mb-1-big; partitioner
> org.apache.cassandra.dht.Murmur3Partitioner does not match system partitioner
> org.apache.cassandra.dht.LocalPartitioner. Note that the default partitioner
> starting with Cassandra 1.2 is Murmur3Partitioner, so you will need to edit
> that to match your old partitioner if upgrading.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12418) sstabledump JSON fails after row tombstone
[
https://issues.apache.org/jira/browse/CASSANDRA-12418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432610#comment-15432610
]
Keith Wansbrough commented on CASSANDRA-12418:
--
It looks to me like JsonTransformer#serializePartition needs to call
json.writeEndObject() after serializeDeletion() around line 191. Before this
point there are two json.writeStartObject() calls, but after it in this branch
there is only one json.writeEndObject() (on line 221), whereas if there is no
partition-level deletion there are two (on lines 195 and 221).
> sstabledump JSON fails after row tombstone
> --
>
> Key: CASSANDRA-12418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12418
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>
> sstabledump fails in JSON generation on an sstable containing a row deletion,
> using Cassandra 3.10-SNAPSHOT accf7a4724e244d6f1ba921cb11d2554dbb54a76 from
> 2016-07-26.
> There are two exceptions displayed:
> * Fatal error parsing partition: aye
> org.codehaus.jackson.JsonGenerationException: Can not start an object,
> expecting field name
> * org.codehaus.jackson.JsonGenerationException: Current context not an ARRAY
> but OBJECT
> Steps to reproduce:
> {code}
> cqlsh> create KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> create TABLE foo.bar (id text, str text, primary key (id));
> cqlsh> insert into foo.bar (id, str) values ('aye', 'alpha');
> cqlsh> insert into foo.bar (id, str) values ('bee', 'beta');
> cqlsh> delete from foo.bar where id = 'bee';
> cqlsh> insert into foo.bar (id, str) values ('bee', 'beth');
> cqlsh> select * from foo.bar;
> id | str
> -+---
> bee | beth
> aye | alpha
> (2 rows)
> cqlsh>
> {code}
> Now find the sstable:
> {code}
> $ cassandra/bin/nodetool flush
> $ cassandra/bin/sstableutil foo bar
> [..]
> Listing files...
> [..]
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> [..]
> {code}
> Now check with sstabledump \-d. This works just fine.
> {code}
> $ cassandra/tools/bin/sstabledump -d
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> [bee]@0 deletedAt=1470737827008101, localDeletion=1470737827
> [bee]@0 Row[info=[ts=1470737832405510] ]: | [str=beth ts=1470737832405510]
> [aye]@31 Row[info=[ts=1470737784401778] ]: | [str=alpha ts=1470737784401778]
> {code}
> Now run sstabledump. This should work as well, but it fails as follows:
> {code}
> $ cassandra/tools/bin/sstabledump
> /home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
> ERROR 10:26:07 Fatal error parsing partition: aye
> org.codehaus.jackson.JsonGenerationException: Can not start an object,
> expecting field name
> at
> org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.codehaus.jackson.impl.WriterBasedGenerator._verifyValueWrite(WriterBasedGenerator.java:836)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.codehaus.jackson.impl.WriterBasedGenerator.writeStartObject(WriterBasedGenerator.java:273)
> ~[jackson-core-asl-1.9.2.jar:1.9.2]
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:181)
> ~[main/:na]
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> ~[na:1.8.0_77]
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> ~[na:1.8.0_77]
> at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_77]
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> ~[na:1.8.0_77]
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> ~[na:1.8.0_77]
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> ~[na:1.8.0_77]
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> ~[na:1.8.0_77]
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> ~[na:1.8.0_77]
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:99)
> ~[main/:na]
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:237)
> ~[main/:na]
> [
> {
> "partition" : {
> "key" : [ "bee" ],
> "position" : 0,
> "deletion_info" : { "marked_deleted" : "2016-08-09T10:17:07.008101Z",
> "local_delete_time" :
[jira] [Commented] (CASSANDRA-11198) Materialized view inconsistency
[
https://issues.apache.org/jira/browse/CASSANDRA-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426093#comment-15426093
]
Keith Wansbrough commented on CASSANDRA-11198:
--
I don't think CASSANDRA-11475 can have fixed my issue, because that's marked as
fixed in 3.0.7 which is the version I'm running. Shall I raise a separate issue?
> Materialized view inconsistency
> ---
>
> Key: CASSANDRA-11198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11198
> Project: Cassandra
> Issue Type: Bug
>Reporter: Gábor Auth
>Assignee: Carl Yeksigian
> Attachments: CASSANDRA-11198.trace
>
>
> Here is a materialized view:
> {code}
> > DESCRIBE MATERIALIZED VIEW unit_by_transport ;
> CREATE MATERIALIZED VIEW unit_by_transport AS
> SELECT *
> FROM unit
> WHERE transportid IS NOT NULL AND type IS NOT NULL
> PRIMARY KEY (transportid, id)
> WITH CLUSTERING ORDER BY (id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> I cannot reproduce this but sometimes and somehow happened the same issue
> (https://issues.apache.org/jira/browse/CASSANDRA-10910):
> {code}
> > SELECT transportid, id, type FROM unit_by_transport WHERE
> > transportid=24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 and
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id
> | type
> --+--+--
> 24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 | 99c05a70-d686-11e5-a169-97287061d5d1
> | null
> (1 rows)
> > SELECT transportid, id, type FROM unit WHERE
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id | type
> -++--
> (0 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11198) Materialized view inconsistency
[
https://issues.apache.org/jira/browse/CASSANDRA-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424700#comment-15424700
]
Keith Wansbrough edited comment on CASSANDRA-11198 at 8/17/16 3:20 PM:
---
I've seen something similar, on 3.0.7.1159. We have a base table scs_subscriber
which contains relevant columns ((id), n), phone, revision, and MV table
lus_subscriber_phone which contains ((phone), id, n), revision. Rows in
scs_subscriber are usually updated by a Paxos batch which does a delete of the
whole row followed by an insert, and always have a valid revision. However
sometimes when we read a row in lus_subscriber_phone we see revision null
rather than the expected value. Successive reads flip intermittently between
null and the expected value, even though there is no write activity on the row
at the time. Some details in the following gist, including sstabledumps of the
base table and two differing replicas of the view.
https://gist.github.com/kw217/bdb92bc5261c3d34aebab59d2eb29f72
The update query we do is roughly
{code}
BEGIN BATCH
DELETE FROM .scs_subscriber WHERE id = ?;
UPDATE .scs_subscriber SET data4 = (?, ?), revision = (?, ?), phone =
?, (and some other fields) WHERE id = ? AND n = 0 IF revision = (?, ?);
APPLY BATCH
{code}
was (Author: kw217):
I've seen something similar, on 3.0.7.1159. We have a base table scs_subscriber
which contains relevant columns ((id), n), phone, revision, and MV table
lus_subscriber_phone which contains ((phone), id, n), revision. Rows in
scs_subscriber are always updated by a Paxos batch which does a delete of the
whole row followed by an insert, and always have a valid revision. However
sometimes when we read a row in lus_subscriber_phone we see revision null
rather than the expected value. Successive reads flip intermittently between
null and the expected value, even though there is no write activity on the row
at the time. Some details in the following gist, including sstabledumps of the
base table and two differing replicas of the view.
https://gist.github.com/kw217/bdb92bc5261c3d34aebab59d2eb29f72
The update query we do is roughly
{code}
BEGIN BATCH
DELETE FROM .scs_subscriber WHERE id = ?;
UPDATE .scs_subscriber SET data4 = (?, ?), revision = (?, ?), phone =
?, (and some other fields) WHERE id = ? AND n = 0 IF revision = (?, ?);
APPLY BATCH
{code}
> Materialized view inconsistency
> ---
>
> Key: CASSANDRA-11198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11198
> Project: Cassandra
> Issue Type: Bug
>Reporter: Gábor Auth
>Assignee: Carl Yeksigian
> Attachments: CASSANDRA-11198.trace
>
>
> Here is a materialized view:
> {code}
> > DESCRIBE MATERIALIZED VIEW unit_by_transport ;
> CREATE MATERIALIZED VIEW unit_by_transport AS
> SELECT *
> FROM unit
> WHERE transportid IS NOT NULL AND type IS NOT NULL
> PRIMARY KEY (transportid, id)
> WITH CLUSTERING ORDER BY (id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> I cannot reproduce this but sometimes and somehow happened the same issue
> (https://issues.apache.org/jira/browse/CASSANDRA-10910):
> {code}
> > SELECT transportid, id, type FROM unit_by_transport WHERE
> > transportid=24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 and
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id
> | type
> --+--+--
> 24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 | 99c05a70-d686-11e5-a169-97287061d5d1
> | null
> (1 rows)
> > SELECT transportid, id, type FROM unit WHERE
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id | type
> -++--
> (0 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11198) Materialized view inconsistency
[
https://issues.apache.org/jira/browse/CASSANDRA-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424700#comment-15424700
]
Keith Wansbrough commented on CASSANDRA-11198:
--
I've seen something similar, on 3.0.7.1159. We have a base table scs_subscriber
which contains relevant columns ((id), n), phone, revision, and MV table
lus_subscriber_phone which contains ((phone), id, n), revision. Rows in
scs_subscriber are always updated by a Paxos batch which does a delete of the
whole row followed by an insert, and always have a valid revision. However
sometimes when we read a row in lus_subscriber_phone we see revision null
rather than the expected value. Successive reads flip intermittently between
null and the expected value, even though there is no write activity on the row
at the time. Some details in the following gist, including sstabledumps of the
base table and two differing replicas of the view.
https://gist.github.com/kw217/bdb92bc5261c3d34aebab59d2eb29f72
The update query we do is roughly
{code}
BEGIN BATCH
DELETE FROM .scs_subscriber WHERE id = ?;
UPDATE .scs_subscriber SET data4 = (?, ?), revision = (?, ?), phone =
?, (and some other fields) WHERE id = ? AND n = 0 IF revision = (?, ?);
APPLY BATCH
{code}
> Materialized view inconsistency
> ---
>
> Key: CASSANDRA-11198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11198
> Project: Cassandra
> Issue Type: Bug
>Reporter: Gábor Auth
>Assignee: Carl Yeksigian
> Attachments: CASSANDRA-11198.trace
>
>
> Here is a materialized view:
> {code}
> > DESCRIBE MATERIALIZED VIEW unit_by_transport ;
> CREATE MATERIALIZED VIEW unit_by_transport AS
> SELECT *
> FROM unit
> WHERE transportid IS NOT NULL AND type IS NOT NULL
> PRIMARY KEY (transportid, id)
> WITH CLUSTERING ORDER BY (id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> I cannot reproduce this but sometimes and somehow happened the same issue
> (https://issues.apache.org/jira/browse/CASSANDRA-10910):
> {code}
> > SELECT transportid, id, type FROM unit_by_transport WHERE
> > transportid=24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 and
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id
> | type
> --+--+--
> 24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 | 99c05a70-d686-11e5-a169-97287061d5d1
> | null
> (1 rows)
> > SELECT transportid, id, type FROM unit WHERE
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
> transportid | id | type
> -++--
> (0 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422514#comment-15422514
]
Keith Wansbrough edited comment on CASSANDRA-12424 at 8/16/16 9:50 AM:
---
FWIW: I have now recovered, by renaming /var/lib/cassandra/commitlog/ and
recreated it as an empty directory, and Cassandra started OK. I still have the
old commitlog folder if it's interesting.
was (Author: kw217):
FWIW: I renamed /var/lib/cassandra/commitlog/ and recreated it as an empty
directory, and Cassandra started OK.
> Assertion failure in ViewUpdateGenerator
>
>
> Key: CASSANDRA-12424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
> Project: Cassandra
> Issue Type: Bug
>Reporter: Keith Wansbrough
>Assignee: Carl Yeksigian
> Attachments: cassandra.log
>
>
> Using released apache-cassandra-3.7.0, we have managed to get a node into a
> state where it won't start up. The exception is {{java.lang.AssertionError:
> We shouldn't have got there is the base row had no associated entry}} and it
> appears in
> ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
> I still have the offending node; what diags/data would be useful for
> diagnosis? I've attached the full cassandra.log. In summary, cassandra.log
> contains multiple instances of the following when replaying the commit log on
> startup, leading ultimately to failure to start up.
> {code}
> ERROR 15:24:17 Unknown exception caught while attempting to update
> MaterializedView! edison.scs_subscriber
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN 15:24:17 Uncaught exception on thread
> Thread[SharedPool-Worker-4,5,main]: {}
> {code}
> and ultimately
> {code}
> ERROR 15:24:18 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422514#comment-15422514
]
Keith Wansbrough commented on CASSANDRA-12424:
--
FWIW: I renamed /var/lib/cassandra/commitlog/ and recreated it as an empty
directory, and Cassandra started OK.
> Assertion failure in ViewUpdateGenerator
>
>
> Key: CASSANDRA-12424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
> Project: Cassandra
> Issue Type: Bug
>Reporter: Keith Wansbrough
>Assignee: Carl Yeksigian
> Attachments: cassandra.log
>
>
> Using released apache-cassandra-3.7.0, we have managed to get a node into a
> state where it won't start up. The exception is {{java.lang.AssertionError:
> We shouldn't have got there is the base row had no associated entry}} and it
> appears in
> ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
> I still have the offending node; what diags/data would be useful for
> diagnosis? I've attached the full cassandra.log. In summary, cassandra.log
> contains multiple instances of the following when replaying the commit log on
> startup, leading ultimately to failure to start up.
> {code}
> ERROR 15:24:17 Unknown exception caught while attempting to update
> MaterializedView! edison.scs_subscriber
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN 15:24:17 Uncaught exception on thread
> Thread[SharedPool-Worker-4,5,main]: {}
> {code}
> and ultimately
> {code}
> ERROR 15:24:18 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420903#comment-15420903
]
Keith Wansbrough commented on CASSANDRA-12424:
--
Hi [~carlyeks] - do you need anything from this node before I destroy and
rebuild it?
> Assertion failure in ViewUpdateGenerator
>
>
> Key: CASSANDRA-12424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
> Project: Cassandra
> Issue Type: Bug
>Reporter: Keith Wansbrough
>Assignee: Carl Yeksigian
> Attachments: cassandra.log
>
>
> Using released apache-cassandra-3.7.0, we have managed to get a node into a
> state where it won't start up. The exception is {{java.lang.AssertionError:
> We shouldn't have got there is the base row had no associated entry}} and it
> appears in
> ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
> I still have the offending node; what diags/data would be useful for
> diagnosis? I've attached the full cassandra.log. In summary, cassandra.log
> contains multiple instances of the following when replaying the commit log on
> startup, leading ultimately to failure to start up.
> {code}
> ERROR 15:24:17 Unknown exception caught while attempting to update
> MaterializedView! edison.scs_subscriber
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN 15:24:17 Uncaught exception on thread
> Thread[SharedPool-Worker-4,5,main]: {}
> {code}
> and ultimately
> {code}
> ERROR 15:24:18 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wansbrough updated CASSANDRA-12424:
-
Description:
Using released apache-cassandra-3.7.0, we have managed to get a node into a
state where it won't start up. The exception is {{java.lang.AssertionError: We
shouldn't have got there is the base row had no associated entry}} and it
appears in
ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
I still have the offending node; what diags/data would be useful for diagnosis?
I've attached the full cassandra.log. In summary, cassandra.log contains
multiple instances of the following when replaying the commit log on startup,
leading ultimately to failure to start up.
{code}
ERROR 15:24:17 Unknown exception caught while attempting to update
MaterializedView! edison.scs_subscriber
java.lang.AssertionError: We shouldn't have got there is the base row had no
associated entry
at
org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
~[apache-cassandra-3.7.0.jar:3.7.0]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[apache-cassandra-3.7.0.jar:3.7.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
[apache-cassandra-3.7.0.jar:3.7.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-3.7.0.jar:3.7.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
WARN 15:24:17 Uncaught exception on thread Thread[SharedPool-Worker-4,5,main]:
{}
{code}
and ultimately
{code}
ERROR 15:24:18 Exception encountered during startup
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: We shouldn't have got there is the base row had no
associated entry
{code}
was:
Using released apache-cassandra-3.7.0, we have managed to get a node into a
state where it won't start up. The exception is {{java.lang.AssertionError: We
shouldn't have got there is the base row had no associated entry}} and it
appears in
ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
I still have the offending node; what diags/data would be useful for diagnosis?
I've attached the full cassandra.log. In summary, cassandra.log contains
multiple instances of the following when replacing the commit log on startup,
leading ultimately to failure to start up.
{code}
ERROR 15:24:17 Unknown exception caught while attempting to update
MaterializedView! edison.scs_subscriber
java.lang.AssertionError: We shouldn't have got there is the base row had no
associated entry
at
org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
~[apache-cassandra-3.7.0.jar:3.7.0]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
[apache-cassandra-3.7.0.jar:3.7.0]
at
[jira] [Updated] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
[
https://issues.apache.org/jira/browse/CASSANDRA-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wansbrough updated CASSANDRA-12424:
-
Attachment: cassandra.log
> Assertion failure in ViewUpdateGenerator
>
>
> Key: CASSANDRA-12424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
> Project: Cassandra
> Issue Type: Bug
>Reporter: Keith Wansbrough
> Attachments: cassandra.log
>
>
> Using released apache-cassandra-3.7.0, we have managed to get a node into a
> state where it won't start up. The exception is {{java.lang.AssertionError:
> We shouldn't have got there is the base row had no associated entry}} and it
> appears in
> ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
> I still have the offending node; what diags/data would be useful for
> diagnosis? I've attached the full cassandra.log. In summary, cassandra.log
> contains multiple instances of the following when replacing the commit log on
> startup, leading ultimately to failure to start up.
> {code}
> ERROR 15:24:17 Unknown exception caught while attempting to update
> MaterializedView! edison.scs_subscriber
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_91]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN 15:24:17 Uncaught exception on thread
> Thread[SharedPool-Worker-4,5,main]: {}
> {code}
> and ultimately
> {code}
> ERROR 15:24:18 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.AssertionError: We shouldn't have got there is the base row had no
> associated entry
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12424) Assertion failure in ViewUpdateGenerator
Keith Wansbrough created CASSANDRA-12424:
Summary: Assertion failure in ViewUpdateGenerator
Key: CASSANDRA-12424
URL: https://issues.apache.org/jira/browse/CASSANDRA-12424
Project: Cassandra
Issue Type: Bug
Reporter: Keith Wansbrough
Using released apache-cassandra-3.7.0, we have managed to get a node into a
state where it won't start up. The exception is {{java.lang.AssertionError: We
shouldn't have got there is the base row had no associated entry}} and it
appears in
ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455).
I still have the offending node; what diags/data would be useful for diagnosis?
I've attached the full cassandra.log. In summary, cassandra.log contains
multiple instances of the following when replacing the commit log on startup,
leading ultimately to failure to start up.
{code}
ERROR 15:24:17 Unknown exception caught while attempting to update
MaterializedView! edison.scs_subscriber
java.lang.AssertionError: We shouldn't have got there is the base row had no
associated entry
at
org.apache.cassandra.db.view.ViewUpdateGenerator.computeLivenessInfoForEntry(ViewUpdateGenerator.java:455)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.updateEntry(ViewUpdateGenerator.java:273)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.ViewUpdateGenerator.addBaseTableUpdate(ViewUpdateGenerator.java:127)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.addToViewUpdateGenerators(TableViews.java:403)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.generateViewUpdates(TableViews.java:236)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:140)
~[apache-cassandra-3.7.0.jar:3.7.0]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:514)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:152)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[apache-cassandra-3.7.0.jar:3.7.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_91]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
[apache-cassandra-3.7.0.jar:3.7.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-3.7.0.jar:3.7.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
WARN 15:24:17 Uncaught exception on thread Thread[SharedPool-Worker-4,5,main]:
{}
{code}
and ultimately
{code}
ERROR 15:24:18 Exception encountered during startup
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError: We shouldn't have got there is the base row had no
associated entry
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12418) sstabledump JSON fails after row tombstone
Keith Wansbrough created CASSANDRA-12418:
Summary: sstabledump JSON fails after row tombstone
Key: CASSANDRA-12418
URL: https://issues.apache.org/jira/browse/CASSANDRA-12418
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Keith Wansbrough
sstabledump fails in JSON generation on an sstable containing a row deletion,
using Cassandra 3.10-SNAPSHOT accf7a4724e244d6f1ba921cb11d2554dbb54a76 from
2016-07-26.
There are two exceptions displayed:
* Fatal error parsing partition: aye
org.codehaus.jackson.JsonGenerationException: Can not start an object,
expecting field name
* org.codehaus.jackson.JsonGenerationException: Current context not an ARRAY
but OBJECT
Steps to reproduce:
{code}
cqlsh> create KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
cqlsh> create TABLE foo.bar (id text, str text, primary key (id));
cqlsh> insert into foo.bar (id, str) values ('aye', 'alpha');
cqlsh> insert into foo.bar (id, str) values ('bee', 'beta');
cqlsh> delete from foo.bar where id = 'bee';
cqlsh> insert into foo.bar (id, str) values ('bee', 'beth');
cqlsh> select * from foo.bar;
id | str
-+---
bee | beth
aye | alpha
(2 rows)
cqlsh>
{code}
Now find the sstable:
{code}
$ cassandra/bin/nodetool flush
$ cassandra/bin/sstableutil foo bar
[..]
Listing files...
[..]
/home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
[..]
{code}
Now check with sstabledump \-d. This works just fine.
{code}
$ cassandra/tools/bin/sstabledump -d
/home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
[bee]@0 deletedAt=1470737827008101, localDeletion=1470737827
[bee]@0 Row[info=[ts=1470737832405510] ]: | [str=beth ts=1470737832405510]
[aye]@31 Row[info=[ts=1470737784401778] ]: | [str=alpha ts=1470737784401778]
{code}
Now run sstabledump. This should work as well, but it fails as follows:
{code}
$ cassandra/tools/bin/sstabledump
/home/kw217/cassandra/data/data/foo/bar-407c56f05e1a11e6835def64bf5c656e/mb-1-big-Data.db
ERROR 10:26:07 Fatal error parsing partition: aye
org.codehaus.jackson.JsonGenerationException: Can not start an object,
expecting field name
at
org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
~[jackson-core-asl-1.9.2.jar:1.9.2]
at
org.codehaus.jackson.impl.WriterBasedGenerator._verifyValueWrite(WriterBasedGenerator.java:836)
~[jackson-core-asl-1.9.2.jar:1.9.2]
at
org.codehaus.jackson.impl.WriterBasedGenerator.writeStartObject(WriterBasedGenerator.java:273)
~[jackson-core-asl-1.9.2.jar:1.9.2]
at
org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:181)
~[main/:na]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
~[na:1.8.0_77]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[na:1.8.0_77]
at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_77]
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
~[na:1.8.0_77]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
~[na:1.8.0_77]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
~[na:1.8.0_77]
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
~[na:1.8.0_77]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
~[na:1.8.0_77]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[na:1.8.0_77]
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
~[na:1.8.0_77]
at
org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:99)
~[main/:na]
at
org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:237)
~[main/:na]
[
{
"partition" : {
"key" : [ "bee" ],
"position" : 0,
"deletion_info" : { "marked_deleted" : "2016-08-09T10:17:07.008101Z",
"local_delete_time" : "2016-08-09T10:17:07Z" }
}
}
]org.codehaus.jackson.JsonGenerationException: Current context not an ARRAY but
OBJECT
at
org.codehaus.jackson.impl.JsonGeneratorBase._reportError(JsonGeneratorBase.java:480)
at
org.codehaus.jackson.impl.WriterBasedGenerator.writeEndArray(WriterBasedGenerator.java:257)
at
org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:100)
at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:237)
{code}
If possible, please can this be fixed in the 3.0.x stream as well as trunk?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12177) sstabledump fails if sstable path includes dot
[
https://issues.apache.org/jira/browse/CASSANDRA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413186#comment-15413186
]
Keith Wansbrough commented on CASSANDRA-12177:
--
Thanks! Please could someone backport this to 3.0.x?
> sstabledump fails if sstable path includes dot
> --
>
> Key: CASSANDRA-12177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12177
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>
> If there is a dot in the file path passed to sstabledump, it fails with an
> error {{partitioner org.apache.cassandra.dht.Murmur3Partitioner does not
> match system partitioner org.apache.cassandra.dht.LocalPartitioner.}}
> I can work around this by renaming the directory containing the file, but it
> seems like a bug. I expected the directory name to be irrelevant.
> Example (assumes you have a keyspace test containing a table called sport,
> but should repro with any keyspace/table):
> {code}
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> testdir
> $ sstabledump testdir/mb-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "2" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-07-11T10:15:22.766107Z" },
> "cells" : [
> { "name" : "score", "value" : "Golf" },
> { "name" : "sport_type", "value" : "5" }
> ]
> }
> ]
> }
> ]
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> test.dir
> $ sstabledump test.dir/mb-1-big-Data.db
> ERROR 15:02:52 Cannot open /home/centos/test.dir/mb-1-big; partitioner
> org.apache.cassandra.dht.Murmur3Partitioner does not match system partitioner
> org.apache.cassandra.dht.LocalPartitioner. Note that the default partitioner
> starting with Cassandra 1.2 is Murmur3Partitioner, so you will need to edit
> that to match your old partitioner if upgrading.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10876) Alter behavior of batch WARN and fail on single partition batches
[ https://issues.apache.org/jira/browse/CASSANDRA-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393675#comment-15393675 ] Keith Wansbrough commented on CASSANDRA-10876: -- +1 - we're hitting this in 3.0.7.1159. Is there any chance you could merge the fix to 3.0.x? > Alter behavior of batch WARN and fail on single partition batches > - > > Key: CASSANDRA-10876 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10876 > Project: Cassandra > Issue Type: Improvement >Reporter: Patrick McFadin >Assignee: Sylvain Lebresne >Priority: Minor > Labels: lhf > Fix For: 3.6 > > Attachments: 10876.txt > > > In an attempt to give operator insight into potentially harmful batch usage, > Jiras were created to log WARN or fail on certain batch sizes. This ignores > the single partition batch, which doesn't create the same issues as a > multi-partition batch. > The proposal is to ignore size on single partition batch statements. > Reference: > [CASSANDRA-6487|https://issues.apache.org/jira/browse/CASSANDRA-6487] > [CASSANDRA-8011|https://issues.apache.org/jira/browse/CASSANDRA-8011] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12177) sstabledump fails if sstable path includes dot
[
https://issues.apache.org/jira/browse/CASSANDRA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374559#comment-15374559
]
Keith Wansbrough edited comment on CASSANDRA-12177 at 7/13/16 9:03 AM:
---
Apologies - this is version 3.7.0, which is the latest available from the
[download page|http://cassandra.apache.org/download/].
I tried this [3.8-SNAPSHOT|http://cassci.datastax.com/job/cassandra-3.8/]
(build #2, Jul 11, 2016 3:12 AM) and it does indeed seem to be fixed there.
Presumably that means it will be fixed in 3.9; will it also be fixed in the
next 3.0.x release (if the bug exists there; I haven't checked)?
Thanks!
was (Author: kw217):
Apologies - this is version 3.7.0, which is the latest available from the
[download page|http://cassandra.apache.org/download/].
I tried this [3.8-SNAPSHOT|http://cassci.datastax.com/job/cassandra-3.8/]
(build #2, Jul 11, 2016 3:12 AM) and it does indeed seem to be fixed there.
Presumably that means it will be fixed in 3.9; will it also be fixed in the
next 3.0.x release?
Thanks!
> sstabledump fails if sstable path includes dot
> --
>
> Key: CASSANDRA-12177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12177
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>
> If there is a dot in the file path passed to sstabledump, it fails with an
> error {{partitioner org.apache.cassandra.dht.Murmur3Partitioner does not
> match system partitioner org.apache.cassandra.dht.LocalPartitioner.}}
> I can work around this by renaming the directory containing the file, but it
> seems like a bug. I expected the directory name to be irrelevant.
> Example (assumes you have a keyspace test containing a table called sport,
> but should repro with any keyspace/table):
> {code}
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> testdir
> $ sstabledump testdir/mb-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "2" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-07-11T10:15:22.766107Z" },
> "cells" : [
> { "name" : "score", "value" : "Golf" },
> { "name" : "sport_type", "value" : "5" }
> ]
> }
> ]
> }
> ]
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> test.dir
> $ sstabledump test.dir/mb-1-big-Data.db
> ERROR 15:02:52 Cannot open /home/centos/test.dir/mb-1-big; partitioner
> org.apache.cassandra.dht.Murmur3Partitioner does not match system partitioner
> org.apache.cassandra.dht.LocalPartitioner. Note that the default partitioner
> starting with Cassandra 1.2 is Murmur3Partitioner, so you will need to edit
> that to match your old partitioner if upgrading.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12177) sstabledump fails if sstable path includes dot
[
https://issues.apache.org/jira/browse/CASSANDRA-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374559#comment-15374559
]
Keith Wansbrough commented on CASSANDRA-12177:
--
Apologies - this is version 3.7.0, which is the latest available from the
[download page|http://cassandra.apache.org/download/].
I tried this [3.8-SNAPSHOT|http://cassci.datastax.com/job/cassandra-3.8/]
(build #2, Jul 11, 2016 3:12 AM) and it does indeed seem to be fixed there.
Presumably that means it will be fixed in 3.9; will it also be fixed in the
next 3.0.x release?
Thanks!
> sstabledump fails if sstable path includes dot
> --
>
> Key: CASSANDRA-12177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12177
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Keith Wansbrough
>
> If there is a dot in the file path passed to sstabledump, it fails with an
> error {{partitioner org.apache.cassandra.dht.Murmur3Partitioner does not
> match system partitioner org.apache.cassandra.dht.LocalPartitioner.}}
> I can work around this by renaming the directory containing the file, but it
> seems like a bug. I expected the directory name to be irrelevant.
> Example (assumes you have a keyspace test containing a table called sport,
> but should repro with any keyspace/table):
> {code}
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> testdir
> $ sstabledump testdir/mb-1-big-Data.db
> [
> {
> "partition" : {
> "key" : [ "2" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-07-11T10:15:22.766107Z" },
> "cells" : [
> { "name" : "score", "value" : "Golf" },
> { "name" : "sport_type", "value" : "5" }
> ]
> }
> ]
> }
> ]
> $ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
> test.dir
> $ sstabledump test.dir/mb-1-big-Data.db
> ERROR 15:02:52 Cannot open /home/centos/test.dir/mb-1-big; partitioner
> org.apache.cassandra.dht.Murmur3Partitioner does not match system partitioner
> org.apache.cassandra.dht.LocalPartitioner. Note that the default partitioner
> starting with Cassandra 1.2 is Murmur3Partitioner, so you will need to edit
> that to match your old partitioner if upgrading.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12177) sstabledump fails if sstable path includes dot
Keith Wansbrough created CASSANDRA-12177:
Summary: sstabledump fails if sstable path includes dot
Key: CASSANDRA-12177
URL: https://issues.apache.org/jira/browse/CASSANDRA-12177
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Keith Wansbrough
If there is a dot in the file path passed to sstabledump, it fails with an
error {{partitioner org.apache.cassandra.dht.Murmur3Partitioner does not match
system partitioner org.apache.cassandra.dht.LocalPartitioner.}}
I can work around this by renaming the directory containing the file, but it
seems like a bug. I expected the directory name to be irrelevant.
Example (assumes you have a keyspace test containing a table called sport, but
should repro with any keyspace/table):
{code}
$ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
testdir
$ sstabledump testdir/mb-1-big-Data.db
[
{
"partition" : {
"key" : [ "2" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-07-11T10:15:22.766107Z" },
"cells" : [
{ "name" : "score", "value" : "Golf" },
{ "name" : "sport_type", "value" : "5" }
]
}
]
}
]
$ cp -a /var/lib/cassandra/data/test/sport-ebe76350474e11e6879fc5e30fbb0e96
test.dir
$ sstabledump test.dir/mb-1-big-Data.db
ERROR 15:02:52 Cannot open /home/centos/test.dir/mb-1-big; partitioner
org.apache.cassandra.dht.Murmur3Partitioner does not match system partitioner
org.apache.cassandra.dht.LocalPartitioner. Note that the default partitioner
starting with Cassandra 1.2 is Murmur3Partitioner, so you will need to edit
that to match your old partitioner if upgrading.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11194) materialized views - support explode() on collections
[
https://issues.apache.org/jira/browse/CASSANDRA-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251727#comment-15251727
]
Keith Wansbrough edited comment on CASSANDRA-11194 at 4/22/16 9:11 AM:
---
We would also find it very useful to be able to create a materialized view on a
set. The {{explode}} syntax looks good for this:
{code}
CREATE TABLE customers (
id text PRIMARY KEY,
data text,
phones frozen
);
CREATE MATERIALIZED VIEW customers_by_phone AS
SELECT explode(phones), id
FROM customers
WHERE phones IS NOT NULL;
{code}
We have a database of customers with an ID as primary key. Each customer has
zero or more phone numbers. We would like to be able to create a materialized
view so we can look up by phone number.
Our current schema uses a frozen set for this, but either frozen or unfrozen
would be fine.
Edit: PostgreSQL calls this function
[{{unnest}}|http://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE].
was (Author: kw217):
We would also find it very useful to be able to create a materialized view on a
set. The {{explode}} syntax looks good for this:
{code}
CREATE TABLE customers (
id text PRIMARY KEY,
data text,
phones frozen
);
CREATE MATERIALIZED VIEW customers_by_phone AS
SELECT explode(phones), id
FROM customers
WHERE phones IS NOT NULL;
{code}
We have a database of customers with an ID as primary key. Each customer has
zero or more phone numbers. We would like to be able to create a materialized
view so we can look up by phone number.
Our current schema uses a frozen set for this, but either frozen or unfrozen
would be fine.
> materialized views - support explode() on collections
> -
>
> Key: CASSANDRA-11194
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11194
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jon Haddad
>
> I'm working on a database design to model a product catalog. Products can
> belong to categories. Categories can belong to multiple sub categories
> (think about Amazon's complex taxonomies).
> My category table would look like this, giving me individual categories &
> their parents:
> {code}
> CREATE TABLE category (
> category_id uuid primary key,
> name text,
> parents set
> );
> {code}
> To get a list of all the children of a particular category, I need a table
> that looks like the following:
> {code}
> CREATE TABLE categories_by_parent (
> parent_id uuid,
> category_id uuid,
> name text,
> primary key (parent_id, category_id)
> );
> {code}
> The important thing to note here is that a single category can have multiple
> parents.
> I'd like to propose support for collections in materialized views via an
> explode() function that would create 1 row per item in the collection. For
> instance, I'll insert the following 3 rows (2 parents, 1 child) into the
> category table:
> {code}
> insert into category (category_id, name, parents) values
> (009fe0e1-5b09-4efc-a92d-c03720324a4f, 'Parent', null);
> insert into category (category_id, name, parents) values
> (1f2914de-0adf-4afc-b7ad-ddd8dc876ab1, 'Parent2', null);
> insert into category (category_id, name, parents) values
> (1f93bc07-9874-42a5-a7d1-b741dc9c509c, 'Child',
> {009fe0e1-5b09-4efc-a92d-c03720324a4f, 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1
> });
> cqlsh:test> select * from category;
> category_id | name| parents
> --+-+--
> 009fe0e1-5b09-4efc-a92d-c03720324a4f | Parent |
> null
> 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1 | Parent2 |
> null
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> {009fe0e1-5b09-4efc-a92d-c03720324a4f, 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1}
> (3 rows)
> {code}
> Given the following CQL to select the child category, utilizing an explode
> function, I would expect to get back 2 rows, 1 for each parent:
> {code}
> select category_id, name, explode(parents) as parent_id from category where
> category_id = 1f93bc07-9874-42a5-a7d1-b741dc9c509c;
> category_id | name | parent_id
> --+---+--
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> 009fe0e1-5b09-4efc-a92d-c03720324a4f
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1
> (2 rows)
> {code}
> This functionality would ideally apply to materialized views, since the
> ability to control partitioning here would allow us to efficiently query our
[jira] [Commented] (CASSANDRA-11194) materialized views - support explode() on collections
[
https://issues.apache.org/jira/browse/CASSANDRA-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251727#comment-15251727
]
Keith Wansbrough commented on CASSANDRA-11194:
--
We would also find it very useful to be able to create a materialized view on a
set. The {{explode}} syntax looks good for this:
{code}
CREATE TABLE customers (
id text PRIMARY KEY,
data text,
phones frozen
);
CREATE MATERIALIZED VIEW customers_by_phone AS
SELECT explode(phones), id
FROM customers
WHERE phones IS NOT NULL;
{code}
We have a database of customers with an ID as primary key. Each customer has
zero or more phone numbers. We would like to be able to create a materialized
view so we can look up by phone number.
Our current schema uses a frozen set for this, but either frozen or unfrozen
would be fine.
> materialized views - support explode() on collections
> -
>
> Key: CASSANDRA-11194
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11194
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jon Haddad
>
> I'm working on a database design to model a product catalog. Products can
> belong to categories. Categories can belong to multiple sub categories
> (think about Amazon's complex taxonomies).
> My category table would look like this, giving me individual categories &
> their parents:
> {code}
> CREATE TABLE category (
> category_id uuid primary key,
> name text,
> parents set
> );
> {code}
> To get a list of all the children of a particular category, I need a table
> that looks like the following:
> {code}
> CREATE TABLE categories_by_parent (
> parent_id uuid,
> category_id uuid,
> name text,
> primary key (parent_id, category_id)
> );
> {code}
> The important thing to note here is that a single category can have multiple
> parents.
> I'd like to propose support for collections in materialized views via an
> explode() function that would create 1 row per item in the collection. For
> instance, I'll insert the following 3 rows (2 parents, 1 child) into the
> category table:
> {code}
> insert into category (category_id, name, parents) values
> (009fe0e1-5b09-4efc-a92d-c03720324a4f, 'Parent', null);
> insert into category (category_id, name, parents) values
> (1f2914de-0adf-4afc-b7ad-ddd8dc876ab1, 'Parent2', null);
> insert into category (category_id, name, parents) values
> (1f93bc07-9874-42a5-a7d1-b741dc9c509c, 'Child',
> {009fe0e1-5b09-4efc-a92d-c03720324a4f, 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1
> });
> cqlsh:test> select * from category;
> category_id | name| parents
> --+-+--
> 009fe0e1-5b09-4efc-a92d-c03720324a4f | Parent |
> null
> 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1 | Parent2 |
> null
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> {009fe0e1-5b09-4efc-a92d-c03720324a4f, 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1}
> (3 rows)
> {code}
> Given the following CQL to select the child category, utilizing an explode
> function, I would expect to get back 2 rows, 1 for each parent:
> {code}
> select category_id, name, explode(parents) as parent_id from category where
> category_id = 1f93bc07-9874-42a5-a7d1-b741dc9c509c;
> category_id | name | parent_id
> --+---+--
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> 009fe0e1-5b09-4efc-a92d-c03720324a4f
> 1f93bc07-9874-42a5-a7d1-b741dc9c509c | Child |
> 1f2914de-0adf-4afc-b7ad-ddd8dc876ab1
> (2 rows)
> {code}
> This functionality would ideally apply to materialized views, since the
> ability to control partitioning here would allow us to efficiently query our
> MV for all categories belonging to a parent in a complex taxonomy.
> {code}
> CREATE MATERIALIZED VIEW categories_by_parent as
> SELECT explode(parents) as parent_id,
> category_id, name FROM category WHERE parents IS NOT NULL
> {code}
> The explode() function is available in Spark Dataframes and my proposed
> function has the same behavior:
> http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.functions.explode
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
