[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100943#comment-16100943
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Sure [~jasonstack] . If 11500 addresses both the problems ( timestamp issues +
view definition/metadata fix ) , please go ahead and mark this issue as
duplicate. Thanks !
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Dattu Koneru updated CASSANDRA-13657:
-
Labels: materializedviews ttl (was: )
Status: Patch Available (was: In Progress)
Patch for trunk. Will do patches for other branches if this looks okay.
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>Assignee: Krishna Dattu Koneru
> Labels: materializedviews, ttl
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> DELETE FROM t WHERE k=1;
> cqlsh:test> SELECT * from t; SELECT * from mv;
> k | a | b
> ---+---+---
> (0 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that view-entry is now orphanized.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075985#comment-16075985
]
Krishna Dattu Koneru commented on CASSANDRA-13657:
--
This happens because of how the row expiry works . Row exists as long as cells
(even non-pk) are live.
For example, below table (MV is also a table, just maintained by cassandra
automatically.)
{code}
CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k,a));
insert into t (k,a) VALUES (1,1) using ttl 10;
update t using ttl 60 set b=1 where k=1 and a=1;
--- wait for 10 seconds.
select k,a,b,ttl(b) from t;
k | a | b | ttl(b)
---+---+---+
1 | 1 | 1 | 45
(1 rows)
{code}
Row does not expire as {{b}} is still alive.
This causes problem for materialized views.column from base table expires and
view row still exists. View row should expire because, base row will no longer
match the mandatory {{IS NOT NULL}} filter on PK columns.
I made a patch to make sure that non-primary key columns don't outlive view pk
columns.
||Patch||Circleci||
|[trunk|https://github.com/apache/cassandra/compare/trunk...krishna-koneru:CASSANDRA-13657-trunk]|[test|https://circleci.com/gh/krishna-koneru/cassandra/24]|
Comments appreciated.
( I am not sure if there is a way to fix already orphanized view entries )
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>Assignee: Krishna Dattu Koneru
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> DELETE FROM t WHERE k=1;
> cqlsh:test> SELECT * from t; SELECT * from mv;
> k | a | b
> ---+---+---
> (0 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that view-entry is now orphanized.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060483#comment-16060483
]
Krishna Dattu Koneru edited comment on CASSANDRA-13464 at 7/3/17 7:51 AM:
--
EDIT : This patch not valid.new patch in later comment.
-I have triggered tests for the patch on CircleCI .-
||Patch||CircleCI||
|[-3.0- |
https://github.com/apache/cassandra/compare/cassandra-3.0...krishna-koneru:CASSANDRA-13464-3.0]
|[-circleci-3.0-|https://circleci.com/gh/krishna-koneru/cassandra/4]|
|[-3.11- |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:CASSANDRA-13464-3.11]|
[-circleci-3.11-|https://circleci.com/gh/krishna-koneru/cassandra/6]|
|[-trunk- |
https://github.com/apache/cassandra/compare/trunk...krishna-koneru:CASSANDRA-13464-trunk]
|[-circleci-trunk-|https://circleci.com/gh/krishna-koneru/cassandra/5]|
was (Author: krishna.koneru):
I have triggered tests for the patch on CircleCI .
||Patch||CircleCI||
|[3.0 |
https://github.com/apache/cassandra/compare/cassandra-3.0...krishna-koneru:CASSANDRA-13464-3.0]
|[circleci-3.0|https://circleci.com/gh/krishna-koneru/cassandra/4]|
|[3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:CASSANDRA-13464-3.11]|
[circleci-3.11|https://circleci.com/gh/krishna-koneru/cassandra/6]|
|[trunk |
https://github.com/apache/cassandra/compare/trunk...krishna-koneru:CASSANDRA-13464-trunk]
|[circleci-trunk|https://circleci.com/gh/krishna-koneru/cassandra/5]|
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
>
[jira] [Commented] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16072022#comment-16072022
]
Krishna Dattu Koneru commented on CASSANDRA-13464:
--
New patch that does not allow token() function to be used in WHERE clause of
CREATE MV
||Patch||CircleCI||
|[3.0 |
https://github.com/apache/cassandra/compare/cassandra-3.0...krishna-koneru:CASSANDRA-13464-3.0]
|[circleci-3.0|https://circleci.com/gh/krishna-koneru/cassandra/18]|
|[3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:CASSANDRA-13464-3.11]|
[circleci-3.11|https://circleci.com/gh/krishna-koneru/cassandra/16]|
|[trunk |
https://github.com/apache/cassandra/compare/trunk...krishna-koneru:CASSANDRA-13464-trunk]
|[circleci-trunk|https://circleci.com/gh/krishna-koneru/cassandra/17]|
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
>
[jira] [Updated] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Dattu Koneru updated CASSANDRA-13464:
-
Status: Patch Available (was: Open)
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
>
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071984#comment-16071984
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Hi [~jasonstack] ,
Here is another go at patch which addresses {{1.Missing Update}} .With Current
implementation of ColumFamily all columns in metadata will be written to disk.
So,I had to add another class member in {{ViewDefinition.java}} to keep track
of all columns used in the view definition.
And regarding your comment on {{2. incorrect non-pk tombstones}} ,
https://issues.apache.org/jira/browse/CASSANDRA-11500 is already open for this
issue.
||patch||test||
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547]|[circleci|https://circleci.com/gh/krishna-koneru/cassandra/13]|
Thanks !
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail:
[jira] [Comment Edited] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067734#comment-16067734
]
Krishna Dattu Koneru edited comment on CASSANDRA-13547 at 6/29/17 5:31 AM:
---
Thanks [~jasonstack] !
I will try to rework on {{1. Missing Update}} to address your comment.
About
{quote}
Using the greater timestamp from view's columns(pk+non-pk) in base row will
later shadow entire row in view if there is a normal column in base as primary
key in view.
{quote}
This looks like a nasty problem. This patch does not cause this. This is a
existing behaviour that any updates to view's pk columns will make old row
marked as tombstone (at highest timestamp of all columns in base row) and will
create a new row with updated pk.
EDIT : I just saw that https://issues.apache.org/jira/browse/CASSANDRA-11500
exists because of this exact problem.
See view row timestamps in the below example :
Existing behaviour without patch:
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=0 ts=5], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=0 ts=5], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
mv_test1 :
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@30 Row[info=[ts=2] ]: 1 | [c=0 ts=5], [d=1 ts=0]
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
With this patch :
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
view : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
view :
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because
of this patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because of this
patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@30 Row[info=[ts=5] ]: 1 | [c=0 ts=5], [d=1 ts=0] /*-- row ts=5 because of
this patch */
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
I am not sure yet how to fix this issue ... given that if live row and
tombstone have same timestamp , tombstone wins.
Another problem is that all view deletes are marked as shadowable. But then
that is a different problem and I belive it is being fixed in
https://issues.apache.org/jira/browse/CASSANDRA-13409 .
was (Author: krishna.koneru):
Thanks [~jasonstack] !
I will try to rework on {{1. Missing Update}} to address your comment.
About
{quote}
Using the greater timestamp from view's columns(pk+non-pk) in base row will
later shadow entire row in view if there is a normal column in base as primary
key in view.
{quote}
This looks like a nasty problem. This patch does not cause this. This is a
existing behaviour that any updates to view's pk columns will make old row
marked as tombstone (at highest timestamp of all columns in base row) and will
create a new row with updated pk.
See view row timestamps in the below example :
Existing behaviour without patch:
{code}
INSERT INTO test
[jira] [Comment Edited] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067734#comment-16067734
]
Krishna Dattu Koneru edited comment on CASSANDRA-13547 at 6/29/17 4:56 AM:
---
Thanks [~jasonstack] !
I will try to rework on {{1. Missing Update}} to address your comment.
About
{quote}
Using the greater timestamp from view's columns(pk+non-pk) in base row will
later shadow entire row in view if there is a normal column in base as primary
key in view.
{quote}
This looks like a nasty problem. This patch does not cause this. This is a
existing behaviour that any updates to view's pk columns will make old row
marked as tombstone (at highest timestamp of all columns in base row) and will
create a new row with updated pk.
See view row timestamps in the below example :
Existing behaviour without patch:
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=0 ts=5], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=0 ts=5], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
mv_test1 :
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@30 Row[info=[ts=2] ]: 1 | [c=0 ts=5], [d=1 ts=0]
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
With this patch :
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
view : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
view :
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because
of this patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because of this
patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@30 Row[info=[ts=5] ]: 1 | [c=0 ts=5], [d=1 ts=0] /*-- row ts=5 because of
this patch */
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
I am not sure yet how to fix this issue ... given that if live row and
tombstone have same timestamp , tombstone wins.
Another problem is that all view deletes are marked as shadowable. But then
that is a different problem and I belive it is being fixed in
https://issues.apache.org/jira/browse/CASSANDRA-13409 .
was (Author: krishna.koneru):
Thanks [~jasonstack] !
I will try to rework on {{1. Missing Update}} to address your comment.
About
{quote}
Using the greater timestamp from view's columns(pk+non-pk) in base row will
later shadow entire row in view if there is a normal column in base as primary
key in view.
{quote}
This looks like a nasty problem. This patch does not cause this. This is a
existing behaviour that any updates to view's pk columns will make old row
marked as tombstone (at highest timestamp of all columns in base row) and will
create a new row with updated pk.
See view row timestamps in the below example :
Existing behaviour without patch:
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0],
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067734#comment-16067734
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Thanks [~jasonstack] !
I will try to rework on {{1. Missing Update}} to address your comment.
About
{quote}
Using the greater timestamp from view's columns(pk+non-pk) in base row will
later shadow entire row in view if there is a normal column in base as primary
key in view.
{quote}
This looks like a nasty problem. This patch does not cause this. This is a
existing behaviour that any updates to view's pk columns will make old row
marked as tombstone (at highest timestamp of all columns in base row) and will
create a new row with updated pk.
See view row timestamps in the below example :
Existing behaviour without patch:
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=0 ts=5], [d=1 ts=0]
mv_test1 : [1]@0 Row[info=[ts=0] ]: 1 | [c=0 ts=5], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
test : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
mv_test1 :
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=1] ]: 0 | [c=0 ts=5], [d=1 ts=0]
[1]@39 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@30 Row[info=[ts=2] ]: 1 | [c=0 ts=5], [d=1 ts=0]
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707897(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498707556(shadowable) ]: 1 |
{code}
With this patch :
{code}
INSERT INTO test (a, b, c, d) VALUES (1, 1, 1, 1) using timestamp 0;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=0], [c=1 ts=0], [d=1 ts=0]
view : [1]@0 Row[info=[ts=0] ]: 1 | [c=1 ts=0], [d=1 ts=0]
{code}
{code}
UPDATE test using timestamp 5 set c = 0 WHERE a=1;
UPDATE test using timestamp 1 set b = 0 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=0 ts=1], [c=0 ts=5], [d=1 ts=0]
view :
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because
of this patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
{code}
UPDATE test using timestamp 2 set b = 1 WHERE a=1;
table : [1]@0 Row[info=[ts=0] ]: | [b=1 ts=2], [c=0 ts=5], [d=1 ts=0]
View (before compaction)
[1]@0 Row[info=[ts=5] ]: 0 | [c=0 ts=5], [d=1 ts=0] /* row ts=5 because of this
patch */
[1]@38 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@30 Row[info=[ts=5] ]: 1 | [c=0 ts=5], [d=1 ts=0] /*-- row ts=5 because of
this patch */
View (after compaction)
[1]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498709103(shadowable) ]: 0 |
[1]@31 Row[info=[ts=-9223372036854775808] del=deletedAt=5,
localDeletion=1498708886(shadowable) ]: 1 |
{code}
I am not sure yet how to fix this issue ... given that if live row and
tombstone have same timestamp , tombstone wins.
Another problem is that these tombstones should not be marked as shadowable.
But then that is a different problem and I belive it is being fixed in
https://issues.apache.org/jira/browse/CASSANDRA-13409 .
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>
[jira] [Commented] (CASSANDRA-13565) Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048 causes exception
[ https://issues.apache.org/jira/browse/CASSANDRA-13565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065875#comment-16065875 ] Krishna Dattu Koneru commented on CASSANDRA-13565: -- This can be closed as duplicate of https://issues.apache.org/jira/browse/CASSANDRA-13622 , which is opened to fix similar possible overflows. May be [~KurtG] can comment. > Materialized view usage of commit logs requires large mutation but > commitlog_segment_size_in_mb=2048 causes exception > - > > Key: CASSANDRA-13565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13565 > Project: Cassandra > Issue Type: Bug > Components: Configuration, Materialized Views, Streaming and > Messaging > Environment: Cassandra 3.9.0, Windows >Reporter: Tania S Engel > Attachments: CQLforTable.png > > > We will be upgrading to 3.10 for CASSANDRA-11670. However, there is another > scenario (not applyunsafe during JOIN) which leads to : > java.lang.IllegalArgumentException: Mutation of 525.847MiB is too large > for the maximum size of 512.000MiB > at > org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:262) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.Mutation.apply(Mutation.java:227) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:147) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:797) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.view.ViewBuilder.buildKey(ViewBuilder.java:96) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.view.ViewBuilder.run(ViewBuilder.java:165) > ~[apache-cassandra-3.9.0.jar:3.9.0] > at > org.apache.cassandra.db.compaction.CompactionManager$14.run(CompactionManager.java:1591) > [apache-cassandra-3.9.0.jar:3.9.0] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_66] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_66] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_66] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_66] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] > Due to the relationship of max_mutation_size_in_kb and > commitlog_segment_size_in_mb, we increased commitlog_segment_size_in_mb and > left Cassandra to calculate max_mutation_size_in_kb as half the size > commitlog_segment_size_in_mb * 1024. > However, we have found that if we set commitlog_segment_size_in_mb=2048 we > get an exception upon starting Cassandra, when it is creating a new commit > log. > ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 > JVMStabilityInspector.java:82 - Exiting due to error while processing commit > log during initialization. > org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was > made to move the file pointer before the beginning of the file > Perhaps the index you are using is not big enough and it goes negative. > Is the relationship between max_mutation_size_in_kb and > commitlog_segment_size_in_mb important to preserve? In our limited stress > test we are finding mutation size already over 512mb and we expect more data > in our sstables and associated materialized views. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Dattu Koneru updated CASSANDRA-13464:
-
Status: Open (was: Patch Available)
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
>
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064031#comment-16064031
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Hey [~jasonstack] I saw that you are working on related issue
https://issues.apache.org/jira/browse/CASSANDRA-13127 . Could you have a look
at this patch as well ?
||Patch||testall||
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547]|[testall|https://circleci.com/gh/kgreav/cassandra/5]|
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062543#comment-16062543
]
Krishna Dattu Koneru commented on CASSANDRA-13464:
--
Thanks [~zznate] and [~brstgt] . So the recommended fix would be to restrict
CQL syntax to not use token functions in where clause and throw a proper error
message (like its currently done if functions are used in select clause ) ?
{code}
cqlsh:test> CREATE MATERIALIZED VIEW mv2 as select token(a),b,c from t1 where a
is not null and b is not null and c=1 primary key(a,b);
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
use function when defining a materialized view"
{code}
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
>
[jira] [Updated] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Dattu Koneru updated CASSANDRA-13464:
-
Labels: materializedviews (was: )
Reproduced In: 3.0.13, 3.11.x, 4.x (was: 3.0.13)
Status: Patch Available (was: Open)
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
> Labels: materializedviews
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
> [apache-cassandra-3.0.13.jar:3.0.13]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
>
[jira] [Commented] (CASSANDRA-13464) Failed to create Materialized view with a specific token range
[
https://issues.apache.org/jira/browse/CASSANDRA-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060483#comment-16060483
]
Krishna Dattu Koneru commented on CASSANDRA-13464:
--
I have triggered tests for the patch on CircleCI .
||Patch||CircleCI||
|[3.0 |
https://github.com/apache/cassandra/compare/cassandra-3.0...krishna-koneru:CASSANDRA-13464-3.0]
|[circleci-3.0|https://circleci.com/gh/krishna-koneru/cassandra/4]|
|[3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:CASSANDRA-13464-3.11]|
[circleci-3.11|https://circleci.com/gh/krishna-koneru/cassandra/6]|
|[trunk |
https://github.com/apache/cassandra/compare/trunk...krishna-koneru:CASSANDRA-13464-trunk]
|[circleci-trunk|https://circleci.com/gh/krishna-koneru/cassandra/5]|
> Failed to create Materialized view with a specific token range
> --
>
> Key: CASSANDRA-13464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13464
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Natsumi Kojima
>Assignee: Krishna Dattu Koneru
>Priority: Minor
>
> Failed to create Materialized view with a specific token range.
> Example :
> {code:java}
> $ ccm create "MaterializedView" -v 3.0.13
> $ ccm populate -n 3
> $ ccm start
> $ ccm status
> Cluster: 'MaterializedView'
> ---
> node1: UP
> node3: UP
> node2: UP
> $ccm node1 cqlsh
> Connected to MaterializedView at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.13 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> cqlsh> CREATE TABLE test.test ( id text PRIMARY KEY , value1 text , value2
> text, value3 text);
> $ccm node1 ring test
> Datacenter: datacenter1
> ==
> AddressRackStatus State LoadOwns
> Token
>
> 3074457345618258602
> 127.0.0.1 rack1 Up Normal 64.86 KB100.00%
> -9223372036854775808
> 127.0.0.2 rack1 Up Normal 86.49 KB100.00%
> -3074457345618258603
> 127.0.0.3 rack1 Up Normal 89.04 KB100.00%
> 3074457345618258602
> $ ccm node1 cqlsh
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('aaa',
> 'aaa', 'aaa' ,'aaa');
> cqlsh> INSERT INTO test.test (id, value1 , value2, value3 ) VALUES ('bbb',
> 'bbb', 'bbb' ,'bbb');
> cqlsh> SELECT token(id),id,value1 FROM test.test;
> system.token(id) | id | value1
> --+-+
> -4737872923231490581 | aaa |aaa
> -3071845237020185195 | bbb |bbb
> (2 rows)
> cqlsh> CREATE MATERIALIZED VIEW test.test_view AS SELECT value1, id FROM
> test.test WHERE id IS NOT NULL AND value1 IS NOT NULL AND TOKEN(id) >
> -9223372036854775808 AND TOKEN(id) < -3074457345618258603 PRIMARY KEY(value1,
> id) WITH CLUSTERING ORDER BY (id ASC);
> ServerError: java.lang.ClassCastException:
> org.apache.cassandra.cql3.TokenRelation cannot be cast to
> org.apache.cassandra.cql3.SingleColumnRelation
> {code}
> Stacktrace :
> {code:java}
> INFO [MigrationStage:1] 2017-04-19 18:32:48,131 ColumnFamilyStore.java:389 -
> Initializing test.test
> WARN [SharedPool-Worker-1] 2017-04-19 18:44:07,263 FBUtilities.java:337 -
> Trigger directory doesn't exist, please create it and try again.
> ERROR [SharedPool-Worker-1] 2017-04-19 18:46:10,072 QueryMessage.java:128 -
> Unexpected error during query
> java.lang.ClassCastException: org.apache.cassandra.cql3.TokenRelation cannot
> be cast to org.apache.cassandra.cql3.SingleColumnRelation
> at
> org.apache.cassandra.db.view.View.relationsToWhereClause(View.java:275)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.CreateViewStatement.announceMigration(CreateViewStatement.java:219)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[apache-cassandra-3.0.13.jar:3.0.13]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
>
[jira] [Comment Edited] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040527#comment-16040527
]
Krishna Dattu Koneru edited comment on CASSANDRA-13547 at 6/23/17 3:26 AM:
---
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;
id | name | enabled | foo
+--+-+-
1 | One |True | Bar
(1 rows)
{code}
{code:title=Problem 1 - Missing Updates|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv1;
name | id | foo
--++-
(0 rows)
{code}
Logic in ViewUpdateGenerator.java does not update the view row if updated
column is not denormalized in the view.
in the above case,{{enabled}} is not denormalized and so update has not
propagated to the view.View metadata only has pk columns + columns in select
statement of create view.
Now that filtering on non-pk columns is supported , we have to make sure that
all non-primary key columns that have filters are denormalized.With this we can
also make sure that {{ALTER TABLE}} does not drop a column that is used in view.
(delete does not do this check because it does not have to. This is the reason
row delete worked when {{enabled}} is set to false.)
{code:title=Problem 2 - incorrect non-pk tombstones|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
This happens because of the way liveliness/deletion info is computed in the
view.
{{computeTimestampForEntryDeletion())}} method takes the biggest timestamp of
all the columns (including non-pk) in the row and uses it in Deletion info when
deleting.
But,when inserting/updating, {{computeLivenessInfoForEntry()}} uses the biggest
timestamp of the primary keys for liveliness info.
This causes non-pk columns to be treated as deleted because view tombstones
have higher timestamp than live cell from base row.
I have uploaded a [patch for 3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547]branch
which fixes above two issues.
I will make patches for other branches if this patch looks okay.
Comments appreciated !
was (Author: krishna.koneru):
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;
id | name | enabled | foo
+--+-+-
1 | One |True | Bar
(1 rows)
{code}
{code:title=Problem 1 - Missing Updates|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv1;
name | id | foo
--++-
(0 rows)
{code}
Logic in ViewUpdateGenerator.java does not update the view row if updated
column is not denormalized in the view.
in the above case,{{enabled}} is not denormalized and so update has not
propagated to the view.View metadata only has pk columns + columns in select
statement of create view.
Now that filtering on non-pk columns is supported , we have to make sure that
all non-primary key columns that have filters are denormalized.
(delete does not do this check because it does not have to. This is the reason
row delete worked when {{enabled}} is set to false.)
{code:title=Problem 2 - incorrect non-pk tombstones|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
This happens because of the way liveliness/deletion info is computed in the
view.
{{computeTimestampForEntryDeletion())}} method takes the biggest timestamp of
all the columns (including non-pk) in the row and uses it in Deletion info when
deleting.
But,when inserting/updating, {{computeLivenessInfoForEntry()}} uses the biggest
timestamp of the primary keys for liveliness info.
This causes non-pk columns to be treated as deleted because view tombstones
have higher timestamp than live cell from base row.
I have uploaded a [patch for 3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547]branch
which fixes above two issues.
I will make patches for other branches if this patch looks okay.
Comments appreciated !
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the
[jira] [Comment Edited] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040527#comment-16040527
]
Krishna Dattu Koneru edited comment on CASSANDRA-13547 at 6/21/17 6:22 AM:
---
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;
id | name | enabled | foo
+--+-+-
1 | One |True | Bar
(1 rows)
{code}
{code:title=Problem 1 - Missing Updates|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv1;
name | id | foo
--++-
(0 rows)
{code}
Logic in ViewUpdateGenerator.java does not update the view row if updated
column is not denormalized in the view.
in the above case,{{enabled}} is not denormalized and so update has not
propagated to the view.View metadata only has pk columns + columns in select
statement of create view.
Now that filtering on non-pk columns is supported , we have to make sure that
all non-primary key columns that have filters are denormalized.
(delete does not do this check because it does not have to. This is the reason
row delete worked when {{enabled}} is set to false.)
{code:title=Problem 2 - incorrect non-pk tombstones|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
This happens because of the way liveliness/deletion info is computed in the
view.
{{computeTimestampForEntryDeletion())}} method takes the biggest timestamp of
all the columns (including non-pk) in the row and uses it in Deletion info when
deleting.
But,when inserting/updating, {{computeLivenessInfoForEntry()}} uses the biggest
timestamp of the primary keys for liveliness info.
This causes non-pk columns to be treated as deleted because view tombstones
have higher timestamp than live cell from base row.
I have uploaded a [patch for 3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547]branch
which fixes above two issues.
I will make patches for other branches if this patch looks okay.
Comments appreciated !
was (Author: krishna.koneru):
I have uploaded a patch (with unittest) for problem 1 in above comment @
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547
below problem addressed by this patch (from the description)
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;
id | name | enabled | foo
+--+-+-
1 | One |True | Bar
(1 rows)
cqlsh> SELECT * FROM test.table1_mv1;
name | id | foo
--++-
{code}
This happens because view metadata does not have column "enabled" .
The decision "should this base table update, update view as well?" is made by
looking at view metadata (among other things).
Due to this, some updates to base table do not result in updates to MV even if
update qualifies .
This patch disallows usage of any non PK columns in WHERE clause if they are
not in SELECT list.
Comments are appreciated !
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
>
[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space
[ https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056710#comment-16056710 ] Krishna Dattu Koneru commented on CASSANDRA-13068: -- Thanks [~krummas] for review and running tests! Here are latest branches with the patch. * [3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...krishna-koneru:cassandra-3.0-13068] * [3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13068] * [trunk|https://github.com/apache/cassandra/compare/trunk...krishna-koneru:cassandra-trunk-13068] > Fully expired sstable not dropped when running out of disk space > > > Key: CASSANDRA-13068 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13068 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Marcus Eriksson >Assignee: Lerh Chuan Low > Labels: lhf > Fix For: 3.0.x, 3.11.x, 4.x > > > If a fully expired sstable is larger than the remaining disk space we won't > run the compaction that can drop the sstable (ie, in our disk space check > should not include the fully expired sstables) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space
[
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047380#comment-16047380
]
Krishna Dattu Koneru commented on CASSANDRA-13068:
--
Hi [~krummas] , I work with [~Lerh Low] . He wanted me to continue working on
this. I have uploaded patches on github for [trunk|
https://github.com/apache/cassandra/compare/trunk...krishna-koneru:cassandra-trunk-13068
] and [3.11 |
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13068]
branches .
I had to make few more changes to what was initially implemented by [~Lerh Low]
to address your comments.
- renamed {{CompactionTask#checkAvailableDiskSpace()}} to
{{CompactionTask#buildCompactionCandidatesForAvailableDiskSpace()}} as the
method checks and might also modify the list of SSTables that will be compacted.
- {{buildCompactionCandidatesForAvailableDiskSpace()}} does not throw exception
if there is no diskspace to compact non-expired SSTables when there are fully
expired SSTables.This means, instead of throwing exception, we try to compact
fully expired SSTables which might free up some space to allow further
compactions.
> Fully expired sstable not dropped when running out of disk space
>
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't
> run the compaction that can drop the sstable (ie, in our disk space check
> should not include the fully expired sstables)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Dattu Koneru updated CASSANDRA-13547:
-
Status: Patch Available (was: In Progress)
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040527#comment-16040527
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
I have uploaded a patch (with unittest) for problem 1 in above comment @
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547
below problem addressed by this patch (from the description)
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;
id | name | enabled | foo
+--+-+-
1 | One |True | Bar
(1 rows)
cqlsh> SELECT * FROM test.table1_mv1;
name | id | foo
--++-
{code}
This happens because view metadata does not have column "enabled" .
The decision "should this base table update, update view as well?" is made by
looking at view metadata (among other things).
Due to this, some updates to base table do not result in updates to MV even if
update qualifies .
This patch disallows usage of any non PK columns in WHERE clause if they are
not in SELECT list.
Comments are appreciated !
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One |
[jira] [Comment Edited] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036632#comment-16036632
]
Krishna Dattu Koneru edited comment on CASSANDRA-13547 at 6/5/17 7:30 AM:
--
Seems related to https://issues.apache.org/jira/browse/CASSANDRA-13409 which
has a fix
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11].
I ran above test on [commit |
https://github.com/jasonstack/cassandra/commit/26f3e7b96001f6148eab7cc3589b2d496c5830c5]
from this branch and it still fails.
I believe the reason for this
{code:title=tombstone|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
is explained
[here|https://issues.apache.org/jira/browse/CASSANDRA-10261?focusedCommentId=14731266=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14731266]
and
[here|https://issues.apache.org/jira/browse/CASSANDRA-9664?focusedCommentId=14724150=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14724150].
was (Author: krishna.koneru):
Seems related to https://issues.apache.org/jira/browse/CASSANDRA-13409 which
has a fix
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11].
I ran above test on [commit |
https://github.com/jasonstack/cassandra/commit/26f3e7b96001f6148eab7cc3589b2d496c5830c5]
from this and it still fails.
I believe the reason for this
{code:title=tombstone|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
is explained
[here|https://issues.apache.org/jira/browse/CASSANDRA-10261?focusedCommentId=14731266=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14731266]
and
[here|https://issues.apache.org/jira/browse/CASSANDRA-9664?focusedCommentId=14724150=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14724150].
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
>
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036632#comment-16036632
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Seems related to https://issues.apache.org/jira/browse/CASSANDRA-13409 which
has a fix
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11].
I ran above test on [commit |
https://github.com/jasonstack/cassandra/commit/26f3e7b96001f6148eab7cc3589b2d496c5830c5]
from this and it still fails.
I believe the reason for this
{code:title=tombstone|borderStyle=solid}
cqlsh> SELECT * FROM test.table1_mv2;
name | id | enabled | foo
--++-+--
One | 1 |True | null
(1 rows)
{code}
is explained
[here|https://issues.apache.org/jira/browse/CASSANDRA-10261?focusedCommentId=14731266=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14731266]
and
[here|https://issues.apache.org/jira/browse/CASSANDRA-9664?focusedCommentId=14724150=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14724150].
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This
[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036519#comment-16036519
]
Krishna Dattu Koneru commented on CASSANDRA-13547:
--
Hi , I am new here and would like to work on this bug . Any help is appreciated.
I see to two problems here :
1. MV's row is not updated if the updated column in base table is not in the
select list of the view (even if it is in where clause)
2. columns that are not in [updated columns + view PK columns] are not updated
or do not have any data after updates.
For the first problem I see these two possible solutions :
1 - add a restriction to CQL syntax that all the columns that are used in where
clause must be in the select list of MV
2 - Look in where clause (of MV) too to check if update to the base table
column should update view or not.
For the second problem, can anyone point the direction in which I should be
looking ?
It seems that the mutations are built correctly but some how they are not
"applied" correctly.
> Filtered materialized views missing data
>
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Priority: Blocker
> Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
> WITH REPLICATION = {
>'class' : 'SimpleStrategy',
>'replication_factor' : 1
> };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL
> AND name IS NOT NULL
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One',
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> One | 1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> One | 1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One | False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should
> include the record in both materialzed views, however only one view
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are
> not updated. You can see that the foo column has a null value in table1_mv2.
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
> id | name | enabled | foo
> +--+-+-
> 1 | One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
> name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
> name | id | enabled | foo
> --++-+--
> One | 1 |True | null
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
