[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091325#comment-16091325
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/18/17 9:16 AM:
---
[~doanduyhai] yes, it's considered.
There are 3 cases:
TTLed base column as view's PK or view's filter condition will wipe entire view
row.
TTLed base column as selected in view will work as it is
TTLed base column not selected in view, if there is no other live cells or
primary key of view, view row is gone.
please feel free to point out any cases, I will summarize the solution into
CASSANDRA-11500
was (Author: jasonstack):
[~doanduyhai] yes, it's considered.
There are 3 cases:
TTLed base column as view's PK or view's filter condition will wipe entire view
row.
TTLed base column as selected in view will work as it is
TTLed base column not selected in view, if there is no other live cells or
primary key of view, view row is gone.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090024#comment-16090024
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/17/17 4:33 PM:
---
For now, base table with view doesn't support drop columns. but there is a
ticket open to support alter MV.
More precisely only for base column/cell that has timestamp/ttl and not used in
MV. (In cases of insertion, no column timestamp/ttl)
Good to carry column->(timestamp/ttl) mapping.
was (Author: jasonstack):
For now, base table with view doesn't support drop columns. but there is a
ticket open to support alter MV.
More precisely only for base column/cell that has timestamp and not used in MV.
(In cases of insertion, no column timestamp)
Good to carry column->timestamp mapping.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083764#comment-16083764
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/17/17 8:28 AM:
---
{{when updating base's columns which are not used in view.}} is about whether
we should consider `Update semantic the same as Insert`. (for now it's not the
same. update-statement has no primary key liveness info. pk's liveness of
update-statement is depending on liveness of normal columns)
eg. current semantic/behavior:
{quote}
base table: create table ks.base (p int, c int, v int, primary key (p, c))
view table: select p,c from ks.base ... primary key (c, p)
an update query on base table normal column {{v}} will not generate any rows..
a delete query on base table normal column {{v}} will remove the base row.
{quote}
imo, to avoid user confusion, it's better to keep view matched with base
regardless semantic of update..
[~tjake][~slebresne] what do you think?
was (Author: jasonstack):
{{when updating base's columns which are not used in view.}} is about whether
we should consider `Update semantic the same as Insert`. (for now it's not the
same. update-statement has no primary key liveness info. pk's liveness of
update-statement is depending on liveness of normal columns)
eg. current semantic/behavior:
{quote}
base table: create table ks.base (p int, c int, v int, primary key (p, c))
view table: select p,c from ks.base ... primary key (c, p)
an update query on base table normal column {{v}} will not generate any rows..
a delete query on base table normal column {{v}} will remove the base row.
{quote}
imo, to avoid user confusion, it's better to keep view matched with base
regardless semantic of update..(the current patch is not general enough to
cover all such cases yet)
[~tjake][~slebresne] what do you think?
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083764#comment-16083764
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/12/17 10:24 AM:
{{when updating base's columns which are not used in view.}} is about whether
we should consider `Update semantic the same as Insert`. (for now it's not the
same. update-statement has no primary key liveness info. pk's liveness of
update-statement is depending on liveness of normal columns)
eg. current semantic/behavior:
{quote}
base table: create table ks.base (p int, c int, v int, primary key (p, c))
view table: select p,c from ks.base ... primary key (c, p)
an update query on base table normal column {{v}} will not generate any rows..
a delete query on base table normal column {{v}} will remove the base row.
{quote}
imo, to avoid user confusion, it's better to keep view matched with base
regardless semantic of update..(the current patch is not general enough to
cover all such cases yet)
[~tjake][~slebresne] what do you think?
was (Author: jasonstack):
{{when updating base's columns which are not used in view.}} is about whether
we should consider `Update semantic the same as Insert`. (for now it's not the
same. update-statement has no primary key liveness info. pk's liveness of
update-statement is depending on liveness of normal columns)
eg.
{quote}
base table: create table ks.base (p int, c int, v int, primary key (p, c))
view table: select p,c from ks.base ... primary key (c, p)
an update query on base table normal column {{v}} will not generate any rows..
a delete query on base table normal column {{v}} will remove the base row.
{quote}
imo, to avoid user confusion, it's better to keep view matched with base
regardless semantic of update..(the current patch is not general enough to
cover all such cases yet)
[~tjake][~slebresne] what do you think?
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079873#comment-16079873
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/10/17 5:07 AM:
---
[~tjake] yes, it will have to read existing data more often.
the extra cases are: when updating base's columns which are not used in view.
If a user has multiple views on a base, read-before-write is very likely to
happen, the original optimization won't help too much.
another naive way is to ignore the TTL effect on view in the given example or
totally disable TTL on table with MV. because it may not make sense to
denormalize a short-life data..
was (Author: jasonstack):
[~tjake] yes, it will have to read existing data more often.
the extra cases are: when updating base's columns which are not used in view.
If a user has multiple views on a base, read-before-write is very likely to
happen, the original optimization won't help too much.
another naive way is totally disable TTL on table with MV. because it may not
make sense to denormalize a short-life data..
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062967#comment-16062967
]
ZhaoYang edited comment on CASSANDRA-13127 at 7/7/17 9:25 AM:
--
|| source || junit-result || dtest-result||
| [trunk|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/115] |
{{bootstrap_test.TestBootstrap.consistent_range_movement_false_with_rf1_should_succeed_test}}[known|https://issues.apache.org/jira/browse/CASSANDRA-13576]
{{cqlsh_tests.cqlsh_tests.TestCqlsh.test_describe}} |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes.
was (Author: jasonstack):
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/62] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062967#comment-16062967
]
ZhaoYang edited comment on CASSANDRA-13127 at 6/27/17 4:36 AM:
---
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/62] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes.
was (Author: jasonstack):
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/59] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062776#comment-16062776
]
ZhaoYang edited comment on CASSANDRA-13127 at 6/26/17 2:07 PM:
---
one more issue in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo)) // only timestamp is
used to compare
rowInfo = row.primaryKeyLivenessInfo();
{quote}
3. when timestamps are the same, whether TTL is considered or ignored, is
depending which row is luckily compared first.
{quote}
a. insert Row-A using timestamp 1; flush
b. insert Row-A using timestamp 1 and ttl 5; flush
c. sleep 6 second
d. check result, Row-A exists
reversed step a and b and try again. Row-A not longer exists.
{quote}
imo, when timestamps are same, livenessInfo with greater localDeletionTime
should supersede another if both are TTL. if only one is TTL, ttl supersede.
was (Author: jasonstack):
one more issue in Row.Merger:
| if (row.primaryKeyLivenessInfo().supersedes(rowInfo)) // only timestamp is
used to compare
|rowInfo = row.primaryKeyLivenessInfo();
3. when timestamps are the same, whether TTL is considered or ignored, is
depending which row is luckily compared first.
| a. insert Row-A using timestamp 1; flush
| b. insert Row-A using timestamp 1 and ttl 5; flush
| c. sleep 6 second
| d. check result, Row-A exists
| reversed step a and b and try again. Row-A not longer exists.
imo, when timestamps are same, livenessInfo with greater localDeletionTime
should supersede another if both are TTL. if only one is TTL, ttl supersede.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062967#comment-16062967
]
ZhaoYang edited comment on CASSANDRA-13127 at 6/26/17 2:05 PM:
---
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/59] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes.
was (Author: jasonstack):
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/57] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes. (for primary key)
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062776#comment-16062776
]
ZhaoYang edited comment on CASSANDRA-13127 at 6/26/17 2:04 PM:
---
one more issue in Row.Merger:
| if (row.primaryKeyLivenessInfo().supersedes(rowInfo)) // only timestamp is
used to compare
|rowInfo = row.primaryKeyLivenessInfo();
3. when timestamps are the same, whether TTL is considered or ignored, is
depending which row is luckily compared first.
| a. insert Row-A using timestamp 1; flush
| b. insert Row-A using timestamp 1 and ttl 5; flush
| c. sleep 6 second
| d. check result, Row-A exists
| reversed step a and b and try again. Row-A not longer exists.
imo, when timestamps are same, livenessInfo with greater localDeletionTime
should supersede another if both are TTL. if only one is TTL, ttl supersede.
was (Author: jasonstack):
one more issue in Row.Merger:
> if (row.primaryKeyLivenessInfo().supersedes(rowInfo)) // only timestamp is
> used to compare
>rowInfo = row.primaryKeyLivenessInfo();
3. when timestamps are the same, whether TTL is considered or ignored, is
depending which row is luckily compared first.
> a. insert Row-A using timestamp 1; flush
> b. insert Row-A using timestamp 1 and ttl 5; flush
> c. sleep 6 second
> d. check result, Row-A exists
> reversed step a and b and try again. Row-A not longer exists.
imo, when timestamps are same, livenessInfo with greater localDeletionTime
should supersede another.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062967#comment-16062967
]
ZhaoYang edited comment on CASSANDRA-13127 at 6/26/17 1:41 PM:
---
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/57] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes. (for primary key)
was (Author: jasonstack):
|| source || junit-result || dtest-result||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13127-trunk]
| [utest|https://circleci.com/gh/jasonstack/cassandra/57] | |
| [dtest| https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13127
] | \ | \ |
1. View.mayBeAffectedBy will return true as long as view-key-values are not
filtered
2. LivenessInfo.supersedes(another) will check localDeletionTime if timestamps
are the same. greater localDeletionTimestamp supersedes. (for primary key)
3. Conflicts.resolveRegular will return LEFT_WIN if left is alive. (for cells)
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002850#comment-16002850
]
Benjamin Roth edited comment on CASSANDRA-13127 at 5/9/17 3:18 PM:
---
[~zznate] I have never "stumbled upon" it but i was also never taking care of
that. We also only use the default TTLs, so maybe this is a different thing.
Sounds like it is worth investigating on it. I think there should be a
consensus what the expected behaviour should be (especially on partial
updates), then some tests should be written and then the desired behaviour
should be implemented if it is not met, yet.
Unfortunately I don't have the time at the moment to dig deep into that issue
and go through all the details in the code to see what's going on here.
Just from reading the description of the issue it totally looks like a bug - at
least from a user's point of view.
If nobody else is available for testing and debugging, maybe I can take a
deeper look in 1-2 weeks.
was (Author: brstgt):
[~zznate] I have never "stumbled upon" it but i was also never taking care of
that. We also only use the default TTLs, so maybe this is a different thing.
Sounds like it is worth investigating on it. I think there should be a
consensus what the expected behaviour should be, then some tests should be
written and then the desired behaviour should be implemented if it is not met,
yet.
Unfortunately I don't have the time at the moment to dig deep into that issue
and go through all the details in the code to see what's going on here.
Just from reading the description of the issue it totally looks like a bug - at
least from a user's point of view.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 2:26 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. TTL/expireTime needs to be aggregated
separately, not just based on timestamp.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. TTL/expireTime needs to be aggregated
differently, not just based on timestamp.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 2:26 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. TTL/expireTime needs to be aggregated
differently, not just based on timestamp.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL/expireTime
is different.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 2:17 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL/expireTime
is different.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL is different.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 2:16 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In the update statement, there is no row-liveness info associated with
update. ( so no base row liveness info because of Update semantic). it thinks
this update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL is different.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row liveness info...). it thinks this
update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL is different.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 2:16 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
There are two issues here:
1. In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row liveness info...). it thinks this
update query won't affect view, thus no view update is generated.
if we need to fix it, it means we have to generate view update for more kinds
of base modifications. ( "View.mayBeAffectedBy" will mostly return true)
2. another issue is in Row.Merger:
{quote}
if (row.primaryKeyLivenessInfo().supersedes(rowInfo))
rowInfo = row.primaryKeyLivenessInfo();
{quote}
This comparison isn't enough. timestamp could be the same but TTL is different.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row liveness info...). it thinks this
update query won't affect view, thus no view update is generated.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 1:20 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row liveness info...). it thinks this
update query won't affect view, thus no view update is generated.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row lifetime...). it thanks this update
query won't affect view, thus no view update is generated.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002522#comment-16002522
]
ZhaoYang edited comment on CASSANDRA-13127 at 5/9/17 1:18 PM:
--
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row lifetime...). it thanks this update
query won't affect view, thus no view update is generated.
was (Author: jasonstack):
"UPDATE" semantic was designed to be different from "INSERT" to solve some TTL
issue which will remove entire PK when only ttl of column is updated.
In this ticket, in the update statement, there is no row-liveness info
associated with update. ( so no base row lifetime...)
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
