[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090043#comment-16090043
]
Fridtjof Sander commented on CASSANDRA-13127:
-
bq. More precisely only for base column/cell that has timestamp and not used in
MV. (In cases of insertion, no column timestamp)
You win.
bq. For now, base table with view doesn't support drop columns. but there is a
ticket open to support alter MV.
I just checked it, and that restriction is actually only checked via
{{view.includes(columnName)}} which is not the case here.
Anyway, it's probably a little to early in the discussion for that kind of
detail, we can just set a mental mark to remember this, I guess.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1609#comment-1609
]
Fridtjof Sander commented on CASSANDRA-13127:
-
Exactly, but only from those columns not projected into the view. The
timestamps of all projected columns are available from the cells as usual.
Would we also need to carry information to identify from which column a
timestamp comes from, in case a unprojected base-column is dropped and hence
becomes irrelevant?
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon
[
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089940#comment-16089940
]
Fridtjof Sander commented on CASSANDRA-13127:
-
I found another bug, that I believe falls into the same ballpark as this issue:
{code:java}
@Test
public void testFilterViewTombstones() throws Throwable {
execute("USE " + keyspace());
executeNet(protocolVersion, "USE " + keyspace());
createTable("CREATE TABLE %s (k int, c int, a int, b int, PRIMARY KEY
(k, c))");
createView("mv", "CREATE MATERIALIZED VIEW %s AS SELECT k,c FROM %%s
WHERE k IS NOT NULL AND c IS NOT NULL PRIMARY KEY (k,c)");
updateView("UPDATE %s USING TIMESTAMP 10 SET b=1 WHERE k=1 AND c=1");
assertRows(execute("SELECT * from %s"), row(1, 1, null, 1));
assertRows(execute("SELECT * from mv"), row(1, 1));
updateView("DELETE b FROM %s USING TIMESTAMP 11 WHERE k=1 AND c=1");
assertEmpty(execute("SELECT * from %s"));
assertEmpty(execute("SELECT * from mv"));
updateView("UPDATE %s USING TIMESTAMP 1 SET a=1 WHERE k=1 AND c=1");
assertRows(execute("SELECT * from %s"), row(1, 1, 1, null));
assertRows(execute("SELECT * from mv"), row(1, 1));
}
{code}
A view entry is created based on {{b=1@ts=10}} and deleted with {{ts=11}}.
Later another cell is added with {{a=1@ts=1}}. The view's liveness-info with
{{ts=1}} does not supersede the previously created tombstone with {{ts=11}}.
If you disagree on the similarity, I'll open a new ticket for this. Both issues
are about handling the contribution of base-cells to the view-row's liveness
that are projected out of the view.
On trunk, this doesn't even create an mv-entry (because of the empty
liveness-info), but also fails with [~jasonstack]'s patch. I'm not sure, if
this can be fixed at all without keeping hidden timestamp/liveness-data of
discarded columns.
> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" |
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 7
> (1 rows)
> c | p
> ---+---
> 0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
> p | c | ttl(v)
> ---+---+
> 0 | 0 | 3
> (1 rows)
> c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry
> the TTLs are compared instead of the expiration times, but I'm not sure I'm
> getting that far ahead in the code when updating a column that's not in the
> view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9928) Add Support for multiple non-primary key columns in Materialized View primary keys
[
https://issues.apache.org/jira/browse/CASSANDRA-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088582#comment-16088582
]
Fridtjof Sander commented on CASSANDRA-9928:
I seem to be the only one, who doesn't understanding where the actual
difference to the single-column case is:
Consider {{(p=1, a=1)}} with an index-MV on {{a}} and two updates {{a=2}} and
{{a=3}}. One base-replica receives {{a=2}}, deletes view entry {{(a=1, p=1)}}
and inserts {{(a=2, p=1)}}, then dies. Other base-replicas get {{a=3}}, delete
{{(a=1, p=1)}} and insert {{(a=3, p=1)}}. Now, how is {{(a=2, p=1)}} removed
from the view replica that was paired with the dying base-node? I don't get
what's different here. Or does my analog case miss the point?
> Add Support for multiple non-primary key columns in Materialized View primary
> keys
> --
>
> Key: CASSANDRA-9928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9928
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
> Labels: materializedviews
> Fix For: 4.x
>
>
> Currently we don't allow > 1 non primary key from the base table in a MV
> primary key. We should remove this restriction assuming we continue
> filtering out nulls. With allowing nulls in the MV columns there are a lot
> of multiplicative implications we need to think through.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-10230) Remove coordinator batchlog from materialized views
[
https://issues.apache.org/jira/browse/CASSANDRA-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fridtjof Sander updated CASSANDRA-10230:
Comment: was deleted
(was: I seem to be the only one, who doesn't understanding where the actual
difference to the single-column case is:
Consider {{(p=1, a=1)}} with an index-MV on {{a}} and two updates {{a=2}} and
{{a=3}}. One base-replica receives {{a=2}}, deletes view entry {{(a=1, p=1)}}
and inserts {{(a=2, p=1)}}, then dies. Other base-replicas get {{a=3}}, delete
{{(a=1, p=1)}} and insert {{(a=3, p=1)}}. Now, how is {{(a=2, p=1)}} removed
from the view replica that was paired with the dying base-node? I don't get
what's different here. Or does my analog case miss the point?
)
> Remove coordinator batchlog from materialized views
> ---
>
> Key: CASSANDRA-10230
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10230
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination, Materialized Views
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.0.0 rc1
>
>
> We are considering removing or making optional the coordinator batchlog.
> The batchlog primary serves as a way to quickly reach consistency between
> base and view since we don't have any kind of read repair between base and
> view. But we do have repair so as long as you don't lose nodes while writing
> at CL.ONE you will be eventually consistent.
> I've committed to the 3.0 branch a way to disable the coordinator with
> {{-Dcassandra.mv_disable_coordinator_batchlog=true}}
> The majority of the performance hit to throughput is currently the batchlog
> as shown by this chart.
> http://cstar.datastax.com/graph?stats=f794245a-4d9d-11e5-9def-42010af0688f=op_rate=1_user=1_aggregates=true=0=498.52=0=50142.4
> I'd like to have tests run with and without this flag to validate how quickly
> we achieve quorum consistency without repair writing with CL.ONE. Once we
> can see there is little/no impact we can permanently remove the coordinator
> batchlog.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10230) Remove coordinator batchlog from materialized views
[
https://issues.apache.org/jira/browse/CASSANDRA-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088580#comment-16088580
]
Fridtjof Sander commented on CASSANDRA-10230:
-
I seem to be the only one, who doesn't understanding where the actual
difference to the single-column case is:
Consider {{(p=1, a=1)}} with an index-MV on {{a}} and two updates {{a=2}} and
{{a=3}}. One base-replica receives {{a=2}}, deletes view entry {{(a=1, p=1)}}
and inserts {{(a=2, p=1)}}, then dies. Other base-replicas get {{a=3}}, delete
{{(a=1, p=1)}} and insert {{(a=3, p=1)}}. Now, how is {{(a=2, p=1)}} removed
from the view replica that was paired with the dying base-node? I don't get
what's different here. Or does my analog case miss the point?
> Remove coordinator batchlog from materialized views
> ---
>
> Key: CASSANDRA-10230
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10230
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination, Materialized Views
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 3.0.0 rc1
>
>
> We are considering removing or making optional the coordinator batchlog.
> The batchlog primary serves as a way to quickly reach consistency between
> base and view since we don't have any kind of read repair between base and
> view. But we do have repair so as long as you don't lose nodes while writing
> at CL.ONE you will be eventually consistent.
> I've committed to the 3.0 branch a way to disable the coordinator with
> {{-Dcassandra.mv_disable_coordinator_batchlog=true}}
> The majority of the performance hit to throughput is currently the batchlog
> as shown by this chart.
> http://cstar.datastax.com/graph?stats=f794245a-4d9d-11e5-9def-42010af0688f=op_rate=1_user=1_aggregates=true=0=498.52=0=50142.4
> I'd like to have tests run with and without this flag to validate how quickly
> we achieve quorum consistency without repair writing with CL.ONE. Once we
> can see there is little/no impact we can permanently remove the coordinator
> batchlog.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted
[
https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083638#comment-16083638
]
Fridtjof Sander commented on CASSANDRA-11500:
-
I also want to through my stuff into the ring right here :)
Following up from CASSANDRA-13657 on this, [my proposal from
there|https://issues.apache.org/jira/browse/CASSANDRA-13657?focusedCommentId=16080177=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16080177]
could be adapted to solve this by introducing a
{{ShortCircuitingRowDeletion}}: A live, sort-circuiting row-deletion supersedes
all columns no matter their timestamp. It can only be superseded by a
liveness-info with higher timestamp.
That way, we don't need to take maximum timestamp of cells anymore for the
view-tombstone, which prevents the conflict of this issue.
> Obsolete MV entry may not be properly deleted
> -
>
> Key: CASSANDRA-11500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11500
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Sylvain Lebresne
>Assignee: ZhaoYang
>
> When a Materialized View uses a non-PK base table column in its PK, if an
> update changes that column value, we add the new view entry and remove the
> old one. When doing that removal, the current code uses the same timestamp
> than for the liveness info of the new entry, which is the max timestamp for
> any columns participating to the view PK. This is not correct for the
> deletion as the old view entry could have other columns with higher timestamp
> which won't be deleted as can easily shown by the failing of the following
> test:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1;
> SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old
> (invalid) and the new one
> {noformat}
> So the correct timestamp to use for the deletion is the biggest timestamp in
> the old view entry (which we know since we read the pre-existing base row),
> and that is what CASSANDRA-11475 does (the test above thus doesn't fail on
> that branch).
> Unfortunately, even then we can still have problems if further updates
> requires us to overide the old entry. Consider the following case:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 10 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; // This will delete the
> entry for a=1 with timestamp 10
> UPDATE t USING TIMESTAMP 3 SET a = 1 WHERE k = 1; // This needs to re-insert
> an entry for a=1 but shouldn't be deleted by the prior deletion
> UPDATE t USING TIMESTAMP 4 SET a = 2 WHERE k = 1; // ... and we can play this
> game more than once
> UPDATE t USING TIMESTAMP 5 SET a = 1 WHERE k = 1;
> ...
> {noformat}
> In a way, this is saying that the "shadowable" deletion mechanism is not
> general enough: we need to be able to re-insert an entry when a prior one had
> been deleted before, but we can't rely on timestamps being strictly bigger on
> the re-insert. In that sense, this can be though as a similar problem than
> CASSANDRA-10965, though the solution there of a single flag is not enough
> since we can have to replace more than once.
> I think the proper solution would be to ship enough information to always be
> able to decide when a view deletion is shadowed. Which means that both
> liveness info (for updates) and shadowable deletion would need to ship the
> timestamp of any base table column that is part the view PK (so {{a}} in the
> example below). It's doable (and not that hard really), but it does require
> a change to the sstable and intra-node protocol, which makes this a bit
> painful right now.
> But I'll also note that as CASSANDRA-1096 shows, the timestamp is not even
> enough since on equal timestamp the value can be the deciding factor. So in
> theory we'd have to ship the value of those columns (in the case of a
> deletion at least since we have it in the view PK for updates). That said, on
> that last problem, my preference would be that we start prioritizing
> CASSANDRA-6123 seriously so we don't have to care about conflicting timestamp
> anymore, which would make this problem go away.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083630#comment-16083630
]
Fridtjof Sander commented on CASSANDRA-13657:
-
[~KurtG] From my understanding that is not possible, since all PK parts must be
{{NOT NULL}}. But if there is way, or the logic is changed that way, then
you're right. My assumption was: The view-rows's liveness is defined by the
indexed column's liveness. If we want to index on values of dead cells that
doesn't work anymore. Are there any thoughts or discussions going into that
direction, though?
The way I see it, a view-row's liveness _is actually_ different from an
ordinary row's liveness. So, it makes sense to have code representing that
difference, instead of squeezing the new definition into the old mechanism and
constantly having to patch stuff left and right.
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>Assignee: Krishna Dattu Koneru
> Labels: materializedviews, ttl
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> DELETE FROM t WHERE k=1;
> cqlsh:test> SELECT * from t; SELECT * from mv;
> k | a | b
> ---+---+---
> (0 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that view-entry is now orphanized.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080177#comment-16080177
]
Fridtjof Sander commented on CASSANDRA-13657:
-
[~krishna.koneru] Thanks for the swift reaction. Looks good to me.
I don't want to appear unappreciative here, but also having
[CASSANDRA-11500|https://issues.apache.org/jira/browse/CASSANDRA-11500] in
mind, I'm wondering if there isn't a more general solution to our consistency
problems of MVs.
I drafted an idea and created kind of a POC here:
https://github.com/f-sander/apache-cassandra/tree/short_circuiting_liveness_info
The general idea of it is below:
The usual "live" definition for a row is as follows:
{quote}
A row is live iff its liveness-info is live or any of its cells is live
{quote}
For rows of MVs the definition changes (for those moving a column into the MV's
PK aka index views)
{quote}
A MV row is live iff the indexed column's cell in the base row is live
{quote}
Instead of modeling this liveness dependency explicitly, we are trying to
squeeze it into the first definition. In my opinion, that's a source of
complicated code, hence oversights, hence bugs.
So, how would a explicit model of that dependency look like?
What we could do is to add a flag to the liveness-info that enables to
short-circuit the live-definition. The live-definition then changes to:
{quote}
A row is live iff its liveness-info is live or (if its liveness-info is not
flagged AND if any of its cells is live)
{quote}
In other words, the cells' "livenesses" are only considered if the parent row's
liveness-info is not flagged. Otherwise the liveness-info defines the whole
row's liveness.
Then, the view-row's liveness-info can be set equivalent to the indexed cell's
liveness (plus the flag).
I have the impression this would greatly simplify the code and fix any problem
with MV consistency we currently have.
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>Assignee: Krishna Dattu Koneru
> Labels: materializedviews, ttl
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> DELETE FROM t WHERE k=1;
> cqlsh:test> SELECT * from t; SELECT * from mv;
> k | a | b
> ---+---+---
> (0 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that view-entry is now orphanized.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fridtjof Sander updated CASSANDRA-13657:
Description:
{noformat}
CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT
NULL PRIMARY KEY (a, k);
INSERT INTO t (k) VALUES (1);
UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
UPDATE t SET b = 100 WHERE k = 1;
SELECT * from t; SELECT * from mv;
k | a | b
---++-
1 | 10 | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- 5 seconds later
SELECT * from t; SELECT * from mv;
k | a| b
---+--+-
1 | null | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- that view entry's liveness-info is (probably) dead, but the entry is kept
alive by b=100
DELETE b FROM t WHERE k=1;
SELECT * from t; SELECT * from mv;
k | a| b
---+--+--
1 | null | null
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
DELETE FROM t WHERE k=1;
cqlsh:test> SELECT * from t; SELECT * from mv;
k | a | b
---+---+---
(0 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- deleting the base-entry doesn't help, because the view-key can not be
constructed anymore (a=10 already expired)
{noformat}
The problem here is that although the view-entry's liveness-info (probably)
expired correctly a regular column (`b`) keeps the view-entry live. It should
have disappeared since it's indexed column (`a`) expired in the corresponding
base-row. This is pretty severe, since that view-entry is now orphanized.
was:
{noformat}
CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT
NULL PRIMARY KEY (a, k);
INSERT INTO t (k) VALUES (1);
UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
UPDATE t SET b = 100 WHERE k = 1;
SELECT * from t; SELECT * from mv;
k | a | b
---++-
1 | 10 | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- 5 seconds later
SELECT * from t; SELECT * from mv;
k | a| b
---+--+-
1 | null | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- that view entry's liveness-info is (probably) dead, but the entry is kept
alive by b=100
DELETE b FROM t WHERE k=1;
SELECT * from t; SELECT * from mv;
k | a| b
---+--+--
1 | null | null
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- deleting the base-entry doesn't help, because the view-key can not be
constructed anymore (a=10 already expired)
{noformat}
The problem here is that although the view-entry's liveness-info (probably)
expired correctly a regular column (`b`) keeps the view-entry live. It should
have disappeared since it's indexed column (`a`) expired in the corresponding
base-row. This is pretty severe, since that view-entry is now orphanized.
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> DELETE FROM t WHERE k=1;
> cqlsh:test> SELECT * from t; SELECT * from mv;
> k | a | b
> ---+---+---
> (0 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that
[jira] [Updated] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
[
https://issues.apache.org/jira/browse/CASSANDRA-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fridtjof Sander updated CASSANDRA-13657:
Description:
{noformat}
CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT
NULL PRIMARY KEY (a, k);
INSERT INTO t (k) VALUES (1);
UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
UPDATE t SET b = 100 WHERE k = 1;
SELECT * from t; SELECT * from mv;
k | a | b
---++-
1 | 10 | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- 5 seconds later
SELECT * from t; SELECT * from mv;
k | a| b
---+--+-
1 | null | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- that view entry's liveness-info is (probably) dead, but the entry is kept
alive by b=100
DELETE b FROM t WHERE k=1;
SELECT * from t; SELECT * from mv;
k | a| b
---+--+--
1 | null | null
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- deleting the base-entry doesn't help, because the view-key can not be
constructed anymore (a=10 already expired)
{noformat}
The problem here is that although the view-entry's liveness-info (probably)
expired correctly a regular column (`b`) keeps the view-entry live. It should
have disappeared since it's indexed column (`a`) expired in the corresponding
base-row. This is pretty severe, since that view-entry is now orphanized.
was:
```
CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT
NULL PRIMARY KEY (a, k);
INSERT INTO t (k) VALUES (1);
UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
UPDATE t SET b = 100 WHERE k = 1;
SELECT * from t; SELECT * from mv;
k | a | b
---++-
1 | 10 | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- 5 seconds later
SELECT * from t; SELECT * from mv;
k | a| b
---+--+-
1 | null | 100
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- that view entry's liveness-info is (probably) dead, but the entry is kept
alive by b=100
DELETE b FROM t WHERE k=1;
SELECT * from t; SELECT * from mv;
k | a| b
---+--+--
1 | null | null
(1 rows)
a | k | b
+---+-
10 | 1 | 100
(1 rows)
-- deleting the base-entry doesn't help, because the view-key can not be
constructed anymore (a=10 already expired)
```
The problem here is that although the view-entry's liveness-info (probably)
expired correctly a regular column (`b`) keeps the view-entry live. It should
have disappeared since it's indexed column (`a`) expired in the corresponding
base-row. This is pretty severe, since that view-entry is now orphanized.
> Materialized Views: Index MV on TTL'ed column produces orphanized view entry
> if another column keeps entry live
> ---
>
> Key: CASSANDRA-13657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13657
> Project: Cassandra
> Issue Type: Bug
> Components: Materialized Views
>Reporter: Fridtjof Sander
>
> {noformat}
> CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k));
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS
> NOT NULL PRIMARY KEY (a, k);
> INSERT INTO t (k) VALUES (1);
> UPDATE t USING TTL 5 SET a = 10 WHERE k = 1;
> UPDATE t SET b = 100 WHERE k = 1;
> SELECT * from t; SELECT * from mv;
> k | a | b
> ---++-
> 1 | 10 | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- 5 seconds later
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+-
> 1 | null | 100
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- that view entry's liveness-info is (probably) dead, but the entry is kept
> alive by b=100
> DELETE b FROM t WHERE k=1;
> SELECT * from t; SELECT * from mv;
> k | a| b
> ---+--+--
> 1 | null | null
> (1 rows)
> a | k | b
> +---+-
> 10 | 1 | 100
> (1 rows)
> -- deleting the base-entry doesn't help, because the view-key can not be
> constructed anymore (a=10 already expired)
> {noformat}
> The problem here is that although the view-entry's liveness-info (probably)
> expired correctly a regular column (`b`) keeps the view-entry live. It should
> have disappeared since it's indexed column (`a`) expired in the corresponding
> base-row. This is pretty severe, since that view-entry is now orphanized.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13657) Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live
Fridtjof Sander created CASSANDRA-13657: --- Summary: Materialized Views: Index MV on TTL'ed column produces orphanized view entry if another column keeps entry live Key: CASSANDRA-13657 URL: https://issues.apache.org/jira/browse/CASSANDRA-13657 Project: Cassandra Issue Type: Bug Components: Materialized Views Reporter: Fridtjof Sander ``` CREATE TABLE t (k int, a int, b int, PRIMARY KEY (k)); CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT NULL PRIMARY KEY (a, k); INSERT INTO t (k) VALUES (1); UPDATE t USING TTL 5 SET a = 10 WHERE k = 1; UPDATE t SET b = 100 WHERE k = 1; SELECT * from t; SELECT * from mv; k | a | b ---++- 1 | 10 | 100 (1 rows) a | k | b +---+- 10 | 1 | 100 (1 rows) -- 5 seconds later SELECT * from t; SELECT * from mv; k | a| b ---+--+- 1 | null | 100 (1 rows) a | k | b +---+- 10 | 1 | 100 (1 rows) -- that view entry's liveness-info is (probably) dead, but the entry is kept alive by b=100 DELETE b FROM t WHERE k=1; SELECT * from t; SELECT * from mv; k | a| b ---+--+-- 1 | null | null (1 rows) a | k | b +---+- 10 | 1 | 100 (1 rows) -- deleting the base-entry doesn't help, because the view-key can not be constructed anymore (a=10 already expired) ``` The problem here is that although the view-entry's liveness-info (probably) expired correctly a regular column (`b`) keeps the view-entry live. It should have disappeared since it's indexed column (`a`) expired in the corresponding base-row. This is pretty severe, since that view-entry is now orphanized. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
