[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/7/17 6:28 AM:
--
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
[utest|https://circleci.com/gh/jasonstack/cassandra/46?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]
| |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| \ | \ |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
[utest|https://circleci.com/gh/jasonstack/cassandra/46?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]
| |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/7/17 6:28 AM:
--
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
[utest|https://circleci.com/gh/jasonstack/cassandra/46?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]
| |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/6/17 12:54 PM:
---
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
|utest | |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/6/17 12:52 PM:
---
|| source || junit || dtest||
| [3.0|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.0]|
[utest|https://circleci.com/gh/jasonstack/cassandra/42] | |
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
|utest | |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| | |
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/6/17 3:25 AM:
--
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/6/17 3:25 AM:
--
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion as following:
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
was (Author: jasonstack):
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion
{code}
"rows" : [
{
"type" : "row",
"position" : 63,
"clustering" : [ "3" ],
"deletion_info" : {
"shadowable" : { "marked_deleted" : "1970-01-01T00:00:00.04Z",
"local_delete_time" : "2017-05-06T03:21:47Z" },
"regular" : { "marked_deleted" : "1970-01-01T00:00:00.02Z",
"local_delete_time" : "2017-05-06T03:21:47Z" }
},
"cells" : [ ]
}
]
{code}
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998614#comment-15998614
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/5/17 5:24 PM:
--
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion
was (Author: jasonstack):
|| source ||
| [3.11|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
| [dtest|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
|
1. if there is row deletion on base, now it will create regular-row-tombstone
for view. (previously only shadowable-row-tombstones)
2. modify Row.Deletion to retain one more regular-row-tombtone with extra flag
in UnfilteredSerializer
* the most recent regular-row-tombstone which has TS < most recent
shadowable-row-tombstones's TS
* if shadowable-row-tombstone is shadowed by new row, its retained
regular-row-tombstone(could be LIVE) will be used
3. changed sstabledump (aka sstableexport) to support : shadowable/regular
row.deletion
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/4/17 12:07 PM:
---
Thanks. I didn't think about compaction with sub-group
1. deleted cell + shadowable row tombstone: deleted cells should be kept even
if there is more recent shadowable row-tombstones, this could be done without
changing storage format
since it's after gc_grace_seconds, it's less critical.
2. regular row tombstone + shadowable row tombstone: if shadowable is newer,
only most recent regular row tombstone plus shadowable should be kept. maybe
have to change storage format. eg. extra flag to indicate extra deletion.
it's critical..
was (Author: jasonstack):
Thanks. I didn't think about compaction with sub-group
1. deleted cell + shadowable row tombstone: deleted cells should be kept even
if there is more recent shadowable row-tombstones, this could be done without
changing storage format
since it's after gc_grace_seconds, it's less critical. In your example,
compaction of sstable 2/3 is irrelevant. When executing No.4, the existing cell
tombstone of key=2 in view and gc_grace_seconds matter.
2. regular row tombstone + shadowable row tombstone: if shadowable is newer,
only most recent regular row tombstone plus shadowable should be kept. maybe
have to change storage format. eg. extra flag to indicate extra deletion.
it's critical..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/4/17 11:35 AM:
---
Thanks. I didn't think about compaction with sub-group
1. deleted cell + shadowable row tombstone: deleted cells should be kept even
if there is more recent shadowable row-tombstones, this could be done without
changing storage format
since it's after gc_grace_seconds, it's less critical. In your example,
compaction of sstable 2/3 is irrelevant. When executing No.4, the existing cell
tombstone of key=2 in view and gc_grace_seconds matter.
2. regular row tombstone + shadowable row tombstone: if shadowable is newer,
only most recent regular row tombstone plus shadowable should be kept. maybe
have to change storage format. eg. extra flag to indicate extra deletion.
it's critical..
was (Author: jasonstack):
Thanks. I didn't think about compaction with sub-group
1. deleted cell + shadowable row tombstone: deleted cells should be kept even
if there is more recent shadowable row-tombstones, this could be done without
changing storage format
since it's after gc_grace_seconds, it's less critical.
2. regular row tombstone + shadowable row tombstone: if shadowable is newer,
only most recent regular row tombstone plus shadowable should be kept. maybe
have to change storage format. eg. extra flag to indicate extra deletion.
it's critical..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/4/17 10:59 AM:
---
Thanks. I didn't think about compaction with sub-group
1. deleted cell + shadowable row tombstone: deleted cells should be kept even
if there is more recent shadowable row-tombstones, this could be done without
changing storage format
since it's after gc_grace_seconds, it's less critical.
2. regular row tombstone + shadowable row tombstone: if shadowable is newer,
only most recent regular row tombstone plus shadowable should be kept. maybe
have to change storage format. eg. extra flag to indicate extra deletion.
it's critical..
was (Author: jasonstack):
Thanks. I didn't think about compaction with sub-group
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone plus shadowable should be kept. maybe have to
change storage format. eg. extra flag to indicate extra deletion.
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990944#comment-15990944
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/4/17 4:53 AM:
--
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base row deletion should be regular row tombstone
instead of shadowable row tombstone, but unfortunately in this case, they are
created as shadowable...
The regression maybe happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
was (Author: jasonstack):
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base row deletion should be regular tombstone
instead of shadowable tombstone, but unfortunately in this case, they are
created as shadowable...
The regression maybe happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/4/17 4:52 AM:
--
Thanks. I didn't think about compaction with sub-group
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone plus shadowable should be kept. maybe have to
change storage format. eg. extra flag to indicate extra deletion.
was (Author: jasonstack):
Thanks.
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone plus shadowable should be kept. maybe have to
change storage format. eg. extra flag to indicate extra deletion.
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994639#comment-15994639
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 3:35 PM:
--
|| Source || Test ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| [circle-ci | https://circleci.com/gh/jasonstack/cassandra/20] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
| |
was (Author: jasonstack):
|| Source || Test ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| [circle-ci | https://circleci.com/gh/jasonstack/cassandra/18] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
| |
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 12:43 PM:
---
Thanks.
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone plus shadowable should be kept. maybe have to
change storage format. eg. extra flag to indicate extra deletion.
was (Author: jasonstack):
Thanks.
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone should be kept. maybe have to change storage
format. eg. extra flag to indicate extra deletion.
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994766#comment-15994766
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 12:35 PM:
---
Thanks.
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is more recent shadowable row-tombstones, this could be done without
changing storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone should be kept. maybe have to change storage
format. eg. extra flag to indicate extra deletion.
was (Author: jasonstack):
Thanks.
deleted cell + shadowable row tombstone: deleted cells should be kept even if
there is newer shadowable row-tombstones, this could be done without changing
storage format
regular row tombstone + shadowable row tombstone: if shadowable is newer, only
most recent regular row tombstone should be kept. maybe have to change storage
format. eg. extra flag to indicate extra deletion.
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994639#comment-15994639
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 12:06 PM:
---
|| Source || Test ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| [circle-ci | https://circleci.com/gh/jasonstack/cassandra/18] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
| |
was (Author: jasonstack):
|| Source || Test ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| [circle-ci | https://circleci.com/gh/jasonstack/cassandra/18] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
| cassi... sorry no access|
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994639#comment-15994639
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 10:29 AM:
---
|| Source || Test ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409]
| [circle-ci | https://circleci.com/gh/jasonstack/cassandra/18] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11]
| cassi... sorry no access|
was (Author: jasonstack):
|| Source ||
| [3.11|https://github.com/jasonstack/cassandra-dtest/commits/CASSANDRA-13409] |
| [dtest|https://github.com/jasonstack/cassandra/commits/CASSANDRA-13409-3.11] |
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990944#comment-15990944
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/3/17 6:56 AM:
--
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base row deletion should be regular tombstone
instead of shadowable tombstone, but unfortunately in this case, they are
created as shadowable...
The regression maybe happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
was (Author: jasonstack):
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
The regression maybe happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990944#comment-15990944
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/1/17 4:48 PM:
--
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
The regression maybe happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
was (Author: jasonstack):
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990944#comment-15990944
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/1/17 4:47 PM:
--
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
was (Author: jasonstack):
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
The regression is likely happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990944#comment-15990944
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/1/17 4:46 PM:
--
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
View tombstones generated by base deletion should be regular tombstone instead
of shadowable tombstone, but unfortunately in this case, they are created as
shadowable...
The regression is likely happened here
[11475|https://issues.apache.org/jira/browse/CASSANDRA-11475]
will try to fix it this week..
was (Author: jasonstack):
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990650#comment-15990650
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/1/17 3:17 PM:
--
This issue is related to
[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261]. I will look into
it..
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
was (Author: jasonstack):
This issue is related to
[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261]. I will look into
it..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13409) Materialized Views: View cells are resurrected
[
https://issues.apache.org/jira/browse/CASSANDRA-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990650#comment-15990650
]
ZhaoYang edited comment on CASSANDRA-13409 at 5/1/17 3:17 PM:
--
This issue is related to
[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261]. I will look into
it..
was (Author: jasonstack):
This issue is related to
[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261]. I will look into
it..
ok..[10261|https://issues.apache.org/jira/browse/CASSANDRA-10261] actually
fixed this issue, but since no test case for this scenario is provied,
somewhere regression happened..
> Materialized Views: View cells are resurrected
> --
>
> Key: CASSANDRA-13409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13409
> Project: Cassandra
> Issue Type: Bug
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk@0f054fee5c:
> {code:xml}
> echo "create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};" | bin/cqlsh
> echo "create table ks.base (p int primary key, v1 int, v2 int) with
> gc_grace_seconds = 1;" | bin/cqlsh
> echo "create materialized view ks.my_view as select * from ks.base where p is
> not null and v1 is not null primary key (v1, p);" | bin/cqlsh
> echo "insert into ks.base (p, v1, v2) values (3, 1, 3) using timestamp 1;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "delete from ks.base using timestamp 2 where p = 3;" | bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "insert into ks.base (p, v1) values (3, 1) using timestamp 3;" |
> bin/cqlsh
> bin/nodetool flush ks my_view base
> echo "select * from ks.my_view;" | bin/cqlsh
> v1 | p | v2
> +---+
> 1 | 3 | 3
> (1 rows)
> echo "select * from ks.base;" | bin/cqlsh
> p | v1 | v2
> ---++--
> 3 | 1 | null
> (1 rows)
> {code}
> As you can see, this incorrectly brings back cell v2=3.
> There is one definitive problem and a potential one:
> * Merging rows must be commutative. If a shadowable tombstone is applied
> after a row tombstone, it will replace that tombstone; if a row marker
> shadows the shadowable tombstone before the row containing the original data
> is applied, then any dead cells in said data will be resurrected;
> * Shadowable tombstones shouldn't compact away previous row tombstones or
> even deleted cells; if the relevant tombstones have been GCed from the base
> table, then a base table update won't carry them anymore (alongside a newer
> row marker).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
