[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173016#comment-16173016
]
Alex Petrov edited comment on CASSANDRA-12373 at 9/20/17 2:50 PM:
--
bq. Thanks for the changes and you patience on this. My main remaining remark
is that I don't think we should include the SC key and value columns to
partitionColumns (in CFMetaData.rebuild).
This was surprisingly simple to do.
bq. In CFMetaData.renameColumn, in the case of updating the SC key or value
column, I believe we should be updating columnMetadata as well since those
columns are listed in it, but that doesn't seem to be the case (not sure how
important it is, it might be a following call to rebuild fixes that in
practice, but since the method doesn't call rebuild itself, probably better to
make sure we handle it).
I can't see how this can be helpful because of the subsequent {{rebuild}} call,
but this also doesn't break anything, so I went ahead and changed it.
bq. In CFMetaData.makeLegacyDefaultValidator, compact tables with counter will
now return BytesType instead of CounterColumnType, which is kind of technically
incorrect. To be entirely honest, this doesn't matter currently because that
method isn't ever called for non-compact tables (and at this point, probably
never will), but if we're going to rely on this, I'd rather make it an
assertion than returning something somewhat wrong. Personally, I'd just keep
the counter special case and move on, as this has nothing to do with this
ticket, but if you prefer transforming it to a assert !isCompactTable(), no
complain.
I've added the {{isCounter}} back, no strong opinion here, too.
bq. Nit: in CFMetaData.renameColumn, the comment "SuperColumn tables allow
renaming all columns" doesn't match the code entirely anymore.
Yeah, I was implying dense ones, but I don't think this comment is of much use
here anyways.
bq. Nit: in SuperColumnCompatibility.getSuperCfKeyColumn, I don't think the
"3.x created supercolumn family" comment is accurate anymore since in
ThriftConversion you now add the 2nd clustering column (which, in itself, lgtm).
It's still true for pre-12373 3.x thrift-created supercolumn family tables.
We've discussed this offline shortly: there was no good way to force the table
update to make all the table look completely the same, so this is the only
place we still have to special-case. I've added the {{pre 12373}} remark and
hope it's clearer now.
|[3.0
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:12373-3.0]|[3.11
patch|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:12373-3.11]|[dtests|https://github.com/apache/cassandra-dtest/compare/master...ifesdjeen:12373]|[2.2
patch|https://github.com/apache/cassandra/compare/cassandra-2.2...ifesdjeen:12373-2.2]|
UPDATE: updated and rebased both branches, CI looks good.
was (Author: ifesdjeen):
bq. Thanks for the changes and you patience on this. My main remaining remark
is that I don't think we should include the SC key and value columns to
partitionColumns (in CFMetaData.rebuild).
This was surprisingly simple to do.
bq. In CFMetaData.renameColumn, in the case of updating the SC key or value
column, I believe we should be updating columnMetadata as well since those
columns are listed in it, but that doesn't seem to be the case (not sure how
important it is, it might be a following call to rebuild fixes that in
practice, but since the method doesn't call rebuild itself, probably better to
make sure we handle it).
I can't see how this can be helpful because of the subsequent {{rebuild}} call,
but this also doesn't break anything, so I went ahead and changed it.
bq. In CFMetaData.makeLegacyDefaultValidator, compact tables with counter will
now return BytesType instead of CounterColumnType, which is kind of technically
incorrect. To be entirely honest, this doesn't matter currently because that
method isn't ever called for non-compact tables (and at this point, probably
never will), but if we're going to rely on this, I'd rather make it an
assertion than returning something somewhat wrong. Personally, I'd just keep
the counter special case and move on, as this has nothing to do with this
ticket, but if you prefer transforming it to a assert !isCompactTable(), no
complain.
I've added the {{isCounter}} back, no strong opinion here, too.
bq. Nit: in CFMetaData.renameColumn, the comment "SuperColumn tables allow
renaming all columns" doesn't match the code entirely anymore.
Yeah, I was implying dense ones, but I don't think this comment is of much use
here anyways.
bq. Nit: in SuperColumnCompatibility.getSuperCfKeyColumn, I don't think the
"3.x created supercolumn family" comment is accurate anymore since in
ThriftConversion you now add the 2nd clustering column (which, in itse
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166050#comment-16166050
]
Alex Petrov edited comment on CASSANDRA-12373 at 9/18/17 6:33 AM:
--
Thank you for the review and comments.
I agree that having {{column2}} as clustering is better. I've tried to move
most of the special-casing to {{rebuild}} and {{SuperColumnCompatibility}}. I
think the patch got a bit cleaner thanks to that.
bq. In CFMetaData.renameColumn(), we appear to allow renaming every column for
any SCF, including non-dense ones. I don't think that was allowed in 2.x
(renaming non-PK columns of non-dense SCF through CQL) and I suggest
maintaining non supporting it. In fact, I don't think it's entirely safe in
some complex case of users still using thrift and doing schema-changes from it.
Fixed and added corresponding tests to both 3.0 and 2.2
bq. I don't think the change in CFMetaData.makeLegacyDefaultValidator is
correct. That said, I don't think the previous code was correct either. If I'm
not mistaken, what we should be returning in the SCF case is
((MapType)compactValueColumn().type).valueComparator().
I _think_ it can be simplified even further, since {{isCounter}} case will work
is because of the {{compactValueColumn}} (or map value type) and {{isCompact}}
call seems to be redundant alltogether.
bq. In SuperColumnCompatibility.prepareUpdateOperations, after the first loop,
I think we should check that superColumnKey != null (and provide a meaningful
error message if that's not the case). I believe otherwise we might NPE when
handling the {{Operation}}s created.
Fixed and added a couple more negative tests.
bq. In SuperColumnCompatibility.columnNameGenerator, I'm not sure I fully
understand the reason for always excluding "column1" (despite the comment). Not
that it's really a big deal.
This is still a bit of a problem, although just in one case. When upgrade was
done through unpatched 3.x, we end up without {{column2}} and {{value}}
columns. Now, we try renaming {{column1}} to {{column1_renamed}}, and, because
{{column2}} is still "virtual" (since it was not renamed), we may end up with
{{column2}} being called {{column1}} because of the defaults without this line.
I'd like to point out that renaming {{column2}} to {{column2}} and {{value}}
are not allowed even in that case (since all the columns are now in column
metadata map).
bq. for the class javadoc, since things are tricky, when saying "the default
column names are used", I think that's a good place to remind what "column1"
and "column2" means, and that both in term of the internal representation, of
their CQL exposure, and of the thrift correspondance. Or maybe move such
explanation to the SuperColumnCompatibility class javadoc and point to it?
Improved comments in header and for this inner class.
bq. for mutliEQRestriction should be ... AND (column1, column2) = ('value1', 1)
but it currently uses a >.
Fixed
bq. for keyInRestriction, the "This operation does not have a direct Thrift
counterpart" isn't true. And In fact, I'm not sure why we have to fetch
everything and filter: can't we just handle this in getColumnFilter by only
selecting the map entries we want? Note that the one operation that does not
have a Thrift counterpart is mutliSliceRestriction (and, technically, anything
operation on strict bounds since Thrift was always inclusive).
I was under impression that {{ColumnFilter.Builder#select}} allows just a
single collection constraint. Thanks for catching that. You're right, we can
handle it without any filtering, looks much better now.
bq. Nit: there is a few typo in those comments ("prece*e*ding" instead of
"preceding", "exlusive", "enitre", "... in this case since, since ...").
Fixed these and spell-checked to catch a couple more.
bq. in MultiColumnRelation, both methods have List receivers
= receivers(cfm), but then in the next line, they call receivers(cfm) instead
of just reusing receivers.
Fixed.
bq. In Relation, I'd extend the error message to something like "Unsupported
operation (" + this + ") on super column family".
Fixed.
bq. Last else of 2nd loop in SuperColumnCompatibility.prepareUpdateOperations
could use brackets
Fixed this an several other ones.
bq. In MultiColumnRestriction.SliceRestriction, if my IDE don't fool me, it
appears we don't need to make slice public anymore.
You're right. Fixed.
bq. In StatementRestrictions, a few added imports are not needed (including the
NotImplementedException one).
Fixed this and several other cases.
|[3.0
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:12373-3.0]|[3.11
patch|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:12373-3.11]|[dtests|https://github.com/apache/cassandra-dtest/compare/master...ifesdjeen:12373]|[2.2
pa
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166050#comment-16166050
]
Alex Petrov edited comment on CASSANDRA-12373 at 9/14/17 3:22 PM:
--
Thank you for the review and comments.
I agree that having {{column2}} as clustering is better. I've tried to move
most of the special-casing to {{rebuild}} and {{SuperColumnCompatibility}}. I
think the patch got a bit cleaner thanks to that.
bq. In CFMetaData.renameColumn(), we appear to allow renaming every column for
any SCF, including non-dense ones. I don't think that was allowed in 2.x
(renaming non-PK columns of non-dense SCF through CQL) and I suggest
maintaining non supporting it. In fact, I don't think it's entirely safe in
some complex case of users still using thrift and doing schema-changes from it.
Fixed and added corresponding tests to both 3.0 and 2.2
bq. I don't think the change in CFMetaData.makeLegacyDefaultValidator is
correct. That said, I don't think the previous code was correct either. If I'm
not mistaken, what we should be returning in the SCF case is
((MapType)compactValueColumn().type).valueComparator().
I _think_ it can be simplified even further, since {{isCounter}} case will work
is because of the {{compactValueColumn}} (or map value type) and {{isCompact}}
call seems to be redundant alltogether.
bq. In SuperColumnCompatibility.prepareUpdateOperations, after the first loop,
I think we should check that superColumnKey != null (and provide a meaningful
error message if that's not the case). I believe otherwise we might NPE when
handling the {{Operation}}s created.
Fixed and added a couple more negative tests.
bq. In SuperColumnCompatibility.columnNameGenerator, I'm not sure I fully
understand the reason for always excluding "column1" (despite the comment). Not
that it's really a big deal.
This is still a bit of a problem, although just in one case. When upgrade was
done through unpatched 3.x, we end up without {{column2}} and {{value}}
columns. Now, we try renaming {{column1}} to {{column1_renamed}}, and, because
{{column2}} is still "virtual" (since it was not renamed), we may end up with
{{column2}} being called {{column1}} because of the defaults without this line.
I'd like to point out that renaming {{column2}} to {{column2}} and {{value}}
are not allowed even in that case (since all the columns are now in column
metadata map).
bq. for the class javadoc, since things are tricky, when saying "the default
column names are used", I think that's a good place to remind what "column1"
and "column2" means, and that both in term of the internal representation, of
their CQL exposure, and of the thrift correspondance. Or maybe move such
explanation to the SuperColumnCompatibility class javadoc and point to it?
Improved comments in header and for this inner class.
bq. for mutliEQRestriction should be ... AND (column1, column2) = ('value1', 1)
but it currently uses a >.
Fixed
bq. for keyInRestriction, the "This operation does not have a direct Thrift
counterpart" isn't true. And In fact, I'm not sure why we have to fetch
everything and filter: can't we just handle this in getColumnFilter by only
selecting the map entries we want? Note that the one operation that does not
have a Thrift counterpart is mutliSliceRestriction (and, technically, anything
operation on strict bounds since Thrift was always inclusive).
I was under impression that {{ColumnFilter.Builder#select}} allows just a
single collection constraint. Thanks for catching that. You're right, we can
handle it without any filtering, looks much better now.
bq. Nit: there is a few typo in those comments ("prece*e*ding" instead of
"preceding", "exlusive", "enitre", "... in this case since, since ...").
Fixed these and spell-checked to catch a couple more.
bq. in MultiColumnRelation, both methods have List receivers
= receivers(cfm), but then in the next line, they call receivers(cfm) instead
of just reusing receivers.
Fixed.
bq. In Relation, I'd extend the error message to something like "Unsupported
operation (" + this + ") on super column family".
Fixed.
bq. Last else of 2nd loop in SuperColumnCompatibility.prepareUpdateOperations
could use brackets
Fixed this an several other ones.
bq. In MultiColumnRestriction.SliceRestriction, if my IDE don't fool me, it
appears we don't need to make slice public anymore.
You're right. Fixed.
bq. In StatementRestrictions, a few added imports are not needed (including the
NotImplementedException one).
Fixed this and several other cases.
|[3.0
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:12373-3.0]|[3.11
patch|https://github.com/apache/cassandra/compare/cassandra-3.11...ifesdjeen:12373-3.11]|[dtests|https://github.com/apache/cassandra-dtest/compare/master...ifesdjeen:12373]|[2.2
pa
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146940#comment-16146940
]
Alex Petrov edited comment on CASSANDRA-12373 at 8/30/17 12:08 PM:
---
[~slebresne] thank you for the review.
I have fixed the problems with dense/non-dense supercolumns. It looks like from
the CQL perspective non-dense SC tables are treated as if they were "normal"
tables, since the internal map column isn't revealed through the CQL operations
in any way. Of course, from the thrift perspective this is quite different.
While going through the patch I've noticed several more problems, namely:
* counters were not fully supported (they did work for the reads, however
writes were not functional)
* LWTs were not working
* {{SELECT}} queries with the supercolumn key restriction was working
incorrectly
There were several more smaller fixes. All these things are now covered with
tests. I've also updated the patch with your suggestions. Pushed the changes to
the
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:12373-3.0]
branch to possibly save some cycles. 3.11 rebase should be relatively easy,
with some minor changes.
[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...ifesdjeen:12373-2.2]
version of branch contains updated tests to make the comparison with an older
version simpler.
was (Author: ifesdjeen):
[~slebresne] thank you for the review.
I have fixed the problems with dense/non-dense supercolumns. It looks like from
the CQL perspective non-dense SC tables are treated as if they were "normal"
tables, since the internal map column isn't revealed through the CQL operations
in any way. Of course, from the thrift perspective this is quite different.
While going through the patch I've noticed several more problems, namely:
* counters were not fully supported (they did work for the reads, however
writes were not functional)
* LWTs were not working
* {{SELECT}} queries with the supercolumn key restriction was working
incorrectly
There were several more smaller fixes. All these things are now covered with
tests. I've also updated the patch with your suggestions. Pushed the changes to
the
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...ifesdjeen:12373-3.0]
branch to possibly save some cycles. 3.11 rebase should be relatively easy,
with some minor changes.
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.11.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[ https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090121#comment-16090121 ] Alex Petrov edited comment on CASSANDRA-12373 at 7/20/17 9:32 AM: -- I'll rebase on top of the latest branches as it seems like the patch has gotten a bit out of date. was (Author: ifesdjeen): I'll rebase on top of the latest trunk as it seems like the patch has gotten a bit out of date. > 3.0 breaks CQL compatibility with super columns families > > > Key: CASSANDRA-12373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12373 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Sylvain Lebresne >Assignee: Alex Petrov > Fix For: 3.0.x > > > This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column > compatibility. > The details and a proposed solution can be found in the comments of > CASSANDRA-12335 but the crux of the issue is that super column famillies show > up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward > compatibilty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604830#comment-15604830
]
Alex Petrov edited comment on CASSANDRA-12373 at 11/28/16 8:05 AM:
---
I hope I've covered all the corner cases. A small note on the implementation:
currently I'm using hardcoded column names {{column2}} and {{value}} for the
fake columns. If you think that can lead to some sort of collision or problem,
we can try relying on column name generation from {{CompactTables}}, although
it's going to be more or less same. These columns are not persisted together
with the rest of schema, they're completely virtual and created during
{{CFMetadata}} construction.
As [~slebresne] mentioned, {{SelectStatement}}, {{UpdateStatement}} and
{{DeleteStatement}} were modified to special-case supercolumns, converting from
map to two columns and reverse.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12373-trunk-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-testall/]|
|[3.X|https://github.com/ifesdjeen/cassandra/tree/12373-3.X-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-3.X-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-3.X-squashed-testall/]|
|[2.2|https://github.com/ifesdjeen/cassandra/tree/12373-2.2-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-testall/]|
|[dtest patch|https://github.com/ifesdjeen/cassandra-dtest/tree/12373]|
was (Author: ifesdjeen):
I hope I've covered all the corner cases. A small note on the implementation:
currently I'm using hardcoded column names {{column2}} and {{value}} for the
fake columns. If you think that can lead to some sort of collision or problem,
we can try relying on column name generation from {{CompactTables}}, although
it's going to be more or less same. These columns are not persisted together
with the rest of schema, they're completely virtual and created during
{{CFMetadata}} construction.
As [~slebresne] mentioned, {{SelectStatement}}, {{UpdateStatement}} and
{{DeleteStatement}} were modified to special-case supercolumns, converting from
map to two columns and reverse.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12373-trunk-squashed-2]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-2-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-2-testall/]|
|[2.2|https://github.com/ifesdjeen/cassandra/tree/12373-2.2-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-testall/]|
|[dtest patch|https://github.com/ifesdjeen/cassandra-dtest/tree/12373]|
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604830#comment-15604830
]
Alex Petrov edited comment on CASSANDRA-12373 at 11/28/16 7:57 AM:
---
I hope I've covered all the corner cases. A small note on the implementation:
currently I'm using hardcoded column names {{column2}} and {{value}} for the
fake columns. If you think that can lead to some sort of collision or problem,
we can try relying on column name generation from {{CompactTables}}, although
it's going to be more or less same. These columns are not persisted together
with the rest of schema, they're completely virtual and created during
{{CFMetadata}} construction.
As [~slebresne] mentioned, {{SelectStatement}}, {{UpdateStatement}} and
{{DeleteStatement}} were modified to special-case supercolumns, converting from
map to two columns and reverse.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12373-trunk-squashed-2]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-2-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-2-testall/]|
|[2.2|https://github.com/ifesdjeen/cassandra/tree/12373-2.2-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-testall/]|
|[dtest patch|https://github.com/ifesdjeen/cassandra-dtest/tree/12373]|
was (Author: ifesdjeen):
I hope I've covered all the corner cases. A small note on the implementation:
currently I'm using hardcoded column names {{column2}} and {{value}} for the
fake columns. If you think that can lead to some sort of collision or problem,
we can try relying on column name generation from {{CompactTables}}, although
it's going to be more or less same. These columns are not persisted together
with the rest of schema, they're completely virtual and created during
{{CFMetadata}} construction.
As [~slebresne] mentioned, {{SelectStatement}}, {{UpdateStatement}} and
{{DeleteStatement}} were modified to special-case supercolumns, converting from
map to two columns and reverse.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12373-trunk-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-trunk-squashed-testall/]|
|[2.2|https://github.com/ifesdjeen/cassandra/tree/12373-2.2-squashed]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12373-2.2-squashed-testall/]|
|[dtest patch|https://github.com/ifesdjeen/cassandra-dtest/tree/12373]|
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15608436#comment-15608436
]
Alex Petrov edited comment on CASSANDRA-12373 at 10/26/16 1:20 PM:
---
I've added a possible fix for that. In order to allow column renames, we have
to change {{LegacySchemaMigrator}} to pass the columns from older versions in
case of supercolumn family. In 3.0, however, they're removed from
{{clustering}} and {{regular}} and converted back to fake "virtual" columns
(with corresponding {{KIND}} and name, in order to avoid them popping up in
queries. Already upgraded tables will still work, even though their schema
doesn't have the second clustering and compact value columns, as they're
initialised as fake columns if missing.
was (Author: ifesdjeen):
I've added a possible fix for that. In order to allow column renames, we have
to change {{LegacySchemaMigrator}} to pass the columns from older versions in
case of supercolumn family. In 3.0, however, they're removed from
{{clustering}} and {{regular}} and converted back to fake "virtual" columns
(with corresponding {{KIND}} and name, in order to avoid them popping up in
queries. Already upgraded tables will still work, even though their schema
doesn't have the second clustering and compact value columns, as they're added
if missing.
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[ https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592857#comment-15592857 ] Alex Petrov edited comment on CASSANDRA-12373 at 10/20/16 8:11 PM: --- Thank you. LWTs [didn't work|https://github.com/ifesdjeen/cassandra/blob/b769e22899ff09dfaa598270121d3dba07dfebc3/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L234-L243], filtering [didn't work either|https://github.com/ifesdjeen/cassandra/blob/b769e22899ff09dfaa598270121d3dba07dfebc3/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L220-L232], although due to different reasons. I'll just add SuperColumn-specific error messages then. was (Author: ifesdjeen): Thank you. LWTs [didn't work|https://github.com/ifesdjeen/cassandra/blob/b769e22899ff09dfaa598270121d3dba07dfebc3/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L234-L243], filtering [didn't work either|https://github.com/ifesdjeen/cassandra/blob/b769e22899ff09dfaa598270121d3dba07dfebc3/test/unit/org/apache/cassandra/cql3/validation/ThriftIntegrationTest.java#L220-L232], although due to different reasons. I'll just add SuperColumn-specific then. > 3.0 breaks CQL compatibility with super columns families > > > Key: CASSANDRA-12373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12373 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Sylvain Lebresne >Assignee: Alex Petrov > Fix For: 3.0.x, 3.x > > > This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column > compatibility. > The details and a proposed solution can be found in the comments of > CASSANDRA-12335 but the crux of the issue is that super column famillies show > up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward > compatibilty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[ https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585042#comment-15585042 ] Alex Petrov edited comment on CASSANDRA-12373 at 10/18/16 12:39 PM: [~slebresne] [~iamaleksey] do we want to support things like materialized views? (sorry about modification storm). was (Author: ifesdjeen): [~slebresne] [~iamaleksey] do we want to support things like materialized views and group by? Or we just want a basic read support so that people could migrate from this table type? > 3.0 breaks CQL compatibility with super columns families > > > Key: CASSANDRA-12373 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12373 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Sylvain Lebresne >Assignee: Alex Petrov > Fix For: 3.0.x, 3.x > > > This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column > compatibility. > The details and a proposed solution can be found in the comments of > CASSANDRA-12335 but the crux of the issue is that super column famillies show > up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward > compatibilty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585042#comment-15585042
]
Alex Petrov edited comment on CASSANDRA-12373 at 10/18/16 12:31 PM:
[~slebresne] [~iamaleksey] do we want to support things like materialized views
and group by? Or we just want a basic read support so that people could migrate
from this table type?
was (Author: ifesdjeen):
[~slebresne] [~iamaleksey] do we want to also support filtering on {{column2}}
and {{value}} for supercolumns? In {{2.x}} it wasn't supported (predicate on
non-2i column).
Also, do we want to support things like materialized views and group by? Or we
just want a basic read support so that people could migrate from this table
type?
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585042#comment-15585042
]
Alex Petrov edited comment on CASSANDRA-12373 at 10/18/16 11:53 AM:
[~slebresne] [~iamaleksey] do we want to also support filtering on {{column2}}
and {{value}} for supercolumns? In {{2.x}} it wasn't supported (predicate on
non-2i column).
Also, do we want to support things like materialized views and group by? Or we
just want a basic read support so that people could migrate from this table
type?
was (Author: ifesdjeen):
[~slebresne] [~iamaleksey] do we want to also support filtering on {{column2}}
and {{value}} for supercolumns? In {{2.x}} it wasn't supported (predicate on
non-2i column).
Also, do we want to support aggregates (like {{max}} etc), materialized views
and group by? Or we just want a basic read support so that people could migrate
from this table type?
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1025#comment-1025
]
Alex Petrov edited comment on CASSANDRA-12373 at 10/12/16 10:56 AM:
I've started collecting information on what needs to be done. I just want to
clarify the behaviour first:
We would like to change the way schema and the resultset are currently
represented (instead of the {{"" map}} to two actual
columns: {{column}} (depending on the current clustering key size) and
{{value}}, just as it was presented in example in [CASSANDRA-12335], although
preserve their internal representation (internally, map type will still be used
for storage).
In CQL terms
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
"" map,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
would return results in form of
{code}
key | column1 | column2 | value |
--+-+-++
key1 | val1| 1 | value1 |
key1 | val1| 2 | value2 |
key1 | val1| 3 | value3 |
key1 | val2| 1 | value1 |
key1 | val2| 2 | value2 |
key1 | val2| 3 | value3 |
{code}
(note that {{column2}} is not clustering as [~slebresne] described in comment).
And this kind of special-casing will be valid for both read and write paths.
was (Author: ifesdjeen):
I've started collecting information on what needs to be done. I just want to
clarify the behaviour first:
We would like to change the way schema and the resultset are currently
represented (instead of the {{"" map}} to two actual
columns: {{column}} (depending on the current clustering key size) and
{{value}}, just as it was presented in example in [CASSANDRA-12335], although
preserve their internal representation (internally, map type will still be used
for storage).
In CQL terms
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
"" map,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
would become
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
column2 int,
value ascii,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
(note that {{column2}} is not clustering as [~slebresne] described in comment).
And this kind of special-casing will be valid for both read and write paths.
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12373) 3.0 breaks CQL compatibility with super columns families
[
https://issues.apache.org/jira/browse/CASSANDRA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1025#comment-1025
]
Alex Petrov edited comment on CASSANDRA-12373 at 10/12/16 9:31 AM:
---
I've started collecting information on what needs to be done. I just want to
clarify the behaviour first:
We would like to change the way schema and the resultset are currently
represented (instead of the {{"" map}} to two actual
columns: {{column}} (depending on the current clustering key size) and
{{value}}, just as it was presented in example in [CASSANDRA-12335], although
preserve their internal representation (internally, map type will still be used
for storage).
In CQL terms
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
"" map,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
would become
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
column2 int,
value ascii,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
(note that {{column2}} is not clustering as [~slebresne] described in comment).
And this kind of special-casing will be valid for both read and write paths.
was (Author: ifesdjeen):
I've started collecting information on what needs to be done. I just want to
clarify the behaviour first:
We would like to change the way schema and the resultset are currently
represented (instead of the {{"" map}} to two actual
columns: {{column}} (depending on the current clustering key size) and
{{value}}, just as it was presented in example in [CASSANDRA-12335].
In CQL terms
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
"" map,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
would become
{code}
CREATE TABLE tbl (
key ascii,
column1 ascii,
column2 int,
value ascii,
PRIMARY KEY (key, column1))
AND COMPACT STORAGE
{code}
(note that {{column2}} is not clustering as [~slebresne] described in comment).
And this kind of special-casing will be valid for both read and write paths.
> 3.0 breaks CQL compatibility with super columns families
>
>
> Key: CASSANDRA-12373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12373
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Alex Petrov
> Fix For: 3.0.x, 3.x
>
>
> This is a follow-up to CASSANDRA-12335 to fix the CQL side of super column
> compatibility.
> The details and a proposed solution can be found in the comments of
> CASSANDRA-12335 but the crux of the issue is that super column famillies show
> up differently in CQL in 3.0.x/3.x compared to 2.x, hence breaking backward
> compatibilty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
