[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980427#comment-14980427 ] Robert Stupp commented on CASSANDRA-10365: -- [~adutra] yes, that's a know issue in Aleksey's branch AFAIK. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980470#comment-14980470 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- bq. In SchemaKeyspace.fetchKeyspacesOnly, the initial query looks unecessary. If it's there as a way to exclude any unexisting keyspace, a comment explaining so would be welcome. But it's only used after having applied mutation that we knew applied to the keyspace passed to this method, so really, I'm not convinced it's useful. It does look a bit silly, doesn't it. Retroactively, it kind of makes sense - it's possible that one of the mutation involved in merging drops a keyspace, so we need to exclude them. But I don't think I put it there for any particular reason. Its presence doesn't bother me much - it's still huge savings all over - now that we don't need to read the previous state at all, and with reload changes, but I'll have a look. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980475#comment-14980475
]
Alexandre Dutra commented on CASSANDRA-10365:
-
Thanks for the quick reply.
Second nit: it seems that initconds of type text are broken, not sure if it's
due to CASSANDRA-10617:
{code}
cqlsh:test> CREATE FUNCTION concat(s1 text, s2 text) RETURNS NULL ON NULL INPUT
RETURNS text LANGUAGE java AS 'return s1 + s2;';
cqlsh:test> CREATE AGGREGATE group(text) SFUNC concat STYPE text INITCOND
'Hello';
ServerError:
{code}
{noformat}
Caused by: org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL
term: [Hello] reason: SyntaxException line 0:-1 no viable alternative at input
''
at
org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:65)
~[main/:na]
at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51) ~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.createAggregateFromRow(SchemaKeyspace.java:1229)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1208)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1133)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:891)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesOnly(SchemaKeyspace.java:881)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1281)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1260)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$1.runMayThrow(MigrationManager.java:491)
~[main/:na]
{noformat}
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980356#comment-14980356
]
Alexandre Dutra commented on CASSANDRA-10365:
-
[~snazy] While testing the Java driver against your branch I came across this
bug, also reproducible via cqlsh:
{code}
cqlsh> create keyspace test with replication = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
cqlsh> use test;
cqlsh:test> create type "Foo" (f1 int);
cqlsh:test> describe type "Foo";
CREATE TYPE test."Foo" (
f1 'int'
);
cqlsh:test> create table t1 (pk int primary key, c1 frozen<"Foo">);
ServerError:
{code}
It seems that mixed-case identifiers are being lower-cased somewhere.
The server logs show:
{noformat}
Caused by: org.apache.cassandra.exceptions.InvalidRequestException: Unknown
type test.foo
at
org.apache.cassandra.cql3.CQL3Type$Raw$RawUT.prepare(CQL3Type.java:536)
~[main/:na]
at
org.apache.cassandra.cql3.CQL3Type$Raw$RawTuple.prepare(CQL3Type.java:596)
~[main/:na]
at
org.apache.cassandra.schema.CQLTypeParser.parse(CQLTypeParser.java:55)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.createColumnFromColumnRow(SchemaKeyspace.java:1024)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.lambda$fetchColumns$242(SchemaKeyspace.java:1008)
~[main/:na]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_66]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchColumns(SchemaKeyspace.java:1008)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:962)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:941)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:889)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesOnly(SchemaKeyspace.java:881)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1281)
~[main/:na]
at
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1260)
~[main/:na]
{noformat}
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980668#comment-14980668 ] Alexandre Dutra commented on CASSANDRA-10365: - Up-to-date Java driver version [here|https://github.com/datastax/java-driver/pull/467]. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980714#comment-14980714 ] Sylvain Lebresne commented on CASSANDRA-10365: -- bq. Its presence doesn't bother me much - it's still huge savings all over - now that we don't need to read the previous state at all I don't dispute that, and while I suspect we might be able to avoid it, it actually doesn't really bother me. The lack of comment does bother me however given that it looks fishy without context. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980775#comment-14980775
]
Sylvain Lebresne commented on CASSANDRA-10365:
--
Remarks on the 3rd patch (understanding that it's not 100% complete):
* Bit of a nitpick, but I don't love the addition of the keyspace to {{Types}}
because it's not consistent with it's siblings ({{Tables}} and {{Functions}}).
It seems to be there just because we need it, along with the types, in some of
{{CQL3Type.Raw.prepare(Types)}} implementations. Can't we just pass the
keyspace name as a first argument to that method instead? If you care about
leaving it in {{Types}} however, lets at least take it into account in
{{equals}} and {{hashCode}}.
* {{CQL3Type.UserDefined.toString()}} doesn't quote stuffs properly (if the udt
name is a quoted name). {{SchemaKeyspace.bbToString()}} has the same problem.
There is a {{ColumnIdentifier.maybeQuote()}} method that could be reused for
info.
* Can you add a comment to fetchDroppedColumns to remind the reader why using
Types.none() is ok (just a pointer to expandUserTypes is fine).
* The use of {{Types.none()}} in {{CQLSSTableWriter}} striked me as weird. That
made realize however that I wasn't sure if UDT were actually usable with
{{CQLSSTableWriter}} at all, and how if they are. Do you know more?
(side-note: making the remaining change in a new commit would be appreciated :))
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980788#comment-14980788 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- bq. That made realize however that I wasn't sure if UDT were actually usable with CQLSSTableWriter at all, and how if they are. Do you know more? They weren't usable before, and I do not really care to fix it in this ticket, so I just retained the current behavior. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981302#comment-14981302 ] Robert Stupp commented on CASSANDRA-10365: -- Yea - that's completely broken (until CASSANDRA-10617) > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980252#comment-14980252
]
Sylvain Lebresne commented on CASSANDRA-10365:
--
A few early remarks on the 2 first patch (the 2nd patch really, the 1st one is
fine):
* The naming of {{Schema.only()}} doesn't make it extra clear that it expects
keyspace names. Maybe {{getKeyspaces()}} (that would also be more consistent
with the rest of the class namings)?
* I tend to agree that the reloading in {{Keyspace.initCf}} shouldn't be
necessary so I would maybe prefer removing it rather that leaving a TODO that
will surely stay there a long time.
* In {{SchemaKeyspace.fetchKeyspacesOnly}}, the initial query looks unecessary.
If it's there as a way to exclude any unexisting keyspace, a comment explaining
so would be welcome. But it's only used after having applied mutation that we
knew applied to the keyspace passed to this method, so really, I'm not
convinced it's useful.
* There is minor inconsistencies in {{SchemaKeyspace}} method naming: you still
have {{createColumnFromColumnRow}} for instance, but renamed
{{createAggregateFromAggregateRow}} to {{createAggregateFromRow}}. I'd just
rename them all to {{createXFromRow}} really as that's easier. On the
functions/aggregates, you use {{fetchFunctions}} to mean both UDF and UDA, then
{{fetchUDFs}} for UDF but then {{createFunctionFromRow}} for UDF (should rename
the last one). Also, {{createAggregateFromRow}} would be more consistent as
{{createUDAFromRow}}.
* Any reason for removing the {{testConversionsInverses}} in {{CFMetadataTest}}
rather than converting it to the new code? I do feel this was a good test to
have. At the very least, the test was testing the inversion of hrift
conversions which isn't impacted by the patch and should be preserved.
I haven't looked at the 3rd patch yet at all so apologies if any of this is
fixed by that 3rd patch.
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978507#comment-14978507 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- Force-pushed updates to the same branch. Everything besides UDTs that refer to other UDTs should work now (except maybe certain parts of UDFs/UDAs, but I need the driver fixed first to verify them). Otherwise the last remaining bits is cross-UDT dependencies. Finalising that, will push soon-ish. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978555#comment-14978555 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- bq. Curious if you have given any thought to storing system_schema.aggregates.initcond as a text CQL literal. This is the only thing I have come across in my integration thus far that requires the client to parse types in order to reproduce DDL. [~snazy] was kind enough to volunteer to deal with that bit of the ticket - will most likely be done tomorrow, no objections. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978622#comment-14978622 ] Adam Holmberg commented on CASSANDRA-10365: --- Repeating here for additional contributors. Python dev branch for this is [422|https://github.com/datastax/python-driver/tree/422] - Requires CASSANDRA-10513 patch for use with cqlsh - Still testing functionality outside of metadata integration > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979182#comment-14979182
]
Robert Stupp commented on CASSANDRA-10365:
--
Pushed a commit that adds initcond literal to schema table
[here|https://github.com/snazy/cassandra/tree/10365-uda-3.0].
Specifically the patch does:
* add {{initcond_literal}} column to {{system_schema.aggregates}}, which
contains the literal from the _CREATE AGGREGATE_ statement
* add {{initcond_version}} column to {{system_schema.aggregates}}, which
contains the native protocol version used to serialize {{initcond}} value (to
be able to read it in future versions)
* add {{"tuple"}} case to {{CqlTypeParser}} (to get {{AggregationTest}} working)
Unfortunately we cannot reliably re-construct the CQL literal for a number of
reasons:
* there are no {{TypeSerializer}} implementations for tuples/UDTs
* AbstractTextSerializer.getString() returns the non-quoted string
* TupleType/UserType return BytesSerializer - this means that nested type
combinations cannot be reliably stringified (would return a hex representation
instead of CQL tuple/UDT literals)
WRT this ticket - although it is not nice to not have the INITCOND literal in a
_describeAggregate_ method for pre-3.0 UDAs, I *think* it's ok, since UDFs and
UDAs were experimental in 2.2 and it only affects a _describeAggregate_ method,
if the driver is not "smart enough" to reconstruct the literal itself from
{{initcond}}. So - for pre-3.0 UDAs such a _describeAggregate_ would not work
perfectly and might return some hint if {{initcond_literal}} column is {{null}}.
I'm not sure whether it's worth to fix the AbstractType/TypeSerializer
getString/toString functionality (for 3.0) since it's a 'bit' more code to
change.
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979372#comment-14979372
]
Robert Stupp commented on CASSANDRA-10365:
--
Alright - next try, same branch (force pushed):
* {{system_schema.aggregates.initcond}} is of type {{text}} and contains the
CQL term
* does not (yet) work perfectly due to the issues with
AbstractType/TypeSerializer mentioned above (created CASSANDRA-10617 for this)
* two unit tests in {{AggregationTest}} fail due these issues
* migration of 2.2 UDAs will work fine
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976821#comment-14976821
]
Adam Holmberg commented on CASSANDRA-10365:
---
[~iamaleksey] Curious if you have given any thought to storing
{{system_schema.aggregates.initcond}} as a text CQL literal. This is the only
thing I have come across in my integration thus far that requires the client to
parse types in order to reproduce DDL.
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975377#comment-14975377
]
Aleksey Yeschenko commented on CASSANDRA-10365:
---
The third (in-progress) commit switches types in schema tables to their
canonical CQL representation. That includes changes to 'functions' and
'aggregates' tables to no longer keep a surrogate {{signature}} column.
Only non-nested (in anything) UDTs are supported there. Unfortunately there is
an annoying bug in {{TypeParser}} that breaks down for some type that don't
even involve UDT's (think {{frozen,
frozen>>}}). Need a bit more time to resolve that.
In the meantime, this unblocks drivers work/critical path.
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974317#comment-14974317 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- I apologise for the delay. The branch will be (mostly) complete later today, but I'm pushing the preparation commit(s) early [here|https://github.com/iamaleksey/cassandra/commits/10365] in case Sylvain might have an early look. The second commit switches reading schema to pure CQL vs. using commands directly, as before, and rewrites the way merging of before/after is performed. With UDTs (soon) referring other UDTs potentially, we can no longer perform the diff on row-by-row basis, and instead *have to* perform it on full snapshot vs. full snapshot basis. The rest to follow shortly. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958593#comment-14958593 ] Olivier Michallat commented on CASSANDRA-10365: --- The Java driver already has this kind of indirection, for example for state and final functions in an aggregate's metadata. It will make things more brittle in the face of potential event propagation bugs, but if it's an assumed choice I'm fine with it. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954916#comment-14954916 ] Sylvain Lebresne commented on CASSANDRA-10365: -- bq. if the goal is to hide implementation details Who said it was the only goal? It's a goal, but avoiding too much unnecessary repetitions is also a goal. Of course, not making too hard on client implementors is also a goal. That said, I'm still wondering what is so hard about resolving UDTs. If you're going to read the columns definition, what's so hard about reading the type definitions first? I understand it's slightly less convenient but coming up with a new ad hoc notation just for that isn't terribly clean imo. Are you guys really that scared by a small indirection? > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954884#comment-14954884 ] Jorge Bay commented on CASSANDRA-10365: --- I agree: if the goal is to hide implementation details, can we come up with a new notation for UDT that provides name, keyspace and fields, without the need to resolve UDT based on a name? > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955023#comment-14955023 ] Adam Holmberg commented on CASSANDRA-10365: --- No one said hiding implementation was the only goal. I was simply addressing that point. bq. what's so hard about reading the type definitions first? ... Are you guys really that scared by a small indirection? Everyone understands how this should work. My concern is that this changes the metadata model, possibly requiring a complete view of all UDTs for resolving types in other artifacts. It *should* be fine, but I think it makes the metadata model more brittle from the client perspective, considering that async schema change event propagation has been less than perfect. This will require us to deal with the fact that type resolution could fail. I might feel less strongly about this than some -- I think we have a way to avoid parsing these CQL strings at all. However, I know some drivers refer directly to these types in the model when binding UDTs for execution. Those will require parsing and resolving, or and API change to avoid it. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955211#comment-14955211 ] Adam Holmberg commented on CASSANDRA-10365: --- To be clear: I'm still in favor of this update. I simply wanted to explore the possibility to smooth things for some drivers by exposing self-contained types along side the cql types. If that's not tenable, there is still a solution, and we still have plenty of time to integrate. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952785#comment-14952785 ] Sylvain Lebresne commented on CASSANDRA-10365: -- bq. I think it is doable, but if keeping both is possible, it'd make the driver upgrades that much easier. I certainly understand that argument, but the fully-qualified class names are also leaking implementation details, namely which concrete classes are used server side. On the long term, removing that leakage is probably a good idea and might allow some cleanups of type classes at some point, and since we've redone the whole schema table, that sounds like a good time to do so. So while I understand this is a bit more work for the drivers (mostly, you'd have to either make sure you're up-to-date on types before querying columns, or resolve the types you don't know about by querying afterwards), I do feel that not conserving the fqcn version in new tables is the right thing to do. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953362#comment-14953362 ] Adam Holmberg commented on CASSANDRA-10365: --- If the concern is about leaking implementation details, would it be reasonable to instead store User Types as fully-specified abstract types as we do in the results encoding? https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v4.spec#L607-L618 I know this is not CQL, but it would avoid having to resolve inner types. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953572#comment-14953572
]
Adam Holmberg commented on CASSANDRA-10365:
---
That was the thought, yes. From your response I gather that this suggestion is
ridiculous. I'm not familiar with all the touch points, but I was hoping it
wouldn't be much harder to maintain than the current cassandra type string,
which fully specifies the type. For example:
{code}
org.apache.cassandra.db.marshal.UserType(test,74797031,7630:org.apache.cassandra.db.marshal.Int32Type,7631:org.apache.cassandra.db.marshal.UserType(test,74797030,7630:org.apache.cassandra.db.marshal.Int32Type,7631:org.apache.cassandra.db.marshal.Int32Type))
{code}
I was wondering aloud if we could abstract that, but still have the type
fully-specified in place. The most obvious downside is that there would then be
yet another string representation of types.
> Consider storing types by their CQL names in schema tables instead of
> fully-qualified internal class names
> --
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0.0 rc2
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return
> types, and UDT components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951339#comment-14951339 ] Bulat Shakirzyanov commented on CASSANDRA-10365: Just to clarify, the issue is that when we see a "org.apache.cassandra.db.marshal.UserType", it contains all information about its fields. Whereas when reading users.address, we will have to lookup the definition of that UDT separately, which may or may not be present in the driver's schema metadata. This might complicate the process of constructing schema metadata, especially considering that we still have to support previous versions of Cassandra that used fqcn style types. I think it is doable, but if keeping both is possible, it'd make the driver upgrades that much easier. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950408#comment-14950408 ] Alexandre Dutra commented on CASSANDRA-10365: - [~snazy] [~iamaleksey] is there a branch for this ticket where drivers could start testing with? > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950390#comment-14950390 ] Robert Stupp commented on CASSANDRA-10365: -- [~avalanche123] if I got your point, you suspect that UDT and UDF/UDA dependencies might cause problems resolving dependencies (and thus resulting in unresolved/stale types/UDAs), right? IMO this (having unresolvable types/UDFs/UDAs) is not related to whether type information is stored using class names or CQL type signatures. Generally, the situation is the same for the drivers and for C* internally. AFAIR [~aholmber]'s point was about parsing CQL types *back* to class-names (_AbstractType_). Of course, this change is not a trivial thing - neither for C* nor for the drivers. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945353#comment-14945353 ] Bulat Shakirzyanov commented on CASSANDRA-10365: I think this is a very useful metadata to include. My only suggestion is to keep the old representation (fqcn) as well. Currently drivers use this representation to detect type information. Parsing CQL-only representation might not be robust for the reasons mentioned in https://issues.apache.org/jira/browse/CASSANDRA-6717#comment-14791788. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.0 rc2 > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803295#comment-14803295 ] Robert Stupp commented on CASSANDRA-10365: -- I'd appreciate this very much as we "convert" a lot between CQL types and _AbstractType_. IMO we have to do this anyway in the (more or less) near future - and this (as schema tables are changed for 3.0) is a good chance to do it. As soon as AbstractType is no longer used in the schema tables, we are free to "streamline" the code and remove AbstractType completely (as soon as Thrift is gone). > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.x > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803281#comment-14803281 ] Aleksey Yeschenko commented on CASSANDRA-10365: --- /cc [~snazy] [~aholmber] [~adutra] > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.x > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10365) Consider storing types by their CQL names in schema tables instead of fully-qualified internal class names
[ https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804302#comment-14804302 ] Adam Holmberg commented on CASSANDRA-10365: --- I think this would be a welcome improvement to the metadata model. Would be interested in getting it in ahead of 3.0 GA, as it may warrant metadata API changes in the driver release. > Consider storing types by their CQL names in schema tables instead of > fully-qualified internal class names > -- > > Key: CASSANDRA-10365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10365 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Labels: client-impacting > Fix For: 3.0.x > > > Consider saving CQL types names for column, UDF/UDA arguments and return > types, and UDT components. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
