[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610896#comment-17610896 ] Manish Khandelwal commented on CASSANDRA-14825: --- Since partial support for COMPACT STORAGE was restored via https://issues.apache.org/jira/browse/CASSANDRA-16217, so with this change DESC output for "COMPACT STORAGE" is giving a warning *Warning: Table keyspace.tablename omitted because it has constructs not compatible with CQL (was created via legacy API).* *Approximate structure, for reference:* *(this should not be used to reproduce this schema)* Is this the right behavior since Cassandra 4 allows creating table with COMPACT STORAGE and we are not describing the table schema. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta1, 4.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148397#comment-17148397 ] Benjamin Lerer commented on CASSANDRA-14825: A big thanks you at [~djoshi] for his work on the review and to [~cnlwsu] and [~snazy] for their patches. It was a long run but we made it! \o/\o/\o/ > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148105#comment-17148105 ] Dinesh Joshi commented on CASSANDRA-14825: -- Thanks [~blerer]. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147748#comment-17147748 ] Benjamin Lerer commented on CASSANDRA-14825: [~djoshi] I rebased the C* and DTest branch and re-run the [tests|https://app.circleci.com/pipelines/github/blerer/cassandra/21/workflows/57517663-0815-4a00-a42e-e88f2db799a7]. The failing tests are now similar to the one failing in your trunk branch. I think we are good to commit. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146709#comment-17146709 ] Dinesh Joshi commented on CASSANDRA-14825: -- Hi [~blerer], the changes look ok but there are some failures that I think we should investigate. I have kicked off a circleci jobs ([j11|https://circleci.com/workflow-run/7eb01c14-efa7-4f67-9172-c7e31bc42e33] and [j8|https://circleci.com/workflow-run/a2780b93-14f5-4e8e-8eae-2841ac412c07]) on trunk to see if those failures are also on trunk. If they are then we're not introducing a new regression so we can go ahead and merge. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142891#comment-17142891 ] Benjamin Lerer commented on CASSANDRA-14825: Sorry, for the delay. It took me some time to track down all the issues caused by the previous version of the patch. cassandra: [PR|https://github.com/apache/cassandra/pull/633] DTest: [PR|https://github.com/apache/cassandra-dtest/pull/80] circleci [JDK11|https://app.circleci.com/pipelines/github/blerer/cassandra/20/workflows/56f4b700-d5fe-41ec-8082-37fe9ec07f4e] [JDK8|https://app.circleci.com/pipelines/github/blerer/cassandra/20/workflows/311d45ea-c7a2-44d1-bc95-7092c3252e81] Jenkins[https://ci-cassandra.apache.org/job/Cassandra-devbranch/168/] > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135401#comment-17135401 ] Dinesh Joshi commented on CASSANDRA-14825: -- hi [~blerer], I've opened up a [PR|https://github.com/apache/cassandra/pull/633] and left a few comments. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130348#comment-17130348
]
Benjamin Lerer commented on CASSANDRA-14825:
{quote}How's this going? Is anybody actively working on it as we near
completion of all the alpha tickets?{quote}
The status is still accurate. [~djoshi] and I already did part of the review.
Unfortunately we both have been busy with some other work and unable to finish
it.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Robert Stupp
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129847#comment-17129847 ] Jon Meredith commented on CASSANDRA-14825: -- How's this going? Is anybody actively working on it as we near completion of all the alpha tickets? > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101149#comment-17101149 ] Dinesh Joshi commented on CASSANDRA-14825: -- [~blerer] any updates? > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084039#comment-17084039 ] Benjamin Lerer commented on CASSANDRA-14825: [Jenkins ci-run|https://ci-cassandra.apache.org/job/Cassandra-devbranch/56/] > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17083928#comment-17083928 ] Benjamin Lerer commented on CASSANDRA-14825: Marking the fix version as 4.0 after discussion on the dev mailing list. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071202#comment-17071202 ] Chris Lohfink commented on CASSANDRA-14825: --- Yeah i initially thought this should be post-4.0 but if we making table changes it would make life a lot easier on drivers so I like idea of putting it in > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040381#comment-17040381 ] Jon Haddad commented on CASSANDRA-14825: It's unfortunate we hit a stall on this ticket, as the recent changes to some of the terms used mean all the drivers (and cqlsh) need to be updated to use new table options. It would be really great if we could get this into 4.0 and reduce some of the driver complexity. Any thoughts here? > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942660#comment-16942660 ] Benedict Elliott Smith commented on CASSANDRA-14825: I am aware of that. I think you have a different understanding of what that achieved than I do. That knowledge is incorporated into my reading of the situation, and view of how we should proceed, as stated above. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942659#comment-16942659 ] Robert Stupp commented on CASSANDRA-14825: -- It's not. Chris and I discussed the approach in person at ApacheCon. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942624#comment-16942624
]
Benedict Elliott Smith commented on CASSANDRA-14825:
FWIW, I personally think it was a bit rude to produce a patch in replacement of
Chris' patch, when his was blocked only on community consensus on approach. I
would prefer to offer Chris the opportunity to modify his patch to implement
{{DESCRIBE}}, as this is only polite given the time and effort he put into it.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Robert Stupp
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942531#comment-16942531 ] Robert Stupp commented on CASSANDRA-14825: -- [~cnlwsu] wanna take a look at the PRs (code + dtest)? > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934349#comment-16934349
]
Robert Stupp commented on CASSANDRA-14825:
--
I've opened [a PR|https://github.com/apache/cassandra/pull/361] that implements
the solution as a {{DESCRIBE}} statement. The syntax has been borrowed from
what cqlsh implements.
{{DESCRIBE}} returns a result set with a single text column, the number or
returned rows depends on the amount of data and the implementation. Callers
just concatenate the contents of the single column of all rows to get the
requested DDL script.
The paging implementation takes care of paging and ensures that the statement
properly aborts when the schema has changed between two pages with an
appropriate error message.
The implementation itself is ready to run even a {{DESCRIBE FULL SCHEMA WITH
INTERNALS}} over a ton of keyspaces and tables.
cqlsh has been adopted as well.
A {{DESCRIBE ... WITH INTERNALS}} shows the internal things like table ID and
emits a DDL script that re-creates dropped columns.
[All dtests|https://github.com/apache/cassandra-dtest/pull/53] checking the
DESCRIBE output have been migrated into a new unit test (i.e. will be skipped
except for a rudimentary test).
A
[readme|https://github.com/snazy/cassandra/blob/14825-expose-schema-describe/doc/DESCRIBE.md]
is present as well.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924360#comment-16924360
]
Chris Lohfink commented on CASSANDRA-14825:
---
Ok so sounds like we should forward with {{DESCRIBE}} then? I will make that
change along with the additional TableCQLHelper fixes/changes from Andys review.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924320#comment-16924320
]
Aleksey Yeschenko commented on CASSANDRA-14825:
---
I think I've been on all three sides of this argument over the past years.
Ever since I rewrote the schema system tables for 3.0 in CASSANDRA-6717, the
code to generate 3.0+ from system tables has been relatively easy. It will
become even easier in 4.0 with {{COMPACT STORAGE}} gone. So I've never felt
strongly about a need to have a server-side {{DESCRIBE}} variant in any shape
or form.
But, much like [~slebresne], I've come around since then. There is value in a
server-side {{DESCRIBE}} implementation. Less than there was before, and will
be even less once 4.0 is out, but there still is.
Now, I love that we have virtual tables now, and I can see this functionality
implemented using them. But my preference would be slightly in favour of moving
{{cqlsh}} {{DESCRIBE}} into its set of standalone CQL statements, on balance.
It's not at all invasive, IMO, but would allow us to have nice tailored
grammar, more flexible than a generic {{SELECT}} statement could ever be.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924307#comment-16924307
]
Jeremiah Jordan commented on CASSANDRA-14825:
-
bq. That's why the pre-existence (and reasonable adequacy so far) of DESCRIBE
is very relevant to me: since we've not removing DESCRIBE in cqlsh, pushing it
server side is really the only option that does not create 2 ways to "describe
the schema"
This makes sense to me and I think this is a big reason to not do a virtual
table, unless we are going to remove DESCRIBE from cqlsh and start telling
users that they get back the schema with {select * from ...}
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924295#comment-16924295 ] Chris Lohfink commented on CASSANDRA-14825: --- It's not complex given the nearly year long context of this discussion to go either way. Theres been multiple people saying opposite things, and ultimately imho everyone has been correct. I would not just say people are ignoring each other and there is a clear and obvious best solution. I just would like something final so we can move on with the ticket. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924234#comment-16924234
]
Sylvain Lebresne commented on CASSANDRA-14825:
--
{quote}it could support versioning of CQL grammar
{quote}
Good point. And again, that fall into the category of {{DESCRIBE}} makes it
easy to have schema specific stuffs, more so than a virtual table approach.
{quote}{{cqlsh}} having it already does not seem particularly important, and
should not bind our future decisions. It is a single tool, even if it is
bundled.
{quote}
The {{cqlsh}} {{DESCRIBE}} has been _the_ main way to get schema users have
used for the past 7-ish years, it's the one documented everywhere, and afaict,
there is no proposal to remove it, so users will continue to use it. I fail to
see how building on something familiar and not creating multiple way to do
essentially the same thing is not at least a bit relevant, and advantageous.
Don't get me wrong, if something is bad, then sure, we shouldn't build on a bad
idea just because it's there, but are we making the argument that {{DESCRIBE}}
is _bad_?
{quote}Virtual tables are also very capable of surfacing the necessary
information to produce dependent types, for instance as a collection column of
the names of those type.
{quote}
I'm genuinely a bit unsure what you are trying to argue here in the context of
this ticket.
This ticket is explicitly about exposing the schema in CQL form, so as strings
at the end of the day, so I'm not sure how virtual tables brings structure to
that. Btw, the tables exposed by the current patch have no collections
whatsoever and no more structure that what {{DESCRIBE}} would give.
It's very possible you have something in mind that is not the current patch,
but I think you'll need to describe it at least a bit so we can discuss it.
{quote}I can say that I hate features like {{DESCRIBE}} because I have to go
and google the manual. With a virtual table interface, I just {{SELECT}}
{quote}
:)
How do you know which table to {{SELECT}}? Believe it or not, the table in the
current patch to get a given table schema is called
{{system_views.describe_table}} ...
{quote}There is a legitimate case to be made to support both approaches, in my
opinion.
{quote}
To clarify, my *main* position here is that doing both approaches would be a
mistake. Not that the virtual table approach is terrible (it's not), nor that
{{DESCRIBE}} is order of magnitude better.
In fact, my argument is that both approach are not different _enough_ to
justify adding user confusion by having 2 ways to do essentially the same thing
(exposing the schema in CQL form without having drivers rebuilding it manually
from our existing schema tables).
That's why the pre-existence (and reasonable adequacy so far) of {{DESCRIBE}}
is very relevant to me: since we've not removing {{DESCRIBE}} in cqlsh, pushing
it server side is really the only option that does not create 2 ways to
"describe the schema" (and the fact it's better at handling things like
versions or "internal" schema details finishes to convince me it's _at least_
not substantially worth than the virtual table approach, if not a bit better).
I genuinely believe that this, having different ways to do essentially the same
thing, is one of the thing we've been historically bad at and is a contributor
to the (deserved) reputation of C* of being hard to use/learn (obviously, not
the only factor, but one nonetheless). I wish for us to learn from our mistake,
not repeat them. I feel we can easily avoid it here.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924174#comment-16924174
]
Robert Stupp commented on CASSANDRA-14825:
--
We already have a structured representation of the schema in {{system_schema}},
IIRC
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924124#comment-16924124
]
Benedict commented on CASSANDRA-14825:
--
There is a legitimate case to be made to support both approaches, in my opinion.
{{DESCRIBE}}'s main advantage, to my mind, is that it could support versioning
of CQL grammar, so that you can ask for a string compatible with a given CQL
version. This is difficult to do cleanly with a virtual table.
The presented factors in its favour don't seem relevant to me, and please try
to avoid characterising others' disagreement with your points as
misunderstanding or ignoring them:
# {{cqlsh}} having it already does not seem particularly important, and should
not bind our future decisions. It is a single tool, even if it is bundled.
# Virtual tables are also very capable of surfacing the necessary information
to produce dependent types, for instance as a collection column of the names of
those type.
Further to this second point, virtual tables permit surfacing a whole lot more
information in a structured _searchable_ manner. Want to find which tables use
a given type? Search on that collection column.
>From a pure UX, personal preference perspective, I can say that I hate
>features like {{DESCRIBE}} because I have to go and google the manual. With a
>virtual table interface, I just {{SELECT}} and progressively narrow my
>criteria with {{WHERE}} clauses. I preferred this approach in SQL Server than
>I do today's {{DESCRIBE}} in {{cqlsh}}.
So, I can see a case for both. But if I had to choose, I would prefer virtual
tables, and I haven't seen an argument that unequivocally seals the deal one
way or the other.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924088#comment-16924088
]
Robert Stupp commented on CASSANDRA-14825:
--
The implementation in the (Python) driver used by {{cqlsh}} is full of details
of database version specifics. Therefore, the reason to move the {{DESCRIBE}}
functionality to the server-side is absolutely fine. I think, there's consensus.
{{DESCRIBE}} is a well known command in {{cqlsh}}, so something that C* users
know for many years now.
But {{DESCRIBE}} is also a pretty complex implementation that is not that easy
to map to a {{SELECT}}, especially considering exposing database internals that
are necessary for backup/restore use cases. My point here is: it's much easier
to extend the syntax of {{DESCRIBE}} to make it more user friendly than to add
new things via system views. In other words: {{DESCRIBE}} is much more flexible
IMO.
A server side {{DESCRIBE}} doesn't need any protocol changes, it would be
implemented like {{LIST ROLES}} is - i.e. returning a result set with a single
column of type {{text}}. All that users of {{DESCRIBE}} would have to do is to
concatenate the strings from all rows to get a DDL script. Paging is pretty
important in the context of {{DESCRIBE}}, as some people have a ton of tables
in a single keyspace, so the DDL to describe a single keyspace can become quite
big. Even the {{DESCRIBE}} of a single table can become big - some MVs here, a
lot of UDTs there (it shouldn't, yes, but bad data models exist). Thing of a
{{DESCRIBE SCHEMA}} over 200 keyspace with 100 tables in each using 100 UDTs -
it's insane and probably doesn't exist in the wild, but thanks to the schema
code improvements (already committed and future ones), this might become more
and more frequent.
Having two things that do (nearly) the same thing is confusing IMO.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924060#comment-16924060
]
Sylvain Lebresne commented on CASSANDRA-14825:
--
I was trying to have an argumented and as-objective-as-possible discussion on
the pros and cons of each approach. I honestly don't feel there has been much
of that: a lot of the early conversation has dismissed the {{DESCRIBE}}
approach on clear mis-comprehension of the proposal (even the voting email
still talk of protocol changes, which is simply not part of the suggestion at
all), touting advantages of the virtual table approach that just aren't there;
the more recent arguments for why the virtual table approach would be superior
seems to be a tad subjective ("IMO the right UX", "more natural").
And in particular, no-one even acknowledged the points about "how do we
properly expose the internal parts of the schema". This ticket only feels stuck
because you guys have made up your mind and don't want to discuss it anymore,
not because all relevant points have been fully considered.
But with that said, I've done my job of raising the arguments I saw, and If
there is no interest in it, feel free to decide with a vote.
{quote}Given that and that the virtual table implementation is rather
concise(300 lines), I felt it pragmatic to consider doing both.
{quote}
My point against having 2 solutions for the same thing wasn't a line-of-code
consideration (at least not primarily, having 2 times 300 lines of code is
still worth than having only it one time for maintenance). It's a user-centric
concern. Having multiple ways to do the same thing is exactly the type of
things that further C* reputation of being hard to approach.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923751#comment-16923751 ] Chris Lohfink commented on CASSANDRA-14825: --- I'll start a thread on mailing list. Everyone is right and there is no consensus in sight so we just need a definitive one or other we can proceed with review. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923747#comment-16923747 ] Benedict commented on CASSANDRA-14825: -- Perhaps we should break this deadlock with a non-binding vote? Possibly on the mailing list? Since nobody feels strongly, it looks set to go round in circles if we don't just pick an approach and go with it. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923720#comment-16923720
]
Andy Tolbert commented on CASSANDRA-14825:
--
{quote}
I think you misunderstood Robert Stupp's comment above. He's not saying we
should stick to cqlsh-based DESCRIBE, he's mentioning the alternative to the
virtual table approach of having DESCRIBE being a genuine CQL (server-side)
command, one that would return result sets (and that I describe in a number of
comments above).
{quote}
I understand what he's referencing in context of the discussion of providing
server-side {{DESCRIBE}}, that is, not cqlsh {{DESCRIBE}}) support. The
virtual table approach is something that there is an available patch for that
works perfectly fine and will meet a lot of use cases, whereas server-side
{{DESCRIBE}} does not yet exist and may require more discussion. Given that
and that the virtual table implementation is rather concise(300 lines), I felt
it pragmatic to consider doing both.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923709#comment-16923709
]
Sylvain Lebresne commented on CASSANDRA-14825:
--
bq. but is there any reason why we can't also offer a virtual table solution?
That would mean supporting 2 different ways to do the exact same things. That's
usually not considered ideal.
bq. as it allows users to access schema via any driver, and doesn't depend on
the drivers to build schema
I think you misunderstood [~snazy]'s comment above. He's not saying we should
stick to cqlsh-based {{DESCRIBE}}, he's mentioning the alternative to the
virtual table approach of having {{DESCRIBE}} being a genuine CQL (server-side)
command, one that would return result sets (and that I describe in a number of
comments above).
This would still give users access to the schema via any driver (without any
driver change) and would not depend on the drivers to build schema.
And fwiw, I so far continue to believe that this server-side {{DESCRIBE}}
approach is, as objectively as I can put it, better than a virtual-table one.
As I've mentioned before, users already know {{DESCRIBE}} and as {{DESCRIBE}}
is not going away cqlsh-side, the virtual table approach kind of creates 2
separate ways to "describe" schema for users, which again don't feel ideal to
me.
Additionally, and that's the point [~snazy] mentioned, using a specific command
(instead of basically reusing {{SELECT}}) gives us flexibility for schema
specific stuffs much more easily. As Robert says, there is subtleties when it
comes to schema, in particular some things that are kind of "internal" (dropped
columns record, table ID, ...), but can be necessary when needing to recreate
the schema identically. So there is a (genuine afaict) need for both getting
the schema with and without those internal info (something we currently support
badly, but it's not a reason to continue doing so).
With the {{DESCRIBE}} approach, this is simple, just support some {{WITH
INTERNALS}} to decide if those "internals" are returned or not. With the
virtual table approach, not so much. Adding syntax to {{SELECT}} that is only
ever useful when querying a handful of system views is ugly.
As for the argument that virtual tables give you the "full power of SELECT", I
think it's more theoretical than anything when you look into the details. It's
not like SELECT is _that_ flexible in the first place, it's somewhat limited by
what schema we pick for the system view. And the {{DESCRIBE}} syntax already
provides 1) full schema, 2) schema of one keyspace and 3) schema of one
"object" for all our schema objects (table, types, ...). What more do we need
in practice?
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923580#comment-16923580
]
Andy Tolbert commented on CASSANDRA-14825:
--
I do see how {{DESCRIBE}} can have value, but is there any reason why we can't
also offer a virtual table solution? Server providing schema via a select is a
big improvement over what currently exists as it allows users to access schema
via any driver, and doesn't depend on the drivers to build schema which has had
its share of bugs. In addition, the maintenance overhead of the virtual tables
seems pretty low as the code for that is approximately 300 lines. It will
definitely have value until a {{DESCRIBE}}-like implementation is available,
and will still be useful if/when there is something like {{DESCRIBE}}. The PR
also fixes some issues in {{TableCQLHelper}} as well.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923531#comment-16923531
]
Robert Stupp commented on CASSANDRA-14825:
--
Sorry for being very late to the ticket here.
The benefit of {{DESCRIBE}} over {{SELECT}} is that {{DESCRIBE}} can/could emit
a complete DDL script - e.g. a script like one that lands as the {{schema.cql}}
file in a snapshot, with all the dropped-columns. This is particularly
important, when you want to get the _really compatible_ schema for a table or
keyspace in backup/restore use cases or if you intend to move some sstables to
another cluster. User could for example issue a {{DESCRIBE
SCHEMA/KEYSPACE/TABLE foobar WITH INTERNALS}} and would get a complete script
that not just contains the keyspace + table definitions including the dropped
columns but also UDTs, UDFs, UDAs.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923085#comment-16923085
]
Andy Tolbert commented on CASSANDRA-14825:
--
+1. I tried this out again and left some review comments on the [pull
request|https://github.com/apache/cassandra/pull/284]. The only non-trivial
suggestion I have is to make the topic a clustering column (i.e. {{table_name}}
in {{system_views.describe_table}} should be a clustering column) so you can
make a query such as 'show me all tables in this keyspace'. On the other hand
I can see 'give me the entire schema for this keyspace' and 'give me the schema
for this particular thing' as more valuable usecases, this would be more of a
'nice to have'.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821337#comment-16821337
]
Chris Lohfink commented on CASSANDRA-14825:
---
I am not against making the protocol change to add a {{DESCRIBE}} for what its
worth, if anything I am neutral and can change patch to go either direction.
In the initial conversation that brought this ticket up was regrettably in IRC
and lost - It was thought that a virtual table provides easier access to other
drivers (even older) and the difference between a DESCRIBE and a SELECT on the
protocol/overhead perspective is essentially the same in terms of post
processing on driver or string generation on the server. Also virtual tables
doesn't require a protocol change so it could be done initially this way and
added in a future protocol version with little cost and easy migration if we
later decide it's a good idea to change. Consider it a lower risk beta view.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821334#comment-16821334 ] Benedict commented on CASSANDRA-14825: -- +1. This is the API I would prefer as a user - select is more natural to interact with than a command I'll need to consult the docs to understand the parameters to. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821297#comment-16821297 ] Jonathan Ellis commented on CASSANDRA-14825: IMO the right UX for interacting with "what tables and keyspaces are defined" is a system view and SELECT. SELECT composes by design with everything else (WHERE, even UDF) in a way that DESCRIBE does not. (It's fine to then say "we're not going to give you the schema in that view, you need to use DESCRIBE if you want that" but I don't think that's necessarily either better or worse.) I think it's more important to get the composability right at the CQL level, than give an easy way to get a full schema dump. I think the latter scenario is much more niche and I'm fine with requiring a for loop to get it. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683824#comment-16683824
]
Alex Petrov commented on CASSANDRA-14825:
-
This patch still uses compact tables API, which due to be deleted in 4.0. I'd
suggest we remove compact-table specific calls such as {{isStaticCompactTable}
and incompatibility note since it's not used anymore and it is impossible to
start 4.0 with compact tables. Or we can make it dependent on [CASSANDRA-13994].
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683564#comment-16683564 ] Sylvain Lebresne commented on CASSANDRA-14825: -- bq. just to be clear if you query describe_keyspace table you can iterate through the result set to get the entire schema (...) And in case there was doubt, I didn't say it wasn't the case. I'm not saying you can't get schema information through virtual tables. What I'm asking is, why use virtual tables when we could just promote to CQL the `DESCRIBE` statements every user is already familiar and which is, I think, a more fexible/direct approach? By which I mean that you can both get the granular if you want, but also get a full schema dump directly. With virtual tables, you get the granular, but a full schema dump requires a small amount of post-processing (_not_ saying it's hard, but it is harder than no post-processing at all). Additionally, it's very easy to add new options to statements, while once you settle on some virtual table schema, it can harder to evolve. What are the pros in favor of virtual tables that outweigh those 2 pros of promoting `DESCRIBE` (existing familiarity and at least some form of better flexibility; to which I could add not having 2 ways to do the same thing, since afaik, we're not going to remove `DESCRIBE` from cqlsh)? I get that virtual tables are everyone's new shiny hammer, but it's not an objective argument. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683015#comment-16683015
]
Chris Lohfink commented on CASSANDRA-14825:
---
just to be clear if you query describe_keyspace table you can iterate through
the result set to get the entire schema ({{describe schema}} equivalent) in the
correct order of all types, tables and functions. All the existing mechanisms
to make DESCRIBE query are done as different virtual tables.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681885#comment-16681885 ] Sylvain Lebresne commented on CASSANDRA-14825: -- bq. At the least we should provide selecting by keyspace, ideally keyspace and table. Ok, sure. As my comments hopefully make it clear, I agree with you on those options needing to be provided. And I'm sure that's not controversial. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681820#comment-16681820 ] Nate McCall commented on CASSANDRA-14825: - [~slebresne] I'm talking specifically about returning a single (potentially large - which is my concern) row and column. At the least we should provide selecting by keyspace, ideally keyspace and table. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681723#comment-16681723 ] Sylvain Lebresne commented on CASSANDRA-14825: -- bq. I'm -1 on this. Good, bad or ugly, there are a lot of clusters out there that do multi-tennancy or otherwise have > 1k tables for which this would be untenable. I'm confused here. How is it fundamentally that different from the exact same query in cqlsh today? I assume those cluster simply do do it today and _nobody_ would force them to do it either. I assume you've read enough to see that nothing I propose would prevent those users to get their schema in a more incremental fashion. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681707#comment-16681707 ] Chris Lohfink commented on CASSANDRA-14825: --- bq. You require multiple querys and the user has to know how to put them back together in the right order to generate a valid schema cql file for use in a restore It does require 1 query per keyspace but it does put it in the right order too be a valid cql fille, there is no relation between the keyspaces so this is safe. It would be trivial to add one that iterates through keyspaces and provides a describe keyspace each though > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681696#comment-16681696 ] Nate McCall commented on CASSANDRA-14825: - bq. 1. Return a single row and column that just has the entire schema in it. I'm -1 on this. Good, bad or ugly, there are *a lot* of clusters out there that do multi-tennancy or otherwise have > 1k tables for which this would be untenable. Second option could work though as it would facilitate paging, etc. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681667#comment-16681667
]
Sylvain Lebresne commented on CASSANDRA-14825:
--
To maybe clarify/be more precise, when I say "promote the DESCRIBE syntax cqlsh
supports to proper CQL", what I have in mind is that each `DESCRIBE` statement
would return a result set with only one result, but there would many variants,
so:
{noformat}
DESCRIBE SCHEMA; // returns a single string with the whole schema
DESCRIBE KEYSPACE ks; // returns a single string, with all of keyspace 'ks'
DESCRIBE TABLE ks.t // returns a single string with just that one table.
...
{noformat}
Again, I think it's going much more messy to get something approaching that
flexibility with tables (doable, sure, but more messy).
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681660#comment-16681660 ] Jeremiah Jordan commented on CASSANDRA-14825: - Wether we add DESCRIBE or we have a virtual table or even both, we return data in result sets. If you want to be able to query the whole schema in a single query I can think of two reasonable options: 1. Return a single row and column that just has the entire schema in it. 2. Return one row per schema object with columns something like: keyspace, name, type, cql The current patch here proposes a different table per object type. I don’t think that is as usable as a being able to get a single result set with all schema objects in it. You require multiple querys and the user has to know how to put them back together in the right order to generate a valid schema cql file for use in a restore. With a single row containing the whole schema it can be in the right order or with a single resultset for all objects the rows can be returned in the correct order for replay. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681623#comment-16681623
]
Andy Tolbert commented on CASSANDRA-14825:
--
{quote}
[1]: but re-drivers, at least the existing ones, let's say I'm unconvinced this
will simply their life.
{quote}
Not in the near term, as long as drivers support versions older than a release
with this change, which will be a long time, the java and python driver will
still need to generate CQL DDL for those older versions. The benefit to the
drivers is more off in the future. The primary value add I see to this change
is that it unbinds CQL DDL generation from specific driver implementations,
allowing any client capable of connecting to C* to extract schema CQL DDL.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681223#comment-16681223
]
Sylvain Lebresne commented on CASSANDRA-14825:
--
CASSANDRA-8005 was a long time ago, and no-one was truly opposed strongly in
the first place. I also agree in hindsight than providing the canonical
representation server side is a good idea\[1\]. Plus, we didn't have the code
to generate the CQL server side at the time, but now that we have it, this is
really not much effort.
That being said, while I'm in favor of exposing the schema as CQL server side,
I'm a loss less convinced that virtual tables are the right approach.
There is a way and syntax that everyone is already familiar for this, it's the
{{DESCRIBE}} syntax from cqlsh. That syntax/way, on top of having the merit of
existing and being familiar, also makes it very easy to get schema elements in
a granular way: you can easily get the whole schema, a whole keyspace, just a
table, a type, an index, etc... Providing the same granularity\[2\] through
virtual tables feels substantially less convenient: I don't see how we could do
so without either rather complex (and thus inconvenient) schema, or tons of
virtual tables, neither of which sounds awesome to me.
So, why not just promote the {{DESCRIBE}} syntax cqlsh supports to proper CQL?
\[1\]: but re-drivers, at least the existing ones, let's say I'm unconvinced
this will simply their life.
\[2\]: I'm sure we can have debates on how granular is useful and how is too
much, but no matter where on stand, remains than the {{DESCRIBE}} allows
whichever granularity we want easily, virtual tables do not; and besides, we
can't ignore that granularity does exists in cqlsh today, so if one of the goal
is to ultimately make cqlsh use the server-side feature for this, we kind of
have to support the existing granularity.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673538#comment-16673538 ] Benedict commented on CASSANDRA-14825: -- FWIW, I'm +1 the principle of this change. The prior discussion didn't seem to touch on issues of modifications to the CQL syntax, such as renaming poorly named properties or otherwise updating the CQL language. When you multiply this dimension with the dimension of clients having to parse multiple schema versions, it seems unlikely all clients will get this right. That said, Aleksey's proposal of a virtual table holding a data dictionary for mapping things might work out. But I'm unclear on exactly how it would work, it sound like a lot of work to accomplish, and I don't personally see a problem with providing this functionality anyway. I also think there's independent value in a 'canonical' CQL string for a given database object. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662631#comment-16662631 ] Andy Tolbert commented on CASSANDRA-14825: -- I think it is worth revisiting the discussion from [CASSANDRA-8005]. We should emphasize that the goal of this is not to benefit the drivers or that the drivers would use this as part of it's schema parsing (we benefit from [CASSANDRA-6717] and the previous 2.x schema tables for that). Exposing the Schema DDL as a string is something some users and tools need, and I think the server can do a better job representing this correctly and managing changes release to release as the server only needs to concern itself with its current view of the schema vs. the driver having to consider all versions. I think two key points of discussion were: 1. Should an API exist that exposes the Schema creation statements as CQL (DDL)? 2. If yes, where should that exist? Having the drivers generate and maintain this functionality with schema changes happening release to release has not been easy. There is a lot of complexity and it is a big ask for it to be done correctly by multiple driver implementations. It is important that the schema CQL is correct, particularly in the case where this functionality is being used by tools for backup/restore or standing up new environments. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662624#comment-16662624 ] Chris Lohfink commented on CASSANDRA-14825: --- Ill ping you outside the jira [~aholmber] > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662620#comment-16662620 ] Adam Holmberg commented on CASSANDRA-14825: --- Big +1 from me on the notion of having a canonical implementation for reproducing DDL as part of the server project. My arguments are the same as posed on the previous ticket, and similar to the rationale echoed here (although the compendium of problem tickets will have grown). Side bar: [~cnlwsu] if it's not too much trouble, would you be so kind to characterize the deficiencies you observed in the driver implementations, or even open some tickets, so we can make them right in the mean time? > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662579#comment-16662579 ] Chris Lohfink commented on CASSANDRA-14825: --- bq. Let's see first how CASSANDRA-6717 and CASSANDRA-8099 pan out. Those two are completed now and per the conversation in IRC that brought this up - the driver developers are still struggling with this. Even things like gocql that may be entering the project doesn't support it. In implementing this I found errors in both the pythons output and the java drivers output (different ones) which also shows that even with two drivers looking at each others code it can have subtle differences. If it was all in 1 place that would be much easier to keep right. Not to mention more likely to catch issues with Cassandras Table->CQL code (which had errors as well) that currently exists. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662526#comment-16662526 ] Jeremiah Jordan commented on CASSANDRA-14825: - Exposing such information from the server has been a controversial topic in the past. See CASSANDRA-8005 for one such discussion. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652423#comment-16652423 ] Chris Lohfink commented on CASSANDRA-14825: --- |[trunk|https://github.com/clohfink/cassandra/tree/describe_vtables]|[units|https://circleci.com/gh/clohfink/cassandra/422]|[dtests|https://circleci.com/gh/clohfink/cassandra/423]| > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651775#comment-16651775 ] Andy Tolbert commented on CASSANDRA-14825: -- This is a really great idea! I agree that having C* share its view of the schema DDL feels more appropriate than having drivers try to piece it together from schema tables. The process of managing this in the drivers can and has been error prone, so I'd be really excited for these virtual tables to land in C* 4.0 if possible. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651168#comment-16651168 ] Chris Lohfink commented on CASSANDRA-14825: --- Couple other things in this PR that had to be fixed is that the TableMetadata -> CQL was broken (params were invalid), and extensions params are ignored from CREATE TABLE statements. Also note that currently cqlsh/python/java driver provide UDF and UDAs in DESCRIBE that are invalid and miss some quoting in the body that this handles so the output is different. Other than virtual tables option is to create a DESCRIBE query, although it will still return a ResultSet that the driver would need to parse and format so it wont be much difference from drivers perspective. In IRC agreement was to go with a virtual table instead as its less invasive. > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
