[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029787#comment-16029787
]
Robert Stupp commented on CASSANDRA-7396:
-
As discussed offline with Benjamin, let's keep it as it is.
Note for other readers: Adding _any_ restriction after a slice restriction is
effectively pointless. For example, {{m\[1..\]\[..2\]}} can and should be
written as {{m\[1..2\]}}, {{m\[1..\]\[2\]}} as {{m\[1..\]}}.
Therefore: +1, ship it!
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028127#comment-16028127
]
Benjamin Lerer commented on CASSANDRA-7396:
---
Forbidding these selectors is, in my opinion, different from what is done for
clustering keys and inequality predicates. Before to {{3.0}} we were preventing
any clustering restrictions after an inequality predicate, simply because we
had not put in place a mechanism to handle those queries. Since {{3.0}}, we
allow them as long as {{ALLOW FILTERING}} is specified.
CQL allow things like {{SELECT v + 1 - 1 FROM myTable}}, even if it is
inefficient. One of the problem of CQL is that it has a lot of corner cases,
that makes the language difficult to use and I have seen a lot of complains
about it. Which is why I am not in favor rejecting some specific mix of
selectors.
Moreover, you do not really want to forbid further operations after a slice as
a query like {{SELECT filterMap(m\[1..\])\[..2\] FROM ...}} can totally make
sense. And blocking a query like {{SELECT m\[1..\]\[..2\] FROM ...}} but not a
query like {{SELECT filterMap(m\[1..\])\[..2\] FROM ...}}, even if the
{{filterMap}} function is not doing any filtering, is also inconsistent.
So, from consistency point of view, I would argue that we should keep the
current behavior and not forbid further restrictions after a slice.
The operations perfomed by the selectors are straight forward:
* {{\[?\]}}: returns a collection element
* {{\[?..?\]}}: returns a slice of the collection
There is no notion of {{inner}} or {{outer}} collections because the selectors
can be used on a collection that does not contains another collection.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023597#comment-16023597
]
Robert Stupp commented on CASSANDRA-7396:
-
Well, restricting the first slice with a second slice ({{m\[1..\]\[2..\]}})
doesn't buy anything ({{headMap().tailMap()}} is not the same as what a slice
with absolute bounds does).
The confusing thing is that the second restriction for {{m\[1\]\[2\]}} refers
to the "inner" map but the second restriction in for {{m\[1..\]\[2..\]}} still
refers to the "outer" map. As you said, that requires "manual filtering"
(cannot be expressed as a cell filter). That's way I proposed to forbid further
restrictions after a slice. This would be consistent to what we already do for
clustering keys and inequality predicates.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023559#comment-16023559 ] Benjamin Lerer commented on CASSANDRA-7396: --- bq. Just want to ensure that we have the option to do that later, if there's demand for it. Using the same syntax to perform different type of operations is, in my opinion, extremely confusing. If we want to do it later, we will have to use another syntax or an other approach. As it is a type of operations where we cannot really optimize the fetching, I would be in favor of doing it as scalar functions. It is in my opinion the perfect place to do it. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql, docs-impacting > Fix For: 4.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023407#comment-16023407
]
Robert Stupp commented on CASSANDRA-7396:
-
bq. {{WRITETIME() + TTL()}} ... require some significant refactoring
Alright, let's tackle that in 8877.
The review-fixes LGTM and testall + dtests look fine.
Regarding the "double slice" thing:
I'd be fine if we don't allow any more restrictions after a slice selection.
Mean, there's currently no way to do what I intended to do with
{{m\[1..\]\[2\]}}. Some examples for what I'm thinking of:
* {{m\[1..\]\[2\]}} --> not allowed
* {{m\[1\]\[2\]}} --> allowed
* {{m\[1\]\[2..\]}} --> allowed
* {{m\[1..\]\[2..\]}} --> not allowed
* {{m\[1\]\[2..\]\[3\]}} --> not allowed
Just want to ensure that we have the option to do that later, if there's demand
for it.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022496#comment-16022496
]
Benjamin Lerer commented on CASSANDRA-7396:
---
bq. {{WRITETIME()}} + {{TTL()}} don’t work with element/slice selections, but
UDFs + CAST do work with element/slice selections.
This will require some significant refactoring to the {{Selector}} classes as
well as to the {{Selection}} and {{ResultSetBuilder}} classes. Due to that I
think it is best to do that improvement within CASSANDRA-8877.
I have pushed a new version of the patch
[here|https://github.com/apache/cassandra/compare/trunk...blerer:7396-trunk] to
fix the remaining issues. I ran the tests on our internal CI and the results
look good.
I renamed {{isTerminal}} to {{areAllFetchedColumnsKnown}} as the original name
was misleading. If we have a query like: {{SELECT m\['1'\] + ? FROM myTable}}
we do not need to have all the parameters bound to build the {{ColumnFilter}}.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021072#comment-16021072
]
Benjamin Lerer commented on CASSANDRA-7396:
---
For me the current behavior make sense. It is similar to
{{navigableMap.tailMap(1).get(2)}} and {{navigableMap.tailMap(1).heapMap(2)}}.
People need to reason in term of operations and not in term of nested level.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021028#comment-16021028
]
Robert Stupp commented on CASSANDRA-7396:
-
Yes, you're right ({{[2]}} technically refers to the slice).
Does that make sense that two restrictions ({{m[1..][2]}} and {{m[1..][..2]}})
are applied to the same map? {{m[1..][2]}} could simply be {{m[2]}} and
{{m[1..][..2]}} could be {{m[1..2]}}. Functionally, I expected that for
{{m[1..][2]}} {{m[1..]}} is applied to the "outer" map and {{[2]}} to the
"inner" map (i.e. the next level).
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020946#comment-16020946
]
Benjamin Lerer commented on CASSANDRA-7396:
---
{quote}There's some incorrectness with nested slice/element selections:{quote}
I had a look at the queries and the results are actually the good ones.
{code}
cqlsh> select m[1..][2] from foo.t1 where id='1';
m[1..][2]
---
null
{code}
{{m\[1..\]}} is not returning a single value. It is returning a slice of the
map which is also in this case the original map: {{*\{1:\{ 2: 'one-two'\} \}*}}.
{{\[2\]}} requires the map entry with the key {{2}} which does not exist. Due
to that a {{null}} is returned.
{code}
cqlsh> select m[1][..2] from foo.t1 where id='1';
m[1][..2]
{2: 'one-two'}
{code}
{{m\[1\]}} returns the map-entry with the key 1: {{*\{2: 'one-two'\}*}}.
{{\[..2\]}} return a slice of that map which include the map-entry with the key
, so also: {{*\{2: 'one-two'\}*}}
{code}
cqlsh> select m[1..][..2] from foo.t1 where id='1';
m[1..][..2]
-
{1: {2: 'one-two'}}
{code}
{{m\[1..\]}} is returning a slice of the map which is also in this case the
original map: {{*\{\{1: \{2: 'one-two'\} \}*}} and {{\[..2\]}} returns another
slice of the map which is also the original map.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015829#comment-16015829
]
Robert Stupp commented on CASSANDRA-7396:
-
Overall this patch looks pretty solid! Very nice work! Few comments and one bug
though:
* Having unit tests for UDTs (frozen and non frozen) that include frozen and
non-frozen collections would be great.
* the javadoc param {{from}} is duplicated in {{getSliceFromSerialized}}
* Wonder, whether to return the {{baseType}} if {{ReversedType.getInstance()}}
is called with a {{ReversedType}} is more efficient than creating a
{{ReversedType(ReversedType(someType))}}.
* {{WRITETIME()}} + {{TTL()}} don’t work with element/slice selections, but
UDFs + CAST do work with element/slice selections.
* IMO you can remove the copied javadocs on overrides/implementations
* A couple of “ugly” methods & constructs have been refactored and became way
more readable.
* There's some incorrectness with nested slice/element selections:
The unit and dtests look fine so far.
{code}
create keyspace foo WITH replication = {'class': 'NetworkTopologyStrategy',
'datacenter1':1};
CREATE TABLE foo.t1 (
id text PRIMARY KEY,
m map>>,
s set
);
insert into foo.t1 (id,m) values ('1', {1: {2: 'one-two'}});
cqlsh> select m[1][2] from foo.t1 where id='1';
m[1][2]
-
one-two
(1 rows)
cqlsh> select m[1..][2] from foo.t1 where id='1';
m[1..][2]
---
null
(1 rows)
cqlsh> select m[1][..2] from foo.t1 where id='1';
m[1][..2]
{2: 'one-two'}
(1 rows)
cqlsh> select m[1..][..2] from foo.t1 where id='1';
m[1..][..2]
-
{1: {2: 'one-two'}}
(1 rows)
cqlsh> select m[1..][2] from foo.t1 where id='1';
m[1..][2]
---
null
(1 rows)
{code}
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995494#comment-15995494
]
Benjamin Lerer commented on CASSANDRA-7396:
---
All my apologizes for the delay.
My main concern with the patch is the split of {{SelectStatement}} into
{{SelectStatement}} and {{SelectOptions}} as well as the introduction of the
{{Finalized}} in {{Selection}} and {{SelectOptions}}. I fully agree that we
should simply {{SelectStatement}} and that we need to have a nice way to deal
with the creation of {{ColumnFilter}} instances. Nevertheless, the patch
approach makes, in my opinion, the logic harder to follow.
I pushed a new version of the patch
[here|https://github.com/apache/cassandra/compare/trunk...blerer:7396-trunk].
The approach is to have {{Selection}} returning a {{Selectors}} interface. The
{{ColumnFilter}} is then retrieved from the {{Selectors}} interface.
If the {{ColumnFilter}} can be created at preparation time it will be created
only one. Otherwise it will be created upon request.
The {{ResultSetBuilder}} can then be created by passing the {{Selectors}}
interface as a constructor argument.
The patch also fix the ANTLR issues and add extra tests.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824574#comment-15824574 ] Benjamin Lerer commented on CASSANDRA-7396: --- The ticket is on my list but CASSANDRA-11935 has only been committed 2 months ago and I have been buzy elsewhere since then. I should be able to work on it in the coming weeks. Now, due to change in the release cycle it will be for the version 4.0. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824198#comment-15824198 ] Vassil Lunchev commented on CASSANDRA-7396: --- This ticket was last updated more than 6 months ago. Is it still in progress? > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360307#comment-15360307 ] Benjamin Lerer commented on CASSANDRA-7396: --- I can have a look at the ANTLR problems. While working on CASSANDRA-11935 I already had to refactor some part of the ANTLR code to be able to use syntactic predicates. By consequence, it might be best to wait a bit for that patch to avoid having do duplicate the efforts. The patch should be ready soon. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359122#comment-15359122
]
Sylvain Lebresne commented on CASSANDRA-7396:
-
Remarks on the patch:
* As this basically uses terms in select clauses, this should be rebased on top
of CASSANDRA-10783, rather than redoing it's own thing. I'm in particular not
at all a fan of the "dynamic" thing, especially when have already the concept
of {{Terminal}} and {{NonTerminal}} terms to deal with the same thing.
* This only allows element/slice selection directly on a column name, and
without nesting, which is imo overly restrictive (we don't have that
restriction for field selection for instance). That does change a bit how we
want this to work.
* The way {{SelectStatement}} deals with {{ColumnFilter}} feels a bit hacky to
me. I understand that we cannot always compute the {{ColumnFilter}} at
preparation time anymore, and that we may want to avoid doing it at execution
time if we can, but I think that could be more cleanly abstracted.
* The patch seems to use {{null}} to handle the absence of {{from}} or {{to}}
in the slice syntax. I'm not sure about that. I think we should refuse {{null}}
but accept {{unset}} and make it equivalent to not having a value. That's more
logical imo.
* I'm not sure about passing the Cell to the {{ResultSetBuilder}}. First
because having an {{Object}} array is somewhat ugly, but also because I think
trying to push along this line in CASSANDRA-7826 will get complicated. I think
it's simpler to serialize what we get from the storage blindly, and let
selector extract subselections from the serialized form aferwards (which they
can do without deserializing, working directly on the serialized form).
* It's a bit of an edge case, but {{SELECT m, m\['4'..'6'\] FROM ...}} wasn't
working as expected, as the {{ColumnFilter}} only ended up querying the
selected slice, ignoring the full column selection.
* There is also a problem with {{SELECT m\[3..4\] FROM ...}} because the parser
parse {{3.}} as a float and fails to recognize the slice syntax afterwards. Mor
eon tat below.
I took a shot at fixing those
[here|https://github.com/pcmanus/cassandra/commits/7396-trunk], which ends up
looking quite a bit different. I did however struggled with ANTLR, and there is
currently still a few parsing issue that prevent this from being "patch
available":
# The problem with {{SELECT m\[3..4\] FROM ...}} where {{3.}} is lexed as a
float. I tried to change the lexer using ANTLR a syntactic predicate to
presumably not lex {{3.}} as a float if it's followed by another {{.}}, but I
must have gotten that wrong as it didn't work. I also tried fixing in the
parser, making the accept '\[' term '.' term '\]' and rejecting that
afterwards if the left-most term isn't what it should, but ANTLR ended with
crazy conflicts. Anyway, I'm currently a bit out of options.
# For some weird reason, ANTLR also started complaining about {{DISTINCT}} and
{{JSON}} not being reserved function names. That it complains isn't all that
weird, since after all, something like {{SELECT DISTINCT (3, 4) FROM .. }} *is*
ambiguous (it could either be a DISTINCT query on a tuple, or a function call),
but what is weird is that it complains following the changes made by that
patch, which ought to be unrelated. It should have complained in
CASSANDRA-10783 where the ambiguity was created, but somehow didn't. I've
currently resolved that by make the keywords reserved, which is strictly
speaking a potential breaking change. That said, that's one problem I'm
personally willing to live with: in hindsight it sounds like a bad idea to not
have those be reserved, and there is an upgrade path for the few users that
might use them as unreserved.
# I wasn't able to make ANTLR accept the new syntax in it's more general form.
Basically, we only allow column names on the left-hand side of the new syntax.
That is, we accept {{SELECT m\[3\]\['foo'..'bar'\] FROM}}, but not {{SELECT
f(c)\[3\]}} for instance. I'd really rather avoid that limitation as we don't
have it for UDT field selection, but I was unable to have ANTLR be reasonable.
Anyway, the patch is currently "blocked" by those parsing issues and if someone
knowledgeable with ANTLR feels like having a look, I certainly wouldn't mind.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql, docs-impacting
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
>
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278398#comment-15278398 ] Robert Stupp commented on CASSANDRA-7396: - Good point! Removed the slice deletions from the patch, pushed it and triggered CI. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278243#comment-15278243 ] Sylvain Lebresne commented on CASSANDRA-7396: - bq. there's an idea to postpone the slice deletes I *definitively* think we shouldn't support slice deletions here as we can't really do it properly. Doing a read-before-write is not only inefficient, it's wrong because what is deleted depends on when that read is done, but messages are never guaranteed to arrive in order to replicas. In other words, a read-before-write deletes will sometime not delete stuffs it should. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276175#comment-15276175 ] Alex Petrov commented on CASSANDRA-7396: After talking with [~snazy], there's an idea to postpone the slice deletes we have range tombstones supported for the Cells (which as I understood has to wait until the next version of the storage format). This will avoid read-before write for the delete operations. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273929#comment-15273929
]
Alex Petrov commented on CASSANDRA-7396:
Wanted to point out that {{serializeForNativeProtocol}} and
{{reserializeForNativeProtocol}} might have some bigger implications for
[https://issues.apache.org/jira/browse/CASSANDRA-7826], especially
{{reserializeForNativeProtocol}}.
The previous version of {{serializedValues}} in collections was taking a
"short" path as it was simply packing things together. Now, it has some more
logic which includes comparing. It might be good to leave {{serializedValues}}
intact (possibly making it protected?) and filtering out the needed Cells or
wrapping it into some sort of filtering iterator if required.
As for {{reserializeForNativeProtocol}}, I ran into similar problem of logic
duplication between {{Maps}}/{{Sets}}/{{Lists}} {{::refromSerialized}} classes,
which currently have decoding logic when creating {{Value}} classes and {{get}}
method.
This might mean that we'll have:
* decoding the value from native protocol and composes {{Cells}} in
{{Maps/Value::fromSerialized}}
* serializing {{Cell}} for native protocol in {{MapType::serializedValues}}
* serialising a full {{Value}} for native protocol in {{Maps/Value::get}}
* re-serialising, which is kind of of combination of {{fromSerialised}} and
{{Value::get}} with filtering out values in-between.
Having all of that together, I'm not sure what would be the best way to
abstract or generalise it all. I just wanted to point out that it might be
multiplying complexity together with nested cells, so finding some common
grounds would ease integration of both features.
One possible solution for all that might be having some sort of intermediate
representation similar to {{Cell}} that would encapsulate logic, and yielding
{{Cell}} when applicable or serving as a container when we just need to take a
ByteBuffer from one format and yield ByteBuffer of the other.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql, docs-impacting
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246032#comment-15246032 ] Robert Stupp commented on CASSANDRA-7396: - Removed all the lists stuff as promised earlier today (except what was supported pre-7396). Got a bit confused with what's actually supported (LWTs + deletions). > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245726#comment-15245726 ] Aleksey Yeschenko commented on CASSANDRA-7396: -- Had an offline conversation with Robert a while ago asking for exactly this: let's not add any more convenience to lists, let's make it more obvious that they are a second class citizen and should be avoided - even if that means some inconsistency/surprise at the language level. Reiterating it here, publicly - feel free to treat the comment as a binding -1 to that subset of the ticket. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245704#comment-15245704 ] Sylvain Lebresne commented on CASSANDRA-7396: - My preference would be to add nothing more to lists, so no single element selection either. Again, because the performance of it will be surprising (as we can't optimize it at all) and lists performance is surprising enough as it is. Of course, if others are strongly _for_ single element selection on lists, I'd certainly prefer that and no slices than adding both. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245699#comment-15245699 ] Robert Stupp commented on CASSANDRA-7396: - We already have some element usages in LWTs and deletion. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245670#comment-15245670 ] Jonathan Ellis commented on CASSANDRA-7396: --- To clarify, you said you're against slices, not single element selection. Right? > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245284#comment-15245284 ] Sylvain Lebresne commented on CASSANDRA-7396: - bq. It allows single element and slice selections on sets, maps and lists I'll re-said that I'm still kind of -1 on lists for the reasons of my previous comment. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql, docs-impacting > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224355#comment-15224355
]
Robert Stupp commented on CASSANDRA-7396:
-
Yea, we cannot support lists as efficiently as maps+sets. I already have code
to deal with frozen collections that does the slice/element selection of these
and just got a first piece of code running with {{addSubSelection}}. (That
could also help with non-frozen UDTs in the future.)
That crazy {{?}} syntax is already gone.
Just noticed, that {{addSubSelection}} seems to return all elements (not just
those within the bounds) but just without the values. Is that behavior correct?
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224236#comment-15224236
]
Sylvain Lebresne commented on CASSANDRA-7396:
-
bq. am I correct, that
org.apache.cassandra.db.filter.ColumnFilter.Builder#addSubSelection would be
the "key" method to query individual elements / slices from collections (sets +
maps)?
You are.
But I'll note that for this ticket my preference would be to stick to basically
expose what {{ColumnFilter.Builder#addSubSelection}} basically provides
internally. In particular, I'd rather not add anything for lists as we can't
make dealing with lists efficient. In fact, I'd rather make it clear that lists
are a second class citizen that is rarely useful and only if you know what
you're doing.
I'm also not entirely convinced by the {{?}} syntax suggested above. In
particular, having {{map?\[key\]}} return a boolean but {{map?\[key1...key2\]}}
a set of value is a bit inconsistent. I'm also not sure that syntax is common
enough that it's worth adding specific syntax for it. We could for instance
have {{has(set, key1)}} if we just want to test an element existence. For
{{map?\[key1...key2\]}}, an alternative could be to have a {{keys()}} function
that applied to a map returns the set of keys.
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223215#comment-15223215
]
Robert Stupp commented on CASSANDRA-7396:
-
[~slebresne] am I correct, that
{{org.apache.cassandra.db.filter.ColumnFilter.Builder#addSubSelection}} would
be the "key" method to query individual elements / slices from collections
(sets + maps)?
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Robert Stupp
> Labels: cql
> Fix For: 3.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069646#comment-15069646 ] Robert Stupp commented on CASSANDRA-7396: - I still have [a branch|https://github.com/snazy/cassandra/tree/7396-coll-slice] regularly rebased on trunk. Not sure, if it still works but it definitely needs some more functionality to read only those "cells" that are actually requested - i.e. only read the collection slice and not the whole collection. The current impl reads the whole collection and just returns the slice. However, we could split this into a separate ticket. > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067487#comment-15067487 ] Jim Witschey commented on CASSANDRA-7396: - Since 8099's been merged, it might be time to look at this and decide its fate. Are we moving forward with it? [~snazy]? > Allow selecting Map key, List index > --- > > Key: CASSANDRA-7396 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Jonathan Ellis >Assignee: Robert Stupp > Labels: cql > Fix For: 3.x > > Attachments: 7396_unit_tests.txt > > > Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield > is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205368#comment-14205368 ] Robert Stupp commented on CASSANDRA-7396: - [~philipthompson] from a quick look this looks good - nice :) Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Assignee: Robert Stupp Labels: cql Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169271#comment-14169271 ] Robert Stupp commented on CASSANDRA-7396: - Discussed offline with [~slebresne] - we'll wait until CASSANDRA-8099 has been implemented because it simplifies access patterns and makes access optimizations for this one much easier. Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Assignee: Robert Stupp Labels: cql Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132615#comment-14132615
]
Robert Stupp commented on CASSANDRA-7396:
-
I'm currently testing whether a kind of CellFilter works for this. Such a
CellFilter could be used in {{SelectStatement#addValue}} to pre-select only
those elements, that shall be part of the result set. This would reduce the
serialization overhead - and it can also be used to optimize for list slice
requests. Altogether this would be the general purpose optimization for slice
requests.
Additionally there could be more optimizations in terms of column-slicing,
which IMO could work for sets and maps but not for lists.
Allow selecting Map key, List index
---
Key: CASSANDRA-7396
URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Jonathan Ellis
Labels: cql
Fix For: 3.0
Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield
is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129082#comment-14129082
]
Sylvain Lebresne commented on CASSANDRA-7396:
-
Not really a fan of the {{?}} variants: you can just query and see if it
returns null and it's not standard enough that it's worth cluttering the syntax
with (we can always provide a {{isNull}} method if we really want to).
Regarding the slicing syntax, I guess it's not crazy for maps and sets but I'd
much rather not adding anything for lists because 1) I don't think there is any
easy way to optimize this under the hood and 2) I'm not all that convinced that
it's that useful in practice.
Allow selecting Map key, List index
---
Key: CASSANDRA-7396
URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Jonathan Ellis
Labels: cql
Fix For: 3.0
Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield
is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127503#comment-14127503
]
Robert Stupp commented on CASSANDRA-7396:
-
Proposal for {{SELECT}} (map used for map columns, set for set columns,
list for list columns):
* {{map\[key]}} to return single map value or {{null}}
* {{map?\[key]}} tests whether single map key exists ({{true}} or {{false}})
* {{map\[key1..key2]}} to return sliced map
* {{map?\[key1..key2]}} to return set with keys that are present in the map
* {{map\[..key2]}} to return sliced map
* {{map?\[..key2]}} to return set with keys that are present in the map
* {{map\[key1..]}} to return sliced map
* {{map?\[key1..]}} to return set with keys that are present in the map
* {{set\[key]}} returns {{true}} or {{false}} whether a single element exists
* {{set\[key1..key2]}} to return sliced set (check values of multiple elements)
* {{set\[key1..]}} to return sliced set (check values of multiple elements)
* {{set\[..key2]}} to return sliced set (check values of multiple elements)
* {{list\[idx]}} to return single list element
* {{list?\[idx]}} tests whether single list element exists ({{true}} or
{{false}})
* {{list\[idx1..idx2]}} to return sliced list - list has length {{idx2-idx1+1}}
* {{list?\[idx1..idx2]}} to return sliced list with {{true}} or {{false}}
indicating whether an element exists (is no tnull)
* {{sizeof(map)}}, {{sizeof(set)}}, {{sizeof(list)}} to return size of
map/set/list
Restriction for {{key1..key2}} : {{key1}} must be less than {{key2}}
Using these collection operations on parts of primary key columns would require
special handling.
Primary key columns are stored in their serialized byte representation - so it
would need deserialzation, handling, 2nd serialization.
All would be added to {{unaliasedSelector}} in {{Cql.g}}.
Other thoughts?
I've built a POC for slicing (except {{sizeof}} functions) availabile at github
at https://github.com/snazy/cassandra/tree/7396-coll-slice to illustrate
functionality.
That POC does not use column slicing to read only the requested values. In fact
I do not know how to read only a map key „foobar“ from a table with clustering
keys. AFAIK will the internal column name“ something like
clusteringKey:mapColumnName:mapColumnKey“. Such a read optimization could be
done for tables that not not have a clustering key.
But there’s room for optimization to prevent unnecessary internal
serialization-deserialization-serialization sequences for the above collection
operations by using only those {{Cell}}s that will be returned (stuff around
{{SelectStatement#addValue}}).
Allow selecting Map key, List index
---
Key: CASSANDRA-7396
URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Jonathan Ellis
Labels: cql
Fix For: 3.0
Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield
is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041900#comment-14041900 ] Robert Stupp commented on CASSANDRA-7396: - bq. It's definitively worth having proper syntactic sugar here agree UDFs are not a requirement for this one - but may help to quickly add new functionality with the option to optimize that later (using CASSANDRA-7085, optimized slice queries etc) bq. slicing capabilities map[key1..key2] maybe also to check which keys are present for a slice (or at all) and return these keys and some containsKey(key1..key2) (return a boolean or an int with the count of the existing keys) Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14040909#comment-14040909
]
Robert Stupp commented on CASSANDRA-7396:
-
Is this something for CASSANDRA-7395 ?
Want to say: wouldn't it be easier to add some C*-UDF-stdlib and implement it
there?
The drawback: it would look like {{select stdlib:mapGet( map, 'key' ),
stdlib:listGet( list, 42 ), stdlib:setContains( set, 'value' ) ...}}
Or add some operator functionality to UDF : {{+ - / * ! [x]}}
Allow selecting Map key, List index
---
Key: CASSANDRA-7396
URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Jonathan Ellis
Fix For: 3.0
Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield
is already supported.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041049#comment-14041049 ] Sylvain Lebresne commented on CASSANDRA-7396: - It's definitively worth having proper syntactic sugar here imo and that's really mainly what this issue is about. We can indeed make this sugar generate a function call behind the scene but we don't need CASSANDRA-7395 per-se to add such function. Also, ultimately we can and should make those operator be efficient (at least for map and sets, for lists there is no much we can do) by not querying the whole CQL row if we don't need to (which amount to extending what CASSANDRA-7085 will introduce) so we'll almost surely end up with having those operation being special cased anyway, not just simple function calls. Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041080#comment-14041080 ] Dave Brosius commented on CASSANDRA-7396: - It seems to me a natural extension would be extending this to slicing capabilities map[key1..key2] or whatever syntax was appropriate. Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[ https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029695#comment-14029695 ] Jonathan Ellis commented on CASSANDRA-7396: --- What should we do for does element X exist in set Y syntax? Allow selecting Map key, List index --- Key: CASSANDRA-7396 URL: https://issues.apache.org/jira/browse/CASSANDRA-7396 Project: Cassandra Issue Type: New Feature Components: API Reporter: Jonathan Ellis Fix For: 3.0 Allow SELECT map['key] and SELECT list[index]. (Selecting a UDT subfield is already supported.) -- This message was sent by Atlassian JIRA (v6.2#6252)
