[ 
https://issues.apache.org/jira/browse/CASSANDRA-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13933227#comment-13933227
 ] 

Ondřej Černoš commented on CASSANDRA-6587:
------------------------------------------

I replayed the example Jan provided above with tracing on. This is the result:

{noformat}
cqlsh> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 
10000000;

 a      | d
--------+--------
 a14521 | d14521


Tracing session: 4c221710-aab2-11e3-80fa-5f25e090b27a

 activity                                                                       
                                                      | timestamp    | source   
 | source_elapsed
--------------------------------------------------------------------------------------------------------------------------------------+--------------+-----------+----------------
                                                                                
                                   execute_cql3_query | 14:20:50,947 | 
127.0.0.1 |              0
                                           Parsing SELECT a, d FROM t.t WHERE b 
= 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 10000000; | 14:20:50,947 | 
127.0.0.1 |           1377
                                                                                
                                  Preparing statement | 14:20:50,948 | 
127.0.0.1 |           1943
                                                                                
                        Determining replicas to query | 14:20:50,948 | 
127.0.0.1 |           2169
 Candidate index mean cardinalities are 
org.apache.cassandra.db.index.composites.CompositesIndex@1527e0c3:13470. 
Scanning with t.t_b. | 14:20:50,949 | 127.0.0.1 |           2723
                                                      Executing indexed scan 
for (min(9221619202503120164), max(9221619202503120164)] | 14:20:50,949 | 
127.0.0.1 |           2858
 Candidate index mean cardinalities are 
org.apache.cassandra.db.index.composites.CompositesIndex@1527e0c3:13470. 
Scanning with t.t_b. | 14:20:50,949 | 127.0.0.1 |           2920
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,949 | 
127.0.0.1 |           2970
                                                                                
                         Acquiring sstable references | 14:20:50,949 | 
127.0.0.1 |           2991
                                                                                
                          Merging memtable tombstones | 14:20:50,949 | 
127.0.0.1 |           3009
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,949 | 
127.0.0.1 |           3055
                                                                                
          Seeking to partition beginning in data file | 14:20:50,949 | 
127.0.0.1 |           3087
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,950 | 
127.0.0.1 |           3532
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,951 | 
127.0.0.1 |           5506
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,952 | 
127.0.0.1 |           6463
                                                                                
                         Acquiring sstable references | 14:20:50,952 | 
127.0.0.1 |           6475
                                                                                
                          Merging memtable tombstones | 14:20:50,952 | 
127.0.0.1 |           6490
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,953 | 
127.0.0.1 |           6534
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,953 | 
127.0.0.1 |           6570
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,953 | 
127.0.0.1 |           6598
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,955 | 
127.0.0.1 |           9511
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,956 | 
127.0.0.1 |          10388
                                                                                
                         Acquiring sstable references | 14:20:50,956 | 
127.0.0.1 |          10397
                                                                                
                          Merging memtable tombstones | 14:20:50,956 | 
127.0.0.1 |          10413
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,956 | 
127.0.0.1 |          10439
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,956 | 
127.0.0.1 |          10452
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,956 | 
127.0.0.1 |          10479
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,960 | 
127.0.0.1 |          14153
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,961 | 
127.0.0.1 |          15152
                                                                                
                         Acquiring sstable references | 14:20:50,961 | 
127.0.0.1 |          15175
                                                                                
                          Merging memtable tombstones | 14:20:50,961 | 
127.0.0.1 |          15197
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,961 | 
127.0.0.1 |          15232
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,961 | 
127.0.0.1 |          15243
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,961 | 
127.0.0.1 |          15278
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,966 | 
127.0.0.1 |          19874
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,967 | 
127.0.0.1 |          21303
                                                                                
                         Acquiring sstable references | 14:20:50,967 | 
127.0.0.1 |          21322
                                                                                
                          Merging memtable tombstones | 14:20:50,967 | 
127.0.0.1 |          21342
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,967 | 
127.0.0.1 |          21375
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,967 | 
127.0.0.1 |          21386
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,967 | 
127.0.0.1 |          21418
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,971 | 
127.0.0.1 |          24567
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,972 | 
127.0.0.1 |          25588
                                                                                
                         Acquiring sstable references | 14:20:50,972 | 
127.0.0.1 |          25604
                                                                                
                          Merging memtable tombstones | 14:20:50,972 | 
127.0.0.1 |          25627
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,972 | 
127.0.0.1 |          25663
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,972 | 
127.0.0.1 |          25674
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,972 | 
127.0.0.1 |          25709
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,976 | 
127.0.0.1 |          29614
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,977 | 
127.0.0.1 |          31160
                                                                                
                         Acquiring sstable references | 14:20:50,977 | 
127.0.0.1 |          31178
                                                                                
                          Merging memtable tombstones | 14:20:50,977 | 
127.0.0.1 |          31202
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,977 | 
127.0.0.1 |          31241
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,977 | 
127.0.0.1 |          31253
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,977 | 
127.0.0.1 |          31288
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,982 | 
127.0.0.1 |          36282
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,984 | 
127.0.0.1 |          37747
                                                                                
                         Acquiring sstable references | 14:20:50,984 | 
127.0.0.1 |          37765
                                                                                
                          Merging memtable tombstones | 14:20:50,984 | 
127.0.0.1 |          37787
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,984 | 
127.0.0.1 |          37822
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,984 | 
127.0.0.1 |          37834
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,984 | 
127.0.0.1 |          37869
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,989 | 
127.0.0.1 |          43474
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,991 | 
127.0.0.1 |          45065
                                                                                
                         Acquiring sstable references | 14:20:50,991 | 
127.0.0.1 |          45085
                                                                                
                          Merging memtable tombstones | 14:20:50,991 | 
127.0.0.1 |          45107
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,991 | 
127.0.0.1 |          45147
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,991 | 
127.0.0.1 |          45159
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,991 | 
127.0.0.1 |          45196
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,994 | 
127.0.0.1 |          48485
                                                                                
            Executing single-partition query on t.t_b | 14:20:50,996 | 
127.0.0.1 |          49997
                                                                                
                         Acquiring sstable references | 14:20:50,996 | 
127.0.0.1 |          50019
                                                                                
                          Merging memtable tombstones | 14:20:50,996 | 
127.0.0.1 |          50043
                                                                                
                          Key cache hit for sstable 1 | 14:20:50,996 | 
127.0.0.1 |          50085
                                                                                
    Seeking to partition indexed section in data file | 14:20:50,996 | 
127.0.0.1 |          50097
                                                                                
           Merging data from memtables and 1 sstables | 14:20:50,996 | 
127.0.0.1 |          50133
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:50,999 | 
127.0.0.1 |          53368
                                                                                
            Executing single-partition query on t.t_b | 14:20:51,001 | 
127.0.0.1 |          54913
                                                                                
                         Acquiring sstable references | 14:20:51,002 | 
127.0.0.1 |          55640
                                                                                
                          Merging memtable tombstones | 14:20:51,002 | 
127.0.0.1 |          55674
                                                                                
                          Key cache hit for sstable 1 | 14:20:51,002 | 
127.0.0.1 |          55720
                                                                                
    Seeking to partition indexed section in data file | 14:20:51,002 | 
127.0.0.1 |          55733
                                                                                
           Merging data from memtables and 1 sstables | 14:20:51,002 | 
127.0.0.1 |          55770
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:51,005 | 
127.0.0.1 |          58956
                                                                                
            Executing single-partition query on t.t_b | 14:20:51,006 | 
127.0.0.1 |          60343
                                                                                
                         Acquiring sstable references | 14:20:51,006 | 
127.0.0.1 |          60358
                                                                                
                          Merging memtable tombstones | 14:20:51,006 | 
127.0.0.1 |          60377
                                                                                
                          Key cache hit for sstable 1 | 14:20:51,006 | 
127.0.0.1 |          60474
                                                                                
    Seeking to partition indexed section in data file | 14:20:51,006 | 
127.0.0.1 |          60487
                                                                                
           Merging data from memtables and 1 sstables | 14:20:51,006 | 
127.0.0.1 |          60517
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:51,011 | 
127.0.0.1 |          64764
                                                                                
            Executing single-partition query on t.t_b | 14:20:51,012 | 
127.0.0.1 |          66425
                                                                                
                         Acquiring sstable references | 14:20:51,012 | 
127.0.0.1 |          66451
                                                                                
                          Merging memtable tombstones | 14:20:51,012 | 
127.0.0.1 |          66473
                                                                                
                          Key cache hit for sstable 1 | 14:20:51,012 | 
127.0.0.1 |          66528
                                                                                
    Seeking to partition indexed section in data file | 14:20:51,013 | 
127.0.0.1 |          66541
                                                                                
           Merging data from memtables and 1 sstables | 14:20:51,013 | 
127.0.0.1 |          66576
                                                                                
                 Read 743 live and 0 tombstoned cells | 14:20:51,015 | 
127.0.0.1 |          68672
                                                                                
            Executing single-partition query on t.t_b | 14:20:51,016 | 
127.0.0.1 |          70159
                                                                                
                         Acquiring sstable references | 14:20:51,016 | 
127.0.0.1 |          70179
                                                                                
                          Merging memtable tombstones | 14:20:51,016 | 
127.0.0.1 |          70201
                                                                                
                          Key cache hit for sstable 1 | 14:20:51,016 | 
127.0.0.1 |          70247
                                                                                
    Seeking to partition indexed section in data file | 14:20:51,016 | 
127.0.0.1 |          70258
                                                                                
           Merging data from memtables and 1 sstables | 14:20:51,016 | 
127.0.0.1 |          70290
                                                                                
                 Read 367 live and 0 tombstoned cells | 14:20:51,018 | 
127.0.0.1 |          72316
                                                                                
                Executing single-partition query on t | 14:20:51,019 | 
127.0.0.1 |          73081
                                                                                
                         Acquiring sstable references | 14:20:51,019 | 
127.0.0.1 |          73096
                                                                                
                          Merging memtable tombstones | 14:20:51,019 | 
127.0.0.1 |          73115
                                                                                
                          Key cache hit for sstable 1 | 14:20:51,019 | 
127.0.0.1 |          73179
                                                                                
          Seeking to partition beginning in data file | 14:20:51,019 | 
127.0.0.1 |          73190
                                                                                
           Merging data from memtables and 1 sstables | 14:20:51,019 | 
127.0.0.1 |          73214
                                                                                
                   Read 1 live and 0 tombstoned cells | 14:20:51,019 | 
127.0.0.1 |          73241
                                                                                
                         Scanned 1 rows and matched 1 | 14:20:51,019 | 
127.0.0.1 |          73332
                                                                                
                                     Request complete | 14:20:51,020 | 
127.0.0.1 |          73449
{noformat}

And the slow one:

{noformat}
cqlsh> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 
10;

 a      | d
--------+--------
 a14521 | d14521


Tracing session: 52362e70-aab2-11e3-80fa-5f25e090b27a

 activity                                                                       
                                                      | timestamp    | source   
 | source_elapsed
--------------------------------------------------------------------------------------------------------------------------------------+--------------+-----------+----------------
                                                                                
                                   execute_cql3_query | 14:21:01,143 | 
127.0.0.1 |              0
                                                 Parsing SELECT a, d FROM t.t 
WHERE b = 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 10; | 14:21:01,144 | 
127.0.0.1 |             56
                                                                                
                                  Preparing statement | 14:21:01,144 | 
127.0.0.1 |            166
                                                                                
                        Determining replicas to query | 14:21:01,144 | 
127.0.0.1 |            363
 Candidate index mean cardinalities are 
org.apache.cassandra.db.index.composites.CompositesIndex@1527e0c3:13470. 
Scanning with t.t_b. | 14:21:01,144 | 127.0.0.1 |            845
                                                      Executing indexed scan 
for (min(9221619202503120164), max(9221619202503120164)] | 14:21:01,144 | 
127.0.0.1 |            957
 Candidate index mean cardinalities are 
org.apache.cassandra.db.index.composites.CompositesIndex@1527e0c3:13470. 
Scanning with t.t_b. | 14:21:01,144 | 127.0.0.1 |           1020
                                                                                
            Executing single-partition query on t.t_b | 14:21:01,145 | 
127.0.0.1 |           1122
                                                                                
                         Acquiring sstable references | 14:21:01,145 | 
127.0.0.1 |           1136
                                                                                
                          Merging memtable tombstones | 14:21:01,145 | 
127.0.0.1 |           1163
                                                                                
                          Key cache hit for sstable 1 | 14:21:01,145 | 
127.0.0.1 |           1219
                                                                                
          Seeking to partition beginning in data file | 14:21:01,145 | 
127.0.0.1 |           1232
                                                                                
           Merging data from memtables and 1 sstables | 14:21:01,145 | 
127.0.0.1 |           1652
                                                                                
                   Read 3 live and 0 tombstoned cells | 14:21:01,145 | 
127.0.0.1 |           1724
                                                                                
            Executing single-partition query on t.t_b | 14:21:01,145 | 
127.0.0.1 |           1793
                                                                                
                         Acquiring sstable references | 14:21:01,145 | 
127.0.0.1 |           1806
                                                                                
                          Merging memtable tombstones | 14:21:01,145 | 
127.0.0.1 |           1824
                                                                                
                          Key cache hit for sstable 1 | 14:21:01,145 | 
127.0.0.1 |           1868
                                                                                
    Seeking to partition indexed section in data file | 14:21:01,145 | 
127.0.0.1 |           1880
                                                                                
           Merging data from memtables and 1 sstables | 14:21:01,145 | 
127.0.0.1 |           1913
                                                                                
                   Read 3 live and 0 tombstoned cells | 14:21:01,146 | 
127.0.0.1 |           2997
                                                                                
            Executing single-partition query on t.t_b | 14:21:01,147 | 
127.0.0.1 |           3063
                                                                                
                         Acquiring sstable references | 14:21:01,147 | 
127.0.0.1 |           3074
                                                                                
                          Merging memtable tombstones | 14:21:01,147 | 
127.0.0.1 |           3093
                                                                                
                          Key cache hit for sstable 1 | 14:21:01,147 | 
127.0.0.1 |           3124
                                                                                
    Seeking to partition indexed section in data file | 14:21:01,147 | 
127.0.0.1 |           3135
                                                                                
           Merging data from memtables and 1 sstables | 14:21:01,147 | 
127.0.0.1 |           3165
                                                                                
                   Read 3 live and 0 tombstoned cells | 14:21:01,148 | 
127.0.0.1 |           4559
                                                                                
            Executing single-partition query on t.t_b | 14:21:01,148 | 
127.0.0.1 |           4635
                                                                                
                         Acquiring sstable references | 14:21:01,148 | 
127.0.0.1 |           4648
                                                                                
                          Merging memtable tombstones | 14:21:01,148 | 
127.0.0.1 |           4669
                                                                                
                          Key cache hit for sstable 1 | 14:21:01,148 | 
127.0.0.1 |           4702
                                                                                
    Seeking to partition indexed section in data file | 14:21:01,148 | 
127.0.0.1 |           4714
                                                                                
           Merging data from memtables and 1 sstables | 14:21:01,148 | 
127.0.0.1 |           4745
                                                                                
                   Read 3 live and 0 tombstoned cells | 14:21:01,150 | 
127.0.0.1 |           6155
                                                                                
            Executing single-partition query on t.t_b | 14:21:01,150 | 
127.0.0.1 |           6231
                                                                                
                         Acquiring sstable references | 14:21:01,150 | 
127.0.0.1 |           6244
                                                                                
                          Merging memtable tombstones | 14:21:01,150 | 
127.0.0.1 |           6267
                                                                                
                          Key cache hit for sstable 1 | 14:21:01,150 | 
127.0.0.1 |           6313
... skiping a lot of Read 3 live and 0 tombstoned cells blocks ...
                                                                                
    Seeking to partition indexed section in data file | 14:21:15,975 | 
127.0.0.1 |       14831792
                                                                                
           Merging data from memtables and 1 sstables | 14:21:15,975 | 
127.0.0.1 |       14831798
                                                                                
                   Read 3 live and 0 tombstoned cells | 14:21:15,977 | 
127.0.0.1 |       14833423
                                                                                
                Executing single-partition query on t | 14:21:15,977 | 
127.0.0.1 |       14833554
                                                                                
                         Acquiring sstable references | 14:21:15,977 | 
127.0.0.1 |       14833561
                                                                                
                          Merging memtable tombstones | 14:21:15,977 | 
127.0.0.1 |       14833567
                                                                                
                          Key cache hit for sstable 1 | 14:21:15,977 | 
127.0.0.1 |       14833587
                                                                                
          Seeking to partition beginning in data file | 14:21:15,977 | 
127.0.0.1 |       14833598
                                                                                
           Merging data from memtables and 1 sstables | 14:21:15,977 | 
127.0.0.1 |       14833709
                                                                                
                   Read 1 live and 0 tombstoned cells | 14:21:15,977 | 
127.0.0.1 |       14833747
                                                                                
            Executing single-partition query on t.t_b | 14:21:15,977 | 
127.0.0.1 |       14833825
                                                                                
                         Acquiring sstable references | 14:21:15,977 | 
127.0.0.1 |       14833827
                                                                                
                          Merging memtable tombstones | 14:21:15,977 | 
127.0.0.1 |       14833832
                                                                                
                          Key cache hit for sstable 1 | 14:21:15,977 | 
127.0.0.1 |       14833839
                                                                                
    Seeking to partition indexed section in data file | 14:21:15,977 | 
127.0.0.1 |       14833842
                                                                                
           Merging data from memtables and 1 sstables | 14:21:15,977 | 
127.0.0.1 |       14833850
                                                                                
                   Read 2 live and 0 tombstoned cells | 14:21:15,979 | 
127.0.0.1 |       14835443
                                                                                
                         Scanned 1 rows and matched 1 | 14:21:15,979 | 
127.0.0.1 |       14835533
                                                                                
                                     Request complete | 14:21:15,978 | 
127.0.0.1 |       14835809
{noformat}

> Slow query when using token range and secondary index
> -----------------------------------------------------
>
>                 Key: CASSANDRA-6587
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6587
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Jan Chochol
>
> We are using token ranges to simulate pagination on external API. To achieve 
> this, we use similar queries:
> {noformat}
> SELECT * FROM table WHERE TOKEN(partition_key) > TOKEN('offset') AND 
> secondary_key = 'value' LIMIT 1000;
> {noformat}
> We found that such statement is quite ineffective, and we do not know how to 
> solve it.
> Let's try some example.
> You can fill Cassandra with folowing script:
> {noformat}
> perl -e "print(\"DROP KEYSPACE t;\nCREATE KEYSPACE t WITH replication = 
> {'class': 'SimpleStrategy', 'replication_factor' : 1};\nuse t;\nCREATE TABLE 
> t (a varchar PRIMARY KEY, b varchar, c varchar, d varchar);\nCREATE INDEX t_b 
> ON t (b);\nCREATE INDEX t_c ON t (c);\nCREATE INDEX t_d ON t (d);\n\");\$max 
> = 100000; for(\$i = 0; \$i < \$max; \$i++) { \$j = int(\$i * 10 / \$max); \$k 
> = int(\$i * 100 / \$max); print(\"INSERT INTO t (a, b, c, d) VALUES ('a\$i', 
> 'b\$j', 'c\$k', 'd\$i');\n\")}; for(\$i = 0; \$i < \$max; \$i++) { 
> print(\"INSERT INTO t (a, b, c, d) VALUES ('e\$i', 'f\$j', 'g\$k', 
> 'h\$i');\n\")}" | cqlsh
> {noformat}
> First we looked for last but one parition key:
> {noformat}
> [root@jch3-devel:~/c4] echo "SELECT a FROM t.t WHERE b = 'b1' LIMIT 100000;" 
> | cqlsh | tail
>  a18283
>  a11336
>  a14712
>  a11476
>  a19396
>  a14269
>  a10719
>  a14521
>  a13934
> {noformat}
> Than we issue following commands for some interesting behaviour:
> {noformat}
> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 
> 10000000;
> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) = TOKEN('a14521') LIMIT 10;
> SELECT a, d FROM t.t WHERE b = 'b1' AND a = 'a14521' LIMIT 10;
> {noformat}
> And here is result:
> {noformat}
> [root@jch3-devel:~/c4] time echo "SELECT a, d FROM t.t WHERE b = 'b1' AND 
> TOKEN(a) = TOKEN('a14521') LIMIT 10000000;" | cqlsh
>  a      | d
> --------+--------
>  a14521 | d14521
> real    0m0.647s
> user    0m0.307s
> sys     0m0.076s
> [root@jch3-devel:~/c4] time echo "SELECT a, d FROM t.t WHERE b = 'b1' AND 
> TOKEN(a) = TOKEN('a14521') LIMIT 10;" | cqlsh
>  a      | d
> --------+--------
>  a14521 | d14521
> real    0m16.454s
> user    0m0.341s
> sys     0m0.090s
> [root@jch3-devel:~/c4] time echo "SELECT a, d FROM t.t WHERE b = 'b1' AND a = 
> 'a14521' LIMIT 10;" | cqlsh
>  a      | d
> --------+--------
>  a14521 | d14521
> real    0m0.404s
> user    0m0.309s
> sys     0m0.071s
> {noformat}
> Problem with {{LIMIT}} is described in CASSANDRA-6348, and is quite funny - 
> lower the limit, slower the requst (and with different structure of data it 
> can be even worse).
> This query is quite silly in reality (asking with secondary key, when you 
> have primary key), but is close as possible to our use case:
> {noformat}
> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) > TOKEN('a14521') LIMIT 10;
> {noformat}
> But we simply can not do:
> {noformat}
> SELECT a, d FROM t.t WHERE b = 'b1' AND a > 'a14521' LIMIT 10;
> {noformat}
> As this is unsupported.
> {{CompositesSearcher.java}} gives us some clue about the problem:
> {noformat}
>         /*
>          * XXX: If the range requested is a token range, we'll have to start 
> at the beginning (and stop at the end) of
>          * the indexed row unfortunately (which will be inefficient), because 
> we have not way to intuit the small
>          * possible key having a given token. A fix would be to actually 
> store the token along the key in the
>          * indexed row.
>          */
> {noformat}
> Index row contains parition keys in partion key ordering (ordering exposed in 
> CQL3 as {{TOKEN(partition_key)}}), so these two request are expected to 
> return same values:
> {noformat}
> SELECT a, d FROM t.t WHERE b = 'b1' AND TOKEN(a) > TOKEN('a14521') LIMIT 1;
> SELECT a, d FROM t.t WHERE b = 'b1' AND a > 'a14521' LIMIT 1;
> {noformat}
> But the second is not supported.
> Currently we are considering to go to our production with this patch:
> {noformat}
> diff --git 
> a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
> b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
> index 44a1e64..0228c3a 100644
> --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
> +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
> @@ -1123,8 +1123,10 @@ public class SelectStatement implements CQLStatement
>                          stmt.keyIsInRelation = true;
>                      }
>                  }
> -                else
> +                else if (stmt.metadataRestrictions.isEmpty())
>                  {
> +                    // This is poor man heuristic, and probably far from 
> correct, but it allows us these requests:
> +                    // SELECT * FROM table WHERE partition_key > 'offset' 
> AND secondary_key = 'value'
>                      throw new InvalidRequestException("Only EQ and IN 
> relation are supported on the partition key (you will need to use the token() 
> function for non equality based relation)");
>                  }
>                  previous = cname;
> diff --git 
> a/src/java/org/apache/cassandra/db/index/composites/CompositesSearcher.java 
> b/src/java/org/apache/cassandra/db/index/composites/CompositesSearcher.java
> index 5ab1df6..2ba2845 100644
> --- 
> a/src/java/org/apache/cassandra/db/index/composites/CompositesSearcher.java
> +++ 
> b/src/java/org/apache/cassandra/db/index/composites/CompositesSearcher.java
> @@ -190,7 +190,8 @@ public class CompositesSearcher extends 
> SecondaryIndexSearcher
>  
>              private int meanColumns = 
> Math.max(index.getIndexCfs().getMeanColumns(), 1);
>              // We shouldn't fetch only 1 row as this provides buggy paging 
> in case the first row doesn't satisfy all clauses
> -            private final int rowsPerQuery = 
> Math.max(Math.min(filter.maxRows(), filter.maxColumns() / meanColumns), 2);
> +            // We  are not too interested in memory consumption, as we are 
> using only "small data"
> +            private final int rowsPerQuery = 
> Math.max(Math.min(filter.maxRows(), filter.maxColumns()), 2);
>  
>              public boolean needsFiltering()
>              {
> {noformat}
> It is probably far from correct, but at least it gives us acceptable 
> performance:
> {noformat}
> [root@jch3-devel:~/c4] time echo "SELECT a, d FROM t.t WHERE b = 'b1' AND 
> TOKEN(a) > TOKEN('a14521') LIMIT 1;" | cqlsh
>  a      | d
> --------+--------
>  a13934 | d13934
> real    0m15.359s
> user    0m0.313s
> sys     0m0.090s
> [root@jch3-devel:~/c4] time echo "SELECT a, d FROM t.t WHERE b = 'b1' AND a > 
> 'a14521' LIMIT 1;" | cqlsh
>  a      | d
> --------+--------
>  a13934 | d13934
> real    0m0.449s
> user    0m0.323s
> sys     0m0.106s
> {noformat}
> We do not think that {{TOKEN}} range requests with secondary indexes should 
> be so much slower.
> Can you please exentend CQL3 with possibility to parition keys range requests 
> with secondary indexes (than change this issue to feature request), or make 
> {{TOKEN}} range requests with secondary indexes faster?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to