[jira] [Commented] (CASSANDRA-12962) SASI: Index are rebuilt on restart

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943400#comment-15943400
 ] 

Alex Petrov commented on CASSANDRA-12962:
-

Composed a simple patch that would put an empty file in place of an index that 
doesn't hold any values and ensure that it doesn't participate neither in query 
process nor is attempted to be rebuilt every restart.

|[patch|https://github.com/ifesdjeen/cassandra/tree/12962-trunk]|[utest|https://cassci.datastax.com/job/ifesdjeen-12962-trunk-testall/]|[dtest|https://cassci.datastax.com/job/ifesdjeen-12962-trunk-dtest/]|

> SASI: Index are rebuilt on restart
> --
>
> Key: CASSANDRA-12962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12962
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Corentin Chary
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: screenshot-1.png
>
>
> Apparently when cassandra any index that does not index a value in *every* 
> live SSTable gets rebuild. The offending code can be found in the constructor 
> of SASIIndex.
> You can easilly reproduce it:
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '1'}  AND durable_writes = true;
> CREATE TABLE test.test (
> a text PRIMARY KEY,
> b text,
> c text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX test_b_idx ON test.test (b) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX test_c_idx ON test.test (c) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> INSERT INTO test.test (a, b) VALUES ('a', 'b');
> {code}
> Log (I added additional traces):
> {code}
> INFO  [main] 2016-11-28 15:32:21,191 ColumnFamilyStore.java:406 - 
> Initializing test.test
> DEBUG [SSTableBatchOpen:1] 2016-11-28 15:32:21,192 SSTableReader.java:505 - 
> Opening 
> /mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big 
> (0.034KiB)
> DEBUG [main] 2016-11-28 15:32:21,194 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@2f661b1a[id=6b00489b-7010-396e-9348-9f32f5167f88,name=test_b_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=b}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> INFO  [main] 2016-11-28 15:32:21,194 DataTracker.java:152 - 
> SSTableIndex.open(column: b, minTerm: value, maxTerm: value, minKey: key, 
> maxKey: key, sstable: BigTableReader(path='/mnt/ssd/tmp/data/data/test/test\
> -229e6380b57711e68407158fde22e121/mc-1-big-Data.db'))
> DEBUG [main] 2016-11-28 15:32:21,195 SASIIndex.java:129 - Rebuilding SASI 
> Indexes: {}
> DEBUG [main] 2016-11-28 15:32:21,195 ColumnFamilyStore.java:895 - Enqueuing 
> flush of IndexInfo: 0.386KiB (0%) on-heap, 0.000KiB (0%) off-heap
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:465 - Writing Memtable-IndexInfo@748981977(0.054KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = (min(-9223\
> 372036854775808), max(9223372036854775807)]
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:494 - Completed flushing 
> /mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db
>  (0.035KiB) for\
>  commitlog position CommitLogPosition(segmentId=1480343535479, position=15652)
> DEBUG [MemtableFlushWriter:1] 2016-11-28 15:32:21,224 
> ColumnFamilyStore.java:1200 - Flushed to 
> [BigTableReader(path='/mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db\
> ')] (1 sstables, 4.838KiB), biggest 4.838KiB, smallest 4.838KiB
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@12f3d291[id=45fcb286-b87a-3d18-a04b-b899a9880c91,name=test_c_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=c}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> 

[jira] [Commented] (CASSANDRA-12661) Make gc_log and gc_warn settable at runtime

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943409#comment-15943409
 ] 

Edward Capriolo commented on CASSANDRA-12661:
-

This ticket is approaching 7 months old.

> Make gc_log and gc_warn settable at runtime
> ---
>
> Key: CASSANDRA-12661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12661
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
>
> Changes:
> * Move gc_log_threshold_in_ms and gc_warn_threshold_in_ms close together in 
> the config
> * rename variables to match properties
> * add unit tests to ensure hybration
> * add unit tests to ensure variables are set propertly
> * minor perf (do not consturct string from buffer f not logging)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943455#comment-15943455
 ] 

Paulo Motta commented on CASSANDRA-11530:
-

cc [~adejanovski] [~scottbale] this may be of interest to you and other 
integrators since it removes the old repair JMX api on 4.0

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Alexander Dejanovski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943492#comment-15943492
 ] 

Alexander Dejanovski commented on CASSANDRA-11530:
--

Thanks for mentioning [~pauloricardomg]

I have an open issue to stop using deprecated methods and will make sure it's 
done before 4.0 comes out.

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11530:

Status: Ready to Commit  (was: Patch Available)

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13366) Possible AssertionError in UnfilteredRowIteratorWithLowerBound

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943476#comment-15943476
 ] 

Philip Thompson commented on CASSANDRA-13366:
-

If the test is going to take more than a few days after the ticket, could we 
get a separate jira ticket, maybe a subtask for that? I would hate to see it be 
forgotten and then no test added.

> Possible AssertionError in UnfilteredRowIteratorWithLowerBound
> --
>
> Key: CASSANDRA-13366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13366
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.0
>
>
> In the code introduced by CASSANDRA-8180, we build a lower bound for a 
> partition (sometimes) based on the min clustering values of the stats file. 
> We can't do that if the sstable has and range tombston marker and the code 
> does check that this is the case, but unfortunately the check is done using 
> the stats {{minLocalDeletionTime}} but that value isn't populated properly in 
> pre-3.0. This means that if you upgrade from 2.1/2.2 to 3.4+, you may end up 
> getting an exception like
> {noformat}
> WARN  [ReadStage-2] 2017-03-20 13:29:39,165  
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread 
> Thread[ReadStage-2,5,main]: {}
> java.lang.AssertionError: Lower bound [INCL_START_BOUND(Foo, 
> -9223372036854775808, -9223372036854775808) ]is bigger than first returned 
> value [Marker INCL_START_BOUND(Foo)@1490013810540999] for sstable 
> /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/system-size_estimates-ka-1-Data.db
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:122)
> {noformat}
> and this until the sstable is upgraded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13065) Consistent range movements to not require MV updates to go through write paths

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943405#comment-15943405
 ] 

Paulo Motta commented on CASSANDRA-13065:
-

Thanks for the ping, I didn't get the JIRA notification for some reason. :(

bq. According to the feedback of Joshua McKenzie, I didn't add anything for 
CDC. Hope this is ok.

Right now the commit-log segment discarding is tied to memtable flushing, so 
I'm not sure how bypassing the memtable will affect commitlog segment 
discarding for mutations originated from streaming so it would be nice to have 
a test for that. Furthermore, right now there's no test for CDC behavior on 
streaming so this is a good opportunity to add a dtest verifying that behavior 
(leaving the house cleaner than you found it).

Since this may be considerable effort, I'm fine if we split the cdc component 
of this ticket into another ticket.

bq. So now you have MV data on C that belonged to a base table on A which is 
very likely to have an inconsistent relation not to it's base table data on D.

That's right, but after inconsistent range movements you are expected to run 
repair in both the base table and the view, which should make the rebuilt view 
consistent with the base again. So, in this scenario you would repair the bases 
A and D first, making them consistent, and then repair the views B and C, thus 
making base D consistent with view C.

I looked the updated patch and looks good, except the following nits:
* reuse {{requiresWritepath}} variable on before {{sendThroughWritePath}}
* no need to have a special {{UNIT_TEST}} {{StreamOperation}}, can just reuse 
one of the existing types
* I think you renamed mbean names ({{StorageProxy}}, {{StorageService}}, 
{{StreamManager}}) from {{type}} to {{streamOperation}} by mistake
* There are still a few places using {{streamOperation.toString()}} instead of 
{{streamOperation.getDescription()}}

> Consistent range movements to not require MV updates to go through write 
> paths 
> ---
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13382) cdc column addition strikes again

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943430#comment-15943430
 ] 

Joshua McKenzie commented on CASSANDRA-13382:
-

Try not to --author my name on this one. :)

> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13382) cdc column addition strikes again

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13382:

Reviewer: Aleksey Yeschenko

> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943446#comment-15943446
 ] 

Paulo Motta commented on CASSANDRA-11530:
-

LGTM, before committing can you just add a note on {{NEWS.txt}} with pointers 
to the new API and notification system? Thanks!

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13382) cdc column addition strikes again

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943557#comment-15943557
 ] 

Jeff Jirsa edited comment on CASSANDRA-13382 at 3/27/17 4:27 PM:
-

Minor nit that the comment is grammatically confusing (probably accidental if 
-> is):

{code}
+ * The tables to which we added the cdc column. This is used in {@link 
#makeUpdateForSchema} below to make sure we skip that
+ * column is cdc is disabled as the columns breaks pre-cdc to post-cdc 
upgrades (typically, 3.0 -> 3.X).
{code}




was (Author: jjirsa):
Minor nit that the comment is grammatically confusing (probably accidental if 
-> is):

{quote}
+ * The tables to which we added the cdc column. This is used in {@link 
#makeUpdateForSchema} below to make sure we skip that
+ * column is cdc is disabled as the columns breaks pre-cdc to post-cdc 
upgrades (typically, 3.0 -> 3.X).
{quote}



> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13382) cdc column addition strikes again

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943557#comment-15943557
 ] 

Jeff Jirsa commented on CASSANDRA-13382:


Minor nit that the comment is grammatically confusing (probably accidental if 
-> is):

{quote}
+ * The tables to which we added the cdc column. This is used in {@link 
#makeUpdateForSchema} below to make sure we skip that
+ * column is cdc is disabled as the columns breaks pre-cdc to post-cdc 
upgrades (typically, 3.0 -> 3.X).
{quote}



> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta reassigned CASSANDRA-13229:
---

Assignee: Alex Petrov  (was: Paulo Motta)
Reviewer: Paulo Motta

> dtest failure in topology_test.TestTopology.size_estimates_multidc_test
> ---
>
> Key: CASSANDRA-13229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
>  Labels: dtest, test-failure
> Fix For: 4.0
>
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 
> - Exception in thread Thread[MemtablePostFlush:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
>   at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
>  ~[main/:na]
>   at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
> ~[main/:na]
>   at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Unexpected error in node1 log, error: 
> ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
>  ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
>  ~[main/:na]
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
>   at 
> org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 

[jira] [Commented] (CASSANDRA-13382) cdc column addition strikes again

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943484#comment-15943484
 ] 

Jeremiah Jordan commented on CASSANDRA-13382:
-

+1 Looks good to me.  addViewToSchemaMutation calls addTableParamsToRowBuilder 
which was covered by the fix in CASSANDRA-12236.  So the only MV cdc thing that 
still needs catching is in makeUpdateForSchema which this patch covers.

> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11471) Add SASL mechanism negotiation to the native protocol

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-11471:
---
Fix Version/s: 4.x

> Add SASL mechanism negotiation to the native protocol
> -
>
> Key: CASSANDRA-11471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11471
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Sam Tunnicliffe
>Assignee: Ben Bromhead
>  Labels: client-impacting
> Fix For: 4.x
>
> Attachments: CASSANDRA-11471
>
>
> Introducing an additional message exchange into the authentication sequence 
> would allow us to support multiple authentication schemes and [negotiation of 
> SASL mechanisms|https://tools.ietf.org/html/rfc4422#section-3.2]. 
> The current {{AUTHENTICATE}} message sent from Client to Server includes the 
> java classname of the configured {{IAuthenticator}}. This could be superceded 
> by a new message which lists the SASL mechanisms supported by the server. The 
> client would then respond with a new message which indicates it's choice of 
> mechanism.  This would allow the server to support multiple mechanisms, for 
> example enabling both {{PLAIN}} for username/password authentication and 
> {{EXTERNAL}} for a mechanism for extracting credentials from SSL 
> certificates\* (see the example in 
> [RFC-4422|https://tools.ietf.org/html/rfc4422#appendix-A]). Furthermore, the 
> server could tailor the list of supported mechanisms on a per-connection 
> basis, e.g. only offering certificate based auth to encrypted clients. 
> The client's response should include the selected mechanism and any initial 
> response data. This is mechanism-specific; the {{PLAIN}} mechanism consists 
> of a single round in which the client sends encoded credentials as the 
> initial response data and the server response indicates either success or 
> failure with no futher challenges required.
> From a protocol perspective, after the mechanism negotiation the exchange 
> would continue as in protocol v4, with one or more rounds of 
> {{AUTH_CHALLENGE}} and {{AUTH_RESPONSE}} messages, terminated by an 
> {{AUTH_SUCCESS}} sent from Server to Client upon successful authentication or 
> an {{ERROR}} on auth failure. 
> XMPP performs mechanism negotiation in this way, 
> [RFC-3920|http://tools.ietf.org/html/rfc3920#section-6] includes a good 
> overview.
> \* Note: this would require some a priori agreement between client and server 
> over the implementation of the {{EXTERNAL}} mechanism.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Reopened] (CASSANDRA-13234) Add histogram for delay to deliver hints

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reopened CASSANDRA-13234:


Reopening - [~aweisberg] correctly notes that there's a (likely rare) edge case 
where FD thinks the host is up, we try to replay hints, hint replay fails, but 
we update metrics incorrectly anyway.

Submitting a modification where we update the hint delay histograms in the 
response callback, rather than before sending, just in case the hint dispatch 
fails.

[~spo...@gmail.com] or [~aweisberg] - either of you care to review the 
modification (pushed to the same trunk cassandra-13234 branch above). 

> Add histogram for delay to deliver hints
> 
>
> Key: CASSANDRA-13234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13234
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.0
>
>
> There is very little visibility into hint delivery in general - having 
> histograms available to understand how long it takes to deliver hints is 
> useful for operators to better identify problems. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13234) Add histogram for delay to deliver hints

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943647#comment-15943647
 ] 

Ariel Weisberg commented on CASSANDRA-13234:


+1

> Add histogram for delay to deliver hints
> 
>
> Key: CASSANDRA-13234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13234
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.0
>
>
> There is very little visibility into hint delivery in general - having 
> histograms available to understand how long it takes to deliver hints is 
> useful for operators to better identify problems. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13316) Build error because of dependent jar (byteman-install-3.0.3.jar) currupted

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang resolved CASSANDRA-13316.

   Resolution: Fixed
Fix Version/s: (was: 3.10)
   4.0
   3.11.0
   2.2.10

> Build error because of dependent jar (byteman-install-3.0.3.jar) currupted 
> ---
>
> Key: CASSANDRA-13316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13316
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Platform: Amd64
> OS: CentOS Linux 7
>Reporter: Sam Ding
> Fix For: 2.2.10, 3.11.0, 4.0
>
>
> When build  cassandra 3.10 on amd64, CentOS Linux 7, there is a build error 
> caused by corrupted jar file (byteman-install-3.0.3.jar).
> Here is the replicated steps:
> After install necessary dependent packages and apache-ant, git clone 
> cassandra 3.10:
> 1)
>   git clone https://github.com/apache/cassandra.git
>   cd cassandra
>   git checkout cassandra-3.10
>   ant 
> Then gets errors like:
> "
> build-project:
>  [echo] apache-cassandra: /cassandra/build.xml
> [javac] Compiling 45 source files to /cassandra/build/classes/thrift
> [javac] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> [javac] error: error reading 
> /cassandra/build/lib/jars/byteman-install-3.0.3.jar; error in opening zip file
> [javac] Compiling 1474 source files to /cassandra/build/classes/main
> [javac] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> [javac] error: error reading 
> /cassandra/build/lib/jars/byteman-install-3.0.3.jar; error in opening zip file
> [javac] Creating empty 
> /cassandra/build/classes/main/org/apache/cassandra/hints/package-info.class 
> "
> 2) 
> To check the jar and get:
> # jar -i /cassandra/build/lib/jars/byteman-install-3.0.3.jar
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> java.util.zip.ZipException: error in opening zip file
> at java.util.zip.ZipFile.open(Native Method)
> at java.util.zip.ZipFile.(ZipFile.java:219)
> at java.util.zip.ZipFile.(ZipFile.java:149)
> at java.util.jar.JarFile.(JarFile.java:166)
> at java.util.jar.JarFile.(JarFile.java:103)
> at sun.tools.jar.Main.getJarPath(Main.java:1163)
> at sun.tools.jar.Main.genIndex(Main.java:1195)
> at sun.tools.jar.Main.run(Main.java:317)
> at sun.tools.jar.Main.main(Main.java:1288)
> 3) if download the jar and replace it, the build will be successful.
> wget http://downloads.jboss.org/byteman/3.0.3/byteman-download-3.0.3-bin.zip
>   unzip byteman-download-3.0.3-bin.zip -d /tmp
>   rm  -f  build/lib/jars/byteman-install-3.0.3.jar
>   cp  /tmp/byteman-download-3.0.3/lib/byteman-install.jar 
> build/lib/jars/byteman-install-3.0.3.jar
>   
>   ant
> 
> BUILD SUCCESSFUL
> Total time: 36 seconds



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: Fix NPEs in original CASSANDRA-13324 commit

[aweisberg]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3dabeeaa2 -> c86de2a98


Fix NPEs in original CASSANDRA-13324 commit

Patch by Ariel Weisberg; Reviewed by Marcus Eriksson


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c86de2a9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c86de2a9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c86de2a9

Branch: refs/heads/trunk
Commit: c86de2a9817aa45930afe181ae1891d2363393c7
Parents: 3dabeea
Author: Ariel Weisberg 
Authored: Fri Mar 24 17:48:24 2017 -0400
Committer: Ariel Weisberg 
Committed: Mon Mar 27 13:24:36 2017 -0400

--
 .../apache/cassandra/net/MessagingService.java  | 40 +---
 1 file changed, 26 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c86de2a9/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 55604d0..b7d4329 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -536,8 +536,6 @@ public final class MessagingService implements 
MessagingServiceMBean
 if (cp != null)
 cp.incrementTimeout();
 
-
getConnectionPool(expiredCallbackInfo.target).incrementTimeout();
-
 if (expiredCallbackInfo.callback.supportsBackPressure())
 {
 updateBackPressureOnReceive(expiredCallbackInfo.target, 
expiredCallbackInfo.callback, true);
@@ -607,8 +605,12 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 if (DatabaseDescriptor.backPressureEnabled() && 
callback.supportsBackPressure())
 {
-BackPressureState backPressureState = 
getConnectionPool(host).getBackPressureState();
-backPressureState.onMessageSent(message);
+OutboundTcpConnectionPool cp = getConnectionPool(host);
+if (cp != null)
+{
+BackPressureState backPressureState = 
cp.getBackPressureState();
+backPressureState.onMessageSent(message);
+}
 }
 }
 
@@ -623,11 +625,15 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 if (DatabaseDescriptor.backPressureEnabled() && 
callback.supportsBackPressure())
 {
-BackPressureState backPressureState = 
getConnectionPool(host).getBackPressureState();
-if (!timeout)
-backPressureState.onResponseReceived();
-else
-backPressureState.onResponseTimeout();
+OutboundTcpConnectionPool cp = getConnectionPool(host);
+if (cp != null)
+{
+BackPressureState backPressureState = 
cp.getBackPressureState();
+if (!timeout)
+backPressureState.onResponseReceived();
+else
+backPressureState.onResponseTimeout();
+}
 }
 }
 
@@ -644,10 +650,16 @@ public final class MessagingService implements 
MessagingServiceMBean
 {
 if (DatabaseDescriptor.backPressureEnabled())
 {
-backPressure.apply(StreamSupport.stream(hosts.spliterator(), false)
-.filter(h -> !h.equals(FBUtilities.getBroadcastAddress()))
-.map(h -> getConnectionPool(h).getBackPressureState())
-.collect(Collectors.toSet()), timeoutInNanos, 
TimeUnit.NANOSECONDS);
+Set states = new HashSet();
+for (InetAddress host : hosts)
+{
+if (host.equals(FBUtilities.getBroadcastAddress()))
+continue;
+OutboundTcpConnectionPool cp = getConnectionPool(host);
+if (cp != null)
+states.add(cp.getBackPressureState());
+}
+backPressure.apply(states, timeoutInNanos, TimeUnit.NANOSECONDS);
 }
 }
 
@@ -679,7 +691,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 if (cp != null)
 {
 logger.trace("Resetting pool for {}", ep);
-getConnectionPool(ep).reset();
+cp.reset();
 }
 else
 {

[jira] [Updated] (CASSANDRA-13373) Provide additional speculative retry statistics

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13373:

Reviewer: Blake Eggleston

> Provide additional speculative retry statistics
> ---
>
> Key: CASSANDRA-13373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13373
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.x
>
>
> Right now there is a single metric for speculative retry on reads that is the 
> number of speculative retries attempted. You can't tell how many of those 
> actually succeeded in salvaging the read.
> The metric is also per table and there is no keyspace level rollup as there 
> is for several other metrics.
> Add a metric that counts reads that attempt to speculate but fail to complete 
> before the timeout (ignoring read errors).
> Add a rollup metric for the current count of speculation attempts as well as 
> the count of failed speculations.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13316) Build error because of dependent jar (byteman-install-3.0.3.jar) currupted

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943664#comment-15943664
 ] 

Jay Zhuang commented on CASSANDRA-13316:


The problem is fixed 
[c0f99c|https://github.com/apache/cassandra/commit/c0f99c4e68da8e4e7f3d430b2cb45a762c1ff9f2]
 by [~mshuler]

> Build error because of dependent jar (byteman-install-3.0.3.jar) currupted 
> ---
>
> Key: CASSANDRA-13316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13316
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Platform: Amd64
> OS: CentOS Linux 7
>Reporter: Sam Ding
> Fix For: 3.10
>
>
> When build  cassandra 3.10 on amd64, CentOS Linux 7, there is a build error 
> caused by corrupted jar file (byteman-install-3.0.3.jar).
> Here is the replicated steps:
> After install necessary dependent packages and apache-ant, git clone 
> cassandra 3.10:
> 1)
>   git clone https://github.com/apache/cassandra.git
>   cd cassandra
>   git checkout cassandra-3.10
>   ant 
> Then gets errors like:
> "
> build-project:
>  [echo] apache-cassandra: /cassandra/build.xml
> [javac] Compiling 45 source files to /cassandra/build/classes/thrift
> [javac] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> [javac] error: error reading 
> /cassandra/build/lib/jars/byteman-install-3.0.3.jar; error in opening zip file
> [javac] Compiling 1474 source files to /cassandra/build/classes/main
> [javac] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> [javac] error: error reading 
> /cassandra/build/lib/jars/byteman-install-3.0.3.jar; error in opening zip file
> [javac] Creating empty 
> /cassandra/build/classes/main/org/apache/cassandra/hints/package-info.class 
> "
> 2) 
> To check the jar and get:
> # jar -i /cassandra/build/lib/jars/byteman-install-3.0.3.jar
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
> java.util.zip.ZipException: error in opening zip file
> at java.util.zip.ZipFile.open(Native Method)
> at java.util.zip.ZipFile.(ZipFile.java:219)
> at java.util.zip.ZipFile.(ZipFile.java:149)
> at java.util.jar.JarFile.(JarFile.java:166)
> at java.util.jar.JarFile.(JarFile.java:103)
> at sun.tools.jar.Main.getJarPath(Main.java:1163)
> at sun.tools.jar.Main.genIndex(Main.java:1195)
> at sun.tools.jar.Main.run(Main.java:317)
> at sun.tools.jar.Main.main(Main.java:1288)
> 3) if download the jar and replace it, the build will be successful.
> wget http://downloads.jboss.org/byteman/3.0.3/byteman-download-3.0.3-bin.zip
>   unzip byteman-download-3.0.3-bin.zip -d /tmp
>   rm  -f  build/lib/jars/byteman-install-3.0.3.jar
>   cp  /tmp/byteman-download-3.0.3/lib/byteman-install.jar 
> build/lib/jars/byteman-install-3.0.3.jar
>   
>   ant
> 
> BUILD SUCCESSFUL
> Total time: 36 seconds



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg resolved CASSANDRA-13324.

Resolution: Fixed

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13324:
---
Fix Version/s: 4.0

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13368) Exception Stack not Printed as Intended in Error Logs

[William R. Speirs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William R. Speirs updated CASSANDRA-13368:
--
Resolution: Invalid
Status: Resolved  (was: Patch Available)

It turns out that our organization is running the Datastax version of 
Cassandra, and since I cannot find the code for it, I'm going to assume they 
changed the log lines.

We'll switch over to the Apache version, and see if we notice this issue moving 
forward. If so, then I'll reopen this ticket and investigate further. Thanks!

> Exception Stack not Printed as Intended in Error Logs
> -
>
> Key: CASSANDRA-13368
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13368
> Project: Cassandra
>  Issue Type: Bug
>Reporter: William R. Speirs
>Priority: Trivial
>  Labels: lhf
> Fix For: 2.1.x
>
> Attachments: cassandra-13368-2.1.patch
>
>
> There are a number of instances where it appears the programmer intended to 
> print a stack trace in an error message, but it is not actually being 
> printed. For example, in {{BlacklistedDirectories.java:54}}:
> {noformat}
> catch (Exception e)
> {
> JVMStabilityInspector.inspectThrowable(e);
> logger.error("error registering MBean {}", MBEAN_NAME, e);
> //Allow the server to start even if the bean can't be registered
> }
> {noformat}
> The logger will use the second argument for the braces, but will ignore the 
> exception {{e}}. It would be helpful to have the stack traces of these 
> exceptions printed. I propose adding a second line that prints the full stack 
> trace: {{logger.error(e.getMessage(), e);}}
> On the 2.1 branch, I found 8 instances of these types of messages:
> {noformat}
> db/BlacklistedDirectories.java:54:logger.error("error registering 
> MBean {}", MBEAN_NAME, e);
> io/sstable/SSTableReader.java:512:logger.error("Corrupt sstable 
> {}; skipped", descriptor, e);
> net/OutboundTcpConnection.java:228:logger.error("error 
> processing a message intended for {}", poolReference.endPoint(), e);
> net/OutboundTcpConnection.java:314:logger.error("error 
> writing to {}", poolReference.endPoint(), e);
> service/CassandraDaemon.java:231:logger.error("Exception in 
> thread {}", t, e);
> service/CassandraDaemon.java:562:logger.error("error 
> registering MBean {}", MBEAN_NAME, e);
> streaming/StreamSession.java:512:logger.error("[Stream #{}] 
> Streaming error occurred", planId(), e);
> transport/Server.java:442:logger.error("Problem retrieving 
> RPC address for {}", endpoint, e);
> {noformat}
> And one where it'll print the {{toString()}} version of the exception:
> {noformat}
> db/Directories.java:689:logger.error("Could not calculate the 
> size of {}. {}", input, e);
> {noformat}
> I'm happy to create a patch for each branch, just need a little guidance on 
> how to do so. We're currently running 2.1 so I started there.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11471) Add SASL mechanism negotiation to the native protocol

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943625#comment-15943625
 ] 

Ariel Weisberg commented on CASSANDRA-11471:


Congrats on thew new baby! If your back this soon I'll take it as a good sign. 

* [Typo in 
associated|https://github.com/apache/cassandra/compare/trunk...benbromhead:11471#diff-f4a806982d4fb565a8ceb5476cfb5978R64]
* [Should this be <= since we are going to change the v5-beta 
protocol?|https://github.com/apache/cassandra/compare/trunk...benbromhead:11471#diff-a6c0d16b6f8e72454c376efc0adf51f9R103]

So I am generally speaking OK with the code modulo the nit and the question. 
Kicked off the tests just to see what happens with the Java driver part not 
done. I asked in #datastax-drivers IRC about how we land a change to the 
v5-beta version of the protocol.

||Code|utests|dtests||
|[trunk|https://github.com/benbromhead/cassandra/compare/trunk...aweisberg:cassandra-11471-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-11471-trunk-testall/1/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-cassandra-11471-trunk-dtest/1/]|


> Add SASL mechanism negotiation to the native protocol
> -
>
> Key: CASSANDRA-11471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11471
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Sam Tunnicliffe
>Assignee: Ben Bromhead
>  Labels: client-impacting
> Fix For: 4.x
>
> Attachments: CASSANDRA-11471
>
>
> Introducing an additional message exchange into the authentication sequence 
> would allow us to support multiple authentication schemes and [negotiation of 
> SASL mechanisms|https://tools.ietf.org/html/rfc4422#section-3.2]. 
> The current {{AUTHENTICATE}} message sent from Client to Server includes the 
> java classname of the configured {{IAuthenticator}}. This could be superceded 
> by a new message which lists the SASL mechanisms supported by the server. The 
> client would then respond with a new message which indicates it's choice of 
> mechanism.  This would allow the server to support multiple mechanisms, for 
> example enabling both {{PLAIN}} for username/password authentication and 
> {{EXTERNAL}} for a mechanism for extracting credentials from SSL 
> certificates\* (see the example in 
> [RFC-4422|https://tools.ietf.org/html/rfc4422#appendix-A]). Furthermore, the 
> server could tailor the list of supported mechanisms on a per-connection 
> basis, e.g. only offering certificate based auth to encrypted clients. 
> The client's response should include the selected mechanism and any initial 
> response data. This is mechanism-specific; the {{PLAIN}} mechanism consists 
> of a single round in which the client sends encoded credentials as the 
> initial response data and the server response indicates either success or 
> failure with no futher challenges required.
> From a protocol perspective, after the mechanism negotiation the exchange 
> would continue as in protocol v4, with one or more rounds of 
> {{AUTH_CHALLENGE}} and {{AUTH_RESPONSE}} messages, terminated by an 
> {{AUTH_SUCCESS}} sent from Server to Client upon successful authentication or 
> an {{ERROR}} on auth failure. 
> XMPP performs mechanism negotiation in this way, 
> [RFC-3920|http://tools.ietf.org/html/rfc3920#section-6] includes a good 
> overview.
> \* Note: this would require some a priori agreement between client and server 
> over the implementation of the {{EXTERNAL}} mechanism.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943683#comment-15943683
 ] 

Ariel Weisberg commented on CASSANDRA-13324:


Merged additional changes in 
[c86de2a9817aa45930afe181ae1891d2363393c7|https://github.com/apache/cassandra/commit/c86de2a9817aa45930afe181ae1891d2363393c7]

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.0
>
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943870#comment-15943870
 ] 

Ariel Weisberg commented on CASSANDRA-13327:


OK. So to remove the node from {{TokenMetadata.pendingEndpointsFor}} then it 
must appear as a natural endpoint from {{StorageService.getNaturalEndpoints()}} 
otherwise writes would never make to that participant at all. I guess I need to 
look up when that transition occurs now and whether doing that transition 
sooner makes sense.



> Pending endpoints size check for CAS doesn't play nicely with 
> writes-on-replacement
> ---
>
> Key: CASSANDRA-13327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13327
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> Consider this ring:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.4   MR DOWN NORMAL -7148113328562451251
> where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to 
> the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and 
> making no progress.
> Then the down node was replaced so we had:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.5   MR UP JOINING -7148113328562451251
> It’s confusing in the ring - the first JOINING is a genuine bootstrap, the 
> second is a replacement. We now had CAS unavailables (but no non-CAS 
> unvailables). I think it’s because the pending endpoints check thinks that 
> 127.0.0.5 is gaining a range when it’s just replacing.
> The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t 
> unnecessarily fail these requests.
> It also appears like required participants is bumped by 1 during a host 
> replacement so if the replacing host fails you will get unavailables and 
> timeouts.
> This is related to the check added in CASSANDRA-8346



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11530) Remove deprecated repair method in 4.0

[Scott Bale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943844#comment-15943844
 ] 

Scott Bale commented on CASSANDRA-11530:


[~pauloricardomg] thanks. Because old repair JMX is broken anyway 
(CASSANDRA-13121), fortunately we've already migrated off of it

> Remove deprecated repair method in 4.0
> --
>
> Key: CASSANDRA-11530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11530
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 4.x
>
>
> Once we hit 4.0, we should remove all deprecated repair JMX API.
> (nodetool has been using only new API since it is introduced.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12962) SASI: Index are rebuilt on restart

[Corentin Chary (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943403#comment-15943403
 ] 

Corentin Chary commented on CASSANDRA-12962:


Looks robust enough to me :)

> SASI: Index are rebuilt on restart
> --
>
> Key: CASSANDRA-12962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12962
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Corentin Chary
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: screenshot-1.png
>
>
> Apparently when cassandra any index that does not index a value in *every* 
> live SSTable gets rebuild. The offending code can be found in the constructor 
> of SASIIndex.
> You can easilly reproduce it:
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '1'}  AND durable_writes = true;
> CREATE TABLE test.test (
> a text PRIMARY KEY,
> b text,
> c text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX test_b_idx ON test.test (b) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX test_c_idx ON test.test (c) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> INSERT INTO test.test (a, b) VALUES ('a', 'b');
> {code}
> Log (I added additional traces):
> {code}
> INFO  [main] 2016-11-28 15:32:21,191 ColumnFamilyStore.java:406 - 
> Initializing test.test
> DEBUG [SSTableBatchOpen:1] 2016-11-28 15:32:21,192 SSTableReader.java:505 - 
> Opening 
> /mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big 
> (0.034KiB)
> DEBUG [main] 2016-11-28 15:32:21,194 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@2f661b1a[id=6b00489b-7010-396e-9348-9f32f5167f88,name=test_b_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=b}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> INFO  [main] 2016-11-28 15:32:21,194 DataTracker.java:152 - 
> SSTableIndex.open(column: b, minTerm: value, maxTerm: value, minKey: key, 
> maxKey: key, sstable: BigTableReader(path='/mnt/ssd/tmp/data/data/test/test\
> -229e6380b57711e68407158fde22e121/mc-1-big-Data.db'))
> DEBUG [main] 2016-11-28 15:32:21,195 SASIIndex.java:129 - Rebuilding SASI 
> Indexes: {}
> DEBUG [main] 2016-11-28 15:32:21,195 ColumnFamilyStore.java:895 - Enqueuing 
> flush of IndexInfo: 0.386KiB (0%) on-heap, 0.000KiB (0%) off-heap
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:465 - Writing Memtable-IndexInfo@748981977(0.054KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = (min(-9223\
> 372036854775808), max(9223372036854775807)]
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:494 - Completed flushing 
> /mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db
>  (0.035KiB) for\
>  commitlog position CommitLogPosition(segmentId=1480343535479, position=15652)
> DEBUG [MemtableFlushWriter:1] 2016-11-28 15:32:21,224 
> ColumnFamilyStore.java:1200 - Flushed to 
> [BigTableReader(path='/mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db\
> ')] (1 sstables, 4.838KiB), biggest 4.838KiB, smallest 4.838KiB
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@12f3d291[id=45fcb286-b87a-3d18-a04b-b899a9880c91,name=test_c_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=c}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:121 - to rebuild: index: 
> BigTableReader(path='/mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big-Data.db'),
>  sstable: org.apache.cassa\
> ndra.index.sasi.conf.ColumnIndex@6cbb6b0e
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:129 - Rebuilding SASI 
> Indexes: 
> 

[jira] [Commented] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942887#comment-15942887
 ] 

Alex Petrov commented on CASSANDRA-13229:
-

The test in [CASSANDRA-9639] has surfaced a problem with the size estimates 
(which is although not very realistic given we have more startup tokens). When 
the tokens are given manually in the test, we have to split 3 ranges into 3 
parts (in dtest we use 3 data directories to ensure we check multi-disk 
setups). Since the given tokens aren't distributed evenly, we end up having 
just 1 range instead of three.

In reality (however improbable the scenario of this happening is), this fix 
would make sure that all the given disks are utilised. 

|[patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:9639-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9639-trunk-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9639-trunk-dtest/]|[novnode|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9639-trunk-novnode_dtest/]|

> dtest failure in topology_test.TestTopology.size_estimates_multidc_test
> ---
>
> Key: CASSANDRA-13229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Alex Petrov
>  Labels: dtest, test-failure
> Fix For: 4.0
>
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 
> - Exception in thread Thread[MemtablePostFlush:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
>   at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
>  ~[main/:na]
>   at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
> ~[main/:na]
>   at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Unexpected error in node1 log, error: 
> ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
>  ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
>  ~[main/:na]
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
>   at 
> org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> 

[jira] [Updated] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13229:

Assignee: Paulo Motta  (was: Alex Petrov)
  Status: Patch Available  (was: Open)

> dtest failure in topology_test.TestTopology.size_estimates_multidc_test
> ---
>
> Key: CASSANDRA-13229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Paulo Motta
>  Labels: dtest, test-failure
> Fix For: 4.0
>
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 
> - Exception in thread Thread[MemtablePostFlush:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
>   at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
>  ~[main/:na]
>   at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
> ~[main/:na]
>   at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Unexpected error in node1 log, error: 
> ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
>  ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
>  ~[main/:na]
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
>   at 
> org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 

[jira] [Commented] (CASSANDRA-13197) +=/-= shortcut syntax bugs/inconsistencies

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942968#comment-15942968
 ] 

Andrés de la Peña commented on CASSANDRA-13197:
---

The patch looks good to me. Maybe the error message for using {{+=}} trying to 
add a set to a map could improved in an analogous way as it has been done for 
{{ -=}} trying to remove a map from a map (case 2). So, this:
{code}
UPDATE simplex.collection_table SET d_m += {1} WHERE k=0;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
set literal for d_m of type map"
{code}
May be improved to:
{code}
UPDATE simplex.collection_table SET d_m += {1} WHERE k=0;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Value for 
a map addition has to be a map, but was: '{1}"
{code}
What do you think? Does it make sense?

> +=/-= shortcut syntax bugs/inconsistencies
> --
>
> Key: CASSANDRA-13197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13197
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kishan Karunaratne
>Assignee: Alex Petrov
>
> CASSANDRA-12232 introduced (+=/-=) shortcuts for counters and collection 
> types. I ran into some bugs/consistencies.
> Given the schema:
> {noformat}
> CREATE TABLE simplex.collection_table (k int PRIMARY KEY, d_l List, d_s 
> Set, d_m Map, d_t Tuple);
> {noformat}
> 1) Using -= on a list column removes all elements that match the value, 
> instead of the first or last occurrence of it. Is this expected?
> {noformat}
> Given d_l = [0, 1, 2, 1, 1]
> UPDATE collection_table SET d_l -= [1] WHERE k=0;
> yields 
> [0, 2]
> {noformat}
> 2) I can't seem to remove a map key/value pair:
> {noformat}
> Given d_m = {0: 0, 1: 1}
> UPDATE collection_table SET d_m -= {1:1} WHERE k=0;
> yields
> Invalid map literal for d_m of type frozen
> {noformat}
> However {noformat}UPDATE collection_table SET d_m -= {1} WHERE k=0;{noformat} 
> does work.
> 3) Tuples are immutable so it make sense that +=/-= doesn't apply. However 
> the error message could be better, now that other collection types are 
> allowed:
> {noformat}
> UPDATE collection_table SET d_t += (1) WHERE k=0;
> yields
> Invalid operation (d_t = d_t + (1)) for non counter column d_t
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[3/6] cassandra git commit: Dropping column results in "corrupt" SSTable

[slebresne]

Dropping column results in "corrupt" SSTable

patch by Sylvain Lebresne; reviewed by Alex Petrov for CASSANDRA-13337


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5262bb17
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5262bb17
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5262bb17

Branch: refs/heads/trunk
Commit: 5262bb17b46fc8c02f9f836ddf9317d0de2698cd
Parents: 6311622
Author: Sylvain Lebresne 
Authored: Mon Mar 20 15:49:27 2017 +0100
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 11:58:32 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  3 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 144 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c5573a..0b1bb01 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
  * Bugs handling range tombstones in the sstable iterators (CASSANDRA-13340)
  * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java 
b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
index 9bcca48..fa337c0 100644
--- a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
@@ -123,20 +123,27 @@ public class SSTableIterator extends 
AbstractSSTableIterator
 {
 assert deserializer != null;
 
-// We use a same reasoning as in handlePreSliceData regarding the 
strictness of the inequality below.
-// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
-// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
-// (EXCL_END(x) == INCL_START(x) for instance) and in that case we 
don't want to return start bound because
-// it's fundamentally excluded. And if the bound is a  end (for a 
range tombstone), it means it's exactly
-// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
-// an open marker" code in hasNextInterna
-if (!deserializer.hasNext() || deserializer.compareNextTo(end) >= 
0)
-return null;
-
-Unfiltered next = deserializer.readNext();
-if (next.kind() == Unfiltered.Kind.RANGE_TOMBSTONE_MARKER)
-updateOpenMarker((RangeTombstoneMarker)next);
-return next;
+while (true)
+{
+// We use a same reasoning as in handlePreSliceData regarding 
the strictness of the inequality below.
+// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
+// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
+// (EXCL_END(x) == INCL_START(x) for instance) and in that 
case we don't want to return start bound because
+// it's fundamentally excluded. And if the bound is a  end 
(for a range tombstone), it means it's exactly
+// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
+// an open marker" code in hasNextInterna
+if (!deserializer.hasNext() || deserializer.compareNextTo(end) 
>= 0)
+return null;
+
+Unfiltered next = deserializer.readNext();
+// We may get empty row for the same reason expressed on 
UnfilteredSerializer.deserializeOne.
+if (next.isEmpty())
+continue;
+
+if (next.kind() == Unfiltered.Kind.RANGE_TOMBSTONE_MARKER)
+

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Dropping column results in "corrupt" SSTable


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0ac928d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0ac928d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0ac928d

Branch: refs/heads/cassandra-3.11
Commit: c0ac928d985a59d00310fed124c5a2b3655db8e0
Parents: ee7023e 5262bb1
Author: Sylvain Lebresne 
Authored: Mon Mar 27 11:59:06 2017 +0200
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 11:59:06 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  3 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 144 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/CHANGES.txt
--
diff --cc CHANGES.txt
index 071dd1a,0b1bb01..98ed992
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,21 @@@
 -3.0.13
 +3.11.0
 + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
 + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns 
(CASSANDRA-13247)
 + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern 
(CASSANDRA-13317)
 + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound 
(CASSANDRA-13366)
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 +Merged from 3.0:
+  * Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
   * Bugs handling range tombstones in the sstable iterators (CASSANDRA-13340)
   * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
   * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/RangeTombstoneMarker.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/Unfiltered.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/UnfilteredSerializer.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--

[1/6] cassandra git commit: Dropping column results in "corrupt" SSTable

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 631162271 -> 5262bb17b
  refs/heads/cassandra-3.11 ee7023e32 -> c0ac928d9
  refs/heads/trunk 23cd27fcf -> 3dabeeaa2


Dropping column results in "corrupt" SSTable

patch by Sylvain Lebresne; reviewed by Alex Petrov for CASSANDRA-13337


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5262bb17
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5262bb17
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5262bb17

Branch: refs/heads/cassandra-3.0
Commit: 5262bb17b46fc8c02f9f836ddf9317d0de2698cd
Parents: 6311622
Author: Sylvain Lebresne 
Authored: Mon Mar 20 15:49:27 2017 +0100
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 11:58:32 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  3 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 144 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c5573a..0b1bb01 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
  * Bugs handling range tombstones in the sstable iterators (CASSANDRA-13340)
  * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java 
b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
index 9bcca48..fa337c0 100644
--- a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
@@ -123,20 +123,27 @@ public class SSTableIterator extends 
AbstractSSTableIterator
 {
 assert deserializer != null;
 
-// We use a same reasoning as in handlePreSliceData regarding the 
strictness of the inequality below.
-// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
-// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
-// (EXCL_END(x) == INCL_START(x) for instance) and in that case we 
don't want to return start bound because
-// it's fundamentally excluded. And if the bound is a  end (for a 
range tombstone), it means it's exactly
-// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
-// an open marker" code in hasNextInterna
-if (!deserializer.hasNext() || deserializer.compareNextTo(end) >= 
0)
-return null;
-
-Unfiltered next = deserializer.readNext();
-if (next.kind() == Unfiltered.Kind.RANGE_TOMBSTONE_MARKER)
-updateOpenMarker((RangeTombstoneMarker)next);
-return next;
+while (true)
+{
+// We use a same reasoning as in handlePreSliceData regarding 
the strictness of the inequality below.
+// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
+// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
+// (EXCL_END(x) == INCL_START(x) for instance) and in that 
case we don't want to return start bound because
+// it's fundamentally excluded. And if the bound is a  end 
(for a range tombstone), it means it's exactly
+// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
+// an open marker" code in hasNextInterna
+if (!deserializer.hasNext() || deserializer.compareNextTo(end) 
>= 0)
+return null;
+
+Unfiltered next = deserializer.readNext();
+// We may get empty row for the same reason expressed on 
UnfilteredSerializer.deserializeOne.
+  

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[slebresne]

Merge branch 'cassandra-3.11' into trunk

* cassandra-3.11:
  Dropping column results in "corrupt" SSTable


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3dabeeaa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3dabeeaa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3dabeeaa

Branch: refs/heads/trunk
Commit: 3dabeeaa29bfb6757e79f17031cb467b7b052fcd
Parents: 23cd27f c0ac928
Author: Sylvain Lebresne 
Authored: Mon Mar 27 12:04:20 2017 +0200
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 12:04:20 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  4 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 145 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
--
diff --cc 
src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
index e67d164,88d415b..837562a
--- 
a/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
+++ 
b/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
@@@ -223,8 -228,12 +223,10 @@@ public class SSTableReversedIterator ex
 && !stopReadingDisk())
  {
  Unfiltered unfiltered = deserializer.readNext();
- buffer.add(unfiltered);
+ // We may get empty row for the same reason expressed on 
UnfilteredSerializer.deserializeOne.
 -if (!unfiltered.isEmpty() && (!isFirst || includeFirst))
++if (!unfiltered.isEmpty())
+ buffer.add(unfiltered);
  
 -isFirst = false;
 -
  if (unfiltered.isRangeTombstoneMarker())
  updateOpenMarker((RangeTombstoneMarker)unfiltered);
  }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/src/java/org/apache/cassandra/db/rows/Unfiltered.java
--
diff --cc src/java/org/apache/cassandra/db/rows/Unfiltered.java
index 39d9e75,e75c632..e1158e2
--- a/src/java/org/apache/cassandra/db/rows/Unfiltered.java
+++ b/src/java/org/apache/cassandra/db/rows/Unfiltered.java
@@@ -53,11 -53,13 +53,13 @@@ public interface Unfiltered extends Clu
   * invalid (some value is invalid for its column type, or some field
   * is nonsensical).
   */
 -public void validateData(CFMetaData metadata);
 +public void validateData(TableMetadata metadata);
  
+ public boolean isEmpty();
+ 
 -public String toString(CFMetaData metadata);
 -public String toString(CFMetaData metadata, boolean fullDetails);
 -public String toString(CFMetaData metadata, boolean includeClusterKeys, 
boolean fullDetails);
 +public String toString(TableMetadata metadata);
 +public String toString(TableMetadata metadata, boolean fullDetails);
 +public String toString(TableMetadata metadata, boolean 
includeClusterKeys, boolean fullDetails);
  
  default boolean isRow()
  {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/src/java/org/apache/cassandra/db/rows/UnfilteredSerializer.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3dabeeaa/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Dropping column results in "corrupt" SSTable


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0ac928d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0ac928d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0ac928d

Branch: refs/heads/trunk
Commit: c0ac928d985a59d00310fed124c5a2b3655db8e0
Parents: ee7023e 5262bb1
Author: Sylvain Lebresne 
Authored: Mon Mar 27 11:59:06 2017 +0200
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 11:59:06 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  3 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 144 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/CHANGES.txt
--
diff --cc CHANGES.txt
index 071dd1a,0b1bb01..98ed992
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,21 @@@
 -3.0.13
 +3.11.0
 + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
 + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns 
(CASSANDRA-13247)
 + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern 
(CASSANDRA-13317)
 + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound 
(CASSANDRA-13366)
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range 
(CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 +Merged from 3.0:
+  * Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
   * Bugs handling range tombstones in the sstable iterators (CASSANDRA-13340)
   * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
   * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/columniterator/SSTableReversedIterator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/RangeTombstoneMarker.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/Unfiltered.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/src/java/org/apache/cassandra/db/rows/UnfilteredSerializer.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0ac928d/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--

[2/6] cassandra git commit: Dropping column results in "corrupt" SSTable

[slebresne]

Dropping column results in "corrupt" SSTable

patch by Sylvain Lebresne; reviewed by Alex Petrov for CASSANDRA-13337


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5262bb17
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5262bb17
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5262bb17

Branch: refs/heads/cassandra-3.11
Commit: 5262bb17b46fc8c02f9f836ddf9317d0de2698cd
Parents: 6311622
Author: Sylvain Lebresne 
Authored: Mon Mar 20 15:49:27 2017 +0100
Committer: Sylvain Lebresne 
Committed: Mon Mar 27 11:58:32 2017 +0200

--
 CHANGES.txt |  1 +
 .../db/columniterator/SSTableIterator.java  | 78 
 .../columniterator/SSTableReversedIterator.java |  3 +-
 .../cassandra/db/rows/RangeTombstoneMarker.java |  6 ++
 .../apache/cassandra/db/rows/Unfiltered.java|  2 +
 .../cassandra/db/rows/UnfilteredSerializer.java | 43 +--
 .../cql3/validation/operations/AlterTest.java   | 51 +
 7 files changed, 144 insertions(+), 40 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c5573a..0b1bb01 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.13
+ * Dropping column results in "corrupt" SSTable (CASSANDRA-13337)
  * Bugs handling range tombstones in the sstable iterators (CASSANDRA-13340)
  * Fix CONTAINS filtering for null collections (CASSANDRA-13246)
  * Applying: Use a unique metric reservoir per test run when using 
Cassandra-wide metrics residing in MBeans (CASSANDRA-13216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5262bb17/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java 
b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
index 9bcca48..fa337c0 100644
--- a/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SSTableIterator.java
@@ -123,20 +123,27 @@ public class SSTableIterator extends 
AbstractSSTableIterator
 {
 assert deserializer != null;
 
-// We use a same reasoning as in handlePreSliceData regarding the 
strictness of the inequality below.
-// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
-// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
-// (EXCL_END(x) == INCL_START(x) for instance) and in that case we 
don't want to return start bound because
-// it's fundamentally excluded. And if the bound is a  end (for a 
range tombstone), it means it's exactly
-// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
-// an open marker" code in hasNextInterna
-if (!deserializer.hasNext() || deserializer.compareNextTo(end) >= 
0)
-return null;
-
-Unfiltered next = deserializer.readNext();
-if (next.kind() == Unfiltered.Kind.RANGE_TOMBSTONE_MARKER)
-updateOpenMarker((RangeTombstoneMarker)next);
-return next;
+while (true)
+{
+// We use a same reasoning as in handlePreSliceData regarding 
the strictness of the inequality below.
+// We want to exclude deserialized unfiltered equal to end, 
because 1) we won't miss any rows since those
+// woudn't be equal to a slice bound and 2) a end bound can be 
equal to a start bound
+// (EXCL_END(x) == INCL_START(x) for instance) and in that 
case we don't want to return start bound because
+// it's fundamentally excluded. And if the bound is a  end 
(for a range tombstone), it means it's exactly
+// our slice end, but in that  case we will properly close the 
range tombstone anyway as part of our "close
+// an open marker" code in hasNextInterna
+if (!deserializer.hasNext() || deserializer.compareNextTo(end) 
>= 0)
+return null;
+
+Unfiltered next = deserializer.readNext();
+// We may get empty row for the same reason expressed on 
UnfilteredSerializer.deserializeOne.
+if (next.isEmpty())
+continue;
+
+if (next.kind() == Unfiltered.Kind.RANGE_TOMBSTONE_MARKER)
+

[jira] [Commented] (CASSANDRA-13379) SASI index returns duplicate rows

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943012#comment-15943012
 ] 

Igor Novgorodov commented on CASSANDRA-13379:
-

Yep, it's paging:
{code}
> PAGING 2
Page size: 2
> select * from bulks_recipients where bulk_id_idx = 
> fdeca186-e9aa-41a7-a086-eb377bb656a5;

 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 4 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

(9 rows)
{code}

9 rows instead of 5:
{code}
> select * from bulks_recipients ;

 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 4 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

(5 rows)
{code}

> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select 

[jira] [Resolved] (CASSANDRA-13332) wrong number of rows in cqlsh

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov resolved CASSANDRA-13332.
-
Resolution: Duplicate

Looks very much like a duplicate, as Stefan has pointed out.

> wrong number of rows in cqlsh
> -
>
> Key: CASSANDRA-13332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13332
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Windows 10
>Reporter: Bogusz Jelinski
>Priority: Trivial
> Attachments: cqlsh.jpg
>
>
> when I send a query via JDBC I receive fewer rows than via CQLSH. What I can 
> see is that at each page/list break the last row is duplicated - replicated 
> to the next page of the list. I have 8 page breaks and the summary of my 
> query gives 8 rows more than my Java client. 
> It might be of no importance - I am querying a table with "WHERE columnA LIKE 
> '%ABCD%'" for which I created an index: 
> CREATE CUSTOM INDEX columnA_idx ON ab.cdef (columnA) using 
> 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = { 'mode': 
> 'CONTAINS' };



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13379) SASI index returns duplicate rows

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942849#comment-15942849
 ] 

Alex Petrov commented on CASSANDRA-13379:
-

Thank you for reporting. I suspect this is a paging issue.

Could you please check if the duplicates are on the page boundaries? For that 
you could run {{paging 10}} in cqlsh and then try running your request again: 

{code}
select * from bulks_recipients where bulk_id_idx = 
fedd95ec-2cc8-4040-8619-baf69647700b;
{code}

Then skipping through the pages. If the last entry of each page is the same as 
the first entry of the next page, that'd be it.

Thank you for assistance. 

> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select count(*) from bulks_recipients ;
>  count
> -
>  100
> (1 rows)
> {code}
> Ok, it's fine here. Now let's query by SASI:
> {code}
> > select count(*) from bulks_recipients where bulk_id_idx = 
> > fedd95ec-2cc8-4040-8619-baf69647700b;
>  count
> -
>  1010101
> (1 rows)
> {code}
> Hmm, very strange count - 10101 extra rows.
> Ok, i've dumped the query result into a text file:
> {code}
> # cat sasi.txt | wc -l
> 1000200
> {code}
> Here we have 200 extra rows for some reason.
> Let's check if these are duplicates:
> {code}
> # cat sasi.txt | sort | uniq | wc -l
> 100
> {code}
> Yep, looks like.
> Recreating index does not help. If i issue the very same query (against 
> partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12962) SASI: Index are rebuilt on restart

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12962:

Issue Type: Bug  (was: Improvement)

> SASI: Index are rebuilt on restart
> --
>
> Key: CASSANDRA-12962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12962
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Corentin Chary
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 3.11.x
>
> Attachments: screenshot-1.png
>
>
> Apparently when cassandra any index that does not index a value in *every* 
> live SSTable gets rebuild. The offending code can be found in the constructor 
> of SASIIndex.
> You can easilly reproduce it:
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '1'}  AND durable_writes = true;
> CREATE TABLE test.test (
> a text PRIMARY KEY,
> b text,
> c text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX test_b_idx ON test.test (b) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX test_c_idx ON test.test (c) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> INSERT INTO test.test (a, b) VALUES ('a', 'b');
> {code}
> Log (I added additional traces):
> {code}
> INFO  [main] 2016-11-28 15:32:21,191 ColumnFamilyStore.java:406 - 
> Initializing test.test
> DEBUG [SSTableBatchOpen:1] 2016-11-28 15:32:21,192 SSTableReader.java:505 - 
> Opening 
> /mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big 
> (0.034KiB)
> DEBUG [main] 2016-11-28 15:32:21,194 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@2f661b1a[id=6b00489b-7010-396e-9348-9f32f5167f88,name=test_b_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=b}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> INFO  [main] 2016-11-28 15:32:21,194 DataTracker.java:152 - 
> SSTableIndex.open(column: b, minTerm: value, maxTerm: value, minKey: key, 
> maxKey: key, sstable: BigTableReader(path='/mnt/ssd/tmp/data/data/test/test\
> -229e6380b57711e68407158fde22e121/mc-1-big-Data.db'))
> DEBUG [main] 2016-11-28 15:32:21,195 SASIIndex.java:129 - Rebuilding SASI 
> Indexes: {}
> DEBUG [main] 2016-11-28 15:32:21,195 ColumnFamilyStore.java:895 - Enqueuing 
> flush of IndexInfo: 0.386KiB (0%) on-heap, 0.000KiB (0%) off-heap
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:465 - Writing Memtable-IndexInfo@748981977(0.054KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = (min(-9223\
> 372036854775808), max(9223372036854775807)]
> DEBUG [PerDiskMemtableFlushWriter_0:1] 2016-11-28 15:32:21,204 
> Memtable.java:494 - Completed flushing 
> /mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db
>  (0.035KiB) for\
>  commitlog position CommitLogPosition(segmentId=1480343535479, position=15652)
> DEBUG [MemtableFlushWriter:1] 2016-11-28 15:32:21,224 
> ColumnFamilyStore.java:1200 - Flushed to 
> [BigTableReader(path='/mnt/ssd/tmp/data/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-4256-big-Data.db\
> ')] (1 sstables, 4.838KiB), biggest 4.838KiB, smallest 4.838KiB
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:118 - index: 
> org.apache.cassandra.schema.IndexMetadata@12f3d291[id=45fcb286-b87a-3d18-a04b-b899a9880c91,name=test_c_idx,kind=CUSTOM,options={class_name=org.a\
> pache.cassandra.index.sasi.SASIIndex, target=c}], base CFS(Keyspace='test', 
> ColumnFamily='test'), tracker 
> org.apache.cassandra.db.lifecycle.Tracker@15900b83
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:121 - to rebuild: index: 
> BigTableReader(path='/mnt/ssd/tmp/data/data/test/test-229e6380b57711e68407158fde22e121/mc-1-big-Data.db'),
>  sstable: org.apache.cassa\
> ndra.index.sasi.conf.ColumnIndex@6cbb6b0e
> DEBUG [main] 2016-11-28 15:32:21,224 SASIIndex.java:129 - Rebuilding SASI 
> Indexes: 
> 

[jira] [Created] (CASSANDRA-13382) cdc column addition strikes again

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-13382:


 Summary: cdc column addition strikes again
 Key: CASSANDRA-13382
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 3.11.x


This is a followup of CASSANDRA-12697, where the patch mistakenly only handled 
the {{system_schema.tables}} table, while the {{cdc}} column has been added to 
{{system_schema.views}} table.

The patch is pretty trivial, though this highlight that we don't seem to have 
upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13382) cdc column addition strikes again

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13382:
-
Status: Patch Available  (was: Open)

Simple patch below:
| [13382-3.11|https://github.com/pcmanus/cassandra/commits/13382-3.11] | 
[utests|http://cassci.datastax.com/job/pcmanus-13382-3.11-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-13382-3.11-dtest] |

I'll try to write a new upgrade dtest to show this, but as I've never really 
been able to run upgrade tests locally, I won't promise to sink 2 days of my 
time into it if it doesn't go smoothly.

> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13337) Dropping column results in "corrupt" SSTable

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13337:
-
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.0
   3.0.13
Reproduced In: 3.10, 3.0.12  (was: 3.0.12, 3.10)
   Status: Resolved  (was: Ready to Commit)

Committed, thanks.

> Dropping column results in "corrupt" SSTable
> 
>
> Key: CASSANDRA-13337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13337
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Jonas Borgström
>Assignee: Sylvain Lebresne
> Fix For: 3.0.13, 3.11.0
>
>
> It seems like dropping a column can make SSTables containing rows with writes 
> to only the dropped column will become uncompactable.
> Also Cassandra <= 3.9 and <= 3.0.11 will even refuse to start with the same 
> stack trace
> {code}
> cqlsh -e "create keyspace test with replication = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 }"
> cqlsh -e "create table test.test(pk text primary key, x text, y text)"
> cqlsh -e "update test.test set x='1' where pk='1'"
> nodetool flush
> cqlsh -e "update test.test set x='1', y='1' where pk='1'"
> nodetool flush
> cqlsh -e "alter table test.test drop x"
> nodetool compact test test
> error: Corrupt empty row found in unfiltered partition
> -- StackTrace --
> java.io.IOException: Corrupt empty row found in unfiltered partition
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:382)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:87)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:65)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.doCompute(SSTableIdentityIterator.java:123)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:369)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:189)
>   at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:158)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:509)
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:369)
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>   at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:58)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:67)
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:26)
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:610)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> 

[jira] [Commented] (CASSANDRA-13379) SASI index returns duplicate rows

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943004#comment-15943004
 ] 

Igor Novgorodov commented on CASSANDRA-13379:
-

Will test ASAP. The default pagesize is 5000 and i get 200 extra rows, which 
sums up to excatly 1 million, so it looks like a reasonable explaination.

BTW, if i change the table structure to
{code}
CREATE TABLE bulks_recipients (
bulk_id uuid,
recipient text,
PRIMARY KEY (recipient, bulk_id)
)
{code}

and create SASI on bulk_id - it works fine and i don't get any duplicates.

> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select count(*) from bulks_recipients ;
>  count
> -
>  100
> (1 rows)
> {code}
> Ok, it's fine here. Now let's query by SASI:
> {code}
> > select count(*) from bulks_recipients where bulk_id_idx = 
> > fedd95ec-2cc8-4040-8619-baf69647700b;
>  count
> -
>  1010101
> (1 rows)
> {code}
> Hmm, very strange count - 10101 extra rows.
> Ok, i've dumped the query result into a text file:
> {code}
> # cat sasi.txt | wc -l
> 1000200
> {code}
> Here we have 200 extra rows for some reason.
> Let's check if these are duplicates:
> {code}
> # cat sasi.txt | sort | uniq | wc -l
> 100
> {code}
> Yep, looks like.
> Recreating index does not help. If i issue the very same query (against 
> partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13065) Consistent range movements to not require MV updates to go through write paths

[Benjamin Roth (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942722#comment-15942722
 ] 

Benjamin Roth commented on CASSANDRA-13065:
---

[~pauloricardomg] Did you notice my update?

> Consistent range movements to not require MV updates to go through write 
> paths 
> ---
>
> Key: CASSANDRA-13065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13065
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
>Priority: Critical
> Fix For: 4.0
>
>
> Booting or decommisioning nodes with MVs is unbearably slow as all streams go 
> through the regular write paths. This causes read-before-writes for every 
> mutation and during bootstrap it causes them to be sent to batchlog.
> The makes it virtually impossible to boot a new node in an acceptable amount 
> of time.
> Using the regular streaming behaviour for consistent range movements works 
> much better in this case and does not break the MV local consistency contract.
> Already tested on own cluster.
> Bootstrap case is super easy to handle, decommission case requires 
> CASSANDRA-13064



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13138) SASI tries to fetch an extra page when resultset size is same size as page size

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-13138:
---

Assignee: Alex Petrov

> SASI tries to fetch an extra page when resultset size is same size as page 
> size
> ---
>
> Key: CASSANDRA-13138
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13138
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> For example, in a dataset that would return 10 rows, SASI would try (and 
> return an empty page) to fetch the next page, while filtering and 2i will 
> return results correctly:
> {code}
>  pk | ck1 | ck2 | reg1 | reg2 | reg3
> +-+-+--+--+--
>   6 |   5 |   5 |5 |5 |   10
>   7 |   5 |   5 |5 |5 |   10
>   9 |   5 |   5 |5 |5 |   10
>   4 |   5 |   5 |5 |5 |   10
>   3 |   5 |   5 |5 |5 |   10
>   5 |   5 |   5 |5 |5 |   10
>   0 |   5 |   5 |5 |5 |   10
>   8 |   5 |   5 |5 |5 |   10
>   2 |   5 |   5 |5 |5 |   10
>   1 |   5 |   5 |5 |5 |   10
> ---MORE---
> (10 rows)
> {code}
> (that {{--MORE--}} shouldn't have been there) 
> This might be an inherent limitation, although even if it is we can opt out 
> for fetching limit+1 if the data limits aren't exhausted. Although it seems 
> that there should be a solution for it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13379) SASI index returns duplicate rows

[Igor Novgorodov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943012#comment-15943012
 ] 

Igor Novgorodov edited comment on CASSANDRA-13379 at 3/27/17 10:23 AM:
---

Yep, it's paging:
{code}
> PAGING 2
Page size: 2
> select * from bulks_recipients where bulk_id_idx = 
> fdeca186-e9aa-41a7-a086-eb377bb656a5;

 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 4 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

---MORE---
 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

(9 rows)
{code}

9 rows instead of 5:
{code}
> select * from bulks_recipients ;

 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 4 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 5 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 3 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 2 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null

(5 rows)
{code}

By the way, it does not explain {noformat}select count(*){noformat} error, 
looks like it's a separate issue, no?
Or count does paging internally too?



was (Author: blind_oracle):
Yep, it's paging:
{code}
> PAGING 2
Page size: 2
> select * from bulks_recipients where bulk_id_idx = 
> fdeca186-e9aa-41a7-a086-eb377bb656a5;

 bulk_id  | recipient | bulk_id_idx 
 | final_time | request_id | send_time | status
--+---+--+++---+
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 4 | 
fdeca186-e9aa-41a7-a086-eb377bb656a5 |   null |   null |  null |   
null
 fdeca186-e9aa-41a7-a086-eb377bb656a5 | 1 | 

[jira] [Updated] (CASSANDRA-13379) SASI index returns duplicate rows

[Igor Novgorodov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Novgorodov updated CASSANDRA-13379:

Description: 
{code}
CREATE TABLE bulks_recipients (
bulk_id uuid,
recipient text,
bulk_id_idx uuid,
PRIMARY KEY ((bulk_id, recipient))
)
{code}

*bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
partition key component at all for some reason.

{code}
CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients (bulk_id_idx) 
USING 'org.apache.cassandra.index.sasi.SASIIndex';
{code}

Then i insert 1 million rows with the same *bulk_id* and different *recipient*. 
Then 

{code}
> select count(*) from bulks_recipients ;

 count
-
 100

(1 rows)
{code}

Ok, it's fine here. Now let's query by SASI:
{code}
> select count(*) from bulks_recipients where bulk_id_idx = 
> fedd95ec-2cc8-4040-8619-baf69647700b;

 count
-
 1010101

(1 rows)
{code}
Hmm, very strange count - 10101 extra rows.
Ok, i've dumped the query result into a text file:
{code}
# cat sasi.txt | wc -l
1000200
{code}
Here we have 200 extra rows for some reason.

Let's check if these are duplicates:
{code}
# cat sasi.txt | sort | uniq | wc -l
100
{code}
Yep, looks like.

Recreating index does not help. If i issue the very same query (against 
partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.

  was:
{code}
CREATE TABLE bulks_recipients (
bulk_id uuid,
recipient text,
bulk_id_idx uuid,
PRIMARY KEY ((bulk_id, recipient))
)
{code}

*bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
partition key component at all for some reason.

{code}
CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients (bulk_id_idx) 
USING 'org.apache.cassandra.index.sasi.SASIIndex';
{code}

Then i insert 1 million rows with the same *bulk_id* and different *recipient*. 
Then 

{code}
> select count(*) from bulks_recipients ;

 count
-
 100

(1 rows)
{code}

Ok, it's fine here. Now let's query by SASI:
{code}
> select count(*) from bulks_recipients where bulk_id_idx = 
> fedd95ec-2cc8-4040-8619-baf69647700b;

 count
-
 1010101

(1 rows)
{code}
Hmm, very strange count - 10101 extra rows.
Ok, i've dumped the query result into a text file:
{code}
# cat sasi.txt | wc -l
1000200
{code}
Here we have 200 extra rows for some reason.

Let's check if these are duplicates:
{code}
# cat sasi.txt | sort | uniq | wc -l
100
{code}
Yep, looks like.

Recreating index does not help. If i drop the index and issue the very same 
query (against partition key *bulk_id*, not *bulk_id_idx*) - i get correct 
results.


> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select count(*) from bulks_recipients ;
>  count
> -
>  100
> (1 rows)
> {code}
> Ok, it's fine here. Now let's query by SASI:
> {code}
> > select count(*) from bulks_recipients where bulk_id_idx = 
> > fedd95ec-2cc8-4040-8619-baf69647700b;
>  count
> -
>  1010101
> (1 rows)
> {code}
> Hmm, very strange count - 10101 extra rows.
> Ok, i've dumped the query result into a text file:
> {code}
> # cat sasi.txt | wc -l
> 1000200
> {code}
> Here we have 200 extra rows for some reason.
> Let's check if these are duplicates:
> {code}
> # cat sasi.txt | sort | uniq | wc -l
> 100
> {code}
> Yep, looks like.
> Recreating index does not help. If i issue the very same query (against 
> partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13324) Outbound TCP connections ignore internode authenticator

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942731#comment-15942731
 ] 

Marcus Eriksson commented on CASSANDRA-13324:
-

+1

> Outbound TCP connections ignore internode authenticator
> ---
>
> Key: CASSANDRA-13324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13324
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> When creating an outbound connection pool and connecting from within 
> andOutboundTcpConnection it doesn't check if internode authenticator will 
> allow the connection. In practice this can cause a bunch of orphaned threads 
> perpetually attempting to reconnect to an endpoint that is will never accept 
> the connection.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13379) SASI index returns duplicate rows

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov resolved CASSANDRA-13379.
-
Resolution: Duplicate

> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select count(*) from bulks_recipients ;
>  count
> -
>  100
> (1 rows)
> {code}
> Ok, it's fine here. Now let's query by SASI:
> {code}
> > select count(*) from bulks_recipients where bulk_id_idx = 
> > fedd95ec-2cc8-4040-8619-baf69647700b;
>  count
> -
>  1010101
> (1 rows)
> {code}
> Hmm, very strange count - 10101 extra rows.
> Ok, i've dumped the query result into a text file:
> {code}
> # cat sasi.txt | wc -l
> 1000200
> {code}
> Here we have 200 extra rows for some reason.
> Let's check if these are duplicates:
> {code}
> # cat sasi.txt | sort | uniq | wc -l
> 100
> {code}
> Yep, looks like.
> Recreating index does not help. If i issue the very same query (against 
> partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña resolved CASSANDRA-13141.
---
Resolution: Fixed

> test failure in 
> cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
> -
>
> Key: CASSANDRA-13141
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13141
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Andrés de la Peña
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test
> {code}
> Error Message
> Error from server: code=2200 [Invalid query] message="Altering of types is 
> not allowed"
> {code}{code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in 
> prepared_statement_invalidation_test
> session.execute("ALTER TABLE test ALTER d TYPE blob")
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943076#comment-15943076
 ] 

Andrés de la Peña commented on CASSANDRA-13141:
---

This has been already solved by [~blerer] in [this 
commit|https://github.com/riptano/cassandra-dtest/commit/21ba1f7ea785fe2cb6f84e1b595b8f608650ceb2].

[~vishnuprasadh], 
[CASSANDRA-12443|https://issues.apache.org/jira/browse/CASSANDRA-12443] removed 
ALTER TYPE support [from 
3.0.11|https://github.com/apache/cassandra/commit/5f66d48b38d32d3768c78f16753ed4f2095bbede].

> test failure in 
> cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
> -
>
> Key: CASSANDRA-13141
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13141
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Andrés de la Peña
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test
> {code}
> Error Message
> Error from server: code=2200 [Invalid query] message="Altering of types is 
> not allowed"
> {code}{code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in 
> prepared_statement_invalidation_test
> session.execute("ALTER TABLE test ALTER d TYPE blob")
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13379) SASI index returns duplicate rows

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943099#comment-15943099
 ] 

Alex Petrov commented on CASSANDRA-13379:
-

bq. By the way, it does not explain select count(*)

It does use paging internally, yes.

bq. Yep, it's paging

Thank you for confirming it. I'll close this one as a duplicate then and will 
try to fix the paging issue as soon as possible.

> SASI index returns duplicate rows
> -
>
> Key: CASSANDRA-13379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13379
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Igor Novgorodov
>
> {code}
> CREATE TABLE bulks_recipients (
> bulk_id uuid,
> recipient text,
> bulk_id_idx uuid,
> PRIMARY KEY ((bulk_id, recipient))
> )
> {code}
> *bulk_id_idx* is just a copy of *bulk_id* because SASI does not work on 
> partition key component at all for some reason.
> {code}
> CREATE CUSTOM INDEX bulks_recipients_bulk_id ON bulks_recipients 
> (bulk_id_idx) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> Then i insert 1 million rows with the same *bulk_id* and different 
> *recipient*. Then 
> {code}
> > select count(*) from bulks_recipients ;
>  count
> -
>  100
> (1 rows)
> {code}
> Ok, it's fine here. Now let's query by SASI:
> {code}
> > select count(*) from bulks_recipients where bulk_id_idx = 
> > fedd95ec-2cc8-4040-8619-baf69647700b;
>  count
> -
>  1010101
> (1 rows)
> {code}
> Hmm, very strange count - 10101 extra rows.
> Ok, i've dumped the query result into a text file:
> {code}
> # cat sasi.txt | wc -l
> 1000200
> {code}
> Here we have 200 extra rows for some reason.
> Let's check if these are duplicates:
> {code}
> # cat sasi.txt | sort | uniq | wc -l
> 100
> {code}
> Yep, looks like.
> Recreating index does not help. If i issue the very same query (against 
> partition key *bulk_id*, not *bulk_id_idx*) - i get correct results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943160#comment-15943160
 ] 

Jason Brown commented on CASSANDRA-8457:


- I really like what you've done with {{HandshakeProtocol}}. It's a nice 
clarification/centralization of the handshake data.
- I'm cool with ditching the {{DisabledCoalescingStrategy}}, but didn't like 
passing {{null}} around. Thus, I switched to passing 
{{Optional}} as the intent is a bit more explicit.
- With the refactorings, it made it easier to remove {{OutboundConnection}} as 
it's reduced functionality can now live simply in 
{{OutboundMessagingConnection}}, without complicating it.
- addressed the comments about timeouts on the outbound handshake side, and 
reduced them to a single timeout (the one that was already in 
{{OutboundMessagingConnection}})
- upgraded to netty 4.1.9
- updated a lot of comments and documentation.
- added in a lot more tests. code coverage is now > 90% for the 
{{org.apache.cassandra.net.async}} package.

One remaining open issue is the current patch does not expire messages on the 
producer thread, like what we used to do in {{OutboundTcpConnection#enqueue}}. 
I'm awaiting the outcome of CASSANDRA-13324, and will apply the result here.
   

bq. In {{connectionTimeout()}}, what happens when a connection attempt timeout?

This is what 
[{{OutboundTcpConnnection}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L248]
 does, and I've replicated it here. Note that there could be ssveral connection 
attempts before the connection timeout triggers.

bq. in the DISCONNECT/NEGOTIATION_FAILURE case, we don't seem to actively 
trying to reconect

The {{DISCONNECT}} case handles the cases when the protocol versions do not 
match. In {{OutboundTcpConnnection}}, if the [peer's version is lesser 
than|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L465]
 what we expected it to be, we clear the backlog and do not attempt to 
reconnect until more messages are {{#enqueue()}}'d. On the contrary, if the 
[peer's version is greater 
than|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L488]
 what we expected it to be, we'll finish the handshake, send what's currently 
in the backlog, and then disconnect. 

TBH, I think both of these behaviors are incorrect. If the peer's version is 
lesser than what we expect, we know how to send messages (both handshake and 
message serialization), so we could send what we have in the backlog without 
throwing them away. If the peer's version is greater than what we expect, if we 
try to send messages it might fail on the receiver side due to a change in the 
handshake sequence (unlikely) or messaging framing/format (likely). WDYT? 
Either way, I'll need to update {{OutboundMessageConnection}} as it's not quite 
correct rn.

{{NEGOTIATION_FAILURE}} does what 
[{{OutboundTcpConnnection}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L453]:
 if we can't get a version from the peer, either we couldn't connect or the 
handshake failed. I think the current behavior (throw away the backlog) is 
desirable.

bq. In {{InboundHandshakeHandler.handleMessagingStartResponse()}}, maybe we 
should check the return of handshakeTimeout.cancel()

As the {{handshakeTimeout}} executes in the netty event loop, it won't complete 
with the {{#decode}} method. As {{#failHandshake()}} closes the channel, we 
won't even get the {{ThirdHandshakeMessage}} into 
{{handleMessagingStartResponse}} as {{#decode()}} would never be triggered.

bq. Regarding the sending and receiving buffer size handling, 

I agree with all your points and have implemented them.

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944078#comment-15944078
 ] 

Tom van der Woerdt edited comment on CASSANDRA-13348 at 3/27/17 9:31 PM:
-

Murmur3 simulator, yes. GossipingPropertyFileSnitch, if it matters.

I don't recall exactly how this cluster was built, but it was something like 
this :
 * Provision 5 nodes per DC, all but one with "-Dcassandra.join_ring=false". 
Keyspace with rf= dc1:2 dc2:2
 * "nodetool join" one at a time (random order)
 * Provision 30 nodes in dc1 -- all have "allocate_tokens_for_keyspace" set
 * "nodetool join" ~10
 * Decommission the first five, so we're now left with dc1:10
 * "nodetool join" the rest
 * Ditto for dc2, so we now have dc1:30 dc2:30

There's a lot of automation involved, a human may take a different route to 
doing this. I decommissioned the 10 initial nodes which had non-ideal hardware, 
and they made place for 60 more powerful machines.

The "nodetool join" batches to join the final 20 in the DC caused the bad 
tokens.


was (Author: tvdw):
Murmur3 simulator, yes. GossipingPropertyFileSnitch, if it matters.

I don't recall exactly how this cluster was built, but it was something like 
this :
 * Provision 5 nodes per DC, all but one with "-Dcassandra.join_ring=false". 
Keyspace with rf= dc1:2 dc2:2
 * "nodetool join" one at a time (random order)
 * Provision 30 nodes in dc1 -- all have "allocate_tokens_for_keyspace" set
 * "nodetool join" ~10
 * Decommission the first five, so we're now left with dc1:10
 * "nodetool join" the rest
 * Ditto for dc2, so we now have dc1:30 dc2:30

There's a lot of automation involved, a human may take a different route to 
doing this. I decommissioned the 10 initial nodes which had non-ideal hardware, 
and they made place for 60 more powerful machines.

The last "nodetool join" batch produced two or three machines with bad tokens.

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 

[jira] [Commented] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944078#comment-15944078
 ] 

Tom van der Woerdt commented on CASSANDRA-13348:


Murmur3 simulator, yes. GossipingPropertyFileSnitch, if it matters.

I don't recall exactly how this cluster was built, but it was something like 
this :
 * Provision 5 nodes per DC, all but one with "-Dcassandra.join_ring=false". 
Keyspace with rf= dc1:2 dc2:2
 * "nodetool join" one at a time (random order)
 * Provision 30 nodes in dc1 -- all have "allocate_tokens_for_keyspace" set
 * "nodetool join" ~10
 * Decommission the first five, so we're now left with dc1:10
 * "nodetool join" the rest
 * Ditto for dc2, so we now have dc1:30 dc2:30

There's a lot of automation involved, a human may take a different route to 
doing this. I decommissioned the 10 initial nodes which had non-ideal hardware, 
and they made place for 60 more powerful machines.

The last "nodetool join" batch produced two or three machines with bad tokens.

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,630 
> TokenAllocation.java:61 - Selected tokens [.., 2890709530010722764, 
> -2416006722819773829, -5820248611267569511, -5990139574852472056, 
> 1645259890258332163, 

[jira] [Commented] (CASSANDRA-13373) Provide additional speculative retry statistics

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944055#comment-15944055
 ] 

Blake Eggleston commented on CASSANDRA-13373:
-

I'd prefer that the speculative retry logic be confined to the speculative 
retry executor, if possible. If SpeculatingReadExecutor created a 
SpeculatingReadCallback, you could just override awaitResults and increment the 
failed counter if a timeout exception is called.

> Provide additional speculative retry statistics
> ---
>
> Key: CASSANDRA-13373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13373
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 4.x
>
>
> Right now there is a single metric for speculative retry on reads that is the 
> number of speculative retries attempted. You can't tell how many of those 
> actually succeeded in salvaging the read.
> The metric is also per table and there is no keyspace level rollup as there 
> is for several other metrics.
> Add a metric that counts reads that attempt to speculate but fail to complete 
> before the timeout (ignoring read errors).
> Add a rollup metric for the current count of speculation attempts as well as 
> the count of failed speculations.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944058#comment-15944058
 ] 

Tom van der Woerdt commented on CASSANDRA-13348:


My replication factor was dc1:2 dc2:2, with ~30 nodes per dc. As for the logs, 
I don't have those anymore, got rotated away by now, though there wasn't 
anything interesting there other than the excerpts I pasted.

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,630 
> TokenAllocation.java:61 - Selected tokens [.., 2890709530010722764, 
> -2416006722819773829, -5820248611267569511, -5990139574852472056, 
> 1645259890258332163, 9135021011763659240, -5451286144622276797, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,794 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.02 min 0.98 stddev 0.
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,795 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:53,149 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 

[jira] [Commented] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944054#comment-15944054
 ] 

Dikang Gu commented on CASSANDRA-13348:
---

[~tvdw], what's your replication factor? Can you send me the full log of the 
node 1 and node 2?

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,630 
> TokenAllocation.java:61 - Selected tokens [.., 2890709530010722764, 
> -2416006722819773829, -5820248611267569511, -5990139574852472056, 
> 1645259890258332163, 9135021011763659240, -5451286144622276797, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,794 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.02 min 0.98 stddev 0.
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,795 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:53,149 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:56:23,149 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> eg. 7604192574911909354 has been chosen by 

[jira] [Commented] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944062#comment-15944062
 ] 

Dikang Gu commented on CASSANDRA-13348:
---

[~tvdw], okay, I will try to run a simulator. I assume you are using Murmur3 
partitioner? 

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,514 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,630 
> TokenAllocation.java:61 - Selected tokens [.., 2890709530010722764, 
> -2416006722819773829, -5820248611267569511, -5990139574852472056, 
> 1645259890258332163, 9135021011763659240, -5451286144622276797, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,794 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.02 min 0.98 stddev 0.
> WARN  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,795 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:53,149 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:56:23,149 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> eg. 7604192574911909354 has been chosen by both.

[jira] [Comment Edited] (CASSANDRA-13348) Duplicate tokens after bootstrap

[Tom van der Woerdt (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944078#comment-15944078
 ] 

Tom van der Woerdt edited comment on CASSANDRA-13348 at 3/27/17 9:32 PM:
-

Murmur3 partitioner, yes. GossipingPropertyFileSnitch, if it matters.

I don't recall exactly how this cluster was built, but it was something like 
this :
 * Provision 5 nodes per DC, all but one with "-Dcassandra.join_ring=false". 
Keyspace with rf= dc1:2 dc2:2
 * "nodetool join" one at a time (random order)
 * Provision 30 nodes in dc1 -- all have "allocate_tokens_for_keyspace" set
 * "nodetool join" ~10
 * Decommission the first five, so we're now left with dc1:10
 * "nodetool join" the rest
 * Ditto for dc2, so we now have dc1:30 dc2:30

There's a lot of automation involved, a human may take a different route to 
doing this. I decommissioned the 10 initial nodes which had non-ideal hardware, 
and they made place for 60 more powerful machines.

The "nodetool join" batches to join the final 20 in the DC caused the bad 
tokens.


was (Author: tvdw):
Murmur3 simulator, yes. GossipingPropertyFileSnitch, if it matters.

I don't recall exactly how this cluster was built, but it was something like 
this :
 * Provision 5 nodes per DC, all but one with "-Dcassandra.join_ring=false". 
Keyspace with rf= dc1:2 dc2:2
 * "nodetool join" one at a time (random order)
 * Provision 30 nodes in dc1 -- all have "allocate_tokens_for_keyspace" set
 * "nodetool join" ~10
 * Decommission the first five, so we're now left with dc1:10
 * "nodetool join" the rest
 * Ditto for dc2, so we now have dc1:30 dc2:30

There's a lot of automation involved, a human may take a different route to 
doing this. I decommissioned the 10 initial nodes which had non-ideal hardware, 
and they made place for 60 more powerful machines.

The "nodetool join" batches to join the final 20 in the DC caused the bad 
tokens.

> Duplicate tokens after bootstrap
> 
>
> Key: CASSANDRA-13348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13348
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tom van der Woerdt
>Priority: Blocker
> Fix For: 3.0.x
>
>
> This one is a bit scary, and probably results in data loss. After a bootstrap 
> of a few new nodes into an existing cluster, two new nodes have chosen some 
> overlapping tokens.
> In fact, of the 256 tokens chosen, 51 tokens were already in use on the other 
> node.
> Node 1 log :
> {noformat}
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: waiting for schema information to complete
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,461 
> StorageService.java:1160 - JOINING: schema complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: waiting for pending range calculation
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: calculation complete, ready to bootstrap
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,462 
> StorageService.java:1160 - JOINING: getting bootstrap token
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,564 
> TokenAllocation.java:61 - Selected tokens [, 2959334889475814712, 
> 3727103702384420083, 7183119311535804926, 6013900799616279548, 
> -1222135324851761575, 1645259890258332163, -1213352346686661387, 
> 7604192574911909354]
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:65 - Replicated node load in datacentre before 
> allocation max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:66 - Replicated node load in datacentre after allocation 
> max 1.00 min 1.00 stddev 0.
> WARN  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:43,729 
> TokenAllocation.java:70 - Unexpected growth in standard deviation after 
> allocation.
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:42:44,150 
> StorageService.java:1160 - JOINING: sleeping 3 ms for pending range setup
> INFO  [RMI TCP Connection(107)-127.0.0.1] 2017-03-09 07:43:14,151 
> StorageService.java:1160 - JOINING: Starting to bootstrap...
> {noformat}
> Node 2 log:
> {noformat}
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:51,937 
> StorageService.java:971 - Joining ring by operator request
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 15:55:52,513 
> StorageService.java:1160 - JOINING: waiting for ring information
> INFO  [RMI TCP Connection(380)-127.0.0.1] 2017-03-17 

[jira] [Commented] (CASSANDRA-13333) Cassandra does not start on Windows due to 'JNA link failure'

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944106#comment-15944106
 ] 

Michael Kjellman commented on CASSANDRA-1:
--

+1 [~blerer] LGTM!

> Cassandra does not start on Windows due to 'JNA link failure'
> -
>
> Key: CASSANDRA-1
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Blocker
>
> Cassandra 3.0 HEAD does not start on Windows. The only error in the logs is: 
> {{ERROR 16:30:10 JNA failing to initialize properly.}} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13333) Cassandra does not start on Windows due to 'JNA link failure'

[Michael Kjellman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Kjellman updated CASSANDRA-1:
-
Status: Ready to Commit  (was: Patch Available)

> Cassandra does not start on Windows due to 'JNA link failure'
> -
>
> Key: CASSANDRA-1
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Blocker
>
> Cassandra 3.0 HEAD does not start on Windows. The only error in the logs is: 
> {{ERROR 16:30:10 JNA failing to initialize properly.}} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-10520) Compressed writer and reader should support non-compressed data.

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943380#comment-15943380
 ] 

Andrés de la Peña commented on CASSANDRA-10520:
---

Any news on this? This is blocking test failures such as 
[CASSANDRA-13250|https://issues.apache.org/jira/browse/CASSANDRA-13250]

> Compressed writer and reader should support non-compressed data.
> 
>
> Key: CASSANDRA-10520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10520
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>  Labels: messaging-service-bump-required
> Fix For: 4.x
>
> Attachments: ReadWriteTestCompression.java
>
>
> Compressing uncompressible data, as done, for instance, to write SSTables 
> during stress-tests, results in chunks larger than 64k which are a problem 
> for the buffer pooling mechanisms employed by the 
> {{CompressedRandomAccessReader}}. This results in non-negligible performance 
> issues due to excessive memory allocation.
> To solve this problem and avoid decompression delays in the cases where it 
> does not provide benefits, I think we should allow compressed files to store 
> uncompressed chunks as alternative to compressed data. Such a chunk could be 
> written after compression returns a buffer larger than, for example, 90% of 
> the input, and would not result in additional delays in writing. On reads it 
> could be recognized by size (using a single global threshold constant in the 
> compression metadata) and data could be directly transferred into the 
> decompressed buffer, skipping the decompression step and ensuring a 64k 
> buffer for compressed data always suffices.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13369) If there are multiple values for a key, CQL grammar choses last value. This should not be silent or should not be allowed.

[Nachiket Patil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nachiket Patil updated CASSANDRA-13369:
---
Attachment: (was: 3.X.diff)

> If there are multiple values for a key, CQL grammar choses last value. This 
> should not be silent or should not be allowed.
> --
>
> Key: CASSANDRA-13369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13369
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Nachiket Patil
>Assignee: Nachiket Patil
>Priority: Minor
> Attachments: 3.X.diff, trunk.diff
>
>
> If through CQL, multiple values are specified for a key, grammar parses the 
> map and last value for the key wins. This behavior is bad.
> e.g. 
> {code}
> CREATE KEYSPACE Excalibur WITH REPLICATION = {'class': 
> 'NetworkTopologyStrategy', 'dc1': 2, 'dc1': 5};
> {code}
> Parsing this statement, 'dc1' gets RF = 5. This can be catastrophic, may even 
> result in loss of data. This behavior should not be silent or not be allowed 
> at all.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13369) If there are multiple values for a key, CQL grammar choses last value. This should not be silent or should not be allowed.

[Nachiket Patil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nachiket Patil updated CASSANDRA-13369:
---
Attachment: (was: trunk.diff)

> If there are multiple values for a key, CQL grammar choses last value. This 
> should not be silent or should not be allowed.
> --
>
> Key: CASSANDRA-13369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13369
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Nachiket Patil
>Assignee: Nachiket Patil
>Priority: Minor
> Attachments: 3.X.diff, trunk.diff
>
>
> If through CQL, multiple values are specified for a key, grammar parses the 
> map and last value for the key wins. This behavior is bad.
> e.g. 
> {code}
> CREATE KEYSPACE Excalibur WITH REPLICATION = {'class': 
> 'NetworkTopologyStrategy', 'dc1': 2, 'dc1': 5};
> {code}
> Parsing this statement, 'dc1' gets RF = 5. This can be catastrophic, may even 
> result in loss of data. This behavior should not be silent or not be allowed 
> at all.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13369) If there are multiple values for a key, CQL grammar choses last value. This should not be silent or should not be allowed.

[Nachiket Patil (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nachiket Patil updated CASSANDRA-13369:
---
Attachment: trunk.diff
3.X.diff

> If there are multiple values for a key, CQL grammar choses last value. This 
> should not be silent or should not be allowed.
> --
>
> Key: CASSANDRA-13369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13369
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Nachiket Patil
>Assignee: Nachiket Patil
>Priority: Minor
> Attachments: 3.X.diff, trunk.diff
>
>
> If through CQL, multiple values are specified for a key, grammar parses the 
> map and last value for the key wins. This behavior is bad.
> e.g. 
> {code}
> CREATE KEYSPACE Excalibur WITH REPLICATION = {'class': 
> 'NetworkTopologyStrategy', 'dc1': 2, 'dc1': 5};
> {code}
> Parsing this statement, 'dc1' gets RF = 5. This can be catastrophic, may even 
> result in loss of data. This behavior should not be silent or not be allowed 
> at all.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13327) Pending endpoints size check for CAS doesn't play nicely with writes-on-replacement

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944229#comment-15944229
 ] 

Ariel Weisberg commented on CASSANDRA-13327:


It seems to me that for host replacement there is simply no pending state at 
all. My reading says that the host replacement can only occur when the host 
being replacement is not alive although I imagine at some nodes it might be 
considered alive.

So why can't the replacing node transition directly to a state where it is no 
longer joining, but has replaced it's target node, and in that situation it is 
simply a node that is  badly behind and in need of repair. Bootstrap can then 
run on the replacing node to provide missing data while it serves both writes 
and reads?

Is that an issue for CL.ONE which already has weak guarantees?

> Pending endpoints size check for CAS doesn't play nicely with 
> writes-on-replacement
> ---
>
> Key: CASSANDRA-13327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13327
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> Consider this ring:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.4   MR DOWN NORMAL -7148113328562451251
> where 127.0.0.1 was bootstrapping for cluster expansion. Note that, due to 
> the failure of 127.0.0.4, 127.0.0.1 was stuck trying to stream from it and 
> making no progress.
> Then the down node was replaced so we had:
> 127.0.0.1  MR UP JOINING -7301836195843364181
> 127.0.0.2MR UP NORMAL -7263405479023135948
> 127.0.0.3MR UP NORMAL -7205759403792793599
> 127.0.0.5   MR UP JOINING -7148113328562451251
> It’s confusing in the ring - the first JOINING is a genuine bootstrap, the 
> second is a replacement. We now had CAS unavailables (but no non-CAS 
> unvailables). I think it’s because the pending endpoints check thinks that 
> 127.0.0.5 is gaining a range when it’s just replacing.
> The workaround is to kill the stuck JOINING node, but Cassandra shouldn’t 
> unnecessarily fail these requests.
> It also appears like required participants is bumped by 1 during a host 
> replacement so if the replacing host fails you will get unavailables and 
> timeouts.
> This is related to the check added in CASSANDRA-8346



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13369) If there are multiple values for a key, CQL grammar choses last value. This should not be silent or should not be allowed.

[Nachiket Patil (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944179#comment-15944179
 ] 

Nachiket Patil commented on CASSANDRA-13369:


[~jjirsa] Thanks. Fixed.

> If there are multiple values for a key, CQL grammar choses last value. This 
> should not be silent or should not be allowed.
> --
>
> Key: CASSANDRA-13369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13369
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Nachiket Patil
>Assignee: Nachiket Patil
>Priority: Minor
> Attachments: 3.X.diff, trunk.diff
>
>
> If through CQL, multiple values are specified for a key, grammar parses the 
> map and last value for the key wins. This behavior is bad.
> e.g. 
> {code}
> CREATE KEYSPACE Excalibur WITH REPLICATION = {'class': 
> 'NetworkTopologyStrategy', 'dc1': 2, 'dc1': 5};
> {code}
> Parsing this statement, 'dc1' gets RF = 5. This can be catastrophic, may even 
> result in loss of data. This behavior should not be silent or not be allowed 
> at all.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944223#comment-15944223
 ] 

Jason Brown commented on CASSANDRA-8457:


[~norman] if you want to give it another shot, i'm always open for more input 

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13250) dtest failure in cqlsh_tests.cqlsh_tests.TestCqlsh.test_describe

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943263#comment-15943263
 ] 

Andrés de la Peña commented on CASSANDRA-13250:
---

This seems to be solved by [this 
patch|https://github.com/blambov/cassandra-dtest/commit/c212a30c1affaa3a5a701b7f72831194eeefab40]
 by [~blambov] at 
[CASSANDRA-10520|https://issues.apache.org/jira/browse/CASSANDRA-10520].

> dtest failure in cqlsh_tests.cqlsh_tests.TestCqlsh.test_describe
> 
>
> Key: CASSANDRA-13250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13250
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Andrés de la Peña
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/430/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe
> {code}
> Error Message
> Lists differ: ["CREATE KEYSPACE test WITH re... != ["CREATE KEYSPACE test 
> WITH re...
> First differing element 11:
> "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}"
> "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor', 'min_compress_ratio': 
> '1.1'}"
>   ["CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '1'}  AND durable_writes = true;",
>'CREATE TABLE test.test (',
>'id int,',
>'col int,',
>'val text,',
>'PRIMARY KEY (id, col)',
>') WITH CLUSTERING ORDER BY (col ASC)',
>'AND bloom_filter_fp_chance = 0.01',
>"AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}",
>"AND comment = ''",
>"AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}",
> -  "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}",
> +  "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor', 'min_compress_ratio': 
> '1.1'}",
> ? 
>   +
>'AND crc_check_chance = 1.0',
>'AND dclocal_read_repair_chance = 0.1',
>'AND default_time_to_live = 0',
>'AND gc_grace_seconds = 864000',
>'AND max_index_interval = 2048',
>'AND memtable_flush_period_in_ms = 0',
>'AND min_index_interval = 128',
>'AND read_repair_chance = 0.0',
>"AND speculative_retry = '99PERCENTILE';",
>'CREATE INDEX test_val_idx ON test.test (val);',
>'CREATE INDEX test_col_idx ON test.test (col);',
>'CREATE TABLE test.users (',
>'userid text PRIMARY KEY,',
>'age int,',
>'firstname text,',
>'lastname text',
>') WITH bloom_filter_fp_chance = 0.01',
>"AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}",
>"AND comment = ''",
>"AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}",
> -  "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}",
> +  "AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor', 'min_compress_ratio': 
> '1.1'}",
> ? 
>   +
>'AND crc_check_chance = 1.0',
>'AND dclocal_read_repair_chance = 0.1',
>'AND default_time_to_live = 0',
>'AND gc_grace_seconds = 864000',
>'AND max_index_interval = 2048',
>'AND memtable_flush_period_in_ms = 0',
>'AND min_index_interval = 128',
>'AND read_repair_chance = 0.0',
>"AND speculative_retry = '99PERCENTILE';",
>'CREATE INDEX myindex ON test.users (age);']
> {code}{code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 
> 702, in test_describe
> self.execute(cql="DESCRIBE KEYSPACE test", 
> expected_output=self.get_keyspace_output())
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 
> 1042, in execute
> self.check_response(output, expected_output)
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 
> 1049, in check_response
> self.assertEqual(expected_lines, lines)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File 

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

[Norman Maurer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943172#comment-15943172
 ] 

Norman Maurer commented on CASSANDRA-8457:
--

[~jasobrown] let me know if I should review the code again.

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13382) cdc column addition strikes again

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943217#comment-15943217
 ] 

Sylvain Lebresne commented on CASSANDRA-13382:
--

Alright, dtest to reproduce is 
[here|https://github.com/riptano/cassandra-dtest/pull/1454].

> cdc column addition strikes again
> -
>
> Key: CASSANDRA-13382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13382
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.11.x
>
>
> This is a followup of CASSANDRA-12697, where the patch mistakenly only 
> handled the {{system_schema.tables}} table, while the {{cdc}} column has been 
> added to {{system_schema.views}} table.
> The patch is pretty trivial, though this highlight that we don't seem to have 
> upgrade tests for materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13274) Fix rolling upgrade to 4.0+

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13274:

Reviewer: Sylvain Lebresne

> Fix rolling upgrade to 4.0+
> ---
>
> Key: CASSANDRA-13274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13274
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 3.0.x, 3.11.x
>
>
> A rolling upgrade from 3.* to 4.0 (messaging version {{11}}) unveils a 
> regression caused by CASSANDRA-11128.
> Generally, we store all possible options/attributes including the default 
> values in the schema. This causes (expected) schema-version-mismatches during 
> rolling upgrades and therefore we prevent schema pulls/pushes in this 
> situation, which has been broken by CASSANDRA-11128.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13298) DataOutputBuffer.asNewBuffer broken

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13298:

Reviewer: Stefania

> DataOutputBuffer.asNewBuffer broken
> ---
>
> Key: CASSANDRA-13298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13298
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.11.x
>
>
> The implementation of {{DataOutputBuffer.asNewBuffer()}} reuses the 
> underlying {{ByteBuffer}} array. This is probably not an issue, but it is 
> definitely incorrect and may lead to incorrect/overwritten information 
> returned by {{SystemKeyspace.truncationAsMapEntry}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13383) Confusing error: unconfigured table table2

[Sachin Goyal (JIRA)]

Sachin Goyal created CASSANDRA-13383:


 Summary: Confusing error: unconfigured table table2
 Key: CASSANDRA-13383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13383
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sachin Goyal


*Select * from an_absent_table;* - gives an error:

com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table 
an_absent_table
at 
com.datastax.driver.core.Responses$Error.asException(Responses.java:136) 
at 
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
 
at 
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:177) 
at 
com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:46) 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:799)
 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:633)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1070)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:993)
 
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 


This error should be more descriptive.
At the very least it should mention the keyspace where Cassandra was trying to 
find the table.
It would help debug client code a lot faster if the keyspace was mentioned in 
the above message.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13383) Confusing error: unconfigured table table2

[Sachin Goyal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sachin Goyal updated CASSANDRA-13383:
-
Description: 
Consider the query:

*Select * from an_absent_table;*

It gives an error:

{panel}
com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table 
an_absent_table
at 
com.datastax.driver.core.Responses$Error.asException(Responses.java:136) 
at 
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
 
at 
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:177) 
at 
com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:46) 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:799)
 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:633)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1070)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:993)
 
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 
{panel}

This error should be more descriptive.
At the very least it should mention the keyspace where Cassandra was trying to 
find the table.
It would help debug client code a lot faster if the keyspace was mentioned in 
the above message.

  was:
*Select * from an_absent_table;* - gives an error:

com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table 
an_absent_table
at 
com.datastax.driver.core.Responses$Error.asException(Responses.java:136) 
at 
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
 
at 
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:177) 
at 
com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:46) 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:799)
 
at 
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:633)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1070)
 
at 
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:993)
 
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 


This error should be more descriptive.
At the very least it should mention the keyspace where Cassandra was trying to 
find the table.
It would help debug client code a lot faster if the keyspace was mentioned in 
the above message.


> Confusing error: unconfigured table table2
> --
>
> Key: CASSANDRA-13383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13383
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sachin Goyal
>
> Consider the query:
> *Select * from an_absent_table;*
> It gives an error:
> {panel}
> com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table 
> an_absent_table
>   at 
> com.datastax.driver.core.Responses$Error.asException(Responses.java:136) 
>   at 
> com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
>  
>   at 
> com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:177)
>  
>   at 
> com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:46) 
>   at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:799)
>  
>   at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:633)
>  
>   at 
> com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1070)
>  
>   at 
> com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:993)
>  
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  
> {panel}
> This error should be more descriptive.
> At the very least it should mention the keyspace where Cassandra was trying 
> to find the table.
> It would help debug client code a lot faster if the keyspace was mentioned in 
> the above message.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13384) Legacy caching options can prevent 3.0 upgrade

[Jeff Jirsa (JIRA)]

Jeff Jirsa created CASSANDRA-13384:
--

 Summary: Legacy caching options can prevent 3.0 upgrade
 Key: CASSANDRA-13384
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13384
 Project: Cassandra
  Issue Type: Bug
  Components: Schema
Reporter: Jeff Jirsa
Assignee: Jeff Jirsa
Priority: Minor
 Fix For: 3.0.x, 3.11.x


In 2.1, we wrote caching options as a JSONified map, but we tolerated raw 
strings ["ALL", "KEYS_ONLY", "ROWS_ONLY", and 
"NONE"|https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/cache/CachingOptions.java#L42].

If a 2.1 node with any of these strings is upgraded to 3.0, the legacy schema 
migration will fail.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13274) Fix rolling upgrade to 4.0+

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943300#comment-15943300
 ] 

Sylvain Lebresne commented on CASSANDRA-13274:
--

+1, but make sure to update NEWS file for trunk with the actual version this is 
going to make it in. Also, completely a nit, but for the changelog I'd probably 
rephrase to something like "Fix code to not exchange schema across major 
versions" as that's more precise and avoid referencing a version that doesn't 
exists yet.

> Fix rolling upgrade to 4.0+
> ---
>
> Key: CASSANDRA-13274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13274
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Critical
> Fix For: 3.0.x, 3.11.x
>
>
> A rolling upgrade from 3.* to 4.0 (messaging version {{11}}) unveils a 
> regression caused by CASSANDRA-11128.
> Generally, we store all possible options/attributes including the default 
> values in the schema. This causes (expected) schema-version-mismatches during 
> rolling upgrades and therefore we prevent schema pulls/pushes in this 
> situation, which has been broken by CASSANDRA-11128.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)