[jira] [Updated] (CASSANDRA-10296) Aggregates aren't resolved properly for reversed types

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-10296:
---
Fix Version/s: (was: 2.2.x)
   2.2.2

> Aggregates aren't resolved properly for reversed types
> --
>
> Key: CASSANDRA-10296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10296
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Fix For: 2.2.2
>
>
> When using an aggregate, it won't resolve if the column is reversed.
> {code}
> CREATE TABLE t1 (pk int, ck int, PRIMARY KEY (pk, ck)) WITH CLUSTERING ORDER 
> BY (ck DESC)
> SELECT min(ck) FROM t1 WHERE pk = 1
> {code}
> results in the error:
> {noformat}
> Ambiguous call to function min (can be matched by following signatures: 
> system.min : (varint) -> varint, system.min : (int) -> int, system.min : 
> (blob) -> blob): use type casts to disambiguate
> {noformat}
> For aggregates, we should be treating a reversed type the same as we do a 
> normal type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9061) Add backoff and recovery to cqlsh COPY FROM when write timeouts occur

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-9061:
--
Fix Version/s: (was: 2.1.x)

> Add backoff and recovery to cqlsh COPY FROM when write timeouts occur
> -
>
> Key: CASSANDRA-9061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9061
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Carl Yeksigian
>Priority: Minor
>  Labels: cqlsh
> Attachments: 9061-2.1.txt, 9061-suggested.txt
>
>
> Previous versions of COPY FROM didn't handle write timeouts because it was 
> rarely fast enough for that to matter.  Now that performance has improved, 
> write timeouts are more likely to occur.  We should handle these by backing 
> off and retrying the operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10108) Windows dtest 3.0: sstablesplit_test.py:TestSSTableSplit.split_test fails

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10108:

Fix Version/s: (was: 3.0.x)

> Windows dtest 3.0: sstablesplit_test.py:TestSSTableSplit.split_test fails
> -
>
> Key: CASSANDRA-10108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10108
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Paulo Motta
>  Labels: Windows
>
> Locally:
> {noformat}
> -- ma-28-big-Data.db-
> Exception in thread "main" java.lang.NoClassDefFoundError: 
> org/supercsv/prefs/CsvPreference$Builder
> at org.apache.cassandra.config.Config.(Config.java:240)
> at 
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:105)
> at 
> org.apache.cassandra.service.StorageService.getPartitioner(StorageService.java:220)
> at 
> org.apache.cassandra.service.StorageService.(StorageService.java:206)
> at 
> org.apache.cassandra.service.StorageService.(StorageService.java:211)
> at 
> org.apache.cassandra.schema.LegacySchemaTables.getSchemaPartitionsForTable(LegacySchemaTables.java:295)
> at 
> org.apache.cassandra.schema.LegacySchemaTables.readSchemaFromSystemTables(LegacySchemaTables.java:210)
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:108)
> at 
> org.apache.cassandra.tools.StandaloneSplitter.main(StandaloneSplitter.java:58)
> Caused by: java.lang.ClassNotFoundException: 
> org.supercsv.prefs.CsvPreference$Builder
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 9 more
> Number of sstables after split: 1. expected 21.0
> {noformat}
> on CI:
> {noformat}
> 21.0 not less than or equal to 2
> and
> [node1 ERROR] Exception calling "CompareTo" with "1" argument(s): "Object 
> must be of type 
> String."
> At D:\temp\dtest-i3xwjx\test\node1\conf\cassandra-env.ps1:336 char:9
> + if ($env:JVM_VERSION.CompareTo("1.8.0_40" -eq -1))
> + ~
> + CategoryInfo  : NotSpecified: (:) [], MethodInvocationException
> + FullyQualifiedErrorId : ArgumentException
> -- ma-28-big-Data.db-
> {noformat}
> Failure history: 
> [consistent|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest_win32/lastCompletedBuild/testReport/sstablesplit_test/TestSSTableSplit/split_test/history/]
> Env: both CI and local



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002948#comment-15002948
 ] 

Joel Knighton commented on CASSANDRA-10089:
---

Sorry for missing this in review - it looks like the applied patch could result 
in a NPE (found by Coverity).

On this line in 
[getStateForVersionBiggerThan|https://github.com/apache/cassandra/blob/7d6dbf897cd14e6c5811a0588f25e5c11385a9fd/src/java/org/apache/cassandra/gms/Gossiper.java],
 we try to put a list of new states in reqdEndpointState. In the event that 
we're getting state for an endpoint that has a lower local heartbeat version 
and no states for the endpoint/lower application states values for all values 
in the EndpointState's ApplicationState map, reqdEndpointState will be null.

That said, in practice, {{getStateForVersionBiggerThan}} is only called in 
{{GossipDigestAckVerbHandler}} for endpoints corresponding to a 
{{GossipDigest}} in the {{GossipDigestAck}} message, and digests are only 
included in the {{GossipDigestAck}} message if either the remote heartbeat 
version or some application state has a higher value than the provided version. 
Because of this, under no situation should reqdEndpointState be null at the 
time of adding application states.

I'm still in favor of guarding this with a null check, but a patch is not of 
critical importance. What do you think [~Stefania]?

> NullPointerException in Gossip handleStateNormal
> 
>
> Key: CASSANDRA-10089
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10089
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1
>
> Attachments: node1_debug.log, node2_debug.log, node3_debug.log
>
>
> Whilst comparing dtests for CASSANDRA-9970 I found [this failing 
> dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/]
>  in 2.2:
> {code}
> Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 
> 15:39:57,873 CassandraDaemon.java:183 - Exception in thread 
> Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat 
> org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na] \tat 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[main/:na] \tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_80] \tat 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]']
> {code}
> I wasn't able to find it on unpatched branches  but it is clearly not related 
> to CASSANDRA-9970, if anything it could have been a side effect of 
> CASSANDRA-9871.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10511) Index summary downsampling prevents mmap access of large files after restart

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-10511:
---
Fix Version/s: (was: 3.1)
   (was: 2.2.x)
   (was: 2.1.x)

> Index summary downsampling prevents mmap access of large files after restart
> 
>
> Key: CASSANDRA-10511
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10511
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
>Assignee: Ariel Weisberg
>
> {{SSTableReader.cloneWithNewSummarySampleLevel}} constructs a 
> {{SegmentedFile.Builder}} but never populates it with any boundaries. For 
> files larger than 2Gb, this will result in their being accessed via buffered 
> io after a restart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9741) cfhistograms dtest flaps on trunk and 2.2

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9741:
--
Fix Version/s: (was: 3.0.x)

> cfhistograms dtest flaps on trunk and 2.2
> -
>
> Key: CASSANDRA-9741
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9741
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Ariel Weisberg
>
> {{jmx_test.py:TestJMX.cfhistograms_test}} flaps on CassCI under trunk and 2.2.
> On 2.2, it fails one of its assertions when {{'Unable to compute when 
> histogram overflowed'}} is found in the output of {{nodetool cfhistograms}}. 
> Here's the failure history for 2.2:
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> On trunk, it fails when an error about a {{WriteFailureException}} during 
> hinted handoff is found in the C* logs after the tests run ([example cassci 
> output|http://cassci.datastax.com/view/trunk/job/trunk_dtest/315/testReport/junit/jmx_test/TestJMX/cfhistograms_test/]).
>  Here's the failure history for trunk:
> http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> I haven't seen it fail locally yet, but haven't run the test more than a 
> couple times because it takes a while.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10492) Fix failing upgrade tests on 2.1->2.2 path

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-10492:
---
Fix Version/s: (was: 2.2.x)
   (was: 2.1.x)

> Fix failing upgrade tests on 2.1->2.2 path
> --
>
> Key: CASSANDRA-10492
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10492
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Ariel Weisberg
>  Labels: triage
> Attachments: test_stdout.txt
>
>
> 2 dtests for the storage engine upgrade on the 2.1->2.2 path fail:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/lastCompletedBuild/testReport/
> (If there are 3 failures at that link, don't worry; one was a dtest bug 
> that's been fixed.)
> {{upgrade_tests/cql_tests.py:TestCQL.large_count_test}} flaps when a TRUNCATE 
> operation times out:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/20/testReport/upgrade_tests.cql_tests/TestCQL/large_count_test/history/
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions}}
>  also fails on this upgrade path:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-21_HEAD-22_tarball/15/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> These are in the same ticket for the moment, but if one becomes a longer 
> project, it should be broken out into a separate ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10171) Windows dtest 3.0: materialized_views_test.py:TestMaterializedViews

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10171:

Fix Version/s: (was: 3.0.x)

> Windows dtest 3.0: materialized_views_test.py:TestMaterializedViews
> ---
>
> Key: CASSANDRA-10171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10171
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>
> The following 3.0 dtests have been failing 
> [consistently|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/24/testReport/junit/materialized_views_test/history/]
>  on Windows:
> * materialized_views_test.TestMaterializedViews.complex_repair_test
> * materialized_views_test.TestMaterializedViews.interrupt_build_process_test
> * materialized_views_test.TestMaterializedViews.really_complex_repair_test
> * materialized_views_test.TestMaterializedViews.simple_repair_test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10027) ALTER TABLE TYPE check broken

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002844#comment-15002844
 ] 

Benjamin Lerer commented on CASSANDRA-10027:


Committed in 2.2 at b05fe4ffee8f1c457a79013c2ddc88abcdadc6bb and merged into 
3.0, 3.1 and trunk

> ALTER TABLE TYPE check broken
> -
>
> Key: CASSANDRA-10027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10027
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aaron Ploetz
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.4, 3.0.1, 3.1
>
> Attachments: 10027-2.2.txt
>
>
> I stumbled onto the fact that 2.2.0 will allow you to ALTER TYPE of a 
> {{varint}} to the new {{date}} type.  I thought that was an odd conversion to 
> allow, so I attempted to query it.  I received an error on all subsequent 
> queries, unless I exited or truncated the table.
> After truncating, I could then INSERT and query as normal.  But the new 
> {{varint}} values inserted simply were reflected as an offset of the minimum 
> {{varint}} value.
> I'm not sure why that's happening, but if we could simply prevent type 
> conversion between {{varint}} and {{date}} (and just show the "types are 
> incompatible" message) that should fix this.
> Steps to reproduce:
> {code}
> aploetz@cqlsh:typeconversion> CREATE TABLE varinttest (key int PRIMARY KEY, 
> c1 varint);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
>1 |  1
> (1 rows)
> aploetz@cqlsh:typeconversion> ALTER TABLE varinttest ALTER c1 TYPE date;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1150, in perform_simple_statement
> rows = future.result(self.session.default_timeout)
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 3296, in result
> raise self._final_exception
> error: unpack requires a string argument of length 4
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> NoHostAvailable: ('Unable to complete the operation against any hosts', 
> {: ConnectionShutdown('Connection to 127.0.0.1 is 
> defunct',)})
> aploetz@cqlsh:typeconversion> TRUNCATE varinttest ;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
> (0 rows)
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (2,2);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (3,3);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+-
>1 | -2147483647
>2 | -2147483646
>3 | -2147483645
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003017#comment-15003017
 ] 

Ariel Weisberg commented on CASSANDRA-7217:
---

Performance counters

2000 threads
{code}
Results:
op rate   : 19419 [WRITE:19419]
partition rate: 19419 [WRITE:19419]
row rate  : 19419 [WRITE:19419]
latency mean  : 103.0 [WRITE:103.0]
latency median: 91.3 [WRITE:91.3]
latency 95th percentile   : 179.4 [WRITE:179.4]
latency 99th percentile   : 252.3 [WRITE:252.3]
latency 99.9th percentile : 428.5 [WRITE:428.5]
latency max   : 57651.8 [WRITE:57651.8]
Total partitions  : 1900 [WRITE:1900]
Total errors  : 0 [WRITE:0]
total gc count: 0
total gc mb   : 0
total gc time (s) : 0
avg gc time(ms)   : NaN
stdev gc time(ms) : 0
Total operation time  : 00:16:18
END

 Performance counter stats for './cassandra-stress write n=1900 -rate 
threads=2000 -mode native cql3 -node 192.168.1.9':

 3,320,451,421,007  cycles#2.192 GHz
 [15.41%]
 2,563,758,232,484  instructions  #0.77  insns per cycle

  #0.94  stalled cycles per 
insn [20.47%]
69,188,067,241  cache-references  #   45.664 M/sec  
 [25.56%]
13,456,198,724  cache-misses  #   19.449 % of all cache 
refs [30.60%]
   131,776,347,830  bus-cycles#   86.973 M/sec  
 [35.65%]
 2,415,412,133,089  idle-cycles-frontend  #   72.74% frontend cycles 
idle[40.69%]
 1,750,197,198,741  idle-cycles-backend   #   52.71% backend  cycles 
idle[45.75%]
1514363.238593  cpu-clock (msec)

1515146.390785  task-clock (msec) #1.530 CPUs utilized  

   154,815  page-faults   #0.102 K/sec  

87,357,050  cs#0.058 M/sec  

37,030,093  migrations#0.024 M/sec  

   154,691  minor-faults  #0.102 K/sec  

 0  major-faults  #0.000 K/sec  

 0  alignment-faults  #0.000 K/sec  

 0  emulation-faults  #0.000 K/sec  

   358,579,878,595  branch-instructions   #  236.664 M/sec  
 [45.74%]
 5,088,330,722  branch-misses #1.42% of all branches
 [45.80%]
70,350,080,393  L1-dcache-load-misses #   46.431 M/sec  
 [45.92%]
24,626,765,787  L1-dcache-store-misses#   16.254 M/sec  
 [40.88%]
19,812,757,638  L1-dcache-prefetch-misses #   13.076 M/sec  
 [40.97%]
59,285,911,291  L1-icache-load-misses #   39.129 M/sec  
 [40.92%]
 4,437,071,985  dTLB-load-misses  #2.928 M/sec  
 [40.90%]
   821,151,709  dTLB-store-misses #0.542 M/sec  
 [40.80%]
 1,188,402,914  iTLB-load-misses  #0.784 M/sec  
 [40.66%]
 5,274,857,779  branch-load-misses#3.481 M/sec  
 [40.58%]
39,293,189,238  LLC-loads #   25.934 M/sec  
 [40.47%]
10,625,403,856  LLC-stores#7.013 M/sec  
 [40.45%]
16,978,686,645  LLC-prefetches#   11.206 M/sec  
 [10.08%]

 990.019887601 seconds time elapsed
{code}
500 threads
{code}
Results:
op rate   : 63678 [WRITE:63678]
partition rate: 63678 [WRITE:63678]
row rate  : 63678 [WRITE:63678]
latency mean  : 7.8 [WRITE:7.8]
latency median: 5.6 [WRITE:5.6]
latency 95th percentile   : 16.8 [WRITE:16.8]
latency 99th percentile   : 36.5 [WRITE:36.5]
latency 99.9th percentile : 77.5 [WRITE:77.5]
latency max   : 358.8 [WRITE:358.8]
Total partitions  : 1900 [WRITE:1900]
Total errors  : 0 [WRITE:0]
total gc count: 0
total gc mb   : 0
total gc time (s) : 0
avg gc time(ms)   : NaN
stdev gc time(ms) : 0
Total operation time  : 00:04:58
END

 Performance counter stats for './cassandra-stress write n=1900 -rate 
threads=500 -mode native cql3 -node 192.168.1.9':

 2,055,138,822,781  cycles#2.519 GHz
 [15.25%]
 1,923,953,212,761  instructions  #0.94  insns per cycle

[jira] [Updated] (CASSANDRA-10027) ALTER TABLE TYPE check broken

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10027:
---
Attachment: 10027-2.2.txt

> ALTER TABLE TYPE check broken
> -
>
> Key: CASSANDRA-10027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10027
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aaron Ploetz
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 10027-2.2.txt
>
>
> I stumbled onto the fact that 2.2.0 will allow you to ALTER TYPE of a 
> {{varint}} to the new {{date}} type.  I thought that was an odd conversion to 
> allow, so I attempted to query it.  I received an error on all subsequent 
> queries, unless I exited or truncated the table.
> After truncating, I could then INSERT and query as normal.  But the new 
> {{varint}} values inserted simply were reflected as an offset of the minimum 
> {{varint}} value.
> I'm not sure why that's happening, but if we could simply prevent type 
> conversion between {{varint}} and {{date}} (and just show the "types are 
> incompatible" message) that should fix this.
> Steps to reproduce:
> {code}
> aploetz@cqlsh:typeconversion> CREATE TABLE varinttest (key int PRIMARY KEY, 
> c1 varint);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
>1 |  1
> (1 rows)
> aploetz@cqlsh:typeconversion> ALTER TABLE varinttest ALTER c1 TYPE date;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1150, in perform_simple_statement
> rows = future.result(self.session.default_timeout)
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 3296, in result
> raise self._final_exception
> error: unpack requires a string argument of length 4
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> NoHostAvailable: ('Unable to complete the operation against any hosts', 
> {: ConnectionShutdown('Connection to 127.0.0.1 is 
> defunct',)})
> aploetz@cqlsh:typeconversion> TRUNCATE varinttest ;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
> (0 rows)
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (2,2);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (3,3);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+-
>1 | -2147483647
>2 | -2147483646
>3 | -2147483645
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10027) ALTER TABLE TYPE check broken

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-10027:
---
Attachment: (was: cassandra-2.2-10271.txt)

> ALTER TABLE TYPE check broken
> -
>
> Key: CASSANDRA-10027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10027
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aaron Ploetz
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> I stumbled onto the fact that 2.2.0 will allow you to ALTER TYPE of a 
> {{varint}} to the new {{date}} type.  I thought that was an odd conversion to 
> allow, so I attempted to query it.  I received an error on all subsequent 
> queries, unless I exited or truncated the table.
> After truncating, I could then INSERT and query as normal.  But the new 
> {{varint}} values inserted simply were reflected as an offset of the minimum 
> {{varint}} value.
> I'm not sure why that's happening, but if we could simply prevent type 
> conversion between {{varint}} and {{date}} (and just show the "types are 
> incompatible" message) that should fix this.
> Steps to reproduce:
> {code}
> aploetz@cqlsh:typeconversion> CREATE TABLE varinttest (key int PRIMARY KEY, 
> c1 varint);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
>1 |  1
> (1 rows)
> aploetz@cqlsh:typeconversion> ALTER TABLE varinttest ALTER c1 TYPE date;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1150, in perform_simple_statement
> rows = future.result(self.session.default_timeout)
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 3296, in result
> raise self._final_exception
> error: unpack requires a string argument of length 4
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> NoHostAvailable: ('Unable to complete the operation against any hosts', 
> {: ConnectionShutdown('Connection to 127.0.0.1 is 
> defunct',)})
> aploetz@cqlsh:typeconversion> TRUNCATE varinttest ;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
> (0 rows)
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (2,2);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (3,3);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+-
>1 | -2147483647
>2 | -2147483646
>3 | -2147483645
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10027) ALTER TABLE TYPE check broken

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002744#comment-15002744
 ] 

Benjamin Lerer commented on CASSANDRA-10027:


Sorry, I attached the good patch.
Thanks for the review.

> ALTER TABLE TYPE check broken
> -
>
> Key: CASSANDRA-10027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10027
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aaron Ploetz
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 10027-2.2.txt
>
>
> I stumbled onto the fact that 2.2.0 will allow you to ALTER TYPE of a 
> {{varint}} to the new {{date}} type.  I thought that was an odd conversion to 
> allow, so I attempted to query it.  I received an error on all subsequent 
> queries, unless I exited or truncated the table.
> After truncating, I could then INSERT and query as normal.  But the new 
> {{varint}} values inserted simply were reflected as an offset of the minimum 
> {{varint}} value.
> I'm not sure why that's happening, but if we could simply prevent type 
> conversion between {{varint}} and {{date}} (and just show the "types are 
> incompatible" message) that should fix this.
> Steps to reproduce:
> {code}
> aploetz@cqlsh:typeconversion> CREATE TABLE varinttest (key int PRIMARY KEY, 
> c1 varint);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
>1 |  1
> (1 rows)
> aploetz@cqlsh:typeconversion> ALTER TABLE varinttest ALTER c1 TYPE date;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1150, in perform_simple_statement
> rows = future.result(self.session.default_timeout)
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py",
>  line 3296, in result
> raise self._final_exception
> error: unpack requires a string argument of length 4
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
> NoHostAvailable: ('Unable to complete the operation against any hosts', 
> {: ConnectionShutdown('Connection to 127.0.0.1 is 
> defunct',)})
> aploetz@cqlsh:typeconversion> TRUNCATE varinttest ;
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+
> (0 rows)
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (1,1);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (2,2);
> aploetz@cqlsh:typeconversion> INSERT INTO varinttest (key, c1) VALUES (3,3);
> aploetz@cqlsh:typeconversion> SELECT * FROM varinttest ;
>  key | c1
> -+-
>1 | -2147483647
>2 | -2147483646
>3 | -2147483645
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/4] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0

Conflicts:
CHANGES.txt
test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/464f8f07
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/464f8f07
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/464f8f07

Branch: refs/heads/trunk
Commit: 464f8f076878dd95ae1f77f96edca8e1cd3c5699
Parents: 9f056a9 b05fe4f
Author: blerer 
Authored: Thu Nov 12 21:07:08 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:07:08 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/464f8f07/CHANGES.txt
--
diff --cc CHANGES.txt
index d554323,5705453..f00b36d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,52 -1,7 +1,53 @@@
 -2.2.4
 +3.0.1
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
+  * Fix SimpleDateType type compatibility (CASSANDRA-10027)
   * (Hadoop) fix splits calculation (CASSANDRA-10640)
   * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 +Merged from 2.1:
 + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
 + * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 +
 +
 +3.0
 + * Fix AssertionError while flushing memtable due to materialized views
 +   incorrectly inserting empty rows (CASSANDRA-10614)
 + * Store UDA initcond as CQL literal in the schema table, instead of a blob 
(CASSANDRA-10650)
 + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491)
 + * Fix distinct queries in mixed version cluster (CASSANDRA-10573)
 + * Skip sstable on clustering in names query (CASSANDRA-10571)
 + * Remove value skipping as it breaks read-repair (CASSANDRA-10655)
 + * Fix bootstrapping with MVs (CASSANDRA-10621)
 + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584)
 + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634)
 + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
 + * Fix reading of legacy sstables (CASSANDRA-10590)
 + * Use CQL type names in schema metadata tables (CASSANDRA-10365)
 + * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
 + * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)
 + * Add client address argument to IAuthenticator::newSaslNegotiator 
(CASSANDRA-8068)
 + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602)
 + * Don't use 'names query' read path for counters (CASSANDRA-10572)
 + * Fix backward compatibility for counters (CASSANDRA-10470)
 + * Remove memory_allocator paramter from cassandra.yaml 
(CASSANDRA-10581,10628)
 + * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
 + * Fix thrift cas operations with defined columns (CASSANDRA-10576)
 + * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
 + * Fix thrift get() queries with defined columns (CASSANDRA-10586)
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
   * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)

[jira] [Updated] (CASSANDRA-10652) Tracing prevents startup after upgrading

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-10652:
---
Fix Version/s: (was: 2.2.x)
   2.2.4

> Tracing prevents startup after upgrading
> 
>
> Key: CASSANDRA-10652
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10652
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 2.2.4, 3.0.0
>
>
> After upgrading from 2.1 to 3.0, the {{system_traces.sessions}} table is not 
> properly upgraded to include the {{client}} column added in CASSANDRA-8162. 
> Just upgrading from a clean 2.2 install to 3.0 won't show this error because 
> the column was included in 2.2, it just doesn't break the queries in that 
> release.
> The errors I get when querying {{system_traces.sessions}}:
> {noformat}
> java.lang.RuntimeException: java.lang.IllegalStateException: 
> [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2350)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.lang.IllegalStateException: [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:465) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[main/:na]
>   at 
> 

cassandra git commit: Fix SimpleDateType type compatibility

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 84d448853 -> b05fe4ffe


Fix SimpleDateType type compatibility

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-10027


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b05fe4ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b05fe4ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b05fe4ff

Branch: refs/heads/cassandra-2.2
Commit: b05fe4ffee8f1c457a79013c2ddc88abcdadc6bb
Parents: 84d4488
Author: blerer 
Authored: Thu Nov 12 20:53:42 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 20:53:42 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9c834f3..5705453 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
  * (Hadoop) fix splits calculation (CASSANDRA-10640)
  * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
  * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java 
b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
index 747709e..6e435ee 100644
--- a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
+++ b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
@@ -69,7 +69,7 @@ public class SimpleDateType extends AbstractType
 @Override
 public boolean isValueCompatibleWithInternal(AbstractType otherType)
 {
-return this == otherType || otherType == IntegerType.instance;
+return this == otherType || otherType == Int32Type.instance;
 }
 
 public Term fromJSONObject(Object parsed) throws MarshalException

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 95380f4..566c0ea 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -200,4 +200,21 @@ public class AlterTest extends CQLTester
 assertInvalidSyntaxMessage("no viable alternative at input 
'WITH'", stmt);
 }
 }
+
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date: 
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
 }

[jira] [Updated] (CASSANDRA-10131) consistently sort DCs in nodetool:status

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10131:

Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)
   (was: 2.1.x)
   2.1.10
   2.2.2
   3.0.0 rc1

> consistently sort DCs in nodetool:status
> 
>
> Key: CASSANDRA-10131
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10131
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Burroughs
>Assignee: Chris Burroughs
>Priority: Minor
> Fix For: 2.1.10, 2.2.2, 3.0.0 rc1
>
> Attachments: j10131-2.1-v1.txt, j10131-2.1-v2.txt, j10131-2.2-v1.txt
>
>
> It's kind of annoying that the order flip flops as I look at different 
> clusters.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10457) fix failing jmx_metrics dtest

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-10457:
---
Fix Version/s: (was: 2.2.x)
   2.2.3

> fix failing jmx_metrics dtest
> -
>
> Key: CASSANDRA-10457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10457
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Carl Yeksigian
> Fix For: 2.2.3, 3.0.0 rc2
>
>
> {{jmxmetrics_test.py:TestJMXMetrics.begin_test}} is failing on CassCI; I've 
> reproduced it on OpenStack as well. Here's the failure:
> http://cassci.datastax.com/view/trunk/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/jmxmetrics_test/TestJMXMetrics/begin_test/
> My first thought is that a fix may be as simple as changing the parameters to 
> stress between collecting the before and after metrics. I haven't debugged 
> this further than reproducing it, though.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0

Conflicts:
CHANGES.txt
test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/464f8f07
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/464f8f07
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/464f8f07

Branch: refs/heads/cassandra-3.1
Commit: 464f8f076878dd95ae1f77f96edca8e1cd3c5699
Parents: 9f056a9 b05fe4f
Author: blerer 
Authored: Thu Nov 12 21:07:08 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:07:08 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/464f8f07/CHANGES.txt
--
diff --cc CHANGES.txt
index d554323,5705453..f00b36d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,52 -1,7 +1,53 @@@
 -2.2.4
 +3.0.1
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
+  * Fix SimpleDateType type compatibility (CASSANDRA-10027)
   * (Hadoop) fix splits calculation (CASSANDRA-10640)
   * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 +Merged from 2.1:
 + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
 + * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 +
 +
 +3.0
 + * Fix AssertionError while flushing memtable due to materialized views
 +   incorrectly inserting empty rows (CASSANDRA-10614)
 + * Store UDA initcond as CQL literal in the schema table, instead of a blob 
(CASSANDRA-10650)
 + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491)
 + * Fix distinct queries in mixed version cluster (CASSANDRA-10573)
 + * Skip sstable on clustering in names query (CASSANDRA-10571)
 + * Remove value skipping as it breaks read-repair (CASSANDRA-10655)
 + * Fix bootstrapping with MVs (CASSANDRA-10621)
 + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584)
 + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634)
 + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
 + * Fix reading of legacy sstables (CASSANDRA-10590)
 + * Use CQL type names in schema metadata tables (CASSANDRA-10365)
 + * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
 + * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)
 + * Add client address argument to IAuthenticator::newSaslNegotiator 
(CASSANDRA-8068)
 + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602)
 + * Don't use 'names query' read path for counters (CASSANDRA-10572)
 + * Fix backward compatibility for counters (CASSANDRA-10470)
 + * Remove memory_allocator paramter from cassandra.yaml 
(CASSANDRA-10581,10628)
 + * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
 + * Fix thrift cas operations with defined columns (CASSANDRA-10576)
 + * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
 + * Fix thrift get() queries with defined columns (CASSANDRA-10586)
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
   * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)

[1/3] cassandra git commit: Fix SimpleDateType type compatibility

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1 e11bd0fa8 -> 6688911f4


Fix SimpleDateType type compatibility

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-10027


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b05fe4ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b05fe4ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b05fe4ff

Branch: refs/heads/cassandra-3.1
Commit: b05fe4ffee8f1c457a79013c2ddc88abcdadc6bb
Parents: 84d4488
Author: blerer 
Authored: Thu Nov 12 20:53:42 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 20:53:42 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9c834f3..5705453 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
  * (Hadoop) fix splits calculation (CASSANDRA-10640)
  * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
  * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java 
b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
index 747709e..6e435ee 100644
--- a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
+++ b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
@@ -69,7 +69,7 @@ public class SimpleDateType extends AbstractType
 @Override
 public boolean isValueCompatibleWithInternal(AbstractType otherType)
 {
-return this == otherType || otherType == IntegerType.instance;
+return this == otherType || otherType == Int32Type.instance;
 }
 
 public Term fromJSONObject(Object parsed) throws MarshalException

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 95380f4..566c0ea 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -200,4 +200,21 @@ public class AlterTest extends CQLTester
 assertInvalidSyntaxMessage("no viable alternative at input 
'WITH'", stmt);
 }
 }
+
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date: 
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
 }

[3/3] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.1

[blerer]

Merge branch cassandra-3.0 into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6688911f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6688911f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6688911f

Branch: refs/heads/cassandra-3.1
Commit: 6688911f4a484ac2472fa3179f10358905cb9af8
Parents: e11bd0f 464f8f0
Author: blerer 
Authored: Thu Nov 12 21:13:12 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:13:29 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6688911f/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-10534) CompressionInfo not being fsynced on close

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002883#comment-15002883
 ] 

Ariel Weisberg commented on CASSANDRA-10534:


Ok. +1 LGTM.

It looks like the other components all use SequentialWriter which if you use 
the transaction stuff does the right thing without any extra work. Except we 
don't use the transaction proxy for the other metadata components instead we 
call finish manually which goes through the transaction proxy.

I can't quite tell since it looks like it can throw and cause other code not to 
execute.

> CompressionInfo not being fsynced on close
> --
>
> Key: CASSANDRA-10534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10534
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Stefania
> Fix For: 2.1.x
>
>
> I was seeing SSTable corruption due to a CompressionInfo.db file of size 0, 
> this happened multiple times in our testing with hard node reboots. After 
> some investigation it seems like these file is not being fsynced, and that 
> can potentially lead to data corruption. I am working with version 2.1.9.
> I checked for fsync calls using strace, and found them happening for all but 
> the following components: CompressionInfo, TOC.txt and digest.sha1. All of 
> these but the CompressionInfo seem tolerable. Also a quick look through the 
> code did not reveal any fsync calls. Moreover, I suspect the commit  
> 4e95953f29d89a441dfe06d3f0393ed7dd8586df 
> (https://github.com/apache/cassandra/commit/4e95953f29d89a441dfe06d3f0393ed7dd8586df#diff-b7e48a1398e39a936c11d0397d5d1966R344)
>  has caused the regression, which removed the line
> {noformat}
>  getChannel().force(true);
> {noformat}
> from CompressionMetadata.Writer.close.
> Following is the trace I saw in system.log:
> {noformat}
> INFO  [SSTableBatchOpen:1] 2015-09-29 19:24:39,170 SSTableReader.java:478 - 
> Opening 
> /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-13368
>  (79 bytes)
> ERROR [SSTableBatchOpen:1] 2015-09-29 19:24:39,177 FileUtils.java:447 - 
> Exiting forcefully due to file system exception on startup, disk failure 
> policy "stop"
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:131)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:752) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:703) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:491) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:387) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:534) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_80]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> [na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> Caused by: java.io.EOFException: null
> at 
> java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340) 
> ~[na:1.7.0_80]
> at java.io.DataInputStream.readUTF(DataInputStream.java:589) 
> ~[na:1.7.0_80]
> at java.io.DataInputStream.readUTF(DataInputStream.java:564) 
> ~[na:1.7.0_80]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:106)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> ... 14 common frames omitted
> {noformat}
> Following is the result of ls on the data directory of a corrupted SSTable 
> after 

[jira] [Created] (CASSANDRA-10696) Audit jmx_test.py

[Philip Thompson (JIRA)]

Philip Thompson created CASSANDRA-10696:
---

 Summary: Audit jmx_test.py
 Key: CASSANDRA-10696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10696
 Project: Cassandra
  Issue Type: Test
Reporter: Philip Thompson
Assignee: Jim Witschey
 Fix For: 3.1


It seems that some/many of the jmx tests are not effectively testing the 
tickets they're targeted for. Someone should go through these and refactor them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix SimpleDateType type compatibility

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9f056a917 -> 464f8f076


Fix SimpleDateType type compatibility

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-10027


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b05fe4ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b05fe4ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b05fe4ff

Branch: refs/heads/cassandra-3.0
Commit: b05fe4ffee8f1c457a79013c2ddc88abcdadc6bb
Parents: 84d4488
Author: blerer 
Authored: Thu Nov 12 20:53:42 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 20:53:42 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9c834f3..5705453 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
  * (Hadoop) fix splits calculation (CASSANDRA-10640)
  * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
  * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java 
b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
index 747709e..6e435ee 100644
--- a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
+++ b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
@@ -69,7 +69,7 @@ public class SimpleDateType extends AbstractType
 @Override
 public boolean isValueCompatibleWithInternal(AbstractType otherType)
 {
-return this == otherType || otherType == IntegerType.instance;
+return this == otherType || otherType == Int32Type.instance;
 }
 
 public Term fromJSONObject(Object parsed) throws MarshalException

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 95380f4..566c0ea 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -200,4 +200,21 @@ public class AlterTest extends CQLTester
 assertInvalidSyntaxMessage("no viable alternative at input 
'WITH'", stmt);
 }
 }
+
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date: 
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
 }

[2/2] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0

[blerer]

Merge branch cassandra-2.2 into cassandra-3.0

Conflicts:
CHANGES.txt
test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/464f8f07
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/464f8f07
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/464f8f07

Branch: refs/heads/cassandra-3.0
Commit: 464f8f076878dd95ae1f77f96edca8e1cd3c5699
Parents: 9f056a9 b05fe4f
Author: blerer 
Authored: Thu Nov 12 21:07:08 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:07:08 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/464f8f07/CHANGES.txt
--
diff --cc CHANGES.txt
index d554323,5705453..f00b36d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,52 -1,7 +1,53 @@@
 -2.2.4
 +3.0.1
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
+  * Fix SimpleDateType type compatibility (CASSANDRA-10027)
   * (Hadoop) fix splits calculation (CASSANDRA-10640)
   * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 +Merged from 2.1:
 + * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
 + * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 +
 +
 +3.0
 + * Fix AssertionError while flushing memtable due to materialized views
 +   incorrectly inserting empty rows (CASSANDRA-10614)
 + * Store UDA initcond as CQL literal in the schema table, instead of a blob 
(CASSANDRA-10650)
 + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491)
 + * Fix distinct queries in mixed version cluster (CASSANDRA-10573)
 + * Skip sstable on clustering in names query (CASSANDRA-10571)
 + * Remove value skipping as it breaks read-repair (CASSANDRA-10655)
 + * Fix bootstrapping with MVs (CASSANDRA-10621)
 + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584)
 + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634)
 + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
 + * Fix reading of legacy sstables (CASSANDRA-10590)
 + * Use CQL type names in schema metadata tables (CASSANDRA-10365)
 + * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
 + * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)
 + * Add client address argument to IAuthenticator::newSaslNegotiator 
(CASSANDRA-8068)
 + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602)
 + * Don't use 'names query' read path for counters (CASSANDRA-10572)
 + * Fix backward compatibility for counters (CASSANDRA-10470)
 + * Remove memory_allocator paramter from cassandra.yaml 
(CASSANDRA-10581,10628)
 + * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
 + * Fix thrift cas operations with defined columns (CASSANDRA-10576)
 + * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
 + * Fix thrift get() queries with defined columns (CASSANDRA-10586)
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
   * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)

[jira] [Commented] (CASSANDRA-10632) sstableutil tests failing

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002958#comment-15002958
 ] 

Jim Witschey commented on CASSANDRA-10632:
--

Looks like the failures meant to be fixed by that dtest PR are fixed:

http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/110/testReport/sstableutil_test/SSTableUtilTest/

So now we're on to a new error:

http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/110/testReport/sstableutil_test/SSTableUtilTest/abortedcompaction_test/

If I read the test correctly, it looks like some temp files are handing around 
after an aborted compaction. [~JoshuaMcKenzie] is this expected on Windows, or 
is this a true bug?

> sstableutil tests failing
> -
>
> Key: CASSANDRA-10632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10632
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 3.1
>
>
> {{sstableutil_test.py:SSTableUtilTest.abortedcompaction_test}} and 
> {{sstableutil_test.py:SSTableUtilTest.compaction_test}} fail on Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/abortedcompaction_test/
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/compaction_test/
> This is a pretty simple failure -- looks like the underlying behavior is ok, 
> but string comparison fails when the leading {{d}} in the filename is 
> lowercase as returned by {{sstableutil}} (see the [{{_invoke_sstableutil}} 
> test 
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L128]),
>  but uppercase as returned by {{glob.glob}} (see the [{{_get_sstable_files}} 
> test 
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L160]).
> Do I understand correctly that Windows filenames are case-insensitive, 
> including the drive portion? If that's the case, then we can just lowercase 
> the file names in the test helper functions above when the tests are run on 
> Windows. [~JoshuaMcKenzie] can you confirm? I'll fix this in the tests if so. 
> If I'm wrong, and something in {{sstableutil}} needs to be fixed, could you 
> find an assignee?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10694) Deletion info is dropped on updated rows when notifying secondary index

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002956#comment-15002956
 ] 

Tyler Hobbs commented on CASSANDRA-10694:
-

Patch with unit tests:

||branch||testall||dtest||
|[CASSANDRA-10694|https://github.com/thobbs/cassandra/tree/CASSANDRA-10694]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-dtest]|
|[CASSANDRA-10694-3.1|https://github.com/thobbs/cassandra/tree/CASSANDRA-10694-3.1]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-3.1-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-3.1-dtest]|
|[CASSANDRA-10694-trunk|https://github.com/thobbs/cassandra/tree/CASSANDRA-10694-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10694-trunk-dtest]|

> Deletion info is dropped on updated rows when notifying secondary index
> ---
>
> Key: CASSANDRA-10694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10694
> Project: Cassandra
>  Issue Type: Bug
>  Components: index
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 3.0.1, 3.1
>
> Attachments: index-deletion.patch
>
>
> In {{SecondaryIndexManager.onUpdated()}}, we fail to copy the 
> {{DeletionInfo}} from the existing and new rows before notifying the index of 
> the update.  This leads the index to believe a new, live row has been 
> inserted instead of a single-row deletion.  It looks like this has been a 
> problem since 3.0.0-beta1.
> I've attached a simple patch that fixes the issue.  I'm working on a full 
> patch with tests, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9689) keyspace does not show in describe list, if create query times out

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9689:
---
Fix Version/s: (was: 2.1.x)
   2.1.10
   2.2.2
   3.0.0 rc1

> keyspace does not show in describe list, if create query times out
> --
>
> Key: CASSANDRA-9689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9689
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Roopesh
>Assignee: Paulo Motta
>  Labels: cqlsh
> Fix For: 2.1.10, 2.2.2, 3.0.0 rc1
>
>
> Here is the snapshot of the overall issue :
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm create 'demo2' -v binary:2.1.7 
> -n 3 -s -d 
> Current cluster is now: demo2
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm list
>   demo_1node
>  *demo2
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm status
> Cluster: 'demo2'
> 
> node1: UP
> node3: UP
> node2: UP
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm node2 nodetool version
> ReleaseVersion: 2.1.7
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm node2 stop
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm status
> Cluster: 'demo2'
> 
> node1: UP
> node3: UP
> node2: DOWN
> whiterabbit@whiterabbit:~/cassandra/bin$ ccm node1 cqlsh
> Connected to demo2 at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> describe keyspaces;
> system_traces  system
> cqlsh> create keyspace training with 
> replication={'class':'SimpleStrategy','replication_factor':1};
> OperationTimedOut: errors={}, last_host=127.0.0.1
> cqlsh> create keyspace training with 
> replication={'class':'SimpleStrategy','replication_factor':1};
> AlreadyExists: Keyspace 'training' already exists
> cqlsh> describe keyspaces;
> system_traces  system
> cqlsh> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10388) Windows dtest 3.0: SSL dtests are failing

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-10388:
--
Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)

> Windows dtest 3.0: SSL dtests are failing
> -
>
> Key: CASSANDRA-10388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10388
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Philip Thompson
>Assignee: Joel Knighton
>
> The dtests 
> {{native_transport_ssl_test.NativeTransportSSL.connect_to_ssl_test}} and 
> {{native_transport_ssl_test.NativeTransportSSL.use_custom_ssl_port_test}} are 
> failing on windows, but not linux.
> Stacktrace is
> {code}
>   File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
>   File 
> "D:\jenkins\workspace\cassandra-3.0_dtest_win32\cassandra-dtest\native_transport_ssl_test.py",
>  line 32, in connect_to_ssl_test
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10578) bootstrap_test.py:TestBootstrap.simultaneous_bootstrap_test dtest failing

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10578:
---
Fix Version/s: (was: 3.0.0)
   (was: 2.2.x)
   (was: 2.1.x)

> bootstrap_test.py:TestBootstrap.simultaneous_bootstrap_test dtest failing
> -
>
> Key: CASSANDRA-10578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10578
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
>
> This test fails on 2.1, 2.2, and 3.0 versions tested on CassCI:
> http://cassci.datastax.com/view/cassandra-2.1/job/cassandra-2.1_dtest/lastCompletedBuild/testReport/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/350/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> It fails with the same error, indicating that the third node, which should 
> not start while another node is bootstrapping, started. Oddly, the assertion 
> just before it, looking for a particular error in the logs, succeeds.
> This could be a race condition, where one node successfully completes 
> bootstrapping before the third node is started. However, I don't know how 
> likely that is, since it fails consistently. Unfortunately, we don't have 
> enough history on CassCI to show when the test failure started.
> I'm assigning [~yukim] for now, feel free to reassign. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10538) Assertion failed in LogFile when disk is full

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001830#comment-15001830
 ] 

Stefania commented on CASSANDRA-10538:
--

bq. How does Throwables.perfom handle AssertionError? It looks like it swallows 
it? Seems like AssertionError shouldn't be caught and should be allowed to 
terminate the JVM?

It merges it and passes it to the caller like any other {{Throwable}}, I don't 
think we should change this.

{quote}
To make sure I understand the fix. The issue was that we marked something 
committed in memory when committing (or aborting) fails to persist to disk 
because the disk is full. The fix was to write to disk first then memory, and 
if writing to disk for commit fails we can hit the abort path and then that can 
fail as well.

Or is this hitting abort and abort like you would expect given that the disk is 
full and the transaction probably can't complete successfully?
{quote}

The fix is to update memory only if disk has been already updated so that we 
can try again later on and we reflect the correct on disk status in memory. 
Before the fix the assertion would have prevented retrying. If the disk is full 
the abort record will still not be added, not even on the second attempt during 
the final close, but that's OK since a missing final record means transaction 
aborted anyway. 

Also, neither commit nor abort should throw, it's the caller of commit that may 
decide to abort ({{LifecycleTransaction}}), but {{LogTransaction}} should not 
throw during commit or abort.

> Assertion failed in LogFile when disk is full
> -
>
> Key: CASSANDRA-10538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10538
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: 
> ma_txn_compaction_67311da0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_696059b0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8ac58b70-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8be24610-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_95500fc0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_a41caa90-72b4-11e5-9eb9-b14fa4bbe709.log
>
>
> [~carlyeks] was running a stress job which filled up the disk. At the end of 
> the system logs there are several assertion errors:
> {code}
> ERROR [CompactionExecutor:1] 2015-10-14 20:46:55,467 CassandraDaemon.java:195 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: Insufficient disk space to write 2097152 bytes
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.getWriteDirectory(CompactionAwareWriter.java:156)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:77)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:220)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_40]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_40]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> INFO  [IndexSummaryManager:1] 2015-10-14 21:10:40,099 
> IndexSummaryManager.java:257 - Redistributing index summaries
> ERROR [IndexSummaryManager:1] 2015-10-14 21:10:42,275 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[IndexSummaryManager:1,1,main]
> java.lang.AssertionError: Already completed!
> at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:221) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:376)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> 

[jira] [Commented] (CASSANDRA-10243) Warn or fail when changing cluster topology live

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001851#comment-15001851
 ] 

Stefania commented on CASSANDRA-10243:
--

Actually the startup checks have already been implemented by CASSANDRA-10242.

> Warn or fail when changing cluster topology live
> 
>
> Key: CASSANDRA-10243
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10243
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> Moving a node from one rack to another in the snitch, while it is alive, is 
> almost always the wrong thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10682) Fix timeouts in BeforeFirstTest

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-10682:
-
Attachment: 
TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log

Sync-ing the parent folder does not help, the timeout issues caused by a 
missing table folder are continuing:

http://cassci.datastax.com/job/stef1927-10682-3.0-testall/5/testReport/junit/org.apache.cassandra.db.commitlog/CommitLogFailurePolicyTest/testCommitFailurePolicy_stop/

See CommitLogFailurePolicyTest.log attached.

> Fix timeouts in BeforeFirstTest
> ---
>
> Key: CASSANDRA-10682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10682
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments: 
> TEST-org.apache.cassandra.db.SinglePartitionSliceCommandTest.log, 
> TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log
>
>
> Some unit tests fail with a timeout in {{BeforeFirstTest}}, see for example 
> [here|http://cassci.datastax.com/job/cassandra-3.0_testall/242/testReport/org.apache.cassandra.db/SinglePartitionSliceCommandTest/BeforeFirstTest/].
>  
> In the corresponding log file, attached, there is a {{NoSuchFileException}} 
> which might be the cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8521) RangeSliceQueryPager may fetch too much data in the first partition

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-8521:
--
Fix Version/s: 2.2.x
   2.1.x

> RangeSliceQueryPager may fetch too much data in the first partition
> ---
>
> Key: CASSANDRA-8521
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8521
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tyler Hobbs
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.x
>
>
> As described in CASSANDRA-8087, it looks like RangeSliceQueryPager may fetch 
> more than it needs to in the first partition:
> {quote}
> when we actually query the underlying partition, the slice filter count might 
> be a lot more than what we care for (it could be Integer.MAX_VALUE if there 
> wasn't any LIMIT on the statement in the first place) and if that's the case, 
> we will read a lot more than we should. This will be only true for the first 
> partition, because after that we will update the SliceQueryFilter at the end 
> of the loop of CFS.filter(), but still, it's potentially inefficient for that 
> that first partition and might even end up blowing up the heap if the 
> partition is big, which defeats the purpose of paging. I'll note that 
> provided we don't blow up the heap then the resultSet returned to the user 
> will be fine since we'll trim it in SelectStatement, but it's still a bug 
> (provided I'm not missing something).
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9131) Defining correct behavior during leap second insertion

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001924#comment-15001924
 ] 

Stefan Podkowinski commented on CASSANDRA-9131:
---

Since CASSANDRA-9649 {{state.getTimestamp()}} is now monotonically increasing 
for the VM instead of only for each connection. Operating this way should be 
pretty safe, as people should be well aware of the importance to keep clocks in 
sync across the cluster. Now in that situation I don't really feel comfortable 
with moving timestamp generation to the clients, as now potentially much more 
servers are affected and need to be monitored for clock skew. The chances will 
be much higher to have a single system introduce incorrect timestamp values 
that will corrupt your data. 

> Defining correct behavior during leap second insertion
> --
>
> Key: CASSANDRA-9131
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9131
> Project: Cassandra
>  Issue Type: Bug
> Environment: Linux ip-172-31-0-5 3.2.0-57-virtual #87-Ubuntu SMP Tue 
> Nov 12 21:53:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Jim Witschey
>Priority: Minor
> Fix For: 3.x
>
>
> On Linux platforms, the insertion of a leap second breaks the monotonicity of 
> timestamps. This can make values appear to have been inserted into Cassandra 
> in a different order than they were. I want to know what behavior is expected 
> and desirable for inserts over this discontinuity.
> From a timestamp perspective, an inserted leap second looks like a repeat of 
> the previous second:
> {code}
> $ while true ; do echo "`date +%s%N` `date -u`" ; sleep .5 ; done
> 1435708798171327029 Tue Jun 30 23:59:58 UTC 2015
> 1435708798679392477 Tue Jun 30 23:59:58 UTC 2015
> 1435708799187550335 Tue Jun 30 23:59:59 UTC 2015
> 1435708799695670453 Tue Jun 30 23:59:59 UTC 2015
> 1435708799203902068 Tue Jun 30 23:59:59 UTC 2015
> 1435708799712168566 Tue Jun 30 23:59:59 UTC 2015
> 1435708800220473932 Wed Jul 1 00:00:00 UTC 2015
> 1435708800728908190 Wed Jul 1 00:00:00 UTC 2015
> 1435708801237611983 Wed Jul 1 00:00:01 UTC 2015
> 1435708801746251996 Wed Jul 1 00:00:01 UTC 2015
> {code}
> Note that 23:59:59 repeats itself, and that the timestamps increase during 
> the first time through, then step back down to the beginning of the second 
> and increase again.
> As a result, the timestamps on values inserted during these seconds will be 
> out of order. I set up a 4-node cluster running under Ubuntu 12.04.3 and 
> synced them to shortly before the leap second would be inserted. During the 
> insertion of the leap second, I ran a test with logic something like:
> {code}
> simple_insert = session.prepare(
> 'INSERT INTO test (foo, bar) VALUES (?, ?);')
> for i in itertools.count():
> # stop after midnight
> now = datetime.utcnow()
> last_midnight = now.replace(hour=0, minute=0,
> second=0, microsecond=0)
> seconds_since_midnight = (now - last_midnight).total_seconds()
> if 5 <= seconds_since_midnight <= 15:
> break
> session.execute(simple_insert, [i, i])
> result = session.execute("SELECT bar, WRITETIME(bar) FROM test;")
> {code}
> EDIT: This behavior occurs with server-generated timestamps; in this 
> particular test, I set {{use_client_timestamp}} to {{False}}.
> Under normal circumstances, the values and writetimes would increase 
> together, but when inserted over the leap second, they don't. These {{value, 
> writetime}} pairs are sorted by writetime:
> {code}
> (582, 1435708799285000)
> (579, 1435708799339000)
> (583, 1435708799593000)
> (580, 1435708799643000)
> (584, 1435708799897000)
> (581, 1435708799958000)
> {code}
> The values were inserted in increasing order, but their writetimes are in a 
> different order because of the repeated second. During the first instance of 
> 23:59:59, the values 579, 580, and 581 were inserted at the beginning, 
> middle, and end of the second. During the leap second, which is also 
> 23:59:59, 582, 583, and 584 were inserted, also at the beginning, middle, and 
> end of the second. However, since the two seconds are the same second, they 
> appear interleaved with respect to timestamps, as shown above.
> So, should I consider this behavior correct? If not, how should Cassandra 
> correctly handle the discontinuity introduced by the insertion of a leap 
> second?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003199#comment-15003199
 ] 

Ariel Weisberg commented on CASSANDRA-7217:
---

My takeaway from the counters is that with 2000 threads working through 19 
million writes took more instructions, almost double the number of cache 
references,  more than double the number of context switches, and double the 
number of dcache misses. So there was a big drop in efficiency that could 
explain how this occurs even without contention or starvation.

Now if there is a way to have 2000 threads do this work more efficiently is a 
good question. There are a lot more performance counters that might give 
insight into what having more threads changed as well as profiling. I'll look 
into it tomorrow.

> Native transport performance (with cassandra-stress) drops precipitously past 
> around 1000 threads
> -
>
> Key: CASSANDRA-7217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7217
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
>Assignee: Ariel Weisberg
>  Labels: performance, stress, triaged
> Fix For: 3.1
>
>
> This is obviously bad. Let's figure out why it's happening and put a stop to 
> it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10538) Assertion failed in LogFile when disk is full

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003374#comment-15003374
 ] 

Stefania commented on CASSANDRA-10538:
--

Thanks, I've rebased and launched another CI job.

> Assertion failed in LogFile when disk is full
> -
>
> Key: CASSANDRA-10538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10538
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: 
> ma_txn_compaction_67311da0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_696059b0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8ac58b70-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8be24610-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_95500fc0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_a41caa90-72b4-11e5-9eb9-b14fa4bbe709.log
>
>
> [~carlyeks] was running a stress job which filled up the disk. At the end of 
> the system logs there are several assertion errors:
> {code}
> ERROR [CompactionExecutor:1] 2015-10-14 20:46:55,467 CassandraDaemon.java:195 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: Insufficient disk space to write 2097152 bytes
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.getWriteDirectory(CompactionAwareWriter.java:156)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:77)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:220)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_40]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_40]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> INFO  [IndexSummaryManager:1] 2015-10-14 21:10:40,099 
> IndexSummaryManager.java:257 - Redistributing index summaries
> ERROR [IndexSummaryManager:1] 2015-10-14 21:10:42,275 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[IndexSummaryManager:1,1,main]
> java.lang.AssertionError: Already completed!
> at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:221) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:376)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:259)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:193)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.close(Transactional.java:158)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:242)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:134)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolE
> {code}
> We should not have an assertion if it can happen when the disk is full, we 
> should rather have a runtime exception.
> I also would like to understand exactly what triggered the assertion. 
> {{LifecycleTransaction}} can throw at the beginning of the commit method if 
> it cannot write the record to disk, in which case all we have to do is ensure 
> we update the records in memory after writing to disk 

[jira] [Commented] (CASSANDRA-10485) Missing host ID on hinted handoff write

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002757#comment-15002757
 ] 

Ariel Weisberg commented on CASSANDRA-10485:


+1 LGTM

> Missing host ID on hinted handoff write
> ---
>
> Key: CASSANDRA-10485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10485
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> when I restart one of them I receive the error "Missing host ID":
> {noformat}
> WARN  [SharedPool-Worker-1] 2015-10-08 13:15:33,882 
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.AssertionError: Missing host ID for 63.251.156.141
> at 
> org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:978)
>  ~[apache-cassandra-2.1.3.jar:2.1.3]
> at 
> org.apache.cassandra.service.StorageProxy$6.runMayThrow(StorageProxy.java:950)
>  ~[apache-cassandra-2.1.3.jar:2.1.3]
> at 
> org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:2235)
>  ~[apache-cassandra-2.1.3.jar:2.1.3]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[apache-cassandra-2.1.3.jar:2.1.3]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-2.1.3.jar:2.1.3]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {noformat}
> If I made nodetool status, the problematic node has ID:
> {noformat}
> UN  10.10.10.12  1.3 TB 1   ?   
> 4d5c8fd2-a909-4f09-a23c-4cd6040f338a  rack3
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/4] cassandra git commit: Fix SimpleDateType type compatibility

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7fcf14faa -> 0781bccc9


Fix SimpleDateType type compatibility

patch by Benjamin Lerer; reviewed by Joshua McKenzie for CASSANDRA-10027


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b05fe4ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b05fe4ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b05fe4ff

Branch: refs/heads/trunk
Commit: b05fe4ffee8f1c457a79013c2ddc88abcdadc6bb
Parents: 84d4488
Author: blerer 
Authored: Thu Nov 12 20:53:42 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 20:53:42 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9c834f3..5705453 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.4
+ * Fix SimpleDateType type compatibility (CASSANDRA-10027)
  * (Hadoop) fix splits calculation (CASSANDRA-10640)
  * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
  * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java 
b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
index 747709e..6e435ee 100644
--- a/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
+++ b/src/java/org/apache/cassandra/db/marshal/SimpleDateType.java
@@ -69,7 +69,7 @@ public class SimpleDateType extends AbstractType
 @Override
 public boolean isValueCompatibleWithInternal(AbstractType otherType)
 {
-return this == otherType || otherType == IntegerType.instance;
+return this == otherType || otherType == Int32Type.instance;
 }
 
 public Term fromJSONObject(Object parsed) throws MarshalException

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b05fe4ff/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
index 95380f4..566c0ea 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/operations/AlterTest.java
@@ -200,4 +200,21 @@ public class AlterTest extends CQLTester
 assertInvalidSyntaxMessage("no viable alternative at input 
'WITH'", stmt);
 }
 }
+
+/**
+ * tests CASSANDRA-10027
+ */
+@Test
+public void testAlterColumnTypeToDate() throws Throwable
+{
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 int);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+execute("ALTER TABLE %s ALTER c1 TYPE date;");
+assertRows(execute("SELECT * FROM %s"), row(1, 1));
+
+createTable("CREATE TABLE %s (key int PRIMARY KEY, c1 varint);");
+execute("INSERT INTO %s (key, c1) VALUES (1,1);");
+assertInvalidMessage("Cannot change c1 from type varint to type date: 
types are incompatible.",
+ "ALTER TABLE %s ALTER c1 TYPE date;");
+}
 }

[4/4] cassandra git commit: Merge branch cassandra-3.1 into trunk

[blerer]

Merge branch cassandra-3.1 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0781bccc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0781bccc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0781bccc

Branch: refs/heads/trunk
Commit: 0781bccc92ad0204425c9515564bdacee0942439
Parents: 7fcf14f 6688911
Author: blerer 
Authored: Thu Nov 12 21:14:45 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:14:59 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0781bccc/CHANGES.txt
--

[3/4] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.1

[blerer]

Merge branch cassandra-3.0 into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6688911f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6688911f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6688911f

Branch: refs/heads/trunk
Commit: 6688911f4a484ac2472fa3179f10358905cb9af8
Parents: e11bd0f 464f8f0
Author: blerer 
Authored: Thu Nov 12 21:13:12 2015 +0100
Committer: blerer 
Committed: Thu Nov 12 21:13:29 2015 +0100

--
 CHANGES.txt|  1 +
 .../cassandra/db/marshal/SimpleDateType.java   |  2 +-
 .../cql3/validation/operations/AlterTest.java  | 17 +
 3 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6688911f/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-10538) Assertion failed in LogFile when disk is full

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002861#comment-15002861
 ] 

Ariel Weisberg commented on CASSANDRA-10538:


+1 LGTM

Only catch is the dtests on trunk appear to have gotten healthier than when you 
started with only 10 failures. I don't think you broke anything though looking 
at where it was a few builds ago.

> Assertion failed in LogFile when disk is full
> -
>
> Key: CASSANDRA-10538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10538
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: 
> ma_txn_compaction_67311da0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_696059b0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8ac58b70-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8be24610-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_95500fc0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_a41caa90-72b4-11e5-9eb9-b14fa4bbe709.log
>
>
> [~carlyeks] was running a stress job which filled up the disk. At the end of 
> the system logs there are several assertion errors:
> {code}
> ERROR [CompactionExecutor:1] 2015-10-14 20:46:55,467 CassandraDaemon.java:195 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: Insufficient disk space to write 2097152 bytes
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.getWriteDirectory(CompactionAwareWriter.java:156)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:77)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:220)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_40]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_40]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> INFO  [IndexSummaryManager:1] 2015-10-14 21:10:40,099 
> IndexSummaryManager.java:257 - Redistributing index summaries
> ERROR [IndexSummaryManager:1] 2015-10-14 21:10:42,275 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[IndexSummaryManager:1,1,main]
> java.lang.AssertionError: Already completed!
> at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:221) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:376)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:259)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:193)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.close(Transactional.java:158)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:242)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:134)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolE
> {code}
> We should not have an assertion if it can happen when the disk is full, we 
> should rather have a runtime exception.
> I also would like to understand exactly what triggered the assertion. 
> {{LifecycleTransaction}} can throw at the beginning of 

[jira] [Commented] (CASSANDRA-9474) DC/Rack property changed on live system

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003120#comment-15003120
 ] 

Paulo Motta commented on CASSANDRA-9474:


Thanks for the patch [~molsson]! In fact, the only reason I see for 
PropertyFileSnitch to be reloadable is to add new nodes, but this is not 
necessary on GossipingPropertyFileSnitch as you mentioned.

The patch looks good. We're towards the end of 2.1 development so we're 
avoiding adding changes at this stage, could you update/rebase your patch to 
2.2 and 3.0? A few remarks:
* Register the {{ReconnectableSnitchHelper}} only if prefer_local=true
** I thought we could do the same with broadcasting the INTERNAL_IP, but better 
to always broadcast in case this is needed somewhere else.
* As you noted, CASSANDRA-10242 already added the check to the rack, so if you 
could add the dc check on the top of that it would be nice. Since the 
{{cassandra.ignore_rack}} was introduced and documented, then I think we should 
add an additional property {{cassandra.ignore_dc}} to be consistent.
** Please add a note to {{NEWS.txt}} (on 2.2.4) similar to 
[this|https://github.com/apache/cassandra/blob/29ff1f2ac2a3da16f75ce87555df8f6014c8303e/NEWS.txt#L21]
 of CASSANDRA-10242.

If you have access to a cassandra-dtest environment, it would be nice if you 
could create a variant of [this 
dtest|https://github.com/carlyeks/cassandra-dtest/blob/47ceb7995c6a4f9599c8a257f995c75ec518b241/replication_test.py#L460]
 to check if the startup fails when the DC is changed. Otherwise I can create 
it before accepting the patch.

> DC/Rack property changed on live system
> ---
>
> Key: CASSANDRA-9474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9474
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.5
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
> Fix For: 2.1.x
>
> Attachments: cassandra-2.1-9474.patch, 
> cassandra-2.1-dc_rack_healthcheck.patch
>
>
> When using GossipingPropertyFileSnitch it is possible to change the data 
> center and rack of a live node by changing the cassandra-rackdc.properties 
> file. Should this really be possible? In the documentation at 
> http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html
>  it's stated that you should ??Choose the name carefully; renaming a data 
> center is not possible??, but with this functionality it doesn't seem 
> impossible(maybe a bit hard with changing replication etc.).
> This functionality was introduced by CASSANDRA-5897 so I'm guessing there is 
> some use case for this?
> Personally I would want the DC/rack settings to be as restricted as the 
> cluster name, otherwise if a node could just join another data center without 
> removing it's local information couldn't it mess up the token ranges? And 
> suddenly the old data center/rack would loose 1 replica of all the data that 
> the node contains.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003017#comment-15003017
 ] 

Ariel Weisberg edited comment on CASSANDRA-7217 at 11/12/15 11:24 PM:
--

Performance counters

2000 threads
{code}
Results:
op rate   : 20576 [WRITE:20576]
partition rate: 20576 [WRITE:20576]
row rate  : 20576 [WRITE:20576]
latency mean  : 97.2 [WRITE:97.2]
latency median: 91.0 [WRITE:91.0]
latency 95th percentile   : 179.1 [WRITE:179.1]
latency 99th percentile   : 268.3 [WRITE:268.3]
latency 99.9th percentile : 499.0 [WRITE:499.0]
latency max   : 1123.2 [WRITE:1123.2]
Total partitions  : 1900 [WRITE:1900]
Total errors  : 0 [WRITE:0]
total gc count: 0
total gc mb   : 0
total gc time (s) : 0
avg gc time(ms)   : NaN
stdev gc time(ms) : 0
Total operation time  : 00:15:23
END

 Performance counter stats for './cassandra-stress write n=1900 -rate 
threads=2000 -mode native cql3 -node 192.168.1.9':

 3,236,123,141,155  cycles#2.115 GHz
 [16.14%]
 2,580,132,815,701  instructions  #0.80  insns per cycle

  #0.89  stalled cycles per 
insn [21.45%]
63,994,020,523  cache-references  #   41.828 M/sec  
 [26.72%]
12,523,946,172  cache-misses  #   19.570 % of all cache 
refs [32.00%]
 2,294,356,584,027  idle-cycles-frontend  #   70.90% frontend cycles 
idle[37.28%]
 1,636,932,476,246  idle-cycles-backend   #   50.58% backend  cycles 
idle[42.54%]
1529337.521837  cpu-clock (msec)

1529938.883184  task-clock (msec) #1.635 CPUs utilized  

   129,217  page-faults   #0.084 K/sec  

87,687,956  cs#0.057 M/sec  

36,591,482  migrations#0.024 M/sec  

   129,132  minor-faults  #0.084 K/sec  

   360,467,544,173  branch-instructions   #  235.609 M/sec  
 [47.81%]
 5,205,849,494  branch-misses #1.44% of all branches
 [47.76%]
67,636,847,959  L1-dcache-load-misses #   44.209 M/sec  
 [47.83%]
24,113,350,939  L1-dcache-store-misses#   15.761 M/sec  
 [47.94%]
18,928,905,359  L1-dcache-prefetch-misses #   12.372 M/sec  
 [42.84%]
56,721,903,854  L1-icache-load-misses #   37.075 M/sec  
 [42.94%]
 3,977,754,938  dTLB-load-misses  #2.600 M/sec  
 [42.96%]
   748,817,996  dTLB-store-misses #0.489 M/sec  
 [42.93%]
   791,352,271  iTLB-load-misses  #0.517 M/sec  
 [42.86%]
 5,414,521,445  branch-load-misses#3.539 M/sec  
 [42.80%]
37,275,666,810  LLC-loads #   24.364 M/sec  
 [42.83%]
10,226,436,059  LLC-stores#6.684 M/sec  
 [42.80%]
16,548,689,552  LLC-prefetches#   10.817 M/sec  
 [10.57%]

 935.835191719 seconds time elapsed
{code}
500 threads
{code}
Results:
op rate   : 63563 [WRITE:63563]
partition rate: 63563 [WRITE:63563]
row rate  : 63563 [WRITE:63563]
latency mean  : 7.9 [WRITE:7.9]
latency median: 5.8 [WRITE:5.8]
latency 95th percentile   : 16.2 [WRITE:16.2]
latency 99th percentile   : 36.3 [WRITE:36.3]
latency 99.9th percentile : 74.0 [WRITE:74.0]
latency max   : 422.0 [WRITE:422.0]
Total partitions  : 1900 [WRITE:1900]
Total errors  : 0 [WRITE:0]
total gc count: 0
total gc mb   : 0
total gc time (s) : 0
avg gc time(ms)   : NaN
stdev gc time(ms) : 0
Total operation time  : 00:04:58
END

 Performance counter stats for './cassandra-stress write n=1900 -rate 
threads=500 -mode native cql3 -node 192.168.1.9':

 1,967,800,644,333  cycles#2.424 GHz
 [16.23%]
 1,939,192,725,937  instructions  #0.99  insns per cycle

  #0.67  stalled cycles per 
insn [21.56%]
29,961,702,909  cache-references  #   36.915 M/sec  
 [26.87%]
 7,138,097,546  cache-misses  #   23.824 % of all cache 
refs [32.16%]
 

[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003400#comment-15003400
 ] 

Stefania commented on CASSANDRA-10089:
--

Thank you for your analysis [~jkni]. We should definitely guard with a null 
check, here is the 
[patch|https://github.com/stef1927/cassandra/tree/10089-cov-2.1]. It applies to 
2.2+ without conflicts. Would you mind committing [~JoshuaMcKenzie]?



> NullPointerException in Gossip handleStateNormal
> 
>
> Key: CASSANDRA-10089
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10089
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1
>
> Attachments: node1_debug.log, node2_debug.log, node3_debug.log
>
>
> Whilst comparing dtests for CASSANDRA-9970 I found [this failing 
> dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/]
>  in 2.2:
> {code}
> Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 
> 15:39:57,873 CassandraDaemon.java:183 - Exception in thread 
> Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat 
> org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629)
>  ~[main/:na] \tat 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) 
> ~[main/:na] \tat 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na] \tat 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) 
> ~[main/:na] \tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_80] \tat 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]']
> {code}
> I wasn't able to find it on unpatched branches  but it is clearly not related 
> to CASSANDRA-9970, if anything it could have been a side effect of 
> CASSANDRA-9871.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9474) DC/Rack property changed on live system

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9474:
---
Issue Type: Improvement  (was: Bug)

> DC/Rack property changed on live system
> ---
>
> Key: CASSANDRA-9474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9474
> Project: Cassandra
>  Issue Type: Improvement
> Environment: Cassandra 2.1.5
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
> Fix For: 2.1.x
>
> Attachments: cassandra-2.1-9474.patch, 
> cassandra-2.1-dc_rack_healthcheck.patch
>
>
> When using GossipingPropertyFileSnitch it is possible to change the data 
> center and rack of a live node by changing the cassandra-rackdc.properties 
> file. Should this really be possible? In the documentation at 
> http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html
>  it's stated that you should ??Choose the name carefully; renaming a data 
> center is not possible??, but with this functionality it doesn't seem 
> impossible(maybe a bit hard with changing replication etc.).
> This functionality was introduced by CASSANDRA-5897 so I'm guessing there is 
> some use case for this?
> Personally I would want the DC/rack settings to be as restricted as the 
> cluster name, otherwise if a node could just join another data center without 
> removing it's local information couldn't it mess up the token ranges? And 
> suddenly the old data center/rack would loose 1 replica of all the data that 
> the node contains.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10693) Bootstrapping failure that at least needs improved logging

[Heiko Sommer (JIRA)]

Heiko Sommer created CASSANDRA-10693:


 Summary: Bootstrapping failure that at least needs improved logging
 Key: CASSANDRA-10693
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10693
 Project: Cassandra
  Issue Type: Bug
  Components: Streaming and Messaging
 Environment: Cassandra 2.1.11
Cluster of 2 or 3 xeon servers (8 core, 24 GB RAM each)
Reporter: Heiko Sommer


I'm playing with a test cluster, trying to add a 3rd server while writing to 
the cluster at max rate (executeAsync with pushing back). Single table for time 
series data, with RF=2, STCS.

The bootstrap failure that I repeatedly get may be the same as in 
CASSANDRA-9440 (or also CASSANDRA-6565).
Even if I could join the 3rd node using temporarily modified configuation 
({{streaming_socket_timeout_in_ms}}, {{phi_convict_threshold}} etc), I think 
that at least improving the logs for this failure case could be worthwhile.

When starting the new node (134.171.18.228), the underlying problem seems to 
happen very early in the bootstrapping, on one of the existing cluster nodes 
(134.171.18.231):
{noformat}
INFO  [STREAM-INIT-/134.171.18.228:45331] 2015-11-11 16:21:30,344 
StreamResultFuture.java:109 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc 
ID#0] Creating new streaming plan for Bootstrap
INFO  [STREAM-INIT-/134.171.18.228:45331] 2015-11-11 16:21:30,344 
StreamResultFuture.java:116 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc, 
ID#0] Received streaming plan for Bootstrap
INFO  [STREAM-INIT-/134.171.18.228:45333] 2015-11-11 16:21:30,350 
StreamResultFuture.java:116 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc, 
ID#0] Received streaming plan for Bootstrap

ERROR [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,505 
StreamSession.java:505 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc] 
Streaming error occurred
java.lang.AssertionError: Illegal bounds [2270144..2270152); size: 2270144
at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:328) 
~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:98) 
~[apache-cassandra-2.1.11.jar:2.1.11]
at org.apache.cassandra.io.util.Memory.getLong(Memory.java:249) 
~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.io.compress.CompressionMetadata.getChunksForSections(CompressionMetadata.java:253)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.messages.OutgoingFileMessage.(OutgoingFileMessage.java:70)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.StreamTransferTask.addTransferFile(StreamTransferTask.java:58)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.StreamSession.addTransferFiles(StreamSession.java:378)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.StreamSession.addTransferRanges(StreamSession.java:272)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:521) 
~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:453)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at 
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:256)
 ~[apache-cassandra-2.1.11.jar:2.1.11]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
INFO  [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,510 
StreamResultFuture.java:180 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc] 
Session with /134.171.18.228 is complete
WARN  [STREAM-IN-/134.171.18.228] 2015-11-11 16:21:41,511 
StreamResultFuture.java:207 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc] 
Stream failed
{noformat}

On the joining node, this failure is not reported at all:
{noformat}
INFO  [main] 2015-11-11 16:21:30,000 StorageService.java:1120 - JOINING: 
Starting to bootstrap...
INFO  [main] 2015-11-11 16:21:30,336 StreamResultFuture.java:86 - [Stream 
#43deef20-8890-11e5-a633-c16ac7efddbc] Executing streaming plan for Bootstrap
INFO  [StreamConnectionEstablisher:2] 2015-11-11 16:21:30,337 
StreamSession.java:220 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc] 
Starting streaming to /134.171.18.229
INFO  [StreamConnectionEstablisher:1] 2015-11-11 16:21:30,337 
StreamSession.java:220 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc] 
Starting streaming to /134.171.18.231
INFO  [StreamConnectionEstablisher:1] 2015-11-11 16:21:30,348 
StreamCoordinator.java:209 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc, 
ID#0] Beginning stream session with /134.171.18.231
INFO  [StreamConnectionEstablisher:2] 2015-11-11 16:21:30,348 
StreamCoordinator.java:209 - [Stream #43deef20-8890-11e5-a633-c16ac7efddbc, 
ID#0] Beginning stream session with /134.171.18.229
INFO  

[jira] [Commented] (CASSANDRA-10685) Index transaction dealing with cleanup doesn't notify indexes of partition deletion

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001970#comment-15001970
 ] 

Sergio Bossa commented on CASSANDRA-10685:
--

+1

> Index transaction dealing with cleanup doesn't notify indexes of partition 
> deletion
> ---
>
> Key: CASSANDRA-10685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10685
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.0.1, 3.1
>
>
> {{SecondaryIndexManager.CleanupGCTransaction}} doesn't notify registered 
> indexers of the partition level deletion on commit. Indexers *are* notified 
> of each individual row removal caused by the partition delete, but also 
> informing them of the top level delete would enable them to make potential 
> optimizations during cleanup. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001992#comment-15001992
 ] 

Marcus Eriksson commented on CASSANDRA-10515:
-

note that we do have a problem with upgradesstables losing level info: 
CASSANDRA-10692

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>  Labels: commitlog, triage
> Fix For: 3.1, 2.1.x, 2.2.x
>
> Attachments: C5commitLogIncrease.jpg, CASSANDRA-19579.jpg, 
> CommitLogProblem.jpg, CommitLogSize.jpg, 
> MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, cassandra.yaml, 
> cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10534) CompressionInfo not being fsynced on close

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003615#comment-15003615
 ] 

Stefania commented on CASSANDRA-10534:
--

Thank you for the review and for checking the remaining components. I agree 
with your analysis. 

These are the components defined in {{SSTableWriter.components()}}:

|| Component || Notes |
| Component.DATA | Sync-ed, SequentialWriter | 
| Component.PRIMARY_INDEX  | Sync-ed, SequentialWriter |
| Component.STATS | Sync-ed, SequentialWriter |
| Component.SUMMARY | {{SSTableReader.saveSummary()}}, called in finish, not 
sync-ed but we write a magic number at the end and we regenerate the summary 
when loading it if we don't find this magic number, |
| Component.TOC | Not sync-ed but it's read only by standalone tools | 
| Component.DIGEST | Written by {{DataIntegrityMetadata.ChecksumWriter}}, not 
sync-ed and not used but intended for users so they can validate uncompressed 
data files via sha1sum. In 2.2 this becomes the adler32 checksum that can be 
verified with nodetool verify or the standalone verifier.|
| Component.FILTER | Written and sync-ed manually in {{IndexWrit
er.close()}} |
| Component.COMPRESSION_INFO | To be sync-ed by this patch in 
{{ompressionMetadata.Writer.close()}} |
| Component.CRC | Sync-ed, SequentialWriter |

bq. I can't quite tell since it looks like it can throw and cause other code 
not to execute.

Yes but this was improved in 2.2 with {{LifecycleTransaction}} and in 3.0 even 
further with the removal of unfinished left overs via {{LogTransaction}}.

So, IMO, we could have problems with standalone tools and we should probably 
sync TOC and DIGEST at some point but it is not critical and probaly best 
addressed in another ticket. Shall I open one?

> CompressionInfo not being fsynced on close
> --
>
> Key: CASSANDRA-10534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10534
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Stefania
> Fix For: 2.1.x
>
>
> I was seeing SSTable corruption due to a CompressionInfo.db file of size 0, 
> this happened multiple times in our testing with hard node reboots. After 
> some investigation it seems like these file is not being fsynced, and that 
> can potentially lead to data corruption. I am working with version 2.1.9.
> I checked for fsync calls using strace, and found them happening for all but 
> the following components: CompressionInfo, TOC.txt and digest.sha1. All of 
> these but the CompressionInfo seem tolerable. Also a quick look through the 
> code did not reveal any fsync calls. Moreover, I suspect the commit  
> 4e95953f29d89a441dfe06d3f0393ed7dd8586df 
> (https://github.com/apache/cassandra/commit/4e95953f29d89a441dfe06d3f0393ed7dd8586df#diff-b7e48a1398e39a936c11d0397d5d1966R344)
>  has caused the regression, which removed the line
> {noformat}
>  getChannel().force(true);
> {noformat}
> from CompressionMetadata.Writer.close.
> Following is the trace I saw in system.log:
> {noformat}
> INFO  [SSTableBatchOpen:1] 2015-09-29 19:24:39,170 SSTableReader.java:478 - 
> Opening 
> /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-13368
>  (79 bytes)
> ERROR [SSTableBatchOpen:1] 2015-09-29 19:24:39,177 FileUtils.java:447 - 
> Exiting forcefully due to file system exception on startup, disk failure 
> policy "stop"
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:131)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:752) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:703) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:491) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:387) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> 

[jira] [Updated] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend

[Ivan Burmistrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Burmistrov updated CASSANDRA-10585:

Flags: Patch
   Attachment: SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
   SSTablePerReadHistogram_RowCache-cassandra-2_2.patch
   SSTablePerReadHistogram_RowCache-cassandra-2_1.patch
Fix Version/s: 3.0.x
   2.2.x

> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read 
> from row cache.
> And so, this metric will have big values even almost all requests processed 
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from 
> row cache then we read zero SSTables by this request.
> The patch at the attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10682) Fix timeouts in BeforeFirstTest

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003626#comment-15003626
 ] 

Stefania commented on CASSANDRA-10682:
--

The problem is that {{CQLTester.prepareServer()}} is called twice and each time 
it deletes all data folders. However, the schema instance is initialized only 
once after the first cleanup.

> Fix timeouts in BeforeFirstTest
> ---
>
> Key: CASSANDRA-10682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10682
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x
>
> Attachments: 
> TEST-org.apache.cassandra.db.SinglePartitionSliceCommandTest.log, 
> TEST-org.apache.cassandra.db.commitlog.CommitLogFailurePolicyTest.log
>
>
> Some unit tests fail with a timeout in {{BeforeFirstTest}}, see for example 
> [here|http://cassci.datastax.com/job/cassandra-3.0_testall/242/testReport/org.apache.cassandra.db/SinglePartitionSliceCommandTest/BeforeFirstTest/].
>  
> In the corresponding log file, attached, there is a {{NoSuchFileException}} 
> which might be the cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend

[Ivan Burmistrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003635#comment-15003635
 ] 

Ivan Burmistrov commented on CASSANDRA-10585:
-

I have prepared patches to versions 2.1, 2.2 and 3.0 (and it is not hard to 
prepare patch for trunk).

Important comment for 2.2 and 3.0 versions.
SSTablePerReadHistogram in these versions is EstimatedHistogram now.
But for this implementation of histogram zero values make almost no effect.  
It seems not good, because it is important to know if, for example, we read 0.1 
SSTables per read at average. 
For example, we want to know do 
https://issues.apache.org/jira/browse/CASSANDRA-2498 and 
https://issues.apache.org/jira/browse/CASSANDRA-5514 optimizations works for 
some table. 
EstimatedHistogram returns only integer values and make this scenario 
impossible, while it was possible in versions 2.1 and below.
So in patches for 2.2 and 3.0 I switched SSTablesPerReadHistogram to 
ExponentiallyDecayingHistogram implementation.

> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read 
> from row cache.
> And so, this metric will have big values even almost all requests processed 
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from 
> row cache then we read zero SSTables by this request.
> The patch at the attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend

[Ivan Burmistrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003635#comment-15003635
 ] 

Ivan Burmistrov edited comment on CASSANDRA-10585 at 11/13/15 7:02 AM:
---

I have prepared patches to versions 2.1, 2.2 and 3.0 (and it is not hard to 
prepare patch for trunk).

Important comment for 2.2 and 3.0 versions.
SSTablePerReadHistogram in these versions is EstimatedHistogram now.
But for this implementation of histogram zero values make almost no effect.  
It seems not good, because it is important to know if, for example, we read 0.1 
SSTables per read at average. 
For example, we want to know do 
[CASSANDRA-2498|https://issues.apache.org/jira/browse/CASSANDRA-2498] and 
[CASSANDRA-5514|https://issues.apache.org/jira/browse/CASSANDRA-5514] 
optimizations works for some table. 
EstimatedHistogram returns only integer values and make this scenario 
impossible, while it was possible in versions 2.1 and below.
So in patches for 2.2 and 3.0 I switched SSTablesPerReadHistogram to 
ExponentiallyDecayingHistogram implementation.


was (Author: isburmistrov):
I have prepared patches to versions 2.1, 2.2 and 3.0 (and it is not hard to 
prepare patch for trunk).

Important comment for 2.2 and 3.0 versions.
SSTablePerReadHistogram in these versions is EstimatedHistogram now.
But for this implementation of histogram zero values make almost no effect.  
It seems not good, because it is important to know if, for example, we read 0.1 
SSTables per read at average. 
For example, we want to know do 
https://issues.apache.org/jira/browse/CASSANDRA-2498 and 
https://issues.apache.org/jira/browse/CASSANDRA-5514 optimizations works for 
some table. 
EstimatedHistogram returns only integer values and make this scenario 
impossible, while it was possible in versions 2.1 and below.
So in patches for 2.2 and 3.0 I switched SSTablesPerReadHistogram to 
ExponentiallyDecayingHistogram implementation.

> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read 
> from row cache.
> And so, this metric will have big values even almost all requests processed 
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from 
> row cache then we read zero SSTables by this request.
> The patch at the attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10697) Leak detected while running offline scrub

[mlowicki (JIRA)]

mlowicki created CASSANDRA-10697:


 Summary: Leak detected while running offline scrub
 Key: CASSANDRA-10697
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10697
 Project: Cassandra
  Issue Type: Bug
 Environment: C* 2.1.9 on Debian Wheezy
Reporter: mlowicki
Priority: Critical


I got couple of those:
{code}
ERROR 05:09:15 LEAK DETECTED: a reference 
(org.apache.cassandra.utils.concurrent.Ref$State@3b60e162) to class 
org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1433208674:/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-405434
 was not released before the reference was garbage collected
{code}

and then:
{code}
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

at 
org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:99)

at 
org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)

at 
org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:353)

at java.io.RandomAccessFile.readFully(RandomAccessFile.java:444)

at java.io.RandomAccessFile.readFully(RandomAccessFile.java:424)

at 
org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:378)

at 
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:348)

at 
org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:327)

at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)

at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)

at 
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)

at 
org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)

at 
org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)

at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:120)

at 
org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)

at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)

at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

at 
org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)

at 
org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)

at 
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:192)

at 
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)

at 
org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:158)

at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:220)

at 
org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:116)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10538) Assertion failed in LogFile when disk is full

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003618#comment-15003618
 ] 

Stefania commented on CASSANDRA-10538:
--

Still getting lots of {{OperationTimedOut}}, unsure why.

http://cassci.datastax.com/job/stef1927-10538-3.0-dtest/7/testReport/

[~philipthompson] do you know why by any chance?

It should be safe to commit however, so I am updating progress accordingly. 
Thank you for the review.


> Assertion failed in LogFile when disk is full
> -
>
> Key: CASSANDRA-10538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10538
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: 
> ma_txn_compaction_67311da0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_696059b0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8ac58b70-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_8be24610-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_95500fc0-72b4-11e5-9eb9-b14fa4bbe709.log, 
> ma_txn_compaction_a41caa90-72b4-11e5-9eb9-b14fa4bbe709.log
>
>
> [~carlyeks] was running a stress job which filled up the disk. At the end of 
> the system logs there are several assertion errors:
> {code}
> ERROR [CompactionExecutor:1] 2015-10-14 20:46:55,467 CassandraDaemon.java:195 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: Insufficient disk space to write 2097152 bytes
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.getWriteDirectory(CompactionAwareWriter.java:156)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:77)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:110)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:220)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_40]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_40]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_40]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> INFO  [IndexSummaryManager:1] 2015-10-14 21:10:40,099 
> IndexSummaryManager.java:257 - Redistributing index summaries
> ERROR [IndexSummaryManager:1] 2015-10-14 21:10:42,275 
> CassandraDaemon.java:195 - Exception in thread 
> Thread[IndexSummaryManager:1,1,main]
> java.lang.AssertionError: Already completed!
> at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:221) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:376)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:259)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:193)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.close(Transactional.java:158)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:242)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:134)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolE
> {code}
> We should not have an assertion if it can happen when the disk is full, we 
> should rather have a runtime exception.
> I also would like to understand exactly what triggered the 

[jira] [Updated] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend

[Ivan Burmistrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Burmistrov updated CASSANDRA-10585:

Attachment: (was: cassandra-10585.patch)

> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read 
> from row cache.
> And so, this metric will have big values even almost all requests processed 
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from 
> row cache then we read zero SSTables by this request.
> The patch at the attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend

[Ivan Burmistrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003635#comment-15003635
 ] 

Ivan Burmistrov edited comment on CASSANDRA-10585 at 11/13/15 7:02 AM:
---

I have prepared patches to versions 2.1, 2.2 and 3.0 (and it is not hard to 
prepare patch for trunk).

Important comment for 2.2 and 3.0 versions.
SSTablePerReadHistogram in these versions is EstimatedHistogram now.
But for this implementation of histogram zero values make almost no effect.  
It seems not good, because it is important to know if, for example, we read 0.1 
SSTables per read at average. 
For example, we want to know does 
[CASSANDRA-2498|https://issues.apache.org/jira/browse/CASSANDRA-2498] or 
[CASSANDRA-5514|https://issues.apache.org/jira/browse/CASSANDRA-5514] 
optimization works for some table. 
EstimatedHistogram returns only integer values and make this scenario 
impossible, while it was possible in versions 2.1 and below.
So in patches for 2.2 and 3.0 I switched SSTablesPerReadHistogram to 
ExponentiallyDecayingHistogram implementation.


was (Author: isburmistrov):
I have prepared patches to versions 2.1, 2.2 and 3.0 (and it is not hard to 
prepare patch for trunk).

Important comment for 2.2 and 3.0 versions.
SSTablePerReadHistogram in these versions is EstimatedHistogram now.
But for this implementation of histogram zero values make almost no effect.  
It seems not good, because it is important to know if, for example, we read 0.1 
SSTables per read at average. 
For example, we want to know do 
[CASSANDRA-2498|https://issues.apache.org/jira/browse/CASSANDRA-2498] and 
[CASSANDRA-5514|https://issues.apache.org/jira/browse/CASSANDRA-5514] 
optimizations works for some table. 
EstimatedHistogram returns only integer values and make this scenario 
impossible, while it was possible in versions 2.1 and below.
So in patches for 2.2 and 3.0 I switched SSTablesPerReadHistogram to 
ExponentiallyDecayingHistogram implementation.

> SSTablesPerReadHistogram seems wrong when row cache hit happend
> ---
>
> Key: CASSANDRA-10585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10585
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ivan Burmistrov
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-2_2.patch, 
> SSTablePerReadHistogram_RowCache-cassandra-3_0.patch
>
>
> SSTablePerReadHistogram metric now not considers case when row has been read 
> from row cache.
> And so, this metric will have big values even almost all requests processed 
> by row cache (and without touching SSTables, of course).
> So, it seems that correct behavior is to consider that if we read row from 
> row cache then we read zero SSTables by this request.
> The patch at the attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002051#comment-15002051
 ] 

Marcus Eriksson edited comment on CASSANDRA-10515 at 11/12/15 1:23 PM:
---

it explains the huge amount of sstables in L0

it is not committed or released yet


was (Author: krummas):
it is not committed or released yet

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>  Labels: commitlog, triage
> Fix For: 3.1, 2.1.x, 2.2.x
>
> Attachments: C5commitLogIncrease.jpg, CASSANDRA-19579.jpg, 
> CommitLogProblem.jpg, CommitLogSize.jpg, 
> MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, cassandra.yaml, 
> cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10059) Test Coverage for AbstractBTreePartition and hierarchy

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002166#comment-15002166
 ] 

Branimir Lambov commented on CASSANDRA-10059:
-

New test pushed [here|https://github.com/blambov/cassandra/tree/10059].

The test is currently failing as there are problems in 
{{AbstractBTreePartition}}'s handling of range tombstone markers, especially in 
the reverse case.



> Test Coverage for AbstractBTreePartition and hierarchy
> --
>
> Key: CASSANDRA-10059
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10059
> Project: Cassandra
>  Issue Type: Test
>Reporter: Benedict
>Assignee: Branimir Lambov
> Fix For: 3.1
>
>
> Follow up to CASSANDRA-9932. The test coverage for AbstractBTreePartition and 
> its hierarchy is entirely indirect. That is not to say it is not covered, but 
> we may have some unexplored behaviour. Coverage for BTree is also missing 
> around a couple of edges, and the gaps should be filled in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Jeff Griffith (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002048#comment-15002048
 ] 

Jeff Griffith commented on CASSANDRA-10515:
---

Thanks [~krummas] i assume you mean this explains the large number of sstables 
(55k) we experienced? I see you've fixed it. I have moved to the latest 2.1 so 
this should help with our rollout.

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>  Labels: commitlog, triage
> Fix For: 3.1, 2.1.x, 2.2.x
>
> Attachments: C5commitLogIncrease.jpg, CASSANDRA-19579.jpg, 
> CommitLogProblem.jpg, CommitLogSize.jpg, 
> MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, cassandra.yaml, 
> cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002051#comment-15002051
 ] 

Marcus Eriksson commented on CASSANDRA-10515:
-

it is not committed or released yet

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>  Labels: commitlog, triage
> Fix For: 3.1, 2.1.x, 2.2.x
>
> Attachments: C5commitLogIncrease.jpg, CASSANDRA-19579.jpg, 
> CommitLogProblem.jpg, CommitLogSize.jpg, 
> MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, cassandra.yaml, 
> cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10515) Commit logs back up with move to 2.1.10

[Jeff Griffith (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002085#comment-15002085
 ] 

Jeff Griffith commented on CASSANDRA-10515:
---

Good to know. We'll watch out for it and use the offline leveling trick you 
suggested.

> Commit logs back up with move to 2.1.10
> ---
>
> Key: CASSANDRA-10515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10515
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: redhat 6.5, cassandra 2.1.10
>Reporter: Jeff Griffith
>Assignee: Branimir Lambov
>  Labels: commitlog, triage
> Fix For: 3.1, 2.1.x, 2.2.x
>
> Attachments: C5commitLogIncrease.jpg, CASSANDRA-19579.jpg, 
> CommitLogProblem.jpg, CommitLogSize.jpg, 
> MultinodeCommitLogGrowth-node1.tar.gz, RUN3tpstats.jpg, cassandra.yaml, 
> cfstats-clean.txt, stacktrace.txt, system.log.clean
>
>
> After upgrading from cassandra 2.0.x to 2.1.10, we began seeing problems 
> where some nodes break the 12G commit log max we configured and go as high as 
> 65G or more before it restarts. Once it reaches the state of more than 12G 
> commit log files, "nodetool compactionstats" hangs. Eventually C* restarts 
> without errors (not sure yet whether it is crashing but I'm checking into it) 
> and the cleanup occurs and the commit logs shrink back down again. Here is 
> the nodetool compactionstats immediately after restart.
> {code}
> jgriffith@prod1xc1.c2.bf1:~$ ndc
> pending tasks: 2185
>compaction type   keyspace  table completed
>   totalunit   progress
> Compaction   SyncCore  *cf1*   61251208033   
> 170643574558   bytes 35.89%
> Compaction   SyncCore  *cf2*   19262483904
> 19266079916   bytes 99.98%
> Compaction   SyncCore  *cf3*6592197093
>  6592316682   bytes100.00%
> Compaction   SyncCore  *cf4*3411039555
>  3411039557   bytes100.00%
> Compaction   SyncCore  *cf5*2879241009
>  2879487621   bytes 99.99%
> Compaction   SyncCore  *cf6*   21252493623
> 21252635196   bytes100.00%
> Compaction   SyncCore  *cf7*   81009853587
> 81009854438   bytes100.00%
> Compaction   SyncCore  *cf8*3005734580
>  3005768582   bytes100.00%
> Active compaction remaining time :n/a
> {code}
> I was also doing periodic "nodetool tpstats" which were working but not being 
> logged in system.log on the StatusLogger thread until after the compaction 
> started working again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10059) Test Coverage and related bug-fixes for AbstractBTreePartition and hierarchy

[Branimir Lambov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov updated CASSANDRA-10059:

Summary: Test Coverage and related bug-fixes for AbstractBTreePartition and 
hierarchy  (was: Test Coverage for AbstractBTreePartition and hierarchy)

> Test Coverage and related bug-fixes for AbstractBTreePartition and hierarchy
> 
>
> Key: CASSANDRA-10059
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10059
> Project: Cassandra
>  Issue Type: Test
>Reporter: Benedict
>Assignee: Branimir Lambov
> Fix For: 3.1
>
>
> Follow up to CASSANDRA-9932. The test coverage for AbstractBTreePartition and 
> its hierarchy is entirely indirect. That is not to say it is not covered, but 
> we may have some unexplored behaviour. Coverage for BTree is also missing 
> around a couple of edges, and the gaps should be filled in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10059) Test Coverage and related bug-fixes for AbstractBTreePartition and hierarchy

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002178#comment-15002178
 ] 

Branimir Lambov commented on CASSANDRA-10059:
-

Patch is not ready for review, I am now working on addressing the identified 
issues.

> Test Coverage and related bug-fixes for AbstractBTreePartition and hierarchy
> 
>
> Key: CASSANDRA-10059
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10059
> Project: Cassandra
>  Issue Type: Test
>Reporter: Benedict
>Assignee: Branimir Lambov
> Fix For: 3.1
>
>
> Follow up to CASSANDRA-9932. The test coverage for AbstractBTreePartition and 
> its hierarchy is entirely indirect. That is not to say it is not covered, but 
> we may have some unexplored behaviour. Coverage for BTree is also missing 
> around a couple of edges, and the gaps should be filled in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10619) disallow streaming operations while upgrading

[Russell Bradberry (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002699#comment-15002699
 ] 

Russell Bradberry commented on CASSANDRA-10619:
---

So you are trying to prevent streaming only between nodes that are of 
incompatible versions?  Rather than, what I inferred as, disabling streaming 
cluster-wide until all nodes have been upgraded.

> disallow streaming operations while upgrading
> -
>
> Key: CASSANDRA-10619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10619
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>
> Cassandra should prevent users from doing streaming operations in the middle 
> of a cluster upgrade.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10619) disallow streaming operations while upgrading

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002707#comment-15002707
 ] 

Jon Haddad commented on CASSANDRA-10619:


Yeah, it would make sense to disable it only if the versions are incompatible.

> disallow streaming operations while upgrading
> -
>
> Key: CASSANDRA-10619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10619
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>
> Cassandra should prevent users from doing streaming operations in the middle 
> of a cluster upgrade.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5576f45b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5576f45b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5576f45b

Branch: refs/heads/cassandra-3.1
Commit: 5576f45b99d8f596c3e916ffafa687b00d0c9a04
Parents: 0cafccf b54836b
Author: Yuki Morishita 
Authored: Thu Nov 12 09:45:06 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:45:06 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5576f45b/CHANGES.txt
--

[02/15] cassandra git commit: Add CASSANDRA-10079 change to CHANGES.txt

[yukim]

Add CASSANDRA-10079 change to CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ab1c837
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ab1c837
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ab1c837

Branch: refs/heads/cassandra-2.2
Commit: 9ab1c8372dc9da1e085120bd4377be74d0413759
Parents: 1c3ff92
Author: Yuki Morishita 
Authored: Thu Nov 12 09:43:36 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:43:36 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ab1c837/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 92244a0..d0c0af9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
  * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[05/15] cassandra git commit: Add CASSANDRA-10079 change to CHANGES.txt

[yukim]

Add CASSANDRA-10079 change to CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ab1c837
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ab1c837
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ab1c837

Branch: refs/heads/cassandra-3.1
Commit: 9ab1c8372dc9da1e085120bd4377be74d0413759
Parents: 1c3ff92
Author: Yuki Morishita 
Authored: Thu Nov 12 09:43:36 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:43:36 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ab1c837/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 92244a0..d0c0af9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
  * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[yukim]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d84b42b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d84b42b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d84b42b3

Branch: refs/heads/cassandra-3.1
Commit: d84b42b325f711e9a7cf0ae65c2428b884464936
Parents: e487553 9ab1c83
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:18 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:18 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d84b42b3/CHANGES.txt
--
diff --cc CHANGES.txt
index 0fcf037,d0c0af9..2cfcb55
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
   * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[04/15] cassandra git commit: Add CASSANDRA-10079 change to CHANGES.txt

[yukim]

Add CASSANDRA-10079 change to CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ab1c837
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ab1c837
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ab1c837

Branch: refs/heads/cassandra-3.0
Commit: 9ab1c8372dc9da1e085120bd4377be74d0413759
Parents: 1c3ff92
Author: Yuki Morishita 
Authored: Thu Nov 12 09:43:36 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:43:36 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ab1c837/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 92244a0..d0c0af9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
  * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b54836b1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b54836b1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b54836b1

Branch: refs/heads/trunk
Commit: b54836b120a0acc5d820f3465a7354eea73d9290
Parents: 0de23f2 d84b42b
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:59 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:59 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b54836b1/CHANGES.txt
--
diff --cc CHANGES.txt
index 02dc249,2cfcb55..8f1b163
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -51,7 -6,15 +51,8 @@@ Merged from 2.2
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
 - * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 - * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 - * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 - * Improve json2sstable error reporting on nonexistent columns 
(CASSANDRA-10401)
   * (cqlsh) fix COPY using wrong variable name for time_format 
(CASSANDRA-10633)
   * Do not run SizeEstimatesRecorder if a node is not a member of the ring 
(CASSANDRA-9912)
   * Improve handling of dead nodes in gossip (CASSANDRA-10298)

[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[yukim]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d84b42b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d84b42b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d84b42b3

Branch: refs/heads/cassandra-2.2
Commit: d84b42b325f711e9a7cf0ae65c2428b884464936
Parents: e487553 9ab1c83
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:18 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:18 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d84b42b3/CHANGES.txt
--
diff --cc CHANGES.txt
index 0fcf037,d0c0af9..2cfcb55
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
   * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[yukim]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d84b42b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d84b42b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d84b42b3

Branch: refs/heads/cassandra-3.0
Commit: d84b42b325f711e9a7cf0ae65c2428b884464936
Parents: e487553 9ab1c83
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:18 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:18 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d84b42b3/CHANGES.txt
--
diff --cc CHANGES.txt
index 0fcf037,d0c0af9..2cfcb55
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
   * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[03/15] cassandra git commit: Add CASSANDRA-10079 change to CHANGES.txt

[yukim]

Add CASSANDRA-10079 change to CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ab1c837
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ab1c837
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ab1c837

Branch: refs/heads/trunk
Commit: 9ab1c8372dc9da1e085120bd4377be74d0413759
Parents: 1c3ff92
Author: Yuki Morishita 
Authored: Thu Nov 12 09:43:36 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:43:36 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ab1c837/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 92244a0..d0c0af9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
  * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b54836b1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b54836b1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b54836b1

Branch: refs/heads/cassandra-3.1
Commit: b54836b120a0acc5d820f3465a7354eea73d9290
Parents: 0de23f2 d84b42b
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:59 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:59 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b54836b1/CHANGES.txt
--
diff --cc CHANGES.txt
index 02dc249,2cfcb55..8f1b163
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -51,7 -6,15 +51,8 @@@ Merged from 2.2
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
 - * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 - * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 - * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 - * Improve json2sstable error reporting on nonexistent columns 
(CASSANDRA-10401)
   * (cqlsh) fix COPY using wrong variable name for time_format 
(CASSANDRA-10633)
   * Do not run SizeEstimatesRecorder if a node is not a member of the ring 
(CASSANDRA-9912)
   * Improve handling of dead nodes in gossip (CASSANDRA-10298)

[15/15] cassandra git commit: Merge branch 'cassandra-3.1' into trunk

[yukim]

Merge branch 'cassandra-3.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7cb33aa5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7cb33aa5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7cb33aa5

Branch: refs/heads/trunk
Commit: 7cb33aa59bb0097c5e5956f2ed53bbdded7686fc
Parents: 186efef 5576f45
Author: Yuki Morishita 
Authored: Thu Nov 12 09:45:17 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:45:17 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7cb33aa5/CHANGES.txt
--

[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[yukim]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b54836b1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b54836b1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b54836b1

Branch: refs/heads/cassandra-3.0
Commit: b54836b120a0acc5d820f3465a7354eea73d9290
Parents: 0de23f2 d84b42b
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:59 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:59 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b54836b1/CHANGES.txt
--
diff --cc CHANGES.txt
index 02dc249,2cfcb55..8f1b163
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -51,7 -6,15 +51,8 @@@ Merged from 2.2
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
   * Expose phi values from failure detector via JMX and tweak debug
 and trace logging (CASSANDRA-9526)
 - * Fix RangeNamesQueryPager (CASSANDRA-10509)
 - * Deprecate Pig support (CASSANDRA-10542)
 - * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
  Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
 - * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 - * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 - * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 - * Improve json2sstable error reporting on nonexistent columns 
(CASSANDRA-10401)
   * (cqlsh) fix COPY using wrong variable name for time_format 
(CASSANDRA-10633)
   * Do not run SizeEstimatesRecorder if a node is not a member of the ring 
(CASSANDRA-9912)
   * Improve handling of dead nodes in gossip (CASSANDRA-10298)

[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[yukim]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d84b42b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d84b42b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d84b42b3

Branch: refs/heads/trunk
Commit: d84b42b325f711e9a7cf0ae65c2428b884464936
Parents: e487553 9ab1c83
Author: Yuki Morishita 
Authored: Thu Nov 12 09:44:18 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:44:18 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d84b42b3/CHANGES.txt
--
diff --cc CHANGES.txt
index 0fcf037,d0c0af9..2cfcb55
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
   * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[01/15] cassandra git commit: Add CASSANDRA-10079 change to CHANGES.txt

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 1c3ff9242 -> 9ab1c8372
  refs/heads/cassandra-2.2 e48755357 -> d84b42b32
  refs/heads/cassandra-3.0 0de23f20a -> b54836b12
  refs/heads/cassandra-3.1 0cafccfc5 -> 5576f45b9
  refs/heads/trunk 186efefe8 -> 7cb33aa59


Add CASSANDRA-10079 change to CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ab1c837
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ab1c837
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ab1c837

Branch: refs/heads/cassandra-2.1
Commit: 9ab1c8372dc9da1e085120bd4377be74d0413759
Parents: 1c3ff92
Author: Yuki Morishita 
Authored: Thu Nov 12 09:43:36 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:43:36 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ab1c837/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 92244a0..d0c0af9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
  * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)

[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5576f45b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5576f45b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5576f45b

Branch: refs/heads/trunk
Commit: 5576f45b99d8f596c3e916ffafa687b00d0c9a04
Parents: 0cafccf b54836b
Author: Yuki Morishita 
Authored: Thu Nov 12 09:45:06 2015 -0600
Committer: Yuki Morishita 
Committed: Thu Nov 12 09:45:06 2015 -0600

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5576f45b/CHANGES.txt
--

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84d44885
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84d44885
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84d44885

Branch: refs/heads/cassandra-2.2
Commit: 84d4488537605a86373325f86bbc4cd358921223
Parents: d84b42b ecd66de
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 17:58:38 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:58:38 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  6 
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 59 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/CHANGES.txt
--
diff --cc CHANGES.txt
index 2cfcb55,2eeda94..9c834f3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
   * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --cc src/java/org/apache/cassandra/service/StorageServiceMBean.java
index 2c92d43,1351fea..00060ae
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@@ -239,9 -248,14 +239,14 @@@ public interface StorageServiceMBean ex
  public long trueSnapshotsSize();
  
  /**
+  * Forces refresh of values stored in system.size_estimates of all column 
families.
+  */
+ public void refreshSizeEstimates() throws ExecutionException;
+ 
+ /**
   * Forces major compaction of a single keyspace
   */
 -public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;
 +public void forceKeyspaceCompaction(boolean splitOutput, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException;
  
  /**
   * Trigger a cleanup of keys on a single keyspace

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeProbe.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --cc src/java/org/apache/cassandra/tools/NodeTool.java
index 175b325,1d4a420..fcb6ed0
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@@ -129,23 -187,14 +129,24 @@@ public class NodeToo
  TpStats.class,
  TopPartitions.class,
  SetLoggingLevel.class,
 -GetLoggingLevels.class
 +GetLoggingLevels.class,
- FailureDetectorInfo.class
++FailureDetectorInfo.class,
++RefreshSizeEstimates.class
  );
  
 -Cli parser = Cli.builder("nodetool")
 -.withDescription("Manage your Cassandra cluster")
 +Cli.CliBuilder builder = Cli.builder("nodetool");
 +
 +builder.withDescription("Manage your 

[1/2] cassandra git commit: Add nodetool command to refresh system.size_estimates

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 d84b42b32 -> 84d448853


Add nodetool command to refresh system.size_estimates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecd66de2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecd66de2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecd66de2

Branch: refs/heads/cassandra-2.2
Commit: ecd66de2bdf2ecb88c4276a4616a76f5ec2921bb
Parents: 9ab1c83
Author: Carl Yeksigian 
Authored: Tue Nov 10 17:22:11 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:53:33 2015 +

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/service/StorageService.java|  6 ++
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 src/java/org/apache/cassandra/tools/NodeProbe.java  | 12 
 src/java/org/apache/cassandra/tools/NodeTool.java   | 11 +++
 5 files changed, 36 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d0c0af9..2eeda94 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
@@ -33,6 +34,7 @@
  * Mark nodes as dead even if they've already left (CASSANDRA-10205)
  * Update internal python driver used by cqlsh (CASSANDRA-10161, 
CASSANDRA-10507)
 
+
 2.1.10
  * Bulk Loader API could not tolerate even node failure (CASSANDRA-10347)
  * Avoid misleading pushed notifications when multiple nodes

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 93b1b97..665ce3a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -83,6 +83,7 @@ import org.apache.cassandra.db.MutationVerbHandler;
 import org.apache.cassandra.db.ReadRepairVerbHandler;
 import org.apache.cassandra.db.ReadVerbHandler;
 import org.apache.cassandra.db.SchemaCheckVerbHandler;
+import org.apache.cassandra.db.SizeEstimatesRecorder;
 import org.apache.cassandra.db.SnapshotDetailsTabularData;
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.db.TruncateVerbHandler;
@@ -2604,6 +2605,11 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return total;
 }
 
+public void refreshSizeEstimates() throws ExecutionException
+{
+
FBUtilities.waitOnFuture(ScheduledExecutors.optionalTasks.submit(SizeEstimatesRecorder.instance));
+}
+
 /**
  * @param allowIndexes Allow index CF names to be passed in
  * @param autoAddIndexes Automatically add secondary indexes if a CF has 
them

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index b8582a3..1351fea 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -248,6 +248,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 public long trueSnapshotsSize();
 
 /**
+ * Forces refresh of values stored in system.size_estimates of all column 
families.
+ */
+public void refreshSizeEstimates() throws ExecutionException;
+
+/**
  * Forces major compaction of a single keyspace
  */
 public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index 6f2b6fb..49c493d 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java

[1/3] cassandra git commit: Add nodetool command to refresh system.size_estimates

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 b54836b12 -> 9f056a917


Add nodetool command to refresh system.size_estimates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecd66de2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecd66de2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecd66de2

Branch: refs/heads/cassandra-3.0
Commit: ecd66de2bdf2ecb88c4276a4616a76f5ec2921bb
Parents: 9ab1c83
Author: Carl Yeksigian 
Authored: Tue Nov 10 17:22:11 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:53:33 2015 +

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/service/StorageService.java|  6 ++
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 src/java/org/apache/cassandra/tools/NodeProbe.java  | 12 
 src/java/org/apache/cassandra/tools/NodeTool.java   | 11 +++
 5 files changed, 36 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d0c0af9..2eeda94 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
@@ -33,6 +34,7 @@
  * Mark nodes as dead even if they've already left (CASSANDRA-10205)
  * Update internal python driver used by cqlsh (CASSANDRA-10161, 
CASSANDRA-10507)
 
+
 2.1.10
  * Bulk Loader API could not tolerate even node failure (CASSANDRA-10347)
  * Avoid misleading pushed notifications when multiple nodes

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 93b1b97..665ce3a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -83,6 +83,7 @@ import org.apache.cassandra.db.MutationVerbHandler;
 import org.apache.cassandra.db.ReadRepairVerbHandler;
 import org.apache.cassandra.db.ReadVerbHandler;
 import org.apache.cassandra.db.SchemaCheckVerbHandler;
+import org.apache.cassandra.db.SizeEstimatesRecorder;
 import org.apache.cassandra.db.SnapshotDetailsTabularData;
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.db.TruncateVerbHandler;
@@ -2604,6 +2605,11 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return total;
 }
 
+public void refreshSizeEstimates() throws ExecutionException
+{
+
FBUtilities.waitOnFuture(ScheduledExecutors.optionalTasks.submit(SizeEstimatesRecorder.instance));
+}
+
 /**
  * @param allowIndexes Allow index CF names to be passed in
  * @param autoAddIndexes Automatically add secondary indexes if a CF has 
them

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index b8582a3..1351fea 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -248,6 +248,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 public long trueSnapshotsSize();
 
 /**
+ * Forces refresh of values stored in system.size_estimates of all column 
families.
+ */
+public void refreshSizeEstimates() throws ExecutionException;
+
+/**
  * Forces major compaction of a single keyspace
  */
 public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index 6f2b6fb..49c493d 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java

[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[aleksey]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9f056a91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9f056a91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9f056a91

Branch: refs/heads/cassandra-3.0
Commit: 9f056a91754b3516ad1521d32abb075b554e62c3
Parents: b54836b 84d4488
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 18:05:54 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 18:05:54 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  5 +++
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 58 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9f056a91/CHANGES.txt
--
diff --cc CHANGES.txt
index 8f1b163,9c834f3..d554323
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,51 -1,6 +1,52 @@@
 -2.2.4
 +3.0.1
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
   * (Hadoop) fix splits calculation (CASSANDRA-10640)
   * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 +Merged from 2.1:
++ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
 + * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 +
 +
 +3.0
 + * Fix AssertionError while flushing memtable due to materialized views
 +   incorrectly inserting empty rows (CASSANDRA-10614)
 + * Store UDA initcond as CQL literal in the schema table, instead of a blob 
(CASSANDRA-10650)
 + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491)
 + * Fix distinct queries in mixed version cluster (CASSANDRA-10573)
 + * Skip sstable on clustering in names query (CASSANDRA-10571)
 + * Remove value skipping as it breaks read-repair (CASSANDRA-10655)
 + * Fix bootstrapping with MVs (CASSANDRA-10621)
 + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584)
 + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634)
 + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
 + * Fix reading of legacy sstables (CASSANDRA-10590)
 + * Use CQL type names in schema metadata tables (CASSANDRA-10365)
 + * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
 + * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)
 + * Add client address argument to IAuthenticator::newSaslNegotiator 
(CASSANDRA-8068)
 + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602)
 + * Don't use 'names query' read path for counters (CASSANDRA-10572)
 + * Fix backward compatibility for counters (CASSANDRA-10470)
 + * Remove memory_allocator paramter from cassandra.yaml 
(CASSANDRA-10581,10628)
 + * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
 + * Fix thrift cas operations with defined columns (CASSANDRA-10576)
 + * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
 + * Fix thrift get() queries with defined columns (CASSANDRA-10586)
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
   * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)

[jira] [Updated] (CASSANDRA-9453) NullPointerException on gossip state change during startup

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-9453:
---
Assignee: Joel Knighton

> NullPointerException on gossip state change during startup
> --
>
> Key: CASSANDRA-9453
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9453
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tyler Hobbs
>Assignee: Joel Knighton
>Priority: Minor
> Fix For: 2.2.x
>
> Attachments: logs.tar.gz
>
>
> In the {{consistency_test.TestConsistency.short_read_reversed_test}} dtest 
> where nodes are restarted one-by-one, one of the nodes logged a 
> NullPointerException during startup:
> {noformat}
> INFO  [HANDSHAKE-/127.0.0.3] 2015-05-21 13:48:16,724 
> OutboundTcpConnection.java:489 - Handshaking version with /127.0.0.3
> INFO  [main] 2015-05-21 13:48:16,725 StorageService.java:1862 - Node 
> /127.0.0.2 state jump to normal
> INFO  [main] 2015-05-21 13:48:16,757 CassandraDaemon.java:517 - Waiting for 
> gossip to settle before accepting client requests...
> INFO  [GossipStage:1] 2015-05-21 13:48:16,776 Gossiper.java:995 - Node 
> /127.0.0.1 has restarted, now UP
> INFO  [CompactionExecutor:1] 2015-05-21 13:48:16,780 CompactionTask.java:225 
> - Compacted (085b4380-ffc0-11e4-b28a-efe71ca64a4e) 4 sstables to 
> [/mnt/tmp/dtest-FLOZYC/test/node2/data/system/local-7ad54392bcdd35a684174e047860b377/la-10-big,]
>  to level=0.  1,783 bytes to 1,217 (~68% of original) in 75ms = 0.015475MB/s. 
>  0 total partitions merged to 1.  Partition merge counts were {4:1, }
> INFO  [GossipStage:2] 2015-05-21 13:48:16,786 Gossiper.java:995 - Node 
> /127.0.0.3 has restarted, now UP
> INFO  [HANDSHAKE-/127.0.0.1] 2015-05-21 13:48:16,788 
> OutboundTcpConnection.java:489 - Handshaking version with /127.0.0.1
> ERROR [GossipStage:1] 2015-05-21 13:48:16,790 CassandraDaemon.java:154 - 
> Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1723)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1796)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1850)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1621)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2308) 
> ~[main/:na]
> at 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1017) 
> ~[main/:na]
> at 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1098) 
> ~[main/:na]
> at 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
> ~[main/:na]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> I've attached the logs for the three nodes.  Node 2 was the one with the 
> error.
> This error was on the trunk dtests, but I assume 2.2 is affected at a 
> minimum, so I set the fix version for 2.2.x.  Please check 2.0 and 2.1 for 
> the same potential problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10694) Deletion info is dropped on updated rows when notifying secondary index

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002558#comment-15002558
 ] 

Tyler Hobbs commented on CASSANDRA-10694:
-

Like CASSANDRA-10690, this also doesn't visibly affect the built-in secondary 
indexes due to stale entry handling, although it would eventually have an 
impact on performance if a lot of deletions are performed on indexed data.  
However, for custom secondary indexes, this is problematic.

> Deletion info is dropped on updated rows when notifying secondary index
> ---
>
> Key: CASSANDRA-10694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10694
> Project: Cassandra
>  Issue Type: Bug
>  Components: index
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 3.0.1, 3.1
>
> Attachments: index-deletion.patch
>
>
> In {{SecondaryIndexManager.onUpdated()}}, we fail to copy the 
> {{DeletionInfo}} from the existing and new rows before notifying the index of 
> the update.  This leads the index to believe a new, live row has been 
> inserted instead of a single-row deletion.  It looks like this has been a 
> problem since 3.0.0-beta1.
> I've attached a simple patch that fixes the issue.  I'm working on a full 
> patch with tests, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10311) It looks like our type alterations may be buggy

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002577#comment-15002577
 ] 

Aleksey Yeschenko commented on CASSANDRA-10311:
---

At least {{IntegerType::isValueCompatibleWithInternal}} is certainly broken, 
something else might too.

> It looks like our type alterations may be buggy
> ---
>
> Key: CASSANDRA-10311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10311
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
>
> We should document how type coercion works, in all contexts (UDFs, query 
> responses, merging), and what our criteria are for success. Right now it 
> looks like we perform no conversion, so we should require that they are 
> compared in the same way (if they are clusterings), and that they at least 
> have the same number of bytes (if both fixed width).
> Integer type considers itself value compatible with Int32 and Long, which 
> from an AlterTable point of view at least seems potentially problematic. 
> It's very likely I'm missing something. However as it stands we seem able to 
> read an old type from an sstable, have it make it through a compaction 
> unscathed, and write out the same bytes "as" the new type. If I'm correct 
> about this behaviour, this will corrupt this partition in the new sstable so 
> that it cannot be read.
> Not marking as critical/blocker, as I'm not familiar enough with how this 
> works to say if this brief analysis is correct, but if I am we should raise 
> the priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9474) DC/Rack property changed on live system

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-9474:
---
Assignee: Marcus Olsson
Reviewer: Paulo Motta

> DC/Rack property changed on live system
> ---
>
> Key: CASSANDRA-9474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9474
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.5
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
> Fix For: 2.1.x
>
> Attachments: cassandra-2.1-9474.patch, 
> cassandra-2.1-dc_rack_healthcheck.patch
>
>
> When using GossipingPropertyFileSnitch it is possible to change the data 
> center and rack of a live node by changing the cassandra-rackdc.properties 
> file. Should this really be possible? In the documentation at 
> http://docs.datastax.com/en/cassandra/2.1/cassandra/initialize/initializeMultipleDS.html
>  it's stated that you should ??Choose the name carefully; renaming a data 
> center is not possible??, but with this functionality it doesn't seem 
> impossible(maybe a bit hard with changing replication etc.).
> This functionality was introduced by CASSANDRA-5897 so I'm guessing there is 
> some use case for this?
> Personally I would want the DC/rack settings to be as restricted as the 
> cluster name, otherwise if a node could just join another data center without 
> removing it's local information couldn't it mess up the token ranges? And 
> suddenly the old data center/rack would loose 1 replica of all the data that 
> the node contains.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-10111:

Assignee: Joel Knighton

> reconnecting snitch can bypass cluster name check
> -
>
> Key: CASSANDRA-10111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
> Project: Cassandra
>  Issue Type: Bug
> Environment: 2.0.x
>Reporter: Chris Burroughs
>Assignee: Joel Knighton
>  Labels: gossip
> Fix For: 2.1.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different 
> listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing 
> node in cluster B (due to an out of data DNS entry).  Cluster B  added all of 
> the nodes from cluster A, somehow bypassing the cluster name mismatch check 
> for this nodes.  The first reference to cluster A nodes in cluster B logs is 
> when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node 
> /8.37.70.168 is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept 
> them out with 'ClusterName mismatch'.  Cluster B however tried to send to 
> send reads/writes to cluster A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since 
> the consequence of crazy merged clusters are really bad (the reason there is 
> the name mismatch check in the first place) I think the hole is reasonable to 
> plug.  I'm not sure exactly what the code path is that skips the check in 
> GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10311) It looks like our type alterations may be buggy

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10311:
--
Fix Version/s: 3.0.x
   2.2.x
   2.1.x

> It looks like our type alterations may be buggy
> ---
>
> Key: CASSANDRA-10311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10311
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benedict
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> We should document how type coercion works, in all contexts (UDFs, query 
> responses, merging), and what our criteria are for success. Right now it 
> looks like we perform no conversion, so we should require that they are 
> compared in the same way (if they are clusterings), and that they at least 
> have the same number of bytes (if both fixed width).
> Integer type considers itself value compatible with Int32 and Long, which 
> from an AlterTable point of view at least seems potentially problematic. 
> It's very likely I'm missing something. However as it stands we seem able to 
> read an old type from an sstable, have it make it through a compaction 
> unscathed, and write out the same bytes "as" the new type. If I'm correct 
> about this behaviour, this will corrupt this partition in the new sstable so 
> that it cannot be read.
> Not marking as critical/blocker, as I'm not familiar enough with how this 
> works to say if this brief analysis is correct, but if I am we should raise 
> the priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10694) Deletion info is dropped on updated rows when notifying secondary index

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-10694:
---

 Summary: Deletion info is dropped on updated rows when notifying 
secondary index
 Key: CASSANDRA-10694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10694
 Project: Cassandra
  Issue Type: Bug
  Components: index
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
 Fix For: 3.0.1, 3.1
 Attachments: index-deletion.patch

In {{SecondaryIndexManager.onUpdated()}}, we fail to copy the {{DeletionInfo}} 
from the existing and new rows before notifying the index of the update.  This 
leads the index to believe a new, live row has been inserted instead of a 
single-row deletion.  It looks like this has been a problem since 3.0.0-beta1.

I've attached a simple patch that fixes the issue.  I'm working on a full patch 
with tests, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Add nodetool command to refresh system.size_estimates

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 9ab1c8372 -> ecd66de2b


Add nodetool command to refresh system.size_estimates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecd66de2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecd66de2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecd66de2

Branch: refs/heads/cassandra-2.1
Commit: ecd66de2bdf2ecb88c4276a4616a76f5ec2921bb
Parents: 9ab1c83
Author: Carl Yeksigian 
Authored: Tue Nov 10 17:22:11 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:53:33 2015 +

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/service/StorageService.java|  6 ++
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 src/java/org/apache/cassandra/tools/NodeProbe.java  | 12 
 src/java/org/apache/cassandra/tools/NodeTool.java   | 11 +++
 5 files changed, 36 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d0c0af9..2eeda94 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
@@ -33,6 +34,7 @@
  * Mark nodes as dead even if they've already left (CASSANDRA-10205)
  * Update internal python driver used by cqlsh (CASSANDRA-10161, 
CASSANDRA-10507)
 
+
 2.1.10
  * Bulk Loader API could not tolerate even node failure (CASSANDRA-10347)
  * Avoid misleading pushed notifications when multiple nodes

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 93b1b97..665ce3a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -83,6 +83,7 @@ import org.apache.cassandra.db.MutationVerbHandler;
 import org.apache.cassandra.db.ReadRepairVerbHandler;
 import org.apache.cassandra.db.ReadVerbHandler;
 import org.apache.cassandra.db.SchemaCheckVerbHandler;
+import org.apache.cassandra.db.SizeEstimatesRecorder;
 import org.apache.cassandra.db.SnapshotDetailsTabularData;
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.db.TruncateVerbHandler;
@@ -2604,6 +2605,11 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return total;
 }
 
+public void refreshSizeEstimates() throws ExecutionException
+{
+
FBUtilities.waitOnFuture(ScheduledExecutors.optionalTasks.submit(SizeEstimatesRecorder.instance));
+}
+
 /**
  * @param allowIndexes Allow index CF names to be passed in
  * @param autoAddIndexes Automatically add secondary indexes if a CF has 
them

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index b8582a3..1351fea 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -248,6 +248,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 public long trueSnapshotsSize();
 
 /**
+ * Forces refresh of values stored in system.size_estimates of all column 
families.
+ */
+public void refreshSizeEstimates() throws ExecutionException;
+
+/**
  * Forces major compaction of a single keyspace
  */
 public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index 6f2b6fb..49c493d 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java

[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84d44885
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84d44885
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84d44885

Branch: refs/heads/cassandra-3.0
Commit: 84d4488537605a86373325f86bbc4cd358921223
Parents: d84b42b ecd66de
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 17:58:38 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:58:38 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  6 
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 59 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/CHANGES.txt
--
diff --cc CHANGES.txt
index 2cfcb55,2eeda94..9c834f3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
   * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --cc src/java/org/apache/cassandra/service/StorageServiceMBean.java
index 2c92d43,1351fea..00060ae
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@@ -239,9 -248,14 +239,14 @@@ public interface StorageServiceMBean ex
  public long trueSnapshotsSize();
  
  /**
+  * Forces refresh of values stored in system.size_estimates of all column 
families.
+  */
+ public void refreshSizeEstimates() throws ExecutionException;
+ 
+ /**
   * Forces major compaction of a single keyspace
   */
 -public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;
 +public void forceKeyspaceCompaction(boolean splitOutput, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException;
  
  /**
   * Trigger a cleanup of keys on a single keyspace

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeProbe.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --cc src/java/org/apache/cassandra/tools/NodeTool.java
index 175b325,1d4a420..fcb6ed0
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@@ -129,23 -187,14 +129,24 @@@ public class NodeToo
  TpStats.class,
  TopPartitions.class,
  SetLoggingLevel.class,
 -GetLoggingLevels.class
 +GetLoggingLevels.class,
- FailureDetectorInfo.class
++FailureDetectorInfo.class,
++RefreshSizeEstimates.class
  );
  
 -Cli parser = Cli.builder("nodetool")
 -.withDescription("Manage your Cassandra cluster")
 +Cli.CliBuilder builder = Cli.builder("nodetool");
 +
 +builder.withDescription("Manage your 

[4/5] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.1

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e11bd0fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e11bd0fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e11bd0fa

Branch: refs/heads/trunk
Commit: e11bd0fa8cb2768037143f5ed8c772f8a84c64b4
Parents: 5576f45 9f056a9
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 18:07:10 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 18:07:10 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  5 +++
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 58 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e11bd0fa/CHANGES.txt
--

[1/4] cassandra git commit: Add nodetool command to refresh system.size_estimates

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.1 5576f45b9 -> e11bd0fa8


Add nodetool command to refresh system.size_estimates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecd66de2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecd66de2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecd66de2

Branch: refs/heads/cassandra-3.1
Commit: ecd66de2bdf2ecb88c4276a4616a76f5ec2921bb
Parents: 9ab1c83
Author: Carl Yeksigian 
Authored: Tue Nov 10 17:22:11 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:53:33 2015 +

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/service/StorageService.java|  6 ++
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 src/java/org/apache/cassandra/tools/NodeProbe.java  | 12 
 src/java/org/apache/cassandra/tools/NodeTool.java   | 11 +++
 5 files changed, 36 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d0c0af9..2eeda94 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
@@ -33,6 +34,7 @@
  * Mark nodes as dead even if they've already left (CASSANDRA-10205)
  * Update internal python driver used by cqlsh (CASSANDRA-10161, 
CASSANDRA-10507)
 
+
 2.1.10
  * Bulk Loader API could not tolerate even node failure (CASSANDRA-10347)
  * Avoid misleading pushed notifications when multiple nodes

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 93b1b97..665ce3a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -83,6 +83,7 @@ import org.apache.cassandra.db.MutationVerbHandler;
 import org.apache.cassandra.db.ReadRepairVerbHandler;
 import org.apache.cassandra.db.ReadVerbHandler;
 import org.apache.cassandra.db.SchemaCheckVerbHandler;
+import org.apache.cassandra.db.SizeEstimatesRecorder;
 import org.apache.cassandra.db.SnapshotDetailsTabularData;
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.db.TruncateVerbHandler;
@@ -2604,6 +2605,11 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return total;
 }
 
+public void refreshSizeEstimates() throws ExecutionException
+{
+
FBUtilities.waitOnFuture(ScheduledExecutors.optionalTasks.submit(SizeEstimatesRecorder.instance));
+}
+
 /**
  * @param allowIndexes Allow index CF names to be passed in
  * @param autoAddIndexes Automatically add secondary indexes if a CF has 
them

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index b8582a3..1351fea 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -248,6 +248,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 public long trueSnapshotsSize();
 
 /**
+ * Forces refresh of values stored in system.size_estimates of all column 
families.
+ */
+public void refreshSizeEstimates() throws ExecutionException;
+
+/**
  * Forces major compaction of a single keyspace
  */
 public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index 6f2b6fb..49c493d 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java

[3/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[aleksey]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9f056a91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9f056a91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9f056a91

Branch: refs/heads/cassandra-3.1
Commit: 9f056a91754b3516ad1521d32abb075b554e62c3
Parents: b54836b 84d4488
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 18:05:54 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 18:05:54 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  5 +++
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 58 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9f056a91/CHANGES.txt
--
diff --cc CHANGES.txt
index 8f1b163,9c834f3..d554323
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,51 -1,6 +1,52 @@@
 -2.2.4
 +3.0.1
 + * Keep the file open in trySkipCache (CASSANDRA-10669)
 + * Updated trigger example (CASSANDRA-10257)
 +Merged from 2.2:
   * (Hadoop) fix splits calculation (CASSANDRA-10640)
   * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 +Merged from 2.1:
++ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
 + * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
 + * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
 + * Remove superfluous COUNTER_MUTATION stage mapping (CASSANDRA-10605)
 +
 +
 +3.0
 + * Fix AssertionError while flushing memtable due to materialized views
 +   incorrectly inserting empty rows (CASSANDRA-10614)
 + * Store UDA initcond as CQL literal in the schema table, instead of a blob 
(CASSANDRA-10650)
 + * Don't use -1 for the position of partition key in schema (CASSANDRA-10491)
 + * Fix distinct queries in mixed version cluster (CASSANDRA-10573)
 + * Skip sstable on clustering in names query (CASSANDRA-10571)
 + * Remove value skipping as it breaks read-repair (CASSANDRA-10655)
 + * Fix bootstrapping with MVs (CASSANDRA-10621)
 + * Make sure EACH_QUORUM reads are using NTS (CASSANDRA-10584)
 + * Fix MV replica filtering for non-NetworkTopologyStrategy (CASSANDRA-10634)
 + * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
 + * Fix reading of legacy sstables (CASSANDRA-10590)
 + * Use CQL type names in schema metadata tables (CASSANDRA-10365)
 + * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
 + * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)
 + * Add client address argument to IAuthenticator::newSaslNegotiator 
(CASSANDRA-8068)
 + * Fix implementation of LegacyLayout.LegacyBoundComparator (CASSANDRA-10602)
 + * Don't use 'names query' read path for counters (CASSANDRA-10572)
 + * Fix backward compatibility for counters (CASSANDRA-10470)
 + * Remove memory_allocator paramter from cassandra.yaml 
(CASSANDRA-10581,10628)
 + * Execute the metadata reload task of all registered indexes on CFS::reload 
(CASSANDRA-10604)
 + * Fix thrift cas operations with defined columns (CASSANDRA-10576)
 + * Fix PartitionUpdate.operationCount()for updates with static column 
operations (CASSANDRA-10606)
 + * Fix thrift get() queries with defined columns (CASSANDRA-10586)
 + * Fix marking of indexes as built and removed (CASSANDRA-10601)
 + * Skip initialization of non-registered 2i instances, remove 
Index::getIndexName (CASSANDRA-10595)
 + * Fix batches on multiple tables (CASSANDRA-10554)
 + * Ensure compaction options are validated when updating KeyspaceMetadata 
(CASSANDRA-10569)
 + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975)
 + * Remove token generator (CASSANDRA-5261)
 + * RolesCache should not be created for any authenticator that does not 
requireAuthentication (CASSANDRA-10562)
 + * Fix LogTransaction checking only a single directory for files 
(CASSANDRA-10421)
 + * Fix handling of range tombstones when reading old format sstables 
(CASSANDRA-10360)
 + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367)
 +Merged from 2.2:
   * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
   * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
   * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)

[1/5] cassandra git commit: Add nodetool command to refresh system.size_estimates

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 7cb33aa59 -> 7fcf14faa


Add nodetool command to refresh system.size_estimates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecd66de2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecd66de2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecd66de2

Branch: refs/heads/trunk
Commit: ecd66de2bdf2ecb88c4276a4616a76f5ec2921bb
Parents: 9ab1c83
Author: Carl Yeksigian 
Authored: Tue Nov 10 17:22:11 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:53:33 2015 +

--
 CHANGES.txt |  2 ++
 .../org/apache/cassandra/service/StorageService.java|  6 ++
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 src/java/org/apache/cassandra/tools/NodeProbe.java  | 12 
 src/java/org/apache/cassandra/tools/NodeTool.java   | 11 +++
 5 files changed, 36 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d0c0af9..2eeda94 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.12
+ * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
  * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
  * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
  * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)
@@ -33,6 +34,7 @@
  * Mark nodes as dead even if they've already left (CASSANDRA-10205)
  * Update internal python driver used by cqlsh (CASSANDRA-10161, 
CASSANDRA-10507)
 
+
 2.1.10
  * Bulk Loader API could not tolerate even node failure (CASSANDRA-10347)
  * Avoid misleading pushed notifications when multiple nodes

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 93b1b97..665ce3a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -83,6 +83,7 @@ import org.apache.cassandra.db.MutationVerbHandler;
 import org.apache.cassandra.db.ReadRepairVerbHandler;
 import org.apache.cassandra.db.ReadVerbHandler;
 import org.apache.cassandra.db.SchemaCheckVerbHandler;
+import org.apache.cassandra.db.SizeEstimatesRecorder;
 import org.apache.cassandra.db.SnapshotDetailsTabularData;
 import org.apache.cassandra.db.SystemKeyspace;
 import org.apache.cassandra.db.TruncateVerbHandler;
@@ -2604,6 +2605,11 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return total;
 }
 
+public void refreshSizeEstimates() throws ExecutionException
+{
+
FBUtilities.waitOnFuture(ScheduledExecutors.optionalTasks.submit(SizeEstimatesRecorder.instance));
+}
+
 /**
  * @param allowIndexes Allow index CF names to be passed in
  * @param autoAddIndexes Automatically add secondary indexes if a CF has 
them

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index b8582a3..1351fea 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -248,6 +248,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 public long trueSnapshotsSize();
 
 /**
+ * Forces refresh of values stored in system.size_estimates of all column 
families.
+ */
+public void refreshSizeEstimates() throws ExecutionException;
+
+/**
  * Forces major compaction of a single keyspace
  */
 public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecd66de2/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index 6f2b6fb..49c493d 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java
@@ -901,6 

[2/5] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[aleksey]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84d44885
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84d44885
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84d44885

Branch: refs/heads/trunk
Commit: 84d4488537605a86373325f86bbc4cd358921223
Parents: d84b42b ecd66de
Author: Aleksey Yeschenko 
Authored: Thu Nov 12 17:58:38 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 12 17:58:38 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/service/StorageService.java   |  6 
 .../cassandra/service/StorageServiceMBean.java  |  5 +++
 .../org/apache/cassandra/tools/NodeProbe.java   | 12 +++
 .../org/apache/cassandra/tools/NodeTool.java|  3 +-
 .../tools/nodetool/RefreshSizeEstimates.java| 33 
 6 files changed, 59 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/CHANGES.txt
--
diff --cc CHANGES.txt
index 2cfcb55,2eeda94..9c834f3
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -2.1.12
 +2.2.4
 + * (Hadoop) fix splits calculation (CASSANDRA-10640)
 + * (Hadoop) ensure that Cluster instances are always closed (CASSANDRA-10058)
 + * (cqlsh) show partial trace if incomplete after max_trace_wait 
(CASSANDRA-7645)
 + * Use most up-to-date version of schema for system tables (CASSANDRA-10652)
 + * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581,10628)
 + * Expose phi values from failure detector via JMX and tweak debug
 +   and trace logging (CASSANDRA-9526)
 + * Fix RangeNamesQueryPager (CASSANDRA-10509)
 + * Deprecate Pig support (CASSANDRA-10542)
 + * Reduce contention getting instances of CompositeType (CASSANDRA-10433)
 +Merged from 2.1:
+  * Add a nodetool command to refresh size_estimates (CASSANDRA-9579)
   * Shutdown compaction in drain to prevent leak (CASSANDRA-10079)
   * Invalidate cache after stream receive task is completed (CASSANDRA-10341)
   * Reject counter writes in CQLSSTableWriter (CASSANDRA-10258)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --cc src/java/org/apache/cassandra/service/StorageServiceMBean.java
index 2c92d43,1351fea..00060ae
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@@ -239,9 -248,14 +239,14 @@@ public interface StorageServiceMBean ex
  public long trueSnapshotsSize();
  
  /**
+  * Forces refresh of values stored in system.size_estimates of all column 
families.
+  */
+ public void refreshSizeEstimates() throws ExecutionException;
+ 
+ /**
   * Forces major compaction of a single keyspace
   */
 -public void forceKeyspaceCompaction(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException;
 +public void forceKeyspaceCompaction(boolean splitOutput, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException;
  
  /**
   * Trigger a cleanup of keys on a single keyspace

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeProbe.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84d44885/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --cc src/java/org/apache/cassandra/tools/NodeTool.java
index 175b325,1d4a420..fcb6ed0
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@@ -129,23 -187,14 +129,24 @@@ public class NodeToo
  TpStats.class,
  TopPartitions.class,
  SetLoggingLevel.class,
 -GetLoggingLevels.class
 +GetLoggingLevels.class,
- FailureDetectorInfo.class
++FailureDetectorInfo.class,
++RefreshSizeEstimates.class
  );
  
 -Cli parser = Cli.builder("nodetool")
 -.withDescription("Manage your Cassandra cluster")
 +Cli.CliBuilder builder = Cli.builder("nodetool");
 +
 +builder.withDescription("Manage your