[jira] [Updated] (CASSANDRA-10430) "Load" report from "nodetool status" is inaccurate

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10430:

Reproduced In: 2.1.9
Fix Version/s: 2.1.x

> "Load" report from "nodetool status" is inaccurate
> --
>
> Key: CASSANDRA-10430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
> enabled. 
>Reporter: julia zhang
> Fix For: 2.1.x
>
>
> After running an incremental repair, nodetool status report unbalanced load 
> among cluster. 
> $ nodetool status mykeyspace
> ==
> ||Status|| Address ||Load   ||Tokens  ||Owns (effective)  
> ||Host ID ||  Rack || 
> |UN  |10.1.1.1  |1.13 TB   |256|48.5%
> |a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
> |UN  |10.1.1.2  |2.58 TB   |256 |50.5% 
> |1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
> |UN  |10.1.1.3  |1.49 TB   |256 |51.5% 
> |27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
> |UN  |10.1.1.4  |250.97 GB  |256 |51.9% 
> |9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
> |UN  |10.1.1.5 |1.88 TB  |256 |49.5% 
> |04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
> |UN  |10.1.1.6 |1.3 TB|256 |48.1% 
> |6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|
> It seems that only 10.1.1.4 reports correct "Load". There is no hints in the 
> cluster and report remains the same after running "nodetool cleanup" on each 
> node. "nodetool cfstats" shows number of keys are evenly distributed and 
> Cassandra data physical disk on each node report about the same usage. 
> "nodetool status" report these inaccurate large storage load until we restart 
> each node, after the restart, "Load" report match what we've seen from disk.  
> We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10430) "Load" report from "nodetool status" is inaccurate

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943714#comment-14943714
 ] 

Philip Thompson commented on CASSANDRA-10430:
-

You say that after restart the numbers become correct. How long after that 
until they become invalid again? Can you include the system.log from one of the 
nodes this is affecting?

> "Load" report from "nodetool status" is inaccurate
> --
>
> Key: CASSANDRA-10430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
> enabled. 
>Reporter: julia zhang
> Fix For: 2.1.x
>
>
> After running an incremental repair, nodetool status report unbalanced load 
> among cluster. 
> $ nodetool status mykeyspace
> ==
> ||Status|| Address ||Load   ||Tokens  ||Owns (effective)  
> ||Host ID ||  Rack || 
> |UN  |10.1.1.1  |1.13 TB   |256|48.5%
> |a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
> |UN  |10.1.1.2  |2.58 TB   |256 |50.5% 
> |1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
> |UN  |10.1.1.3  |1.49 TB   |256 |51.5% 
> |27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
> |UN  |10.1.1.4  |250.97 GB  |256 |51.9% 
> |9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
> |UN  |10.1.1.5 |1.88 TB  |256 |49.5% 
> |04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
> |UN  |10.1.1.6 |1.3 TB|256 |48.1% 
> |6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|
> It seems that only 10.1.1.4 reports correct "Load". There is no hints in the 
> cluster and report remains the same after running "nodetool cleanup" on each 
> node. "nodetool cfstats" shows number of keys are evenly distributed and 
> Cassandra data physical disk on each node report about the same usage. 
> "nodetool status" report these inaccurate large storage load until we restart 
> each node, after the restart, "Load" report match what we've seen from disk.  
> We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-7276) Include keyspace and table names in logs where possible

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta reassigned CASSANDRA-7276:
--

Assignee: Paulo Motta  (was: Nitzan Volman)

> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Tyler Hobbs
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 2.1-CASSANDRA-7276-v1.txt, 
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt, 
> cassandra-2.1.9-7276-v2.txt, cassandra-2.1.9-7276.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or 
> table was causing the problem.  For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java 
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at 
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at 
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error 
> messages or logs whenever possible.  This includes reads, writes, 
> compactions, flushes, repairs, and probably more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943824#comment-14943824
 ] 

Fernando Gonçalves commented on CASSANDRA-10233:


I think that jvm assertions should be disabled in production, if we need to 
validate something in production it should be done like [~eitikimura] said.

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10430) "Load" report from "nodetool status" is inaccurate

[julia zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943830#comment-14943830
 ] 

julia zhang commented on CASSANDRA-10430:
-

We have since switched to full repair (nodetool repair -pr keyspace), and 
nodetool no longer reports invalid "Load".

> "Load" report from "nodetool status" is inaccurate
> --
>
> Key: CASSANDRA-10430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
> enabled. 
>Reporter: julia zhang
> Fix For: 2.1.x
>
> Attachments: system.log.2.zip, system.log.3.zip, system.log.4.zip
>
>
> After running an incremental repair, nodetool status report unbalanced load 
> among cluster. 
> $ nodetool status mykeyspace
> ==
> ||Status|| Address ||Load   ||Tokens  ||Owns (effective)  
> ||Host ID ||  Rack || 
> |UN  |10.1.1.1  |1.13 TB   |256|48.5%
> |a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
> |UN  |10.1.1.2  |2.58 TB   |256 |50.5% 
> |1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
> |UN  |10.1.1.3  |1.49 TB   |256 |51.5% 
> |27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
> |UN  |10.1.1.4  |250.97 GB  |256 |51.9% 
> |9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
> |UN  |10.1.1.5 |1.88 TB  |256 |49.5% 
> |04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
> |UN  |10.1.1.6 |1.3 TB|256 |48.1% 
> |6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|
> It seems that only 10.1.1.4 reports correct "Load". There is no hints in the 
> cluster and report remains the same after running "nodetool cleanup" on each 
> node. "nodetool cfstats" shows number of keys are evenly distributed and 
> Cassandra data physical disk on each node report about the same usage. 
> "nodetool status" report these inaccurate large storage load until we restart 
> each node, after the restart, "Load" report match what we've seen from disk.  
> We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8616) sstable2json may result in commit log segments be written

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8616:
--
Reproduced In: 2.1.3, 2.0.10  (was: 2.0.10, 2.1.3)
 Priority: Critical  (was: Major)

Bumping to critical since this can prevent startup when sstable2json creates a 
new segment as root. 

> sstable2json may result in commit log segments be written
> -
>
> Key: CASSANDRA-8616
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8616
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Yuki Morishita
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: 8161-2.0.txt
>
>
> There was a report of sstable2json causing commitlog segments to be written 
> out when run.  I haven't attempted to reproduce this yet, so that's all I 
> know for now.  Since sstable2json loads the conf and schema, I'm thinking 
> that it may inadvertently be triggering the commitlog code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9602) EACH_QUORUM READ support needed

[Daniel Cohen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943818#comment-14943818
 ] 

Daniel Cohen commented on CASSANDRA-9602:
-

[~oseiler]: [~stewartg] and I agree that we all want EACH_QUORUM reads.

I also agree that this was in fact never supported in Cassandra, and at the 
very least the C* v2.0 documentation was a little confusing. Check out this 
link to the OSS C* code:

https://git1-us-west.apache.org/repos/asf?p=cassandra.git;a=commitdiff;h=72199e23ec9d604449bef87733a32e1da9924437

In particular there is a comment regarding a change as of C* v1.1:

{quote}EACH_QUORUM ConsistencyLevel is only supported for writes and will now
+  throw an InvalidRequestException when used for reads.  (Previous
+  versions would silently perform a LOCAL_QUORUM read instead.){quote}

> EACH_QUORUM READ support needed
> ---
>
> Key: CASSANDRA-9602
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9602
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Scott Guminy
>Assignee: Carl Yeksigian
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> EACH_QUORUM consistency for READ should be added.
> This bug https://issues.apache.org/jira/browse/CASSANDRA-3272 says it is not 
> needed ever, however I have a use case where I need it.  I think the decision 
> made was incorrect. Here's why...
>  
>  My application has two key pieces:
>  
>  # *End user actions* which add/modify data in the system.  End users 
> typically access the application from only one Data Center and only see their 
> own data
> # *Scheduled business logic tasks* which run from any node in any data 
> center.  These tasks process data added by the end users in an asynchronous 
> way
>  
>  *End user actions must have the highest degree of availability.*  Users must 
> always be able to add data to the system.  The data will be processed later.  
> To support this, end user actions will use *LOCAL_QUORUM Read and Write 
> Consistency*.
>  
>  Scheduled tasks don't need to have a high degree of availability but MUST 
> operate on the most up to date data.  *The tasks will run with EACH_QUORUM* 
> to ensure that no matter how many data centers we have, we always READ the 
> latest data.  This approach allows us some amount of fault tolerance. 
>  
>  The problem is that EACH_QUORUM is not a valid READ consistency level.  This 
> mean I have no alternative but to use ALL.  ALL will work, but is not the 
> best since it offers support for ZERO failures.  I would prefer EACH_QUORUM 
> since it can support some failures in each data center.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10430) "Load" report from "nodetool status" is inaccurate

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943835#comment-14943835
 ] 

Philip Thompson commented on CASSANDRA-10430:
-

[~yukim], any idea what the issue could be?

> "Load" report from "nodetool status" is inaccurate
> --
>
> Key: CASSANDRA-10430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
> enabled. 
>Reporter: julia zhang
> Fix For: 2.1.x
>
> Attachments: system.log.2.zip, system.log.3.zip, system.log.4.zip
>
>
> After running an incremental repair, nodetool status report unbalanced load 
> among cluster. 
> $ nodetool status mykeyspace
> ==
> ||Status|| Address ||Load   ||Tokens  ||Owns (effective)  
> ||Host ID ||  Rack || 
> |UN  |10.1.1.1  |1.13 TB   |256|48.5%
> |a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
> |UN  |10.1.1.2  |2.58 TB   |256 |50.5% 
> |1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
> |UN  |10.1.1.3  |1.49 TB   |256 |51.5% 
> |27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
> |UN  |10.1.1.4  |250.97 GB  |256 |51.9% 
> |9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
> |UN  |10.1.1.5 |1.88 TB  |256 |49.5% 
> |04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
> |UN  |10.1.1.6 |1.3 TB|256 |48.1% 
> |6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|
> It seems that only 10.1.1.4 reports correct "Load". There is no hints in the 
> cluster and report remains the same after running "nodetool cleanup" on each 
> node. "nodetool cfstats" shows number of keys are evenly distributed and 
> Cassandra data physical disk on each node report about the same usage. 
> "nodetool status" report these inaccurate large storage load until we restart 
> each node, after the restart, "Load" report match what we've seen from disk.  
> We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10393) LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref)

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10393:

Fix Version/s: 2.2.x

> LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref)
> --
>
> Key: CASSANDRA-10393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10393
> Project: Cassandra
>  Issue Type: Bug
> Environment: v 2.2.1 (from apt)
> -> lsb_release -a
> No LSB modules are available.
> Distributor ID:   Debian
> Description:  Debian GNU/Linux 7.8 (wheezy)
> Release:  7.8
> Codename: wheezy
> -> java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>Reporter: Christian Winther
>  Labels: memory-leak, sstable
> Fix For: 2.2.x
>
>
> When trying to repair full on a table with the following schema my nodes 
> stall 
> and end up with spamming this 
> I've recently changed the table from SizeTieredCompactionStrategy to 
> LeveledCompactionStrategy.
> Coming from 2.1.9 -> 2.2.0 -> 2.2.1 i ran upgradesstable without issue as well
> When trying to full repair post compaction change, I got "out of order" 
> errors. A few google searches later, I was told to "scrub" the keyspace - did 
> that during the night (no problems logged, and no data lost)
> Now a repair just stalls and output memory leaks all over the place 
> {code}
> CREATE KEYSPACE sessions WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '3'}  AND durable_writes = true;
> CREATE TABLE sessions.sessions (
> id text PRIMARY KEY,
> client_ip text,
> controller text,
> controller_action text,
> created timestamp,
> data text,
> expires timestamp,
> http_host text,
> modified timestamp,
> request_agent text,
> request_agent_bot boolean,
> request_path text,
> site_id int,
> user_id int
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"NONE", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> {code}
> ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@4428a373) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@184765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@368dd97) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@184765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@66fb78be) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@184765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@9fdd2e6) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big
>  was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@84fcb91) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big
>  was not released before the reference was garbage collected
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10401) Improve json2sstable error reporting on nonexistent column

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10401:
-
Assignee: Paulo Motta

> Improve json2sstable error reporting on nonexistent column
> --
>
> Key: CASSANDRA-10401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10401
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra 2.1.8.621
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
>
> We have the following table...
> {noformat}
> CREATE TABLE keyspace_name.table_name (
> col1 text,
> col2 text,
> col3 text,
> col4 text,
> PRIMARY KEY ((col1, col2), col3)
> ) WITH CLUSTERING ORDER BY (col3 ASC)
> {noformat}
> And the following  json in a file created from sstable2json tool
> {noformat}
> [
> {"key": "This is col1:This is col2,
>  "cells": [["This is col3:","",1443217787319002],
>["This is col3:"col4","This is col4",1443217787319002]]}
> ]
> {noformat}
> Let's say we deleted that record form the DB and wanted to bring it back
> If we try to create an sstable from this data in a json file named 
> test_file.json, we get a NPE 
> {noformat}
> -bash-4.1$ json2sstable -K elp -c table_name-3264cbe063c211e5bc34e746786b7b29 
> test_file.json  
> /var/lib/cassandra/data/keyspace_name/table_name-3264cbe063c211e5bc34e746786b7b29/keyspace_name-table_name-ka-1-Data.db
> Importing 1 keys...
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.tools.SSTableImport.getKeyValidator(SSTableImport.java:442)
>   at 
> org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:316)
>   at 
> org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:287)
>   at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:514)
> ERROR: null
> -bash-4.1$
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-10421:
---
Reviewer: Ariel Weisberg

> Potential issue with LogTransaction as it only checks in a single directory 
> for files
> -
>
> Key: CASSANDRA-10421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Stefania
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> When creating a new LogTransaction we try to create the new logfile in the 
> same directory as the one we are writing to, but as we use 
> {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
>  this might end up in "any" of the configured data directories. If it does, 
> we will not be able to clean up leftovers as we check for files in the same 
> directory as the logfile was created: 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7276) Include keyspace and table names in logs where possible

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-7276:
---
Assignee: Nitzan Volman  (was: Paulo Motta)

> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Tyler Hobbs
>Assignee: Nitzan Volman
>Priority: Minor
>  Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 2.1-CASSANDRA-7276-v1.txt, 
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt, 
> cassandra-2.1.9-7276-v2.txt, cassandra-2.1.9-7276.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or 
> table was causing the problem.  For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java 
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at 
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at 
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at 
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at 
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error 
> messages or logs whenever possible.  This includes reads, writes, 
> compactions, flushes, repairs, and probably more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10430) "Load" report from "nodetool status" is inaccurate

[julia zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

julia zhang updated CASSANDRA-10430:

Attachment: system.log.4.zip
system.log.3.zip
system.log.2.zip

Attached system log files from a node, whose "Load" became 1.1TB after running 
incremental repair

> "Load" report from "nodetool status" is inaccurate
> --
>
> Key: CASSANDRA-10430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10430
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra v2.1.9 running on 6 node Amazon AWS, vnodes 
> enabled. 
>Reporter: julia zhang
> Fix For: 2.1.x
>
> Attachments: system.log.2.zip, system.log.3.zip, system.log.4.zip
>
>
> After running an incremental repair, nodetool status report unbalanced load 
> among cluster. 
> $ nodetool status mykeyspace
> ==
> ||Status|| Address ||Load   ||Tokens  ||Owns (effective)  
> ||Host ID ||  Rack || 
> |UN  |10.1.1.1  |1.13 TB   |256|48.5%
> |a4477534-a5c6-4e3e-9108-17a69aebcfc0|  RAC1|
> |UN  |10.1.1.2  |2.58 TB   |256 |50.5% 
> |1a7c3864-879f-48c5-8dde-bc00cf4b23e6  |RAC2|
> |UN  |10.1.1.3  |1.49 TB   |256 |51.5% 
> |27df5b30-a5fc-44a5-9a2c-1cd65e1ba3f7  |RAC1|
> |UN  |10.1.1.4  |250.97 GB  |256 |51.9% 
> |9898a278-2fe6-4da2-b6dc-392e5fda51e6  |RAC3|
> |UN  |10.1.1.5 |1.88 TB  |256 |49.5% 
> |04aa9ce1-c1c3-4886-8d72-270b024b49b9  |RAC2|
> |UN  |10.1.1.6 |1.3 TB|256 |48.1% 
> |6d5d48e6-d188-4f88-808d-dcdbb39fdca5  |RAC3|
> It seems that only 10.1.1.4 reports correct "Load". There is no hints in the 
> cluster and report remains the same after running "nodetool cleanup" on each 
> node. "nodetool cfstats" shows number of keys are evenly distributed and 
> Cassandra data physical disk on each node report about the same usage. 
> "nodetool status" report these inaccurate large storage load until we restart 
> each node, after the restart, "Load" report match what we've seen from disk.  
> We did not see this behavior until upgrade to v2.1.9



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10437) Remove offheap_objects option until 9472 re-introduces them

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-10437:
---
Reviewer: Ariel Weisberg

> Remove offheap_objects option until 9472 re-introduces them
> ---
>
> Key: CASSANDRA-10437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10437
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Trivial
> Fix For: 3.0.0 rc2
>
> Attachments: 0001-Remove-offheap_objects-option.txt
>
>
> We need to send a meaningful error message if user try to use 
> {{offheap_objects}} in 3.0 since it's not supported currently (pending 
> CASSANDRA-9472). And document the current removal in the NEWS file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9602) EACH_QUORUM READ support needed

[Oliver Seiler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943807#comment-14943807
 ] 

Oliver Seiler commented on CASSANDRA-9602:
--

As the author of CASSANDRA-6970, I'm interested in this having a different 
outcome; I would still like EACH_QUORUM READs, as it would eliminate relying on 
CL=ALL in some places. I have a similar use-case as the reporter (doing 
CL=EACH_QUORUM writes is just out of the question for our environments...)

> In (certain) versions 2.0 this was supported:

It has never been supported, AFAIK; the docs for 2.0 are wrong, something I 
complained about a lot to whomever could stand my ranting (you get an error in 
2.0 trying to run an EACH_QUORUM read).

> EACH_QUORUM READ support needed
> ---
>
> Key: CASSANDRA-9602
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9602
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Scott Guminy
>Assignee: Carl Yeksigian
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> EACH_QUORUM consistency for READ should be added.
> This bug https://issues.apache.org/jira/browse/CASSANDRA-3272 says it is not 
> needed ever, however I have a use case where I need it.  I think the decision 
> made was incorrect. Here's why...
>  
>  My application has two key pieces:
>  
>  # *End user actions* which add/modify data in the system.  End users 
> typically access the application from only one Data Center and only see their 
> own data
> # *Scheduled business logic tasks* which run from any node in any data 
> center.  These tasks process data added by the end users in an asynchronous 
> way
>  
>  *End user actions must have the highest degree of availability.*  Users must 
> always be able to add data to the system.  The data will be processed later.  
> To support this, end user actions will use *LOCAL_QUORUM Read and Write 
> Consistency*.
>  
>  Scheduled tasks don't need to have a high degree of availability but MUST 
> operate on the most up to date data.  *The tasks will run with EACH_QUORUM* 
> to ensure that no matter how many data centers we have, we always READ the 
> latest data.  This approach allows us some amount of fault tolerance. 
>  
>  The problem is that EACH_QUORUM is not a valid READ consistency level.  This 
> mean I have no alternative but to use ALL.  ALL will work, but is not the 
> best since it offers support for ZERO failures.  I would prefer EACH_QUORUM 
> since it can support some failures in each data center.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943821#comment-14943821
 ] 

Fernando Gonçalves commented on CASSANDRA-10233:


[~nutbunnies]: the JVM assertions are disabled!
We just follow the following comment in order to improve the performance: (we 
considered that it was a good practice). 
https://github.com/apache/cassandra/blob/cassandra-2.1/conf/cassandra-env.sh#L172

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10381) NullPointerException in cqlsh paging through CF with static columns

[Michael Keeney (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943895#comment-14943895
 ] 

Michael Keeney commented on CASSANDRA-10381:


[~blerer] Is there a good way to query for this specific type of partition?

> NullPointerException in cqlsh paging through CF with static columns
> ---
>
> Key: CASSANDRA-10381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Michael Keeney
>Assignee: Benjamin Lerer
>  Labels: cqlsh, nullpointerexception, range
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> When running select count( * ) from cqlsh with limit, the following NPE 
> occurs:
> select count( * ) from tbl1 limit 5 ; 
> {code}
> ERROR [SharedPool-Worker-4] 2015-09-16 14:49:43,480 QueryMessage.java:132 - 
> Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.containsPreviousLast(RangeSliceQueryPager.java:99)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.fetchPage(RangeSliceQueryPager.java:37)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.pageCountQuery(SelectStatement.java:286)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:230)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:67)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$StatementExecution.execute(DseQueryHandler.java:291)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithTiming(DseQueryHandler.java:223)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithAuditLogging(DseQueryHandler.java:259)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler.process(DseQueryHandler.java:94)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_75]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
> {code}
> Table definition looks something like:
> {code}
> CREATE TABLE tbl1 (
> field1 bigint,
> field2 int,
> field3 timestamp,
> field4 map,
> field5 text static,
> field6 text static,
> field7 text static
> PRIMARY KEY (field1, field2, field3)
> ) WITH CLUSTERING ORDER BY (field2 ASC, field3 ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>...
> {code}
> Following appears in debug log leading up to the error:
> {code}
> DEBUG [SharedPool-Worker-1] 2015-09-17 15:32:06,484  
> AbstractQueryPager.java:95 - Fetched 101 live 

[jira] [Commented] (CASSANDRA-10381) NullPointerException in cqlsh paging through CF with static columns

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943910#comment-14943910
 ] 

Benjamin Lerer commented on CASSANDRA-10381:


[~Michael Keeney] not that I am aware of.
You will have to scan the entire table and check the static columns and the 
clustering keys. 

> NullPointerException in cqlsh paging through CF with static columns
> ---
>
> Key: CASSANDRA-10381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Michael Keeney
>Assignee: Benjamin Lerer
>  Labels: cqlsh, nullpointerexception, range
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> When running select count( * ) from cqlsh with limit, the following NPE 
> occurs:
> select count( * ) from tbl1 limit 5 ; 
> {code}
> ERROR [SharedPool-Worker-4] 2015-09-16 14:49:43,480 QueryMessage.java:132 - 
> Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.containsPreviousLast(RangeSliceQueryPager.java:99)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.service.pager.RangeSliceQueryPager.fetchPage(RangeSliceQueryPager.java:37)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.pageCountQuery(SelectStatement.java:286)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:230)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:67)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$StatementExecution.execute(DseQueryHandler.java:291)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithTiming(DseQueryHandler.java:223)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithAuditLogging(DseQueryHandler.java:259)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> com.datastax.bdp.cassandra.cql3.DseQueryHandler.process(DseQueryHandler.java:94)
>  ~[dse-4.7.2.jar:4.7.2]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
>  ~[cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_75]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [cassandra-all-2.1.8.621.jar:2.1.8.621]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
> {code}
> Table definition looks something like:
> {code}
> CREATE TABLE tbl1 (
> field1 bigint,
> field2 int,
> field3 timestamp,
> field4 map,
> field5 text static,
> field6 text static,
> field7 text static
> PRIMARY KEY (field1, field2, field3)
> ) WITH CLUSTERING ORDER BY (field2 ASC, field3 ASC)
> AND bloom_filter_fp_chance = 0.1
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>...
> {code}
> Following appears in debug log leading up to the error:
> {code}
> DEBUG [SharedPool-Worker-1] 2015-09-17 

[jira] [Created] (CASSANDRA-10445) Cassandra-stress throws max frame size error when SSL certification is enabled

[Sam Goldberg (JIRA)]

Sam Goldberg created CASSANDRA-10445:


 Summary: Cassandra-stress throws max frame size error when SSL 
certification is enabled
 Key: CASSANDRA-10445
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10445
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Goldberg


Running cassandra-stress when SSL is enabled gives the following error and does 
not finish executing:

cassandra-stress write n=100
Exception in thread "main" java.lang.RuntimeException: 
org.apache.thrift.transport.TTransportException: Frame size (352518912) larger 
than max length (15728640)!
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:144)
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:110)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesThrift(SettingsSchema.java:111)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:59)
at 
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:205)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:55)
at org.apache.cassandra.stress.Stress.main(Stress.java:109)


I was able to reproduce this issue consistently via the following steps:
1) Spin up 3 node cassandra cluster running 2.1.8
2) Perform cassandra-stress write n=100
3) Everything works!
4) Generate keystore and truststore for each node in the cluster and distribute 
appropriately 
5) Modify cassandra.yaml on each node to enable SSL:
client_encryption_options:
enabled: true
keystore: /
# require_client_auth: false
# Set trustore and truststore_password if require_client_auth is true
truststore:  /
truststore_password: 
# More advanced defaults below:
protocol: ssl
6) Restart each node.
7) Perform cassandra-stress write n=100
8) Get Frame Size error, cassandra-stress fails

This may be related to CASSANDRA-9325.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10444) Create an option to forcibly disable tracing

[Brandon Williams (JIRA)]

Brandon Williams created CASSANDRA-10444:


 Summary: Create an option to forcibly disable tracing
 Key: CASSANDRA-10444
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10444
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Priority: Minor


Sometimes people will experience dropped TRACE messages.  Ostensibly, trace is 
disabled on the server and we know it's from some client, somewhere.  With an 
inability to locate exactly where client code is causing this, it would be 
useful to just be able to kill it entirely on the server side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10444) Create an option to forcibly disable tracing

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-10444:
-
 Assignee: Joshua McKenzie
Fix Version/s: 2.1.x

> Create an option to forcibly disable tracing
> 
>
> Key: CASSANDRA-10444
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10444
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Joshua McKenzie
>Priority: Minor
> Fix For: 2.1.x
>
>
> Sometimes people will experience dropped TRACE messages.  Ostensibly, trace 
> is disabled on the server and we know it's from some client, somewhere.  With 
> an inability to locate exactly where client code is causing this, it would be 
> useful to just be able to kill it entirely on the server side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9748) Can't see other nodes when using multiple network interfaces

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943965#comment-14943965
 ] 

Paulo Motta commented on CASSANDRA-9748:


[~RomanB] I've attached a [github 
branch|https://github.com/pauloricardomg/cassandra/tree/9478-2.1] that allows 
setting listen_address to 0.0.0.0, could you please test that? I've tested with 
my wlan0 and eth0 interfaces and it seems to work. After cloning the repository 
you may use {{ant jar}} to create jars (and replace them manually), or {{ant 
artifacts}} to create a cassandra tarball which you can then decompress to 
perform the test.

In addition to setting the {{listen_address}} to {{0.0.0.0}} you should enable 
the {{GossipingPropertyFileSnitch}} option {{prefer_local}} and also set the 
new option of this snitch {{local_address}} to the local ip adress the node 
should be contacted in the private network.

In summary, your configuration should look like:
* *seeds*: public IP (same as used in broadcast_address)
* *listen_address*: 0.0.0.0
* *broadcast_address*: public IP
* *rpc_address*: 0.0.0.0
* *broadcast_rpc_address*: public IP
* *endpoint_snitch*: GossipingPropertyFileSnitch
** *prefer_local*: true
** *local_address*: private IP
 

> Can't see other nodes when using multiple network interfaces
> 
>
> Key: CASSANDRA-9748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9748
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.0.16; multi-DC configuration
>Reporter: Roman Bielik
>Assignee: Paulo Motta
> Attachments: system_node1.log, system_node2.log
>
>
> The idea is to setup a multi-DC environment across 2 different networks based 
> on the following configuration recommendations:
> http://docs.datastax.com/en/cassandra/2.0/cassandra/configuration/configMultiNetworks.html
> Each node has 2 network interfaces. One used as a private network (DC1: 
> 10.0.1.x and DC2: 10.0.2.x). The second one a "public" network where all 
> nodes can see each other (this one has a higher latency). 
> Using the following settings in cassandra.yaml:
> *seeds:* public IP (same as used in broadcast_address)
> *listen_address:* private IP
> *broadcast_address:* public IP
> *rpc_address:* 0.0.0.0
> *endpoint_snitch:* GossipingPropertyFileSnitch
> _(tried different combinations with no luck)_
> No firewall and no SSL/encryption used.
> The problem is that nodes do not see each other (a gossip problem I guess). 
> The nodetool ring/status shows only the local node but not the other ones 
> (even from the same DC).
> When I set listen_address to public IP, then everything works fine, but that 
> is not the required configuration.
> _Note: Not using EC2 cloud!_
> netstat -anp | grep -E "(7199|9160|9042|7000)"
> tcp0  0 0.0.0.0:71990.0.0.0:*   
> LISTEN  3587/java   
> tcp0  0 10.0.1.1:9160   0.0.0.0:*   
> LISTEN  3587/java   
> tcp0  0 10.0.1.1:9042   0.0.0.0:*   
> LISTEN  3587/java   
> tcp0  0 10.0.1.1:7000   0.0.0.0:*   
> LISTEN  3587/java   
> tcp0  0 127.0.0.1:7199  127.0.0.1:52874 
> ESTABLISHED 3587/java   
> tcp0  0 10.0.1.1:7199   10.0.1.1:39650  
> ESTABLISHED 3587/java 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10424) Altering base table column with materialized view causes unexpected server error.

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943005#comment-14943005
 ] 

Sylvain Lebresne commented on CASSANDRA-10424:
--

bq. there shouldn't be a case where we validate on the base table but not the 
view, I think.

There is, if the type we alter to is "value-compatible" with the old type but 
not "compatible" (that is, all values of the old type are valid for the new 
type, but the new type don't sort like the old one). In that case, altering the 
base table would be ok (since the column is not a clustering column), but 
altering the view wouldn't. An example of such transition is int -> blob. Can 
you add a test for that.

Two other small remarks:
* In the {{ALTER TABLE}} tests, there is this comment before every MV creation: 
{{// Cannot use SELECT \*, as those are always handled by the includeAll 
shortcut in View.updateAffectsView}}.  Could you explain how this has any 
impact on {{ALTER}}? If it doesn't, then we should probably remove the comment 
and switch back to {{\*}} so no-one thinks there is something subtle. If it 
does make a difference, we should still test the {{\*}} case additionally since 
it should be working anyway.
* In {{AlterTableStatement.validateAlter}}, I think we should revert the change 
to the error message relating to printing {{reversed(%s)}}. That information 
makes not sense for a CQL3 user and there should be no reason to add it anyway 
since we silently add the {{ReversedType}} if necessary.


> Altering base table column with materialized view causes unexpected server 
> error.
> -
>
> Key: CASSANDRA-10424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10424
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: cassandra-3.0.0-rc1, with python driver 3.0-alpha
>Reporter: Greg Bestland
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc2
>
>
> When attempting to alter column type of base table which has a corresponding  
> materialized view we get an exception from the server.
> Steps to reproduce.
> 1. Create a base table
> {code}
> CREATE TABLE test.scores(
> user TEXT,
> game TEXT,
> year INT,
> month INT,
> day INT,
> score TEXT,
> PRIMARY KEY (user, game, year, month, day)
> )
> {code}
> 2. Create a corresponding materialized view
> {code}
> CREATE MATERIALIZED VIEW test.monthlyhigh AS
> SELECT game, year, month, score, user, day FROM test.scores
> WHERE game IS NOT NULL AND year IS NOT NULL AND month IS NOT 
> NULL AND score IS NOT NULL AND user IS NOT NULL AND day IS NOT NULL
> PRIMARY KEY ((game, year, month), score, user, day)
> WITH CLUSTERING ORDER BY (score DESC, user ASC, day ASC)
> {code}
> 3. Attempt to Alter the base table 
> {code}
> ALTER TABLE test.scores ALTER score TYPE blob
> {code}
> In the python driver we see the following exception returned from the server
> {code}
> Ignoring schedule_unique for already-scheduled task: ( ControlConnection.refresh_schema of  object at 0x100f72c50>>, (), (('keyspace', 'test'), ('target_type', 
> 'KEYSPACE'), ('change_type', 'UPDATED')))
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "./cassandra/cluster.py", line 1623, in execute
> result = future.result()
>   File "./cassandra/cluster.py", line 3205, in result
> raise self._final_exception
> cassandra.protocol.ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.ConfigurationException: Column family 
> comparators do not match or are not compatible (found ClusteringComparator; 
> expected ClusteringComparator).">
> {code}
> On the server I see the following stack trace
> {code}
> INFO  [MigrationStage:1] 2015-09-30 11:45:47,457 ColumnFamilyStore.java:825 - 
> Enqueuing flush of keyspaces: 512 (0%) on-heap, 0 (0%) off-heap
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,457 Memtable.java:362 - 
> Writing Memtable-keyspaces@1714565887(0.146KiB serialized bytes, 1 ops, 0%/0% 
> of on/off-heap limit)
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,463 Memtable.java:395 - 
> Completed flushing 
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/ma-54-big-Data.db
>  (0.109KiB) for commitlog position ReplayPosition(segmentId=1443623481894, 
> position=9812)
> INFO  [MigrationStage:1] 2015-09-30 11:45:47,472 ColumnFamilyStore.java:825 - 
> Enqueuing flush of columns: 877 (0%) on-heap, 0 (0%) off-heap
> INFO  

[jira] [Updated] (CASSANDRA-10443) CQLSStableWriter example fails on 3.0rc1

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10443:
-
Fix Version/s: 3.0.0 rc2

> CQLSStableWriter example fails on 3.0rc1
> 
>
> Key: CASSANDRA-10443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10443
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Tools
>Reporter: Jonathan Shook
> Fix For: 3.0.0 rc2
>
>
> CQLSSTableWriter which works with 2.2.1 does not work with 3.0rc1.
> Something like https://github.com/yukim/cassandra-bulkload-example should be 
> added to the test suite.
> Exception in thread "main" java.lang.RuntimeException: 
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter.close(SSTableSimpleUnsortedWriter.java:136)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter.close(CQLSSTableWriter.java:274)
>   at com.metawiring.sandbox.BulkLoadExample.main(BulkLoadExample.java:160)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: java.lang.ExceptionInInitializerError
>   at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:372)
>   at org.apache.cassandra.db.Keyspace.(Keyspace.java:309)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:133)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.create(SSTableTxnWriter.java:97)
>   at 
> org.apache.cassandra.io.sstable.AbstractSSTableSimpleWriter.createWriter(AbstractSSTableSimpleWriter.java:63)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:206)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1153)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:116)
>   ... 7 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10442) Paging repeats records

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10442:
-
Description: 
Paging repeats records every fetchSize records. The following sample easily 
reproduces the problem on Cassandra 2.0.16 with Java Driver 2.0.11.

{noformat}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build(); 

Session s = c.connect();

s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication 
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_page(id INT, 
sec BIGINT, data VARCHAR static, PRIMARY KEY ((id), sec))");

s.execute("INSERT INTO test.test_page (id, data) VALUES (1, 
'asdfasdfasdfasdfasdfasdf')");

PreparedStatement insert = s.prepare("INSERT INTO 
test.test_page (id, sec) VALUES (1, ?)"); 
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));

PreparedStatement select = s.prepare("SELECT sec FROM 
test.test_page WHERE id = 1");

long lastSec = -1;  
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id 
%d", sec));

lastSec = sec;
}
System.exit(0);
}
}
{noformat}

The program outputs the following:

Duplicated id 299
Duplicated id 598
Duplicated id 897

Note that the static column is required. This bug doesn't occur if you remove 
the column from the schema.

I realize that this may be a driver bug, but I don't really know, so I'm 
logging it here until that can be determined.

  was:
Paging repeats records every fetchSize records. The following sample easily 
reproduces the problem on Cassandra 2.0.16 with Java Driver 2.0.11.

public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();

Cluster c = builder.addContactPoints("192.168.98.190").build();

Session s = c.connect();

s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication 
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");

s.execute("CREATE TABLE IF NOT EXISTS test.test_page(id INT, 
sec BIGINT, data VARCHAR static, PRIMARY KEY ((id), sec))");

s.execute("INSERT INTO test.test_page (id, data) VALUES (1, 
'asdfasdfasdfasdfasdfasdf')");

PreparedStatement insert = s.prepare("INSERT INTO 
test.test_page (id, sec) VALUES (1, ?)");

for (int i = 0; i < 1000; i++)
{
s.execute(insert.bind((long) i));
}

PreparedStatement select = s.prepare("SELECT sec FROM 
test.test_page WHERE id = 1");

long lastSec = -1;

for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");

if (sec == lastSec)
{
System.out.println(String.format("Duplicated id 
%d", sec));
}

lastSec = sec;
}

System.exit(0);
}
}

The program outputs the following:

Duplicated id 299
Duplicated id 598
Duplicated id 897

Note that the static column is required. This bug doesn't occur if you remove 
the column from the schema.

I realize that this may be a driver bug, but I don't really know, so I'm 
logging it here until that can be determined.


> Paging repeats records
> --
>
> Key: CASSANDRA-10442
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10442
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Wille
>
> Paging repeats records every fetchSize records. The following sample easily 
> reproduces the problem on Cassandra 2.0.16 with Java Driver 2.0.11.
> {noformat}
> public class TestPagingBug
> {
>   public static void main(String[] args)
>   {
>   Cluster.Builder builder = Cluster.builder();
>

[jira] [Updated] (CASSANDRA-10442) Paging repeats records

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10442:
-
Assignee: Benjamin Lerer

> Paging repeats records
> --
>
> Key: CASSANDRA-10442
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10442
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Wille
>Assignee: Benjamin Lerer
>
> Paging repeats records every fetchSize records. The following sample easily 
> reproduces the problem on Cassandra 2.0.16 with Java Driver 2.0.11.
> {noformat}
> public class TestPagingBug
> {
>   public static void main(String[] args)
>   {
>   Cluster.Builder builder = Cluster.builder();
>   Cluster c = builder.addContactPoints("192.168.98.190").build(); 
> 
>   Session s = c.connect();
>   
>   s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication 
> = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
>   s.execute("CREATE TABLE IF NOT EXISTS test.test_page(id INT, 
> sec BIGINT, data VARCHAR static, PRIMARY KEY ((id), sec))");
>   s.execute("INSERT INTO test.test_page (id, data) VALUES (1, 
> 'asdfasdfasdfasdfasdfasdf')");
>   
>   PreparedStatement insert = s.prepare("INSERT INTO 
> test.test_page (id, sec) VALUES (1, ?)"); 
>   for (int i = 0; i < 1000; i++)
>   s.execute(insert.bind((long) i));
>   
>   PreparedStatement select = s.prepare("SELECT sec FROM 
> test.test_page WHERE id = 1");
>   
>   long lastSec = -1;  
>   for (Row row : s.execute(select.bind().setFetchSize(300)))
>   {
>   long sec = row.getLong("sec");
>   if (sec == lastSec)
>   System.out.println(String.format("Duplicated id 
> %d", sec));
>   
>   lastSec = sec;
>   }
>   System.exit(0);
>   }
> }
> {noformat}
> The program outputs the following:
> Duplicated id 299
> Duplicated id 598
> Duplicated id 897
> Note that the static column is required. This bug doesn't occur if you remove 
> the column from the schema.
> I realize that this may be a driver bug, but I don't really know, so I'm 
> logging it here until that can be determined.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10443) CQLSStableWriter example fails on 3.0rc1

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10443:
-
Assignee: Carl Yeksigian

> CQLSStableWriter example fails on 3.0rc1
> 
>
> Key: CASSANDRA-10443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10443
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Tools
>Reporter: Jonathan Shook
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc2
>
>
> CQLSSTableWriter which works with 2.2.1 does not work with 3.0rc1.
> Something like https://github.com/yukim/cassandra-bulkload-example should be 
> added to the test suite.
> Exception in thread "main" java.lang.RuntimeException: 
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter.close(SSTableSimpleUnsortedWriter.java:136)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter.close(CQLSSTableWriter.java:274)
>   at com.metawiring.sandbox.BulkLoadExample.main(BulkLoadExample.java:160)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: java.lang.ExceptionInInitializerError
>   at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:372)
>   at org.apache.cassandra.db.Keyspace.(Keyspace.java:309)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:133)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.create(SSTableTxnWriter.java:97)
>   at 
> org.apache.cassandra.io.sstable.AbstractSSTableSimpleWriter.createWriter(AbstractSSTableSimpleWriter.java:63)
>   at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:206)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1153)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:116)
>   ... 7 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-6970) Support CL=EACH_QUORUM for reads

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-6970.
-
Resolution: Duplicate

> Support CL=EACH_QUORUM for reads
> 
>
> Key: CASSANDRA-6970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6970
> Project: Cassandra
>  Issue Type: Wish
> Environment: SLES 11 SP3 x86_64
> Cassandra 2.0.6
>Reporter: Oliver Seiler
>
> SELECTs done at CL=EACH_QUORUM get an InvalidRequestException: with a message 
> of "EACH_QUORUM ConsistencyLevel is only supported for writes".
> The documentation doesn't indicate this would happen, so at minimum this is 
> inconsistent with the current documentation. I'm aware of CASSANDRA-3272, 
> which indicates this has never worked as expected, and made this obvious with 
> the InvalidRequestException.
> I'm not sure why this shouldn't work, regardless of Jonathan Ellis commenting 
> "I ask because I've never seen it to be the 'right' CL yet". I'd like it 
> because: I want to do my writes at LOCAL_QUORUM and (some) reads at 
> EACH_QUORUM, because I require guaranteed consistency for (some) reads where 
> I can be partition intolerant without affecting clients, rather than becoming 
> partition intolerant on writes...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10256) document commitlog segment size's relationship to max write size

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10256:
-
Assignee: Tushar Agrawal

> document commitlog segment size's relationship to max write size
> 
>
> Key: CASSANDRA-10256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Config
>Reporter: Chris Burroughs
>Assignee: Tushar Agrawal
>Priority: Trivial
>  Labels: lhf
> Fix For: 3.0.0 rc2
>
> Attachments: CASSANDRA-10256.txt
>
>
> This is in the code: 
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L57
> But not part of the description in cassandra.yaml



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-10407:
-
Attachment: reBufferStandardTime.png
allocateDirect.png
size.png
3_0_head_std_wo_ra.nps
3_0_head_mmap_wo_ra.nps

> Benchmark and evaluate CASSANDRA-8894 improvements
> --
>
> Key: CASSANDRA-10407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10407
> Project: Cassandra
>  Issue Type: Test
>Reporter: Aleksey Yeschenko
>Assignee: Alan Boudreault
> Fix For: 3.0.0 rc2
>
> Attachments: 3_0_head_mmap_wo_ra.nps, 3_0_head_std_wo_ra.nps, 
> 8894_tiny.yaml, allocateDirect.png, flight-recordings.tar.gz, 
> reBufferStandardTime.png, size.png, test-with-8894-tiny.json, 
> test-without-8894-tiny.json
>
>
> The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months 
> ago. We need to get proper performance tests before GA.
> See [~benedict]'s 
> [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203]
>  for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943083#comment-14943083
 ] 

Stefania commented on CASSANDRA-10407:
--

Thank you for the tests [~aboudreault]. It seems that when the patch was 
committed the performance of std was really bad (44-45k ops/second) compared to 
mmap (66-68k ops/second) and neither the patch nor readahead had any impact. 
However, this has changed quite radically with 3.0 rc1 (52k vs 57-59k). It's 
difficult to know why without bisecting but I think we should forget what the 
code was like back then and focus on the current 3.0 head and aim to have std 
without readahead as good as or better than mmap with readahead. We should also 
bring mmap back to 66-68k ops/second. 

I did some profiling by running mmap vs std locally and I've attached a couple 
of _.nps_ files that can be opened with [visual 
vm|http://visualvm.java.net/download.html]. I've also attached some screenshots.

Here are some things that I've noticed:

* In RAR.overrideLength() the call to channel.size() is expensive, we should 
cache the size in the channel or avoid calling this (it is just to throw an 
IllegalArgumentException)

* Most of the time we spend in {{reBuffer}} follows a call from 
{{SSTableReader.getFileDataInput()}}, which is called when initializing 
iterators. Here a new reader is created so we cannot avoid rebuffering, also we 
spend a lot of time in dealing with checksums, more than we spend reading.

* CRAR is allocating buffers that are too big for the buffer pool (> 64kb) and 
therefore we spend time calling {{allocateDirect()}}: {{INFO  07:57:16 
Requested buffer size 65813 is bigger than 65536, allocating directly}}

We should really address these points first (the first one is trivial but the 
other two I am not as familiar with). However, we could also in parallel try 
running these tests to see if there are any differences with the STD case:

* 3.0 HEAD MMAP with readahead
* 3.0 HEAD STD without readahead and setting this in the yaml: 
{{disk_optimization_page_cross_chance: 0.1}} - this is actually the current 
default
* 3.0 HEAD STD without readahead and setting this in the yaml: 
{{disk_optimization_page_cross_chance: 0.5}}
* 3.0 HEAD STD without readahead and setting this in the yaml: 
{{disk_optimization_page_cross_chance: 0.9}}

[~benedict] WDYT regarding the points I've raised? Is it worth addressing them 
before running more tests or would you expect a big difference in the tests 
anyway? Also, is there anything that needs to be done to disable readahead, 
other than calling {{blockdev --setra}}?


> Benchmark and evaluate CASSANDRA-8894 improvements
> --
>
> Key: CASSANDRA-10407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10407
> Project: Cassandra
>  Issue Type: Test
>Reporter: Aleksey Yeschenko
>Assignee: Alan Boudreault
> Fix For: 3.0.0 rc2
>
> Attachments: 3_0_head_mmap_wo_ra.nps, 3_0_head_std_wo_ra.nps, 
> 8894_tiny.yaml, allocateDirect.png, flight-recordings.tar.gz, 
> reBufferStandardTime.png, size.png, test-with-8894-tiny.json, 
> test-without-8894-tiny.json
>
>
> The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months 
> ago. We need to get proper performance tests before GA.
> See [~benedict]'s 
> [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203]
>  for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10383) Disable auto snapshot on selected tables.

[Tommy Stendahl (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943111#comment-14943111
 ] 

Tommy Stendahl commented on CASSANDRA-10383:


I have created a new patch based on 3.0, its availble 
[here|https://github.com/tommystendahl/cassandra/commits/cassandra-10383-30].

> Disable auto snapshot on selected tables.
> -
>
> Key: CASSANDRA-10383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10383
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tommy Stendahl
>Assignee: Tommy Stendahl
> Fix For: 2.1.x
>
> Attachments: 10383.txt
>
>
> I have a use case where I would like to turn off auto snapshot for selected 
> tables, I don't want to turn it off completely since its a good feature. 
> Looking at the code I think it would be relatively easy to fix.
> My plan is to create a new table property named something like 
> "disable_auto_snapshot". If set to false it will prevent auto snapshot on the 
> table, if set to true auto snapshot will be controlled by the "auto_snapshot" 
> property in the cassandra.yaml. Default would be true.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9304) COPY TO improvements

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943139#comment-14943139
 ] 

Stefania commented on CASSANDRA-9304:
-

I think I should still add one more thing to the patch: we should put a cap on 
the maximum number of connections to protect us against very large clusters. I 
need to address some more urgent tickets before this however, so you can review 
the existing code anyway and then I will implement this along with the code 
review comments if that's OK.

> COPY TO improvements
> 
>
> Key: CASSANDRA-9304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9304
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.1.x
>
>
> COPY FROM has gotten a lot of love.  COPY TO not so much.  One obvious 
> improvement could be to parallelize reading and writing (write one page of 
> data while fetching the next).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10428) select with exact date is not working

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania reassigned CASSANDRA-10428:


Assignee: Stefania

> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status, 
> translated_criteria_status from ez_task_result where 
> submissionid='760dd154670811e58c04005056bb6ff0' and 
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and 
> time>='2015-09-29 20:54:23-0700';
>  submissionid | ezid | name   
>   | time | state | status  | 
> translated_criteria_status
> --+--+--+--+---+-+
>  760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 | 
> run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |   
> run-sanities started
> (1 rows)
> cqlsh:ccc>
> {panel}
> {panel:title=Not 
> working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status, 
> translated_criteria_status from ez_task_result where 
> submissionid='760dd154670811e58c04005056bb6ff0' and 
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and 
> time='2015-09-29 20:54:23-0700';
>  submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path 
> | clientid | end_time | jenkins_path | log_file_path | path_available | 
> path_to_task | required_for_overall_status | start_time | state | status | 
> translated_criteria_status | type
> --+--+--+--+-+---+--+--+--+---++--+-++---+++--
> (0 rows)
> cqlsh:ccc>
> {panel}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[J.P. Eiti Kimura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.P. Eiti Kimura updated CASSANDRA-10233:
-
Attachment: cassandra-2.1.8-10233-v4.txt
cassandra-2.2.1-10233.txt

Hello [~pauloricardomg] you can se the new patches attached the 2.1-v4 and the 
2.2.1 patch for 2.2.X versions. I hope it helps. 

I'm expliciting check the values, logging a warn message and throing an 
AssertionError as you asked. I hope it helps. 
Thanks  

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt, cassandra-2.1.8-10233-v4.txt, 
> cassandra-2.2.1-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[J.P. Eiti Kimura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944338#comment-14944338
 ] 

J.P. Eiti Kimura edited comment on CASSANDRA-10233 at 10/6/15 12:45 AM:


Hello [~pauloricardomg] you can se the new patches attached the 2.1-v4 and the 
2.2.1 patch for 2.2.X versions.
I'm expliciting check the values, logging a warn message and throing an 
AssertionError as you asked. I hope it helps. 
Thanks  


was (Author: eitikimura):
Hello [~pauloricardomg] you can se the new patches attached the 2.1-v4 and the 
2.2.1 patch for 2.2.X versions. I hope it helps. 

I'm expliciting check the values, logging a warn message and throing an 
AssertionError as you asked. I hope it helps. 
Thanks  

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt, cassandra-2.1.8-10233-v4.txt, 
> cassandra-2.2.1-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Robbie Strickland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robbie Strickland updated CASSANDRA-10449:
--
Environment: Ubuntu 14.04, AWS  (was: Ubuntu 14.04)
Description: 
I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and 
500-700GB per node.  SSTable counts are <10 per table.  I am attempting to 
provision additional nodes, but bootstrapping OOMs every time after about 10 
hours with a sudden long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
CassandraDaemon.java:223 - Exception in thread 
Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to 
no avail.

  was:
I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  
SSTable counts are <10 per table.  I am attempting to provision additional 
nodes, but bootstrapping OOMs every time after about 10 hours with a sudden 
long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
CassandraDaemon.java:223 - Exception in thread 
Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to 
no avail.


> OOM on bootstrap due to long GC pause
> -
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Ubuntu 14.04, AWS
>Reporter: Robbie Strickland
>  Labels: gc
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and 
> 500-700GB per node.  SSTable counts are <10 per table.  I am attempting to 
> provision additional nodes, but bootstrapping OOMs every time after about 10 
> hours with a sudden long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
> Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to 
> no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944090#comment-14944090
 ] 

Paulo Motta commented on CASSANDRA-10233:
-

[~eitikimura] I'd prefer not add the try-catch block on 
{{HintedHandOffManager.scheduleAllDeliveries()}} as in the general case stored 
hints shouldn't be corrupted, and it could make hints be silently dropped which 
could lead to more serious issues. Since we already know the issue was caused 
on {{StorageProxy.writeHintForMutation}} I think it suffices to perform the 
check there. And if someone hit this bug before it's fixed, the workaround 
should be truncate hints + repair.

I think what you did on {{StorageProxy.writeHintForMutation}} looks awesome, 
but the thrown exception might be ignored silently by the hints executor, so 
it's better to perform an explicit check, log a warn and throw an 
AssertionError if {{hostId \!= null}}, so we'll be able to track if it happens 
again in the logs. Could you please make these changes and re-submit the patch? 
Please check if your patch apply to cassandra-2.2 branch, and if it doesn't 
please also submit a patch for 2.2. It should not be necessary to create a 
patch for 3.0, as the hints engine was rewritten from scratch.

Thanks for that [~eitikimura]!

[~fhsgoncalves] yep, afaik assertions should be optional in production, but 
they should never happen in the first place. probably this is being caused by 
some other issue I was not able to track in the latest changes, but 
[~eitikimura]'s patch should help us troubleshoot if it happens in the future.

[~nutbunnies] [~mambocab] maybe it would be interesting to have a dtest job 
with assertions disabled, since we rely a lot on assertions for pre-condition 
checking, and many people disable them in production.

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) 

[jira] [Created] (CASSANDRA-10446) Run repair with down replicas

[sankalp kohli (JIRA)]

sankalp kohli created CASSANDRA-10446:
-

 Summary: Run repair with down replicas
 Key: CASSANDRA-10446
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10446
 Project: Cassandra
  Issue Type: Improvement
Reporter: sankalp kohli
Priority: Minor


We should have an option of running repair when replicas are down. We can call 
it -force.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10447) Async logging configuration doesn't result in data flushing when shutdown hook runs

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944265#comment-14944265
 ] 

Ariel Weisberg commented on CASSANDRA-10447:


Alternatively maybe SL4J has a path for this we should be hitting to have it 
shut things down gracefully?

> Async logging configuration doesn't result in data flushing when shutdown 
> hook runs
> ---
>
> Key: CASSANDRA-10447
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10447
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ariel Weisberg
> Fix For: 3.0.0 rc2
>
>
> Stefania discovered that tests that don't produce a lot of log output end up 
> producing 0 debug output to files because the data is not flushed as part of 
> the shutdown hook. I traced through and it looks like the shutdown hook 
> doesn't actually invoke code that does anything useful. It shuts down an 
> executor service in the logging context but doesn't call stop on any 
> appenders.
> A hackish thing we can do is use a status listener to collect all the 
> appenders and then stop them when the shutdown hook runs. Even adding a small 
> delay to the shutdown hook (no code changes on our part) would in let the 
> async appender flush in 90% of cases.
> We still need to fix it for test which uses a different config file and for 
> which a small delay is not desirable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944266#comment-14944266
 ] 

Fernando Gonçalves commented on CASSANDRA-10233:


[~nutbunnies] Thanks!

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7392) Abort in-progress queries that time out

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944284#comment-14944284
 ] 

Ariel Weisberg commented on CASSANDRA-7392:
---

bq.  I don't think it strictly matters if we incorrectly say that there were 
more dropped operations in one interval rather than the next. WDYT?
I agree it's fine.

bq. As for logging during unit tests, I've changed logback-test.xml since we 
cannot change the log level at runtime, I hope that's OK. I think we have a 
problem with the TeeingAppender, I could not see any log messages in the log 
files, only on stdout.
You are right it is dropping log output pretty badly in the test configuration. 
In the production configuration it would drop less, but still doesn't have a 
working shutdown hook.

I dug a lot and finally got it to work by using the status listener to find out 
when file appenders are created, cache them, then stop them on shutdown. I 
think this is a problem for Paulo's work as well because this means the async 
appender won't be stopped. I created CASSANDRA-10447 for it.

So you set the filter in the logging to be more permissive, but only have the 
monitoring logger running at debug? The other loggers won't emit debug 
statements? That sounds fine.

It looks like the NanoTimeToCurrentTimeMillis test can't check the condition it 
was checking before which was that it kinda sort of work to try and propagate 
NTP updates. You could have the test submit the update task to the STPE instead 
of notifying it. That could be a function in NanoTimeToCurrentTimeMillis so the 
details aren't foisted onto the test.

> Abort in-progress queries that time out
> ---
>
> Key: CASSANDRA-7392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7392
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 3.x
>
>
> Currently we drop queries that time out before we get to them (because node 
> is overloaded) but not queries that time out while being processed.  
> (Particularly common for index queries on data that shouldn't be indexed.)  
> Adding the latter and logging when we have to interrupt one gets us a poor 
> man's "slow query log" for free.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10448) "Unknown type 0" Stream failure on Repair

[Omri Iluz (JIRA)]

Omri Iluz created CASSANDRA-10448:
-

 Summary: "Unknown type 0" Stream failure on Repair
 Key: CASSANDRA-10448
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10448
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.2.2
5 Nodes in Google Compute Engine
Java 1.8.0_60
Reporter: Omri Iluz


While running repair after upgrading to 2.2.2 I am getting many stream fail 
errors:
{noformat}
[2015-10-05 23:52:30,353] Repair session 4c181051-6bbb-11e5-acdb-d9a8bbd39330 
for range (59694553044959221,86389982480621619] failed with error [repair 
#4c181051-6bbb-11e5-acdb-d9a8bbd39330 on px/acti
vities, (59694553044959221,86389982480621619]] Sync failed between 
/10.240.81.104 and /10.240.134.221 (progress: 4%)
{noformat}
Logs from both sides of the stream:
Sides 1 -
{noformat}
INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
StreamResultFuture.java:111 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
ID#0] Creating new streaming plan for Repair
INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
ID#0] Received streaming plan for Repair
INFO  [STREAM-INIT-/10.240.81.104:52723] 2015-10-05 23:52:30,063 
StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
ID#0] Received streaming plan for Repair
INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,098 
StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
ID#0] Prepare completed. Receiving 13 files(517391317 bytes), sending 10 
files(469491729 bytes)
ERROR [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,234 StreamSession.java:524 
- [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] Streaming error occurred
java.lang.IllegalArgumentException: Unknown type 0
at 
org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:96)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:57)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
Session with /10.240.81.104 is complete
WARN  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
StreamResultFuture.java:209 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
Stream failed
{noformat}
Side 2 -
{noformat}
INFO  [AntiEntropyStage:1] 2015-10-05 23:52:30,060 StreamResultFuture.java:86 - 
[Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] Executing streaming plan for 
Repair
INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,061 
StreamSession.java:232 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
Starting streaming to /10.240.134.221
INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,063 
StreamCoordinator.java:213 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
ID#0] Beginning stream session with /10.240.134.221
INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,098 
StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
ID#0] Prepare completed. Receiving 10 files(469491729 bytes), sending 13 
files(517391317 bytes)
INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,349 
StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
Session with /10.240.134.221 is complete
ERROR [STREAM-OUT-/10.240.134.221] 2015-10-05 23:52:30,349 
StreamSession.java:524 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
Streaming error occurred
org.apache.cassandra.io.FSReadError: java.io.IOException: Broken pipe
at 
org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144) 
~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:79)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:76)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.applyToChannel(BufferedDataOutputStreamPlus.java:293)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.compress.CompressedStreamWriter.write(CompressedStreamWriter.java:75)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.messages.OutgoingFileMessage.serialize(OutgoingFileMessage.java:96)
 ~[apache-cassandra-2.2.2.jar:2.2.2]
at 
org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:48)
 ~[apache-cassandra-2.2.2.jar:2.2.2]

[jira] [Commented] (CASSANDRA-7392) Abort in-progress queries that time out

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944319#comment-14944319
 ] 

Stefania commented on CASSANDRA-7392:
-

Thanks for spending the time to understand the problem with logging. 

bq. So you set the filter in the logging to be more permissive, but only have 
the monitoring logger running at debug? The other loggers won't emit debug 
statements? 

Correct.

bq. It looks like the NanoTimeToCurrentTimeMillis test can't check the 
condition it was checking before which was that it kinda sort of work to try 
and propagate NTP updates. You could have the test submit the update task to 
the STPE instead of notifying it. That could be a function in 
NanoTimeToCurrentTimeMillis so the details aren't foisted onto the test.

See if [this 
is|https://github.com/stef1927/cassandra/commit/fc42b1e684649fc1cf9499e72c643d5c2a31455d]
 what you meant.

> Abort in-progress queries that time out
> ---
>
> Key: CASSANDRA-7392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7392
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 3.x
>
>
> Currently we drop queries that time out before we get to them (because node 
> is overloaded) but not queries that time out while being processed.  
> (Particularly common for index queries on data that shouldn't be indexed.)  
> Adding the latter and logging when we have to interrupt one gets us a poor 
> man's "slow query log" for free.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10445) Cassandra-stress throws max frame size error when SSL certification is enabled

[Sam Goldberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Goldberg updated CASSANDRA-10445:
-
Description: 
Running cassandra-stress when SSL is enabled gives the following error and does 
not finish executing:

{quote}
cassandra-stress write n=100
Exception in thread "main" java.lang.RuntimeException: 
org.apache.thrift.transport.TTransportException: Frame size (352518912) larger 
than max length (15728640)!
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:144)
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:110)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesThrift(SettingsSchema.java:111)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:59)
at 
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:205)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:55)
at org.apache.cassandra.stress.Stress.main(Stress.java:109)
{quote}

I was able to reproduce this issue consistently via the following steps:
1) Spin up 3 node cassandra cluster running 2.1.8
2) Perform cassandra-stress write n=100
3) Everything works!
4) Generate keystore and truststore for each node in the cluster and distribute 
appropriately 
5) Modify cassandra.yaml on each node to enable SSL:
client_encryption_options:
enabled: true
keystore: /
# require_client_auth: false
# Set trustore and truststore_password if require_client_auth is true
truststore:  /
truststore_password: 
# More advanced defaults below:
protocol: ssl
6) Restart each node.
7) Perform cassandra-stress write n=100
8) Get Frame Size error, cassandra-stress fails

This may be related to CASSANDRA-9325.

  was:
Running cassandra-stress when SSL is enabled gives the following error and does 
not finish executing:

cassandra-stress write n=100
Exception in thread "main" java.lang.RuntimeException: 
org.apache.thrift.transport.TTransportException: Frame size (352518912) larger 
than max length (15728640)!
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:144)
at 
org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:110)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesThrift(SettingsSchema.java:111)
at 
org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:59)
at 
org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:205)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:55)
at org.apache.cassandra.stress.Stress.main(Stress.java:109)


I was able to reproduce this issue consistently via the following steps:
1) Spin up 3 node cassandra cluster running 2.1.8
2) Perform cassandra-stress write n=100
3) Everything works!
4) Generate keystore and truststore for each node in the cluster and distribute 
appropriately 
5) Modify cassandra.yaml on each node to enable SSL:
client_encryption_options:
enabled: true
keystore: /
# require_client_auth: false
# Set trustore and truststore_password if require_client_auth is true
truststore:  /
truststore_password: 
# More advanced defaults below:
protocol: ssl
6) Restart each node.
7) Perform cassandra-stress write n=100
8) Get Frame Size error, cassandra-stress fails

This may be related to CASSANDRA-9325.


> Cassandra-stress throws max frame size error when SSL certification is enabled
> --
>
> Key: CASSANDRA-10445
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10445
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Goldberg
>
> Running cassandra-stress when SSL is enabled gives the following error and 
> does not finish executing:
> {quote}
> cassandra-stress write n=100
> Exception in thread "main" java.lang.RuntimeException: 
> org.apache.thrift.transport.TTransportException: Frame size (352518912) 
> larger than max length (15728640)!
> at 
> org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:144)
> at 
> org.apache.cassandra.stress.settings.StressSettings.getRawThriftClient(StressSettings.java:110)
> at 
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesThrift(SettingsSchema.java:111)
> at 
> org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:59)
> at 
> org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:205)
> at 

[jira] [Updated] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Robbie Strickland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robbie Strickland updated CASSANDRA-10449:
--
Description: 
I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  
SSTable counts are <10 per table.  I am attempting to provision additional 
nodes, but bootstrapping OOMs every time after about 10 hours with a sudden 
long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
CassandraDaemon.java:223 - Exception in thread 
Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to 
no avail.

  was:
I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  
SSTable counts are <10 per node.  I am attempting to provision additional 
nodes, but bootstrapping OOMs every time after about 10 hours with a sudden 
long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
CassandraDaemon.java:223 - Exception in thread 
Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to 
no avail.


> OOM on bootstrap due to long GC pause
> -
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Ubuntu 14.04
>Reporter: Robbie Strickland
>  Labels: gc
>
> I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node. 
>  SSTable counts are <10 per table.  I am attempting to provision additional 
> nodes, but bootstrapping OOMs every time after about 10 hours with a sudden 
> long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
> Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to 
> no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Robbie Strickland (JIRA)]

Robbie Strickland created CASSANDRA-10449:
-

 Summary: OOM on bootstrap due to long GC pause
 Key: CASSANDRA-10449
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 14.04
Reporter: Robbie Strickland


I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  
SSTable counts are <10 per node.  I am attempting to provision additional 
nodes, but bootstrapping OOMs every time after about 10 hours with a sudden 
long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
CassandraDaemon.java:223 - Exception in thread 
Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to 
no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10341) Streaming does not guarantee cache invalidation

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944216#comment-14944216
 ] 

Yuki Morishita commented on CASSANDRA-10341:


Thanks for the patch.
Overall it's good.
I think we can improve constructing bounds to invalidate by sorting SSTables by 
its token bound(see {{SSTableReader.sstableComparator}}), eliminating overlaps.
Also, I like to call Bounds as bounds instead of {{inclusiveRanges}}.

> Streaming does not guarantee cache invalidation
> ---
>
> Key: CASSANDRA-10341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10341
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Paulo Motta
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> Looking at the code, we attempt to invalidate the row cache for any rows we 
> receive via streaming, however we invalidate them immediately, before the new 
> data is available. So, if it is requested (which is likely if it is "hot") in 
> the interval, it will be re-cached and not invalidated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10447) Async logging configuration doesn't result in data flushing when shutdown hook runs

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-10447:
--

 Summary: Async logging configuration doesn't result in data 
flushing when shutdown hook runs
 Key: CASSANDRA-10447
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10447
 Project: Cassandra
  Issue Type: Bug
Reporter: Ariel Weisberg
 Fix For: 3.0.0 rc2


Stefania discovered that tests that don't produce a lot of log output end up 
producing 0 debug output to files because the data is not flushed as part of 
the shutdown hook. I traced through and it looks like the shutdown hook doesn't 
actually invoke code that does anything useful. It shuts down an executor 
service in the logging context but doesn't call stop on any appenders.

A hackish thing we can do is use a status listener to collect all the appenders 
and then stop them when the shutdown hook runs. Even adding a small delay to 
the shutdown hook (no code changes on our part) would in let the async appender 
flush in 90% of cases.

We still need to fix it for test which uses a different config file and for 
which a small delay is not desirable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10449:

Fix Version/s: 2.1.x

> OOM on bootstrap due to long GC pause
> -
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Ubuntu 14.04, AWS
>Reporter: Robbie Strickland
>  Labels: gc
> Fix For: 2.1.x
>
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and 
> 500-700GB per node.  SSTable counts are <10 per table.  I am attempting to 
> provision additional nodes, but bootstrapping OOMs every time after about 10 
> hours with a sudden long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
> Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to 
> no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1497#comment-1497
 ] 

Philip Thompson commented on CASSANDRA-10449:
-

Can you attach a system.log from a node that fails to bootstrap? 

> OOM on bootstrap due to long GC pause
> -
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Ubuntu 14.04, AWS
>Reporter: Robbie Strickland
>  Labels: gc
> Fix For: 2.1.x
>
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and 
> 500-700GB per node.  SSTable counts are <10 per table.  I am attempting to 
> provision additional nodes, but bootstrapping OOMs every time after about 10 
> hours with a sudden long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
> Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to 
> no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10448) "Unknown type 0" Stream failure on Repair

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10448:

Assignee: Yuki Morishita

> "Unknown type 0" Stream failure on Repair
> -
>
> Key: CASSANDRA-10448
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10448
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.2
> 5 Nodes in Google Compute Engine
> Java 1.8.0_60
>Reporter: Omri Iluz
>Assignee: Yuki Morishita
> Fix For: 2.2.x
>
>
> While running repair after upgrading to 2.2.2 I am getting many stream fail 
> errors:
> {noformat}
> [2015-10-05 23:52:30,353] Repair session 4c181051-6bbb-11e5-acdb-d9a8bbd39330 
> for range (59694553044959221,86389982480621619] failed with error [repair 
> #4c181051-6bbb-11e5-acdb-d9a8bbd39330 on px/acti
> vities, (59694553044959221,86389982480621619]] Sync failed between 
> /10.240.81.104 and /10.240.134.221 (progress: 4%)
> {noformat}
> Logs from both sides of the stream:
> Sides 1 -
> {noformat}
> INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:111 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Creating new streaming plan for Repair
> INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Received streaming plan for Repair
> INFO  [STREAM-INIT-/10.240.81.104:52723] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Received streaming plan for Repair
> INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,098 
> StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Prepare completed. Receiving 13 files(517391317 bytes), sending 10 
> files(469491729 bytes)
> ERROR [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,234 
> StreamSession.java:524 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Streaming error occurred
> java.lang.IllegalArgumentException: Unknown type 0
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:96)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:57)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
> StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Session with /10.240.81.104 is complete
> WARN  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
> StreamResultFuture.java:209 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Stream failed
> {noformat}
> Side 2 -
> {noformat}
> INFO  [AntiEntropyStage:1] 2015-10-05 23:52:30,060 StreamResultFuture.java:86 
> - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] Executing streaming plan for 
> Repair
> INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,061 
> StreamSession.java:232 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Starting streaming to /10.240.134.221
> INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,063 
> StreamCoordinator.java:213 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Beginning stream session with /10.240.134.221
> INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,098 
> StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Prepare completed. Receiving 10 files(469491729 bytes), sending 13 
> files(517391317 bytes)
> INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,349 
> StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Session with /10.240.134.221 is complete
> ERROR [STREAM-OUT-/10.240.134.221] 2015-10-05 23:52:30,349 
> StreamSession.java:524 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Streaming error occurred
> org.apache.cassandra.io.FSReadError: java.io.IOException: Broken pipe
>   at 
> org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144) 
> ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:79)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:76)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.applyToChannel(BufferedDataOutputStreamPlus.java:293)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> 

[jira] [Updated] (CASSANDRA-10448) "Unknown type 0" Stream failure on Repair

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-10448:

Fix Version/s: 2.2.x

> "Unknown type 0" Stream failure on Repair
> -
>
> Key: CASSANDRA-10448
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10448
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.2
> 5 Nodes in Google Compute Engine
> Java 1.8.0_60
>Reporter: Omri Iluz
> Fix For: 2.2.x
>
>
> While running repair after upgrading to 2.2.2 I am getting many stream fail 
> errors:
> {noformat}
> [2015-10-05 23:52:30,353] Repair session 4c181051-6bbb-11e5-acdb-d9a8bbd39330 
> for range (59694553044959221,86389982480621619] failed with error [repair 
> #4c181051-6bbb-11e5-acdb-d9a8bbd39330 on px/acti
> vities, (59694553044959221,86389982480621619]] Sync failed between 
> /10.240.81.104 and /10.240.134.221 (progress: 4%)
> {noformat}
> Logs from both sides of the stream:
> Sides 1 -
> {noformat}
> INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:111 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Creating new streaming plan for Repair
> INFO  [STREAM-INIT-/10.240.81.104:52722] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Received streaming plan for Repair
> INFO  [STREAM-INIT-/10.240.81.104:52723] 2015-10-05 23:52:30,063 
> StreamResultFuture.java:118 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Received streaming plan for Repair
> INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,098 
> StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Prepare completed. Receiving 13 files(517391317 bytes), sending 10 
> files(469491729 bytes)
> ERROR [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,234 
> StreamSession.java:524 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Streaming error occurred
> java.lang.IllegalArgumentException: Unknown type 0
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:96)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:57)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> INFO  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
> StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Session with /10.240.81.104 is complete
> WARN  [STREAM-IN-/10.240.81.104] 2015-10-05 23:52:30,302 
> StreamResultFuture.java:209 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Stream failed
> {noformat}
> Side 2 -
> {noformat}
> INFO  [AntiEntropyStage:1] 2015-10-05 23:52:30,060 StreamResultFuture.java:86 
> - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] Executing streaming plan for 
> Repair
> INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,061 
> StreamSession.java:232 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Starting streaming to /10.240.134.221
> INFO  [StreamConnectionEstablisher:6] 2015-10-05 23:52:30,063 
> StreamCoordinator.java:213 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550, 
> ID#0] Beginning stream session with /10.240.134.221
> INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,098 
> StreamResultFuture.java:168 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550 
> ID#0] Prepare completed. Receiving 10 files(469491729 bytes), sending 13 
> files(517391317 bytes)
> INFO  [STREAM-IN-/10.240.134.221] 2015-10-05 23:52:30,349 
> StreamResultFuture.java:182 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Session with /10.240.134.221 is complete
> ERROR [STREAM-OUT-/10.240.134.221] 2015-10-05 23:52:30,349 
> StreamSession.java:524 - [Stream #239d8e60-6bbc-11e5-93ac-31bdef2dc550] 
> Streaming error occurred
> org.apache.cassandra.io.FSReadError: java.io.IOException: Broken pipe
>   at 
> org.apache.cassandra.io.util.ChannelProxy.transferTo(ChannelProxy.java:144) 
> ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:79)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.streaming.compress.CompressedStreamWriter$1.apply(CompressedStreamWriter.java:76)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.applyToChannel(BufferedDataOutputStreamPlus.java:293)
>  ~[apache-cassandra-2.2.2.jar:2.2.2]
>   at 
> 

[jira] [Comment Edited] (CASSANDRA-10449) OOM on bootstrap due to long GC pause

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1497#comment-1497
 ] 

Philip Thompson edited comment on CASSANDRA-10449 at 10/6/15 3:18 AM:
--

Can you attach a system.log from a node that fails to bootstrap? 

[~JoshuaMcKenzie], who should this be assigned to?


was (Author: philipthompson):
Can you attach a system.log from a node that fails to bootstrap? 

> OOM on bootstrap due to long GC pause
> -
>
> Key: CASSANDRA-10449
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Ubuntu 14.04, AWS
>Reporter: Robbie Strickland
>  Labels: gc
> Fix For: 2.1.x
>
>
> I have a 20-node cluster (i2.4xlarge) with vnodes (default of 256) and 
> 500-700GB per node.  SSTable counts are <10 per table.  I am attempting to 
> provision additional nodes, but bootstrapping OOMs every time after about 10 
> hours with a sudden long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old 
> Generation GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 
> CassandraDaemon.java:223 - Exception in thread 
> Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to 
> no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10378) Make skipping more efficient

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943248#comment-14943248
 ] 

Benedict commented on CASSANDRA-10378:
--

* The {{previousUnfilteredSize}} is not propagated to {{serializedRowBodySize}} 
in {{serializedSize}}
* A static method for deciding if we have an extension byte would be nice
* {{skipRowBody}} and {{skipMarkerBody}} have unused parameters

> Make skipping more efficient
> 
>
> Key: CASSANDRA-10378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10378
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc2
>
>
> Following on from the impact of CASSANDRA-10322, we can improve the 
> efficiency of our calls to skipping methods. CASSANDRA-10326 is showing our 
> performance to be in-and-around the same ballpark except for seeks into the 
> middle of a large partition, which suggests (possibly) that the higher 
> density of data we're storing may simply be resulting in a more significant 
> CPU burden as we have more data to skip over (and since CASSANDRA-10322 
> improves performance here really dramatically, further improvements are 
> likely to be of similar benefit).
> I propose doing our best to flatten the skipping of macro data items into as 
> few skip invocations as necessary. One way of doing this would be to 
> introduce a special {{skipUnsignedVInts(int)}} method, that can efficiently 
> skip a number of unsigned vints. Almost the entire body of a cell and row 
> consist of vints now, each data component with their own special {{skipX}} 
> method that invokes {{readUnsignedVint}}. This would permit more efficient 
> despatch.
> We could also potentially avoid the construction of a new {{Columns}} 
> instance for each row skip, since all we need is an iterator over the 
> columns, and share the temporary space used for storing them, which should 
> further reduce the GC burden for skipping many rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10428) select with exact date is not working

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943147#comment-14943147
 ] 

Stefania commented on CASSANDRA-10428:
--

I've added it to my backlog but it may take some time before I get a chance to 
work on this. I've also noticed the 'f' in the timestamp, it's quite strange, I 
will try and reproduce it when I start work on this, so far I simply replaced 
the format directly in the code without reading it from the config file.

> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
>Assignee: Stefania
>  Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status, 
> translated_criteria_status from ez_task_result where 
> submissionid='760dd154670811e58c04005056bb6ff0' and 
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and 
> time>='2015-09-29 20:54:23-0700';
>  submissionid | ezid | name   
>   | time | state | status  | 
> translated_criteria_status
> --+--+--+--+---+-+
>  760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 | 
> run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |   
> run-sanities started
> (1 rows)
> cqlsh:ccc>
> {panel}
> {panel:title=Not 
> working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status, 
> translated_criteria_status from ez_task_result where 
> submissionid='760dd154670811e58c04005056bb6ff0' and 
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and 
> time='2015-09-29 20:54:23-0700';
>  submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path 
> | clientid | end_time | jenkins_path | log_file_path | path_available | 
> path_to_task | required_for_overall_status | start_time | state | status | 
> translated_criteria_status | type
> --+--+--+--+-+---+--+--+--+---++--+-++---+++--
> (0 rows)
> cqlsh:ccc>
> {panel}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/2.2.2-tentative [deleted] ae9b7e052

[jira] [Commented] (CASSANDRA-9625) GraphiteReporter not reporting

[sylvain boily (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943396#comment-14943396
 ] 

sylvain boily commented on CASSANDRA-9625:
--

We are also running into this issue with 2.1.9. Metrics are not reporting after 
a while node by node. 

> GraphiteReporter not reporting
> --
>
> Key: CASSANDRA-9625
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9625
> Project: Cassandra
>  Issue Type: Bug
> Environment: Debian Jessie, 7u79-2.5.5-1~deb8u1, Cassandra 2.1.3
>Reporter: Eric Evans
> Attachments: metrics.yaml, thread-dump.log
>
>
> When upgrading from 2.1.3 to 2.1.6, the Graphite metrics reporter stops 
> working.  The usual startup is logged, and one batch of samples is sent, but 
> the reporting interval comes and goes, and no other samples are ever sent.  
> The logs are free from errors.
> Frustratingly, metrics reporting works in our smaller (staging) environment 
> on 2.1.6; We are able to reproduce this on all 6 of production nodes, but not 
> on a 3 node (otherwise identical) staging cluster (maybe it takes a certain 
> level of concurrency?).
> Attached is a thread dump, and our metrics.yaml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.2.2 [created] 044b6b467

svn commit: r10716 - /release/cassandra/debian/pool/main/c/cassandra/

[jake]

Author: jake
Date: Mon Oct  5 13:38:43 2015
New Revision: 10716

Log:
2.1.10 and 2.2.2 releases

Added:

release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.1.10_all.deb   
(with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.2.2_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.orig.tar.gz 
  (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.2.2.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.2.2.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.2.2.orig.tar.gz  
 (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.2.2.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.2.2_all.deb   
(with props)
Removed:
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.16.diff.gz
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.16.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.16.orig.tar.gz

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.16.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.16_all.deb

Added: 
release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.1.10_all.deb
==
Binary file - no diff available.

Propchange: 
release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.1.10_all.deb
--
svn:mime-type = application/octet-stream

Added: 
release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.2.2_all.deb
==
Binary file - no diff available.

Propchange: 
release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.2.2_all.deb
--
svn:mime-type = application/octet-stream

Added: release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.diff.gz
==
Binary file - no diff available.

Propchange: 
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.diff.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.dsc
==
--- release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.dsc (added)
+++ release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.10.dsc Mon Oct 
 5 13:38:43 2015
@@ -0,0 +1,45 @@
+-BEGIN PGP SIGNED MESSAGE-
+Hash: SHA1
+
+Format: 1.0
+Source: cassandra
+Binary: cassandra, cassandra-tools
+Architecture: all
+Version: 2.1.10
+Maintainer: Eric Evans 
+Uploaders: Sylvain Lebresne 
+Homepage: http://cassandra.apache.org
+Standards-Version: 3.8.3
+Vcs-Browser: https://git-wip-us.apache.org/repos/asf?p=cassandra.git
+Vcs-Git: http://git-wip-us.apache.org/repos/asf/cassandra.git
+Build-Depends: debhelper (>= 5), openjdk-7-jdk | java7-jdk, ant (>= 1.7), 
ant-optional (>= 1.7), python-support, dpatch, bash-completion
+Package-List:
+ cassandra deb misc extra arch=all
+ cassandra-tools deb misc extra arch=all
+Checksums-Sha1:
+ fa61b01bfb87b877c48b926a601b6f067d8c3643 16618425 cassandra_2.1.10.orig.tar.gz
+ a0fddd5458378c1bf3c10dd2f5c060d1347741ed 20 cassandra_2.1.10.diff.gz
+Checksums-Sha256:
+ 8b1cfc1bed49ecfb4567bfee6c798d42fc8dba684bc31d8c35bff827aaf51dc3 16618425 
cassandra_2.1.10.orig.tar.gz
+ f61f27bd17de546264aa58f40f3aafaac7021e0ef69c17f6b1b4cd7664a037ec 20 
cassandra_2.1.10.diff.gz
+Files:
+ 86e520a316f72839273a9ac97ede845a 16618425 cassandra_2.1.10.orig.tar.gz
+ 4a4dd3598707603b3f76a2378a4504aa 20 cassandra_2.1.10.diff.gz
+
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAEBAgAGBQJWDT+vAAoJEHSdbuwDU7Es2joP/iMLUn1X+d+NNcDiqYnLtV8+
+2wjCVtQPZ5YJ/i4qiUpi80vJ988VfzxNH5gKa7sMnCsvdfLV2xPTI3itWkHI+Soh
+mOfb66dF6GJw3X2nYXByDtGfDX5bBy+6hMikhGYbfndhEkk5V8Uc65fun8pWG4xJ
+RQ3ie5HvovfboNGlbLy2tZmb+1EUm8iPrEVN72+D986yeCtbcED4J5x/aavA50d1
+a0K4DAq+ur6r62iBEtfga6fKLnvlkOU13EAS3TBeSQIj7cdfd96qLzTR0IPiA1NJ
+mgWz0FbCKiBY5O45bWtuz9pZUWVVCQQ1A5X2LvAk3s1ltBAmJnZCq6uJX96JNSti
+8Dg8jc53ys5NZJvW0m+DwH6OsMSYwJqteLJSpg3WZrg2ZHz+RpMUwn9phV10Q1Hz
+OMSi0Ldx/bPAk+P3mqLVDinCHJy/iKMluUJ5HhIegAPaN9i7otvcfiPKdrAVkJyB

Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/2.1.10-tentative [deleted] 78f2e7aa0

[jira] [Updated] (CASSANDRA-10293) A node down is not always displayed in nodetool status

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10293:

Fix Version/s: 3.0.0 rc2

> A node down is not always displayed in nodetool status 
> ---
>
> Key: CASSANDRA-10293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10293
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alan Boudreault
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 3.0.0 rc2
>
> Attachments: mv_repair_test.sh
>
>
> I've noticed this while working on another task: a node that was currently 
> down was not showed in my nodetool status. The node had joined the cluster 
> and was displayed in the status before.
> I've attached [^mv_repair_test.sh] to reproduce the issue. After the 
> execution of the script, just try a: ccm node3 nodetool status



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9741) cfhistograms dtest flaps on trunk and 2.2

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-9741:
--
Fix Version/s: (was: 3.0.0 rc2)
   3.0.x

> cfhistograms dtest flaps on trunk and 2.2
> -
>
> Key: CASSANDRA-9741
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9741
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Ariel Weisberg
> Fix For: 3.0.x
>
>
> {{jmx_test.py:TestJMX.cfhistograms_test}} flaps on CassCI under trunk and 2.2.
> On 2.2, it fails one of its assertions when {{'Unable to compute when 
> histogram overflowed'}} is found in the output of {{nodetool cfhistograms}}. 
> Here's the failure history for 2.2:
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> On trunk, it fails when an error about a {{WriteFailureException}} during 
> hinted handoff is found in the C* logs after the tests run ([example cassci 
> output|http://cassci.datastax.com/view/trunk/job/trunk_dtest/315/testReport/junit/jmx_test/TestJMX/cfhistograms_test/]).
>  Here's the failure history for trunk:
> http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> I haven't seen it fail locally yet, but haven't run the test more than a 
> couple times because it takes a while.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9741) cfhistograms dtest flaps on trunk and 2.2

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943416#comment-14943416
 ] 

Jim Witschey commented on CASSANDRA-9741:
-

[~aweisberg] Could you set the fix version to 3.0.x if that's appropriate, 
then? Thanks.

> cfhistograms dtest flaps on trunk and 2.2
> -
>
> Key: CASSANDRA-9741
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9741
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Ariel Weisberg
> Fix For: 3.0.0 rc2
>
>
> {{jmx_test.py:TestJMX.cfhistograms_test}} flaps on CassCI under trunk and 2.2.
> On 2.2, it fails one of its assertions when {{'Unable to compute when 
> histogram overflowed'}} is found in the output of {{nodetool cfhistograms}}. 
> Here's the failure history for 2.2:
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> On trunk, it fails when an error about a {{WriteFailureException}} during 
> hinted handoff is found in the C* logs after the tests run ([example cassci 
> output|http://cassci.datastax.com/view/trunk/job/trunk_dtest/315/testReport/junit/jmx_test/TestJMX/cfhistograms_test/]).
>  Here's the failure history for trunk:
> http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/junit/jmx_test/TestJMX/cfhistograms_test/history/
> I haven't seen it fail locally yet, but haven't run the test more than a 
> couple times because it takes a while.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r10717 - in /release/cassandra: 2.0.16/ 2.1.10/ 2.2.2/ debian/dists/21x/ debian/dists/21x/main/binary-amd64/ debian/dists/21x/main/binary-i386/ debian/dists/21x/main/source/ debian/dists/2

[jake]

Author: jake
Date: Mon Oct  5 13:51:35 2015
New Revision: 10717

Log:
2.1.10 and 2.2.2 releases

Added:
release/cassandra/2.1.10/
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz   (with props)
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.md5
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.sha1
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.md5
release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.sha1
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz   (with props)
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz.asc
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz.asc.md5
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz.asc.sha1
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz.md5
release/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz.sha1
release/cassandra/2.2.2/
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz   (with props)
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.asc
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.asc.md5
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.asc.sha1
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.md5
release/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.sha1
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz   (with props)
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.asc
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.asc.md5
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.asc.sha1
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.md5
release/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.sha1
Removed:
release/cassandra/2.0.16/
Modified:
release/cassandra/debian/dists/21x/InRelease
release/cassandra/debian/dists/21x/Release
release/cassandra/debian/dists/21x/Release.gpg
release/cassandra/debian/dists/21x/main/binary-amd64/Packages
release/cassandra/debian/dists/21x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/21x/main/binary-i386/Packages
release/cassandra/debian/dists/21x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/21x/main/source/Sources.gz
release/cassandra/debian/dists/22x/InRelease
release/cassandra/debian/dists/22x/Release
release/cassandra/debian/dists/22x/Release.gpg
release/cassandra/debian/dists/22x/main/binary-amd64/Packages
release/cassandra/debian/dists/22x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/22x/main/binary-i386/Packages
release/cassandra/debian/dists/22x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/22x/main/source/Sources.gz

Added: release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc
==
--- release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc (added)
+++ release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc Mon Oct  5 
13:51:35 2015
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAABAgAGBQJWDTjvAAoJEHSdbuwDU7EsookP/jGvDOj8c584zrToV4p7+CDm
+mygGE1JXc2eG6EHNYGh1c9B4ByDTFwK6Wpya7iUtCMqiYALeRwMxJBGBYRPgziya
+ljTW7azlZnTiYb7BA7h07eN3BKVMNIItxW8mmdyMdH3+kX3XCdg1OObyBfKEVrqh
+Si+MwaXxY8xtNBfVIzSI91aSI2aOcX9cRYwVWdB8OK9cup+F8nNDbxfpdnnnkF5D
+PzvxmaFXSbenz9WodxLi9VttvfQHrBwxw0g/UZ3npAbxWqw8s1lC36CVK2i0E2e0
+jIU/MUCJaXHPl+pPVOYop0HsiWVuOeMFzRTGqs86KIGUmkUJQtw2I1pqbGzO+eW7
+V5HDBdEtvbMIbwpirhqV3+N4yf39AdXa24R1dclS9Q1vP/3MSQYGHkdNS+WnndPU
+5VxKYo07cr6AlGT3OQZBfabM+GsKSGb3ovqjhaob2YLQz9cjbNmHvb4/iBpJWo1f
+rgO18VNaYrOQfbxRGD+5VrOsdRoLbWDyDIm7QH/aKgSL1xOxZdKS+TLmzvjwZx12
+IgD3SwE/MXj0GoikzH5OxLCpNy7G15whktFV1dSDJax3CSpz0cukrGu5YjslTAPB
+RvB9p0Oppoiy+4LDRvKpfDv97CpBqg/17ZPbMtJH5RzIZ/kZd4wsR9vwBGsiS97G
+BJjkd9c2678qULjM73Ts
+=mHj8
+-END PGP SIGNATURE-

Added: release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.md5
==
--- release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.md5 (added)
+++ release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.md5 Mon Oct 
 5 13:51:35 2015
@@ -0,0 +1 @@
+29cda4a301ce0f249e2e98d43a4f7405
\ No newline at end of file

Added: release/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc.sha1
==
--- 

[jira] [Commented] (CASSANDRA-10388) Windows dtest 3.0: SSL dtests are failing

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943440#comment-14943440
 ] 

Joshua McKenzie commented on CASSANDRA-10388:
-

The fact that we still get the NoHostAvailableException in the driver makes 
this a non-concern to me.

re: the patch - I'd recommend skipping that check on Windows but still 
performing it on Linux rather than removing it entirely. 
[ccm.common.is_win()|https://github.com/pcmanus/ccm/blob/master/ccmlib/common.py#L255-L256]
 is the idiomatic way to branch based on that. 

> Windows dtest 3.0: SSL dtests are failing
> -
>
> Key: CASSANDRA-10388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10388
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Philip Thompson
>Assignee: Joel Knighton
> Fix For: 2.2.x, 3.0.x
>
>
> The dtests 
> {{native_transport_ssl_test.NativeTransportSSL.connect_to_ssl_test}} and 
> {{native_transport_ssl_test.NativeTransportSSL.use_custom_ssl_port_test}} are 
> failing on windows, but not linux.
> Stacktrace is
> {code}
>   File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
>   File 
> "D:\jenkins\workspace\cassandra-3.0_dtest_win32\cassandra-dtest\native_transport_ssl_test.py",
>  line 32, in connect_to_ssl_test
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9625) GraphiteReporter not reporting

[Jean-Francois Gosselin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943468#comment-14943468
 ] 

Jean-Francois Gosselin commented on CASSANDRA-9625:
---

I ran into another assert that breaks the GraphiteReporter on 2.1.9 . When 
SSTableReader.getApproximateKeyCount is called, how can I get in a state where 
the CompactionMetadata is null ?

{code:title=SSTableReader.java|borderStyle=solid}
276try
278{
279CompactionMetadata metadata = (CompactionMetadata) 
sstable.descriptor.getMetadataSerializer().deserialize(sstable.descriptor, 
MetadataType.COMPACTION);
280assert metadata != null : sstable.getFilename();
281if (cardinality == null)
{code}

{noformat}
at 
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:279)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:296)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:290)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:292)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:27)
 
at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28) 
at 
com.yammer.metrics.reporting.GraphiteReporter.printRegularMetrics(GraphiteReporter.java:235)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.run(GraphiteReporter.java:199) 
{noformat}




> GraphiteReporter not reporting
> --
>
> Key: CASSANDRA-9625
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9625
> Project: Cassandra
>  Issue Type: Bug
> Environment: Debian Jessie, 7u79-2.5.5-1~deb8u1, Cassandra 2.1.3
>Reporter: Eric Evans
> Attachments: metrics.yaml, thread-dump.log
>
>
> When upgrading from 2.1.3 to 2.1.6, the Graphite metrics reporter stops 
> working.  The usual startup is logged, and one batch of samples is sent, but 
> the reporting interval comes and goes, and no other samples are ever sent.  
> The logs are free from errors.
> Frustratingly, metrics reporting works in our smaller (staging) environment 
> on 2.1.6; We are able to reproduce this on all 6 of production nodes, but not 
> on a 3 node (otherwise identical) staging cluster (maybe it takes a certain 
> level of concurrency?).
> Attached is a thread dump, and our metrics.yaml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9625) GraphiteReporter not reporting

[Jean-Francois Gosselin (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943468#comment-14943468
 ] 

Jean-Francois Gosselin edited comment on CASSANDRA-9625 at 10/5/15 2:51 PM:


I ran into another assert that breaks the GraphiteReporter on 2.1.9 . When 
SSTableReader.getApproximateKeyCount is called, how can I get in a state where 
the CompactionMetadata is null ?

{code:title=SSTableReader.java|borderStyle=solid}
276  try
278  {
279 CompactionMetadata metadata = (CompactionMetadata) 
sstable.descriptor.getMetadataSerializer().deserialize(sstable.descriptor, 
MetadataType.COMPACTION);
280 assert metadata != null : sstable.getFilename();
281 if (cardinality == null)
{code}

{noformat}
at 
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:279)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:296)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:290)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:292)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:27)
 
at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28) 
at 
com.yammer.metrics.reporting.GraphiteReporter.printRegularMetrics(GraphiteReporter.java:235)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.run(GraphiteReporter.java:199) 
{noformat}





was (Author: jfgosselin):
I ran into another assert that breaks the GraphiteReporter on 2.1.9 . When 
SSTableReader.getApproximateKeyCount is called, how can I get in a state where 
the CompactionMetadata is null ?

{code:title=SSTableReader.java|borderStyle=solid}
276try
278{
279CompactionMetadata metadata = (CompactionMetadata) 
sstable.descriptor.getMetadataSerializer().deserialize(sstable.descriptor, 
MetadataType.COMPACTION);
280assert metadata != null : sstable.getFilename();
281if (cardinality == null)
{code}

{noformat}
at 
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:279)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:296)
 
at 
org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:290)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:292)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.processGauge(GraphiteReporter.java:27)
 
at com.yammer.metrics.core.Gauge.processWith(Gauge.java:28) 
at 
com.yammer.metrics.reporting.GraphiteReporter.printRegularMetrics(GraphiteReporter.java:235)
 
at 
com.yammer.metrics.reporting.GraphiteReporter.run(GraphiteReporter.java:199) 
{noformat}




> GraphiteReporter not reporting
> --
>
> Key: CASSANDRA-9625
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9625
> Project: Cassandra
>  Issue Type: Bug
> Environment: Debian Jessie, 7u79-2.5.5-1~deb8u1, Cassandra 2.1.3
>Reporter: Eric Evans
> Attachments: metrics.yaml, thread-dump.log
>
>
> When upgrading from 2.1.3 to 2.1.6, the Graphite metrics reporter stops 
> working.  The usual startup is logged, and one batch of samples is sent, but 
> the reporting interval comes and goes, and no other samples are ever sent.  
> The logs are free from errors.
> Frustratingly, metrics reporting works in our smaller (staging) environment 
> on 2.1.6; We are able to reproduce this on all 6 of production nodes, but not 
> on a 3 node (otherwise identical) staging cluster (maybe it takes a certain 
> level of concurrency?).
> Attached is a thread dump, and our metrics.yaml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.1.10 [created] 255fbe689

[jira] [Updated] (CASSANDRA-10327) Performance regression in 2.2

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-10327:
---
Fix Version/s: (was: 2.2.2)
   2.2.x

> Performance regression in 2.2
> -
>
> Key: CASSANDRA-10327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10327
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
> Fix For: 2.2.x
>
>
> Related to CASSANDRA-10326, one of the read-only workloads _appears_ to show 
> a regression in 2.2, however it is possible this is simply down to a 
> different compaction result (this shouldn't be very likely given the use of 
> LCS, though, and that we wait for compaction to acquiesce, and while the 
> different is not consistent across both runs, it is consistently worse).
> The query is looking up the last item of a partition.
> [run1|http://cstar.datastax.com/graph?stats=f0a17292-5a13-11e5-847a-42010af0688f=op_rate=3_user=1_aggregates=true=0=155.43=0=13777.5]
> [run2|http://cstar.datastax.com/graph?stats=e250-5a13-11e5-ae0d-42010af0688f=op_rate=3_user=1_aggregates=true=0=74.36=0=34078]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1706873 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py

[jake]

Author: jake
Date: Mon Oct  5 17:00:51 2015
New Revision: 1706873

URL: http://svn.apache.org/viewvc?rev=1706873=rev
Log:
new versions

Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1706873=1706872=1706873=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Mon Oct  5 17:00:51 2015
@@ -55,21 +55,21 @@
There are currently two active releases available:


-  The latest release of Apache Cassandra is 2.2.1
-  (released on 2015-09-01).  If you're just
+  The latest release of Apache Cassandra is 2.2.2
+  (released on 2015-10-05).  If you're just
   starting out and not yet in production, download this one.
 
   

  
http://www.apache.org/dyn/closer.lua/cassandra/2.2.1/apache-cassandra-2.2.1-bin.tar.gz;
+ 
href="http://www.apache.org/dyn/closer.lua/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz;
  onclick="javascript: 
pageTracker._trackPageview('/clicks/binary_download');">
-apache-cassandra-2.2.1-bin.tar.gz
+apache-cassandra-2.2.2-bin.tar.gz

-   [http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-bin.tar.gz.asc;>PGP]
-   [http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-bin.tar.gz.md5;>MD5]
-   [http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-bin.tar.gz.sha1;>SHA1]
+   [http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.asc;>PGP]
+   [http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.md5;>MD5]
+   [http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-bin.tar.gz.sha1;>SHA1]
  
  
http://wiki.apache.org/cassandra/DebianPackaging;>Debian 
installation instructions
@@ -77,16 +77,16 @@

 

- The most stable release of Apache Cassandra is 2.1.9
- (released on 2015-08-28).  If you are in production or planning to be 
soon, download this one.
+ The most stable release of Apache Cassandra is 2.1.10
+ (released on 2015-10-05).  If you are in production or planning to be 
soon, download this one.

 

  
-   http://www.apache.org/dyn/closer.lua/cassandra/2.1.9/apache-cassandra-2.1.9-bin.tar.gz;>apache-cassandra-2.1.9-bin.tar.gz
-   [http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-bin.tar.gz.asc;>PGP]
-   [http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-bin.tar.gz.md5;>MD5]
-   [http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-bin.tar.gz.sha1;>SHA1]
+   http://www.apache.org/dyn/closer.lua/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz;>apache-cassandra-2.1.10-bin.tar.gz
+   [http://www.apache.org/dist/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.asc;>PGP]
+   [http://www.apache.org/dist/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.md5;>MD5]
+   [http://www.apache.org/dist/cassandra/2.1.10/apache-cassandra-2.1.10-bin.tar.gz.sha1;>SHA1]
  
 
 http://wiki.apache.org/cassandra/DebianPackaging;>Debian 
installation instructions
@@ -171,20 +171,20 @@
   
 
 http://www.apache.org/dyn/closer.lua/cassandra/2.2.1/apache-cassandra-2.2.1-src.tar.gz;
+   
href="http://www.apache.org/dyn/closer.lua/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz;
onclick="javascript: 
pageTracker._trackPageview('/clicks/source_download');">
-  apache-cassandra-2.2.1-src.tar.gz
+  apache-cassandra-2.2.2-src.tar.gz
 
-[http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-src.tar.gz.asc;>PGP]
-[http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-src.tar.gz.md5;>MD5]
-[http://www.apache.org/dist/cassandra/2.2.1/apache-cassandra-2.2.1-src.tar.gz.sha1;>SHA1]
+[http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.asc;>PGP]
+[http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.md5;>MD5]
+[http://www.apache.org/dist/cassandra/2.2.2/apache-cassandra-2.2.2-src.tar.gz.sha1;>SHA1]
 
   
 
-http://www.apache.org/dyn/closer.lua/cassandra/2.1.9/apache-cassandra-2.1.9-src.tar.gz;>apache-cassandra-2.1.9-src.tar.gz
-[http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-src.tar.gz.asc;>PGP]
-[http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-src.tar.gz.md5;>MD5]
-[http://www.apache.org/dist/cassandra/2.1.9/apache-cassandra-2.1.9-src.tar.gz.sha1;>SHA1]
+http://www.apache.org/dyn/closer.lua/cassandra/2.1.10/apache-cassandra-2.1.10-src.tar.gz;>apache-cassandra-2.1.10-src.tar.gz
+

[jira] [Updated] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[J.P. Eiti Kimura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.P. Eiti Kimura updated CASSANDRA-10233:
-
Attachment: cassandra-2.1.8-10233-v3.txt

[~pauloricardomg] please, see the file: cassandra-2.1.8-10233-v3.txt
I did the following improvement:

{code:java}
public static void writeHintForMutation(Mutation mutation, long now, int 
ttl, InetAddress target)
{
Preconditions.checkArgument(ttl > 0, "the ttl must be > 0");
UUID hostId = 
StorageService.instance.getTokenMetadata().getHostId(target);
Preconditions.checkNotNull(hostId, "Missing host ID for " + 
target.getHostAddress());
HintedHandOffManager.instance.hintFor(mutation, now, ttl, 
hostId).apply();
StorageMetrics.totalHints.inc();
}
{code}

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, 
> cassandra-2.1.8-10233-v3.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10388) Windows dtest 3.0: SSL dtests are failing

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943521#comment-14943521
 ] 

Joel Knighton commented on CASSANDRA-10388:
---

Thanks for the tip - I've PRed the updated version of the dtest fix with the 
conditional check as [#580|https://github.com/riptano/cassandra-dtest/pull/580].

I'm working on getting the JCE fixes on CassCI machines still.

> Windows dtest 3.0: SSL dtests are failing
> -
>
> Key: CASSANDRA-10388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10388
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Philip Thompson
>Assignee: Joel Knighton
> Fix For: 2.2.x, 3.0.x
>
>
> The dtests 
> {{native_transport_ssl_test.NativeTransportSSL.connect_to_ssl_test}} and 
> {{native_transport_ssl_test.NativeTransportSSL.use_custom_ssl_port_test}} are 
> failing on windows, but not linux.
> Stacktrace is
> {code}
>   File "C:\tools\python2\lib\unittest\case.py", line 329, in run
> testMethod()
>   File 
> "D:\jenkins\workspace\cassandra-3.0_dtest_win32\cassandra-dtest\native_transport_ssl_test.py",
>  line 32, in connect_to_ssl_test
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[J.P. Eiti Kimura (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.P. Eiti Kimura updated CASSANDRA-10233:
-
Attachment: (was: cassandra-2.1.8-10233.txt)

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943601#comment-14943601
 ] 

Jim Witschey commented on CASSANDRA-9126:
-

[~dkblinux98] and [~sgottipati] do you have any more details, INFO-level logs 
in particular?

[~krummas], [~yukim], [~benedict] Any thoughts on next steps here? At the very 
least, is it possible to guide people toward {{nodetool scrub}} with error 
reporting?

> java.lang.RuntimeException: Last written key DecoratedKey >= current key 
> DecoratedKey
> -
>
> Key: CASSANDRA-9126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9126
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: srinivasu gottipati
>Priority: Critical
> Fix For: 2.1.x
>
> Attachments: cassandra-system.log
>
>
> Cassandra V: 2.0.14,
> Getting the following exceptions while trying to compact (I see this issue 
> was raised in earlier versions and marked as closed. However it still appears 
> in 2.0.14). In our case, compaction is not getting succeeded and keep failing 
> with this error.:
> {code}java.lang.RuntimeException: Last written key 
> DecoratedKey(3462767860784856708, 
> 354038323137333038305f3330325f31355f474d4543454f) >= current key 
> DecoratedKey(3462334604624154281, 
> 354036333036353334315f3336315f31355f474d4543454f) writing into {code}
> ...
> Stacktrace:{code}
>   at 
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
>   at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745){code}
> Any help is greatly appreciated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7032) Improve vnode allocation

[Sebastian Estevez (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943678#comment-14943678
 ] 

Sebastian Estevez commented on CASSANDRA-7032:
--

Is there a way we can allow this to work without specifying the keyspace?

How about using a weighted average of ownership across all existing keyspaces?

> Improve vnode allocation
> 
>
> Key: CASSANDRA-7032
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Branimir Lambov
>  Labels: performance, vnodes
> Fix For: 3.0 alpha 1
>
> Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java, 
> TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java, 
> TestVNodeAllocation.java
>
>
> It's been known for a little while that random vnode allocation causes 
> hotspots of ownership. It should be possible to improve dramatically on this 
> with deterministic allocation. I have quickly thrown together a simple greedy 
> algorithm that allocates vnodes efficiently, and will repair hotspots in a 
> randomly allocated cluster gradually as more nodes are added, and also 
> ensures that token ranges are fairly evenly spread between nodes (somewhat 
> tunably so). The allocation still permits slight discrepancies in ownership, 
> but it is bound by the inverse of the size of the cluster (as opposed to 
> random allocation, which strangely gets worse as the cluster size increases). 
> I'm sure there is a decent dynamic programming solution to this that would be 
> even better.
> If on joining the ring a new node were to CAS a shared table where a 
> canonical allocation of token ranges lives after running this (or a similar) 
> algorithm, we could then get guaranteed bounds on the ownership distribution 
> in a cluster. This will also help for CASSANDRA-6696.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10424) Altering base table column with materialized view causes unexpected server error.

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943622#comment-14943622
 ] 

Carl Yeksigian commented on CASSANDRA-10424:


Pushed a new commit that addresses the latest comment.

- Added an int -> blob alter test
- That was a bad copy-paste job, removed and switched back to *
- Removed the {{reversed()}} part of the text

> Altering base table column with materialized view causes unexpected server 
> error.
> -
>
> Key: CASSANDRA-10424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10424
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: cassandra-3.0.0-rc1, with python driver 3.0-alpha
>Reporter: Greg Bestland
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc2
>
>
> When attempting to alter column type of base table which has a corresponding  
> materialized view we get an exception from the server.
> Steps to reproduce.
> 1. Create a base table
> {code}
> CREATE TABLE test.scores(
> user TEXT,
> game TEXT,
> year INT,
> month INT,
> day INT,
> score TEXT,
> PRIMARY KEY (user, game, year, month, day)
> )
> {code}
> 2. Create a corresponding materialized view
> {code}
> CREATE MATERIALIZED VIEW test.monthlyhigh AS
> SELECT game, year, month, score, user, day FROM test.scores
> WHERE game IS NOT NULL AND year IS NOT NULL AND month IS NOT 
> NULL AND score IS NOT NULL AND user IS NOT NULL AND day IS NOT NULL
> PRIMARY KEY ((game, year, month), score, user, day)
> WITH CLUSTERING ORDER BY (score DESC, user ASC, day ASC)
> {code}
> 3. Attempt to Alter the base table 
> {code}
> ALTER TABLE test.scores ALTER score TYPE blob
> {code}
> In the python driver we see the following exception returned from the server
> {code}
> Ignoring schedule_unique for already-scheduled task: ( ControlConnection.refresh_schema of  object at 0x100f72c50>>, (), (('keyspace', 'test'), ('target_type', 
> 'KEYSPACE'), ('change_type', 'UPDATED')))
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "./cassandra/cluster.py", line 1623, in execute
> result = future.result()
>   File "./cassandra/cluster.py", line 3205, in result
> raise self._final_exception
> cassandra.protocol.ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.ConfigurationException: Column family 
> comparators do not match or are not compatible (found ClusteringComparator; 
> expected ClusteringComparator).">
> {code}
> On the server I see the following stack trace
> {code}
> INFO  [MigrationStage:1] 2015-09-30 11:45:47,457 ColumnFamilyStore.java:825 - 
> Enqueuing flush of keyspaces: 512 (0%) on-heap, 0 (0%) off-heap
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,457 Memtable.java:362 - 
> Writing Memtable-keyspaces@1714565887(0.146KiB serialized bytes, 1 ops, 0%/0% 
> of on/off-heap limit)
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,463 Memtable.java:395 - 
> Completed flushing 
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/ma-54-big-Data.db
>  (0.109KiB) for commitlog position ReplayPosition(segmentId=1443623481894, 
> position=9812)
> INFO  [MigrationStage:1] 2015-09-30 11:45:47,472 ColumnFamilyStore.java:825 - 
> Enqueuing flush of columns: 877 (0%) on-heap, 0 (0%) off-heap
> INFO  [MemtableFlushWriter:12] 2015-09-30 11:45:47,472 Memtable.java:362 - 
> Writing Memtable-columns@771367282(0.182KiB serialized bytes, 1 ops, 0%/0% of 
> on/off-heap limit)
> INFO  [MemtableFlushWriter:12] 2015-09-30 11:45:47,478 Memtable.java:395 - 
> Completed flushing 
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/columns-24101c25a2ae3af787c1b40ee1aca33f/ma-51-big-Data.db
>  (0.107KiB) for commitlog position ReplayPosition(segmentId=1443623481894, 
> position=9812)
> INFO  [MigrationStage:1] 2015-09-30 11:45:47,490 ColumnFamilyStore.java:825 - 
> Enqueuing flush of views: 2641 (0%) on-heap, 0 (0%) off-heap
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,490 Memtable.java:362 - 
> Writing Memtable-views@1740452585(0.834KiB serialized bytes, 1 ops, 0%/0% of 
> on/off-heap limit)
> INFO  [MemtableFlushWriter:11] 2015-09-30 11:45:47,496 Memtable.java:395 - 
> Completed flushing 
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/views-9786ac1cdd583201a7cdad556410c985/ma-22-big-Data.db
>  (0.542KiB) for commitlog position ReplayPosition(segmentId=1443623481894, 
> position=9812)
> ERROR [MigrationStage:1] 

[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager

[J.P. Eiti Kimura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943659#comment-14943659
 ] 

J.P. Eiti Kimura commented on CASSANDRA-10233:
--

[~pauloricardomg] I think this method writeHintForMutation is from class 
StorageProxy and not for the HintedHandoffManager, right? 

> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Paulo Motta
> Attachments: cassandra-2.1.8-10233-v2.txt, cassandra-2.1.8-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing 
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
>  ]
> INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 
> sstables to 
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] 
> to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 
> 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were 
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - 
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
>   at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
>   at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) 
> ~[na:1.7.0_79]
>   at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) 
> ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
>  ~[apache-cassandra-2.2.0.jar:2.2.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
>   at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)