[jira] [Commented] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

2017-12-12 Thread Rob Emery (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287278#comment-16287278
 ] 

Rob Emery commented on CASSANDRA-12978:
---

Hi Jeff,

That sounds perfectly acceptable.

Many Thanks,
Rob

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
>  Labels: proposed-wontfix
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

2016-11-30 Thread Rob Emery (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Emery updated CASSANDRA-12978:
--
Priority: Critical  (was: Major)

> mx4j -> HTTP 500 -> ConcurrentModificationException
> ---
>
> Key: CASSANDRA-12978
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 
> 16GB, RAID1 SSD Commit log, RAID10 15k HDD data
>Reporter: Rob Emery
>Priority: Critical
> Fix For: 2.1.6
>
>
> We run some checks from our Monitoring software that rely on mx4j.
> The checks typically grab some xml via HTTP request and parse it. For 
> example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
> using:
> http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily
> The checks run each minute. Periodically they result in a "HTTP 500 internal 
> server error". The HTML body returned is empty.
> Experimentally we ran Cassandra in the foreground on one node and reproduced 
> the problem. this elicited the following stack trace:
> javax.management.RuntimeMBeanException: 
> java.util.ConcurrentModificationException
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
> at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
> at 
> mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
> at 
> mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
> Caused by: java.util.ConcurrentModificationException
> at 
> java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
> at 
> java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
> at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
> at 
> org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
> at 
> org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
> at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at 
> com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
> at 
> com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
> ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-12978) mx4j -> HTTP 500 -> ConcurrentModificationException

2016-11-30 Thread Rob Emery (JIRA)
Rob Emery created CASSANDRA-12978:
-

 Summary: mx4j -> HTTP 500 -> ConcurrentModificationException
 Key: CASSANDRA-12978
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12978
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Debian, Single cluster, 2 data centres, E5-2620 v3, 16GB, 
RAID1 SSD Commit log, RAID10 15k HDD data
Reporter: Rob Emery
 Fix For: 2.1.6


We run some checks from our Monitoring software that rely on mx4j.

The checks typically grab some xml via HTTP request and parse it. For 
example, CF Stats on 'MyKeySpace' and 'MyColumnFamily' are retrieved 
using:

http://cassandra001:8081/mbean?template=identity=org.apache.cassandra.db%3Atype%3DColumnFamilies%2Ckeyspace%3DMyKeySpace%2Ccolumnfamily%3DMyColumnFamily

The checks run each minute. Periodically they result in a "HTTP 500 internal 
server error". The HTML body returned is empty.

Experimentally we ran Cassandra in the foreground on one node and reproduced 
the problem. this elicited the following stack trace:

javax.management.RuntimeMBeanException: 
java.util.ConcurrentModificationException
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at 
mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
at 
mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
at 
mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
Caused by: java.util.ConcurrentModificationException
at 
java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1594)
at 
java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1642)
at 
java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1636)
at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
at 
org.apache.cassandra.utils.StreamingHistogram.sum(StreamingHistogram.java:160)
at 
org.apache.cassandra.io.sstable.metadata.StatsMetadata.getDroppableTombstonesBefore(StatsMetadata.java:113)
at 
org.apache.cassandra.io.sstable.SSTableReader.getDroppableTombstonesBefore(SSTableReader.java:2004)
at 
org.apache.cassandra.db.DataTracker.getDroppableTombstoneRatio(DataTracker.java:507)
at 
org.apache.cassandra.db.ColumnFamilyStore.getDroppableTombstoneRatio(ColumnFamilyStore.java:3089)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at 
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at 
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at 
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
... 4 more




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-12871) Debian package no longer exports EXTRA_CLASSPATH

2016-11-29 Thread Rob Emery (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Emery updated CASSANDRA-12871:
--
Since Version: 2.1.0
  Component/s: Tools

> Debian package no longer exports EXTRA_CLASSPATH
> 
>
> Key: CASSANDRA-12871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12871
> Project: Cassandra
>  Issue Type: Bug
>  Components: Packaging, Tools
> Environment: Debian Jessie/Ubuntu 12.04+; Upgrading 2.0.17 to 2.1.16
>Reporter: Rob Emery
>
> We use mx4j to monitor Cassandra; in 2.0.17 we uncomment the EXTRA_CLASSPATH 
> in /etc/default/cassandra and cassandra loads the mx4j jar and provides the 
> web interface on port 8081.
> In 2.1.16 the export of EXTRA_CLASSPATH has been removed 
> (https://github.com/apache/cassandra/commit/2ba394676dc673b6d66a07247dccd122b64b0578)
>  in this commit meaning that the mx4j jar is never discovered.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-12871) Debian package no longer exports EXTRA_CLASSPATH

2016-11-02 Thread Rob Emery (JIRA)
Rob Emery created CASSANDRA-12871:
-

 Summary: Debian package no longer exports EXTRA_CLASSPATH
 Key: CASSANDRA-12871
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12871
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
 Environment: Debian Jessie/Ubuntu 12.04+; Upgrading 2.0.17 to 2.1.16
Reporter: Rob Emery


We use mx4j to monitor Cassandra; in 2.0.17 we uncomment the EXTRA_CLASSPATH in 
/etc/default/cassandra and cassandra loads the mx4j jar and provides the web 
interface on port 8081.

In 2.1.16 the export of EXTRA_CLASSPATH has been removed 
(https://github.com/apache/cassandra/commit/2ba394676dc673b6d66a07247dccd122b64b0578)
 in this commit meaning that the mx4j jar is never discovered.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-10041) "timeout during write query at consistency ONE" when updating counter at consistency QUORUM and 2 of 3 nodes alive

2016-04-07 Thread Rob Emery (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230017#comment-15230017
 ] 

Rob Emery commented on CASSANDRA-10041:
---

Interesting; from our perspective the nodes that intermittently throw this 
exception are not actually down. They are delivering other workloads without 
any apparent problem. We just get a few of these per day and have been unable 
to track down any cause (there does not appear to be any IO pressure or GC 
pauses etc); therefore for our situation all information, no matter how small, 
is vital for us to be able to diagnose the problem. Currently we have no way of 
tracking the point during the mutation that the issue is actually occurring. 

> "timeout during write query at consistency ONE" when updating counter at 
> consistency QUORUM and 2 of 3 nodes alive
> --
>
> Key: CASSANDRA-10041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10041
> Project: Cassandra
>  Issue Type: Bug
> Environment: centos 6.6 server, java version "1.8.0_45", cassandra 
> 2.1.8, 3 machines, keyspace with replication factor 3
>Reporter: Anton Lebedevich
> Fix For: 2.1.x
>
>
> Test scenario is: kill -9 one node, wait 60 seconds, start it back, wait till 
> it becomes available, wait 120 seconds (during that time all 3 nodes are up), 
> repeat with the next node. Application reads from one table and updates 
> counters in another table with consistency QUORUM. When one node out of 3 is 
> killed application logs this exception for several seconds:
> {noformat}
> Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: 
> Cassandra timeout during write query at consistency ONE (1 replica were 
> required but only 0 acknowledged the write)
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:57) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:204) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:195) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>  [io.netty.netty-codec-4.0.27.Final.jar:4.0.27.Final]
> ... 13 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-10041) "timeout during write query at consistency ONE" when updating counter at consistency QUORUM and 2 of 3 nodes alive

2016-04-06 Thread Rob Emery (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228625#comment-15228625
 ] 

Rob Emery edited comment on CASSANDRA-10041 at 4/6/16 4:52 PM:
---

Hi Joel,

Am I reading this correctly in that this implies that the UPDATE query isn't 
being sent to a replica? Which would imply that we have misconfigured the 
partition key within our queries?

Thanks,


was (Author: re_weavers):
Hi Joel,

Am I reading this correctly in that this imply that the UPDATE query isn't 
being sent to a replica? Which would imply that we have misconfigured the 
partition key within our queries?

Thanks,

> "timeout during write query at consistency ONE" when updating counter at 
> consistency QUORUM and 2 of 3 nodes alive
> --
>
> Key: CASSANDRA-10041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10041
> Project: Cassandra
>  Issue Type: Bug
> Environment: centos 6.6 server, java version "1.8.0_45", cassandra 
> 2.1.8, 3 machines, keyspace with replication factor 3
>Reporter: Anton Lebedevich
> Fix For: 2.1.x
>
>
> Test scenario is: kill -9 one node, wait 60 seconds, start it back, wait till 
> it becomes available, wait 120 seconds (during that time all 3 nodes are up), 
> repeat with the next node. Application reads from one table and updates 
> counters in another table with consistency QUORUM. When one node out of 3 is 
> killed application logs this exception for several seconds:
> {noformat}
> Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: 
> Cassandra timeout during write query at consistency ONE (1 replica were 
> required but only 0 acknowledged the write)
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:57) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:204) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:195) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>  [io.netty.netty-codec-4.0.27.Final.jar:4.0.27.Final]
> ... 13 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-10041) "timeout during write query at consistency ONE" when updating counter at consistency QUORUM and 2 of 3 nodes alive

2016-04-06 Thread Rob Emery (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228625#comment-15228625
 ] 

Rob Emery commented on CASSANDRA-10041:
---

Hi Joel,

Am I reading this correctly in that this imply that the UPDATE query isn't 
being sent to a replica? Which would imply that we have misconfigured the 
partition key within our queries?

Thanks,

> "timeout during write query at consistency ONE" when updating counter at 
> consistency QUORUM and 2 of 3 nodes alive
> --
>
> Key: CASSANDRA-10041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10041
> Project: Cassandra
>  Issue Type: Bug
> Environment: centos 6.6 server, java version "1.8.0_45", cassandra 
> 2.1.8, 3 machines, keyspace with replication factor 3
>Reporter: Anton Lebedevich
> Fix For: 2.1.x
>
>
> Test scenario is: kill -9 one node, wait 60 seconds, start it back, wait till 
> it becomes available, wait 120 seconds (during that time all 3 nodes are up), 
> repeat with the next node. Application reads from one table and updates 
> counters in another table with consistency QUORUM. When one node out of 3 is 
> killed application logs this exception for several seconds:
> {noformat}
> Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: 
> Cassandra timeout during write query at consistency ONE (1 replica were 
> required but only 0 acknowledged the write)
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:57) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Responses$Error$1.decode(Responses.java:37) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:204) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:195) 
> ~[com.datastax.cassandra.cassandra-driver-core-2.1.6.jar:na]
> at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>  [io.netty.netty-codec-4.0.27.Final.jar:4.0.27.Final]
> ... 13 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-2103) expiring counter columns

2015-04-15 Thread Rob Emery (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495843#comment-14495843
 ] 

Rob Emery commented on CASSANDRA-2103:
--

I also agree with Nikolay, Amol and Marco. This seems a little clunky to not 
have any way of expiring counters. In our (presumably fairly common usecase) we 
have different granularities of statistics stored in time buckets, after the 
first day then the minute bucket becomes pointless. Currently the only way for 
us to dispose of the unused data would be to hack it with a cronjob and delete 
the buckets for the previous day, which just feels really unpleasant versus the 
elegance of using TTLs on columns. I would concur with the desired behaviour of 
setting the TTL on the first upsert and then ignoring attempts to set it on 
subsequent updates.

 expiring counter columns
 

 Key: CASSANDRA-2103
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2103
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Affects Versions: 0.8 beta 1
Reporter: Kelvin Kakugawa
 Attachments: 0001-CASSANDRA-2103-expiring-counters-logic-tests.patch


 add ttl functionality to counter columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)