[Cassandra Wiki] Update of Counters by SylvainLebresne
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The Counters page has been changed by SylvainLebresne:
http://wiki.apache.org/cassandra/Counters?action=diffrev1=13rev2=14
== Getting started ==
- In order to use counters in Cassandra you need to first set up a cluster, see
the [[http://wiki.apache.org/cassandra/GettingStarted|Getting started]] guide
for more information. Currently the counters code is slated for a release in
Cassandra 0.8.0.
+ In order to use counters in Cassandra you need to first set up a cluster, see
the [[http://wiki.apache.org/cassandra/GettingStarted|Getting started]] guide
for more information. Also note that counters are available starting with
Cassandra 0.8.0.
Configuration
+ To use counters, you have to define a column family (or super column family)
whose columns will act as counters. To create such a column family `counterCF`
using the CLI, you will have to do:
- When first setting up the cluster you can specify a column or super column
that will act as counters.
- For example using the cassandra.yaml configuration file:
-
{{{
+ [default@test] create column family counterCF with
default_validation_class=CounterColumnType and replicate_on_write=true;
- - name: Counter1
- default_validation_class: CounterColumnType
-
- - name: SuperCounter1
- column_type: Super
- default_validation_class: CounterColumnType
}}}
+ Setting the `default_validation_class` to `CounterColumnType` indicates that
the column will be counters. Setting `replicate_on_write=true` will be optional
starting in 0.8.2, but a bug made it default to false in 0.8.0 and 0.8.1, which
is unsafe.
Incrementing and accessing counters
[Cassandra Wiki] Update of Counters by SylvainLebresne
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The Counters page has been changed by SylvainLebresne.
http://wiki.apache.org/cassandra/Counters?action=diffrev1=10rev2=11
--
And read it back
{{{
- rv = client.get_counter('key1', ColumnPath(column_family='Counter1',
column='c1'), ConsistencyLevel.ONE)
+ rv = client.get('key1', ColumnPath(column_family='Counter1', column='c1'),
ConsistencyLevel.ONE).counter_column.value
}}}
Please read the rest of this wiki page, especially Technical limitations and
Operational considerations to make sure this actually does what you need.
@@ -39, +39 @@
== Interface ==
The interface follows the main API. The main differences are:
- * CounterColumn requires an i64 value (can be negative) and no timestamp, and
+ * CounterColumn requires an i64 value (can be negative) and no timestamp,
+ * Counter can be used inside super columns using the CounterSuperColumn
structure, and
* Deletion, when used on a counter column family, does not use a timestamp.
Internally, the data store generates timestamps on the server to determine
priority of deletion.
@@ -55, +56 @@
2: required listCounterColumn columns
}
- struct Counter {
+ struct ColumnOrSuperColumn {
- 1: optional CounterColumn column,
+ 1: optional Column column,
- 2: optional CounterSuperColumn super_column
+ 2: optional SuperColumn super_column,
+ 3: optional CounterColumn counter_column,
+ 4: optional CounterSuperColumn counter_super_column
}
}}}
+ where the pre-existing ColumnOrSuperColumn has the two new fields, specific
to counters, `counter_column` and `counter_super_column`.
- Moreover, as mentioned previously, then timestamp field of Deletion is now
optional (but
- remain mandatory for non counter column family operation).
- The Mutation has also been updated with a new `counter` field for batch
increment/decrement.
+ Moreover, as mentioned previously, the timestamp field of Deletion is now
optional (but remain mandatory for non counter column family operation).
- The counter operation comprise `batch_mutate` and the following new
operations:
+ The counter operations comprise the usual `batch_mutate`, `get`, `get_slice`,
`multiget_slice`, `multiget_count` and `get_range_slice` (secondary indexes on
counter column family is not supported at the moment), as well as the following
new operations for access to a single counter:
{{{
# counter methods
@@ -79, +81 @@
throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
/**
-* Return the counter at the specified column path.
-*/
- Counter get_counter(1:required binary key,
- 2:required ColumnPath path,
- 3:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
- throws (1:InvalidRequestException ire, 2:NotFoundException nfe,
3:UnavailableException ue, 4:TimedOutException te),
-
- /**
-* Get a list of counters from the specified columns.
-*/
- listCounter get_counter_slice(1:required binary key,
- 2:required ColumnParent column_parent,
- 3:required SlicePredicate predicate,
- 4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
- throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
-
- /**
-* Get counter slices from multiple keys.
-*/
- mapbinary,listCounter multiget_counter_slice(1:required listbinary
keys,
-2:required ColumnParent
column_parent,
-3:required SlicePredicate
predicate,
-4:required
ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
- throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
-
- /**
* Remove a counter at the specified location.
*/
void remove_counter(1:required binary key,
@@ -115, +91 @@
== Technical limitations ==
- * If a write fails unexpectedly (timeout or loss of connection to the
coordinator node) the client will not know if the operation has been performed.
A retry can result in an over count.
+ * If a write fails unexpectedly (timeout or loss of connection to the
coordinator node) the client will not know if the operation has been performed.
A retry can result in an over count
[[https://issues.apache.org/jira/browse/CASSANDRA-2495|CASSANDRA-2495]].
- * Range slices on counter column family is not yet supported
(https://issues.apache.org/jira/browse/CASSANDRA-2342).
== Further reading ==
See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]]
and
[Cassandra Wiki] Update of Counters by SylvainLebresne
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The Counters page has been changed by SylvainLebresne. http://wiki.apache.org/cassandra/Counters?action=diffrev1=11rev2=12 -- == Technical limitations == * If a write fails unexpectedly (timeout or loss of connection to the coordinator node) the client will not know if the operation has been performed. A retry can result in an over count [[https://issues.apache.org/jira/browse/CASSANDRA-2495|CASSANDRA-2495]]. + * Counter removal is intrinsically limited. For instance, if you issue very quickly the sequence increment, remove, increment it is possible for the removal to be lost (if for some reason the remove happens to be the last received messages). Hence, removal of counters is provided for definitive removal, that is when the deleted counter is not increment afterwards. Note that if you need to reset a counter, you can read its ''value'' and insert ''-value''. == Further reading == See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]] and especially the [[https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf|design doc]] for further information about how this works internally.
[Cassandra Wiki] Update of Counters by SylvainLebresne
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Cassandra Wiki for
change notification.
The Counters page has been changed by SylvainLebresne.
http://wiki.apache.org/cassandra/Counters?action=diffrev1=9rev2=10
--
== Interface ==
The interface follows the main API. The main differences are:
- * CounterColumn requires an i64 value and no timestamp, and
+ * CounterColumn requires an i64 value (can be negative) and no timestamp, and
- * CounterDeletion does not take a timestamp.
+ * Deletion, when used on a counter column family, does not use a timestamp.
Internally, the data store generates timestamps on the server to determine
priority of deletion.
+ The new structures for dealing with counters are:
{{{
struct CounterColumn {
1: required binary name,
@@ -53, +54 @@
1: required binary name,
2: required listCounterColumn columns
}
- }}}
- {{{
struct Counter {
1: optional CounterColumn column,
2: optional CounterSuperColumn super_column
}
+ }}}
+ Moreover, as mentioned previously, then timestamp field of Deletion is now
optional (but
+ remain mandatory for non counter column family operation).
+ The Mutation has also been updated with a new `counter` field for batch
increment/decrement.
- struct CounterDeletion {
- 1: optional binary super_column,
- 2: optional SlicePredicate predicate,
- }
+ The counter operation comprise `batch_mutate` and the following new
operations:
- /**
- A CounterMutation is either an insert, represented by filling counter, or
a deletion, represented by filling the deletion attribute.
- @param counter. An insert to a counter column or supercolumn
- @param deletion. A deletion of a counter column or supercolumn
- */
- struct CounterMutation {
- 1: optional Counter counter,
- 2: optional CounterDeletion deletion,
- }
- }}}
-
{{{
# counter methods
@@ -87, +76 @@
2:required ColumnParent column_parent,
3:required CounterColumn column,
4:required ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
-throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
- /**
-* Batch increment or decrement a counter.
-*/
- void batch_add(1:required mapbinary, mapstring, listCounterMutation
update_map,
- 2:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
/**
@@ -132, +115 @@
== Technical limitations ==
- Writes require CL.ONE.
- * The commutative design requires partitionability and idempotence between
partitions.
- * If a write fails unexpectedly (timeout for example) the client will not
know if the operation has been performed. A retry can result in an over count.
+ * If a write fails unexpectedly (timeout or loss of connection to the
coordinator node) the client will not know if the operation has been performed.
A retry can result in an over count.
+ * Range slices on counter column family is not yet supported
(https://issues.apache.org/jira/browse/CASSANDRA-2342).
-
- == Operational considerations ==
- * Reusing IP addresses will cause problems. Since the current version of
the patch uses machine ip addresses in the on disk data format as an identifier
a reuse of an old address will result in incorrect counts.
- * See https://issues.apache.org/jira/browse/CASSANDRA-1944 and
https://issues.apache.org/jira/browse/CASSANDRA-1938
== Further reading ==
- See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]]
and especially the
[[https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf|design
doc]] for further information about how this works.
+ See [[https://issues.apache.org/jira/browse/CASSANDRA-1072|CASSANDRA-1072]]
and especially the
[[https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf|design
doc]] for further information about how this works internally.
