[jira] [Updated] (CASSANDRA-5838) Upgrade metrics-core library
[ https://issues.apache.org/jira/browse/CASSANDRA-5838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Brosius updated CASSANDRA-5838: Attachment: 5838.txt Major changes are 1) MetricsRegistry is no longer static, so i added static c* class that holds the instance 2) No more MetricName class, just strings... adjusted accordingly 3) simple api renamings value() now getValue() etc. 4) Removal of the clear() method from Histogram and ExponentialDecayingSample now ExponentialDecayingReservoir... Not sure how to handle this. In the one place that mattered, i just unregistered, and then reregistered new EDRs. The other case was from a test, that seems to pass even w/o the clears, so removed. > Upgrade metrics-core library > > > Key: CASSANDRA-5838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5838 > Project: Cassandra > Issue Type: Improvement >Reporter: Eugen Paraschiv >Assignee: Dave Brosius >Priority: Minor > Fix For: 2.1 > > Attachments: 5838.txt > > > Cassandra is now using [metrics|https://github.com/codahale/metrics] and is > depending on metrics-core 2.0.3. > It would be great to make the jump to the latest version of the library which > is 3.x - [latest is > 3.0.1|http://search.maven.org/#search|gav|1|g%3A%22com.codahale.metrics%22%20AND%20a%3A%22metrics-core%22]. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5851) Fix 2i on composite components omissions
[ https://issues.apache.org/jira/browse/CASSANDRA-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734196#comment-13734196 ] Aleksey Yeschenko commented on CASSANDRA-5851: -- bq. Can you add a unit test? https://github.com/riptano/cassandra-dtest/commit/26f9cc6ff5e6d95d44784f034a3d8ed974f415b3 > Fix 2i on composite components omissions > > > Key: CASSANDRA-5851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5851 > Project: Cassandra > Issue Type: Bug >Affects Versions: 2.0 beta 2 >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko > Fix For: 2.0 > > Attachments: 5851-extra.txt, 5851.txt > > > There some edge-cases, not covered by CASSANDRA-5125, the attached patch > fixes those: > (Assuming CREATE TABLE test (pk0 int, pk1 int, ck0 int, ck1 int, val int, > PRIMARY KEY ((pk0, pk1), ck0, ck1))) > - could not create a 2i on the first part of a composite partition key (pk0) > - if created, it couldn't work because of getKeyBounds() returning non-empty > bounds > - could create an index on the first clustering key column (ck0), but it > would never actually be triggered on reads > - queries like SELECT * FROM test WHERE pk0 = x AND pk1 = y AND ck1 = z would > throw an exception because COCK.makeIndexColumnNameBuilder() couldn't handle > empty provided columnName > - cqlsh could not describe any of these indexes because it was taking column > aliases and key aliases from schema_columnfamilies and not reading them > directly from schema_columns (had to do the related refactoring). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5865) NPE when you mistakenly set listen_address to 0.0.0.0
Michaël Figuière created CASSANDRA-5865:
---
Summary: NPE when you mistakenly set listen_address to 0.0.0.0
Key: CASSANDRA-5865
URL: https://issues.apache.org/jira/browse/CASSANDRA-5865
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 1.2.8
Reporter: Michaël Figuière
Priority: Trivial
It's clearly stated that setting {{listen_address}} to {{0.0.0.0}} is always
wrong. But if you mistakenly do it anyway you end up with an NPE on 1.2.8 while
it's not the case on 2.0.0-rc1. See bellow:
{code}
INFO 16:34:43,598 JOINING: waiting for ring information
INFO 16:34:44,505 Handshaking version with /127.0.0.1
INFO 16:34:44,533 Handshaking version with /0.0.0.0
INFO 16:35:13,626 JOINING: schema complete, ready to bootstrap
INFO 16:35:13,631 JOINING: getting bootstrap token
ERROR 16:35:13,633 Exception encountered during startup
java.lang.RuntimeException: No other nodes seen! Unable to bootstrap.If you
intended to start a single-node cluster, you should make sure your
broadcast_address (or listen_address) is listed as a seed. Otherwise, you need
to determine why the seed being contacted has no knowledge of the rest of the
cluster. Usually, this can be solved by giving all nodes the same seed list.
at
org.apache.cassandra.dht.BootStrapper.getBootstrapSource(BootStrapper.java:154)
at
org.apache.cassandra.dht.BootStrapper.getBalancedToken(BootStrapper.java:135)
at
org.apache.cassandra.dht.BootStrapper.getBootstrapTokens(BootStrapper.java:115)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:666)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
java.lang.RuntimeException: No other nodes seen! Unable to bootstrap.If you
intended to start a single-node cluster, you should make sure your
broadcast_address (or listen_address) is listed as a seed. Otherwise, you need
to determine why the seed being contacted has no knowledge of the rest of the
cluster. Usually, this can be solved by giving all nodes the same seed list.
at
org.apache.cassandra.dht.BootStrapper.getBootstrapSource(BootStrapper.java:154)
at
org.apache.cassandra.dht.BootStrapper.getBalancedToken(BootStrapper.java:135)
at
org.apache.cassandra.dht.BootStrapper.getBootstrapTokens(BootStrapper.java:115)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:666)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Exception encountered during startup: No other nodes seen! Unable to
bootstrap.If you intended to start a single-node cluster, you should make sure
your broadcast_address (or listen_address) is listed as a seed. Otherwise, you
need to determine why the seed being contacted has no knowledge of the rest of
the cluster. Usually, this can be solved by giving all nodes the same seed
list.
ERROR 16:35:13,668 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at
org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at
org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
at
org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
at
org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:519)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.lang.Thread.run(Thread.java:724)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5864) Scrub should discard the columns from CFMetaData.droppedColumns map (if they are old enough)
[ https://issues.apache.org/jira/browse/CASSANDRA-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734094#comment-13734094 ] Jonathan Ellis commented on CASSANDRA-5864: --- This keeps coming up... is scrub just a "recover damaged information" tool? Or should it also perform other edits that may make sense? Logically it makes more sense to me to put "other edits" into upgradesstables, which also has the benefit of making scrub more predictable when it's doing some fairly hairy recovery logic, but I'm not sure the code organization lends itself to that. > Scrub should discard the columns from CFMetaData.droppedColumns map (if they > are old enough) > > > Key: CASSANDRA-5864 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5864 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Jonathan Ellis >Priority: Minor > Labels: scrub > > CASSANDRA-3919 restored ALTER TABLE DROP support in CQL3 and it would be nice > to make scrub dropped-columns-aware. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5864) Scrub should discard the columns from CFMetaData.droppedColumns map (if they are old enough)
Aleksey Yeschenko created CASSANDRA-5864: Summary: Scrub should discard the columns from CFMetaData.droppedColumns map (if they are old enough) Key: CASSANDRA-5864 URL: https://issues.apache.org/jira/browse/CASSANDRA-5864 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Jonathan Ellis Priority: Minor CASSANDRA-3919 restored ALTER TABLE DROP support in CQL3 and it would be nice to make scrub dropped-columns-aware. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[04/19] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e0efa3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e0efa3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e0efa3d
Branch: refs/heads/cassandra-2.0.0
Commit: 9e0efa3dbbeddf382c8e0f45cfdf2ebb91bc9db0
Parents: ccb32a8
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:51:34 2013 -0500
--
CHANGES.txt | 2 ++
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 859e7f9..30ce976 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,8 @@
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
* (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
(CASSANDRA-5718)
+Merged from 1.1:
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.2.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 616f3c0..7b614f5 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -26,6 +26,7 @@ import java.util.Collection;
import java.util.List;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -287,7 +288,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[15/19] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d044fe6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d044fe6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d044fe6b
Branch: refs/heads/trunk
Commit: d044fe6b155fa828b9e7f0477d47fd07ef0b5d17
Parents: 5411e5f 73b0ffb
Author: Jonathan Ellis
Authored: Thu Aug 8 17:08:14 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:08:14 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java | 13 +++--
1 file changed, 3 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d044fe6b/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index d9e0416,632392f..b0d22fb
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -302,16 -292,10 +302,9 @@@ public class Column implements OnDiskAt
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
--ByteBuffer internalName;
- if (cfdef.isComposite && !cfdef.isCompact)
- {
- CompositeType comparator = (CompositeType) metadata.comparator;
- internalName = comparator.extractLastComponent(name);
- }
- else
- {
- internalName = name;
- }
-internalName = (cfdef.isComposite && !cfdef.isCompact)
- ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
- : name;
++ByteBuffer internalName = (cfdef.isComposite && !cfdef.isCompact)
++? ((CompositeType)
metadata.comparator).extractLastComponent(name)
++: name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
[12/19] git commit: make scrubbing collection-aware
make scrubbing collection-aware
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/73b0ffba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/73b0ffba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/73b0ffba
Branch: refs/heads/cassandra-2.0
Commit: 73b0ffba82f431d6fca2a876504fe9f0631ef0a6
Parents: 9e0efa3
Author: Jonathan Ellis
Authored: Thu Aug 8 17:06:56 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:06:56 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java| 13 +++
.../cassandra/db/marshal/CompositeType.java | 24
2 files changed, 27 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 7b614f5..632392f 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -293,16 +293,9 @@ public class Column implements IColumn
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
ByteBuffer internalName;
-if (cfdef.isComposite && !cfdef.isCompact)
-{
-AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
-}
-else
-{
-internalName = name;
-}
+internalName = (cfdef.isComposite && !cfdef.isCompact)
+ ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
+ : name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 522be13..2a27617 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -124,6 +124,30 @@ public class CompositeType extends AbstractCompositeType
return build(serialized);
}
+// Extract component idx from bb. Return null if there is not enough
component.
+public static ByteBuffer extractComponent(ByteBuffer bb, int idx)
+{
+bb = bb.duplicate();
+int i = 0;
+while (bb.remaining() > 0)
+{
+ByteBuffer c = getWithShortLength(bb);
+if (i == idx)
+return c;
+
+bb.get(); // skip end-of-component
+++i;
+}
+return null;
+}
+
+// Extract CQL3 column name from the full column name.
+public ByteBuffer extractLastComponent(ByteBuffer bb)
+{
+int idx = types.get(types.size() - 1) instanceof
ColumnToCollectionType ? types.size() - 2 : types.size() - 1;
+return extractComponent(bb, idx);
+}
+
@Override
public boolean isCompatibleWith(AbstractType previous)
{
[14/19] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d044fe6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d044fe6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d044fe6b
Branch: refs/heads/cassandra-2.0
Commit: d044fe6b155fa828b9e7f0477d47fd07ef0b5d17
Parents: 5411e5f 73b0ffb
Author: Jonathan Ellis
Authored: Thu Aug 8 17:08:14 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:08:14 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java | 13 +++--
1 file changed, 3 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d044fe6b/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index d9e0416,632392f..b0d22fb
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -302,16 -292,10 +302,9 @@@ public class Column implements OnDiskAt
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
--ByteBuffer internalName;
- if (cfdef.isComposite && !cfdef.isCompact)
- {
- CompositeType comparator = (CompositeType) metadata.comparator;
- internalName = comparator.extractLastComponent(name);
- }
- else
- {
- internalName = name;
- }
-internalName = (cfdef.isComposite && !cfdef.isCompact)
- ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
- : name;
++ByteBuffer internalName = (cfdef.isComposite && !cfdef.isCompact)
++? ((CompositeType)
metadata.comparator).extractLastComponent(name)
++: name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
[07/19] git commit: Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0
Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5411e5ff Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5411e5ff Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5411e5ff Branch: refs/heads/trunk Commit: 5411e5ff850bd1bbbf8d9f0057ff4136402b2bff Parents: fdfdd5c 7a8cbb3 Author: Jonathan Ellis Authored: Thu Aug 8 15:51:58 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:51:58 2013 -0500 -- lib/licenses/disruptor-3.0.1.txt| 201 ++ lib/licenses/thrift-server-0.3.0.txt| 202 +++ .../cassandra/service/CassandraDaemon.java | 14 +- .../cassandra/service/StorageService.java | 5 +- 4 files changed, 406 insertions(+), 16 deletions(-) --
[05/19] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e0efa3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e0efa3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e0efa3d
Branch: refs/heads/trunk
Commit: 9e0efa3dbbeddf382c8e0f45cfdf2ebb91bc9db0
Parents: ccb32a8
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:51:34 2013 -0500
--
CHANGES.txt | 2 ++
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 859e7f9..30ce976 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,8 @@
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
* (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
(CASSANDRA-5718)
+Merged from 1.1:
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.2.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 616f3c0..7b614f5 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -26,6 +26,7 @@ import java.util.Collection;
import java.util.List;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -287,7 +288,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[19/19] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70c0d30d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70c0d30d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70c0d30d Branch: refs/heads/trunk Commit: 70c0d30df966784eb5bae8fa8dfe172bc6cc0d88 Parents: a1212c5 bb3f09c Author: Jonathan Ellis Authored: Thu Aug 8 17:08:41 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 17:08:41 2013 -0500 -- CHANGES.txt | 1 + lib/licenses/disruptor-3.0.1.txt| 201 ++ lib/licenses/thrift-server-0.3.0.txt| 202 +++ src/java/org/apache/cassandra/db/Column.java| 13 +- .../compress/CompressedRandomAccessReader.java | 14 +- .../io/compress/CompressedSequentialWriter.java | 7 +- .../io/compress/CompressionMetadata.java| 6 +- .../apache/cassandra/io/sstable/Descriptor.java | 6 +- .../compress/CompressedInputStream.java | 17 +- .../compress/CompressedStreamReader.java| 2 +- test/data/serialization/2.0/db.RowMutation.bin | Bin 3599 -> 3599 bytes .../CompressedRandomAccessReaderTest.java | 4 +- .../compress/CompressedInputStreamTest.java | 2 +- 13 files changed, 450 insertions(+), 25 deletions(-) --
[06/19] git commit: Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0
Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5411e5ff Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5411e5ff Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5411e5ff Branch: refs/heads/cassandra-2.0 Commit: 5411e5ff850bd1bbbf8d9f0057ff4136402b2bff Parents: fdfdd5c 7a8cbb3 Author: Jonathan Ellis Authored: Thu Aug 8 15:51:58 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:51:58 2013 -0500 -- lib/licenses/disruptor-3.0.1.txt| 201 ++ lib/licenses/thrift-server-0.3.0.txt| 202 +++ .../cassandra/service/CassandraDaemon.java | 14 +- .../cassandra/service/StorageService.java | 5 +- 4 files changed, 406 insertions(+), 16 deletions(-) --
[17/19] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb3f09c4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb3f09c4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb3f09c4 Branch: refs/heads/trunk Commit: bb3f09c4b189d27bf136ad9df044d378019607d5 Parents: 815b238 d044fe6 Author: Jonathan Ellis Authored: Thu Aug 8 17:08:23 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 17:08:23 2013 -0500 -- lib/licenses/disruptor-3.0.1.txt | 201 + lib/licenses/thrift-server-0.3.0.txt | 202 ++ src/java/org/apache/cassandra/db/Column.java | 13 +- 3 files changed, 406 insertions(+), 10 deletions(-) --
[18/19] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bb3f09c4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bb3f09c4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bb3f09c4 Branch: refs/heads/cassandra-2.0 Commit: bb3f09c4b189d27bf136ad9df044d378019607d5 Parents: 815b238 d044fe6 Author: Jonathan Ellis Authored: Thu Aug 8 17:08:23 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 17:08:23 2013 -0500 -- lib/licenses/disruptor-3.0.1.txt | 201 + lib/licenses/thrift-server-0.3.0.txt | 202 ++ src/java/org/apache/cassandra/db/Column.java | 13 +- 3 files changed, 406 insertions(+), 10 deletions(-) --
[16/19] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d044fe6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d044fe6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d044fe6b
Branch: refs/heads/cassandra-2.0.0
Commit: d044fe6b155fa828b9e7f0477d47fd07ef0b5d17
Parents: 5411e5f 73b0ffb
Author: Jonathan Ellis
Authored: Thu Aug 8 17:08:14 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:08:14 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java | 13 +++--
1 file changed, 3 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d044fe6b/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index d9e0416,632392f..b0d22fb
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -302,16 -292,10 +302,9 @@@ public class Column implements OnDiskAt
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
--ByteBuffer internalName;
- if (cfdef.isComposite && !cfdef.isCompact)
- {
- CompositeType comparator = (CompositeType) metadata.comparator;
- internalName = comparator.extractLastComponent(name);
- }
- else
- {
- internalName = name;
- }
-internalName = (cfdef.isComposite && !cfdef.isCompact)
- ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
- : name;
++ByteBuffer internalName = (cfdef.isComposite && !cfdef.isCompact)
++? ((CompositeType)
metadata.comparator).extractLastComponent(name)
++: name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
[02/19] git commit: add thrift-server and LMAX Disruptor licences
add thrift-server and LMAX Disruptor licences Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7a8cbb3e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7a8cbb3e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7a8cbb3e Branch: refs/heads/trunk Commit: 7a8cbb3ea7e0898778ed41a6d19c40d67c065667 Parents: 50b40e9 Author: Pavel Yaskevich Authored: Thu Aug 8 11:30:02 2013 -0700 Committer: Pavel Yaskevich Committed: Thu Aug 8 11:30:02 2013 -0700 -- lib/licenses/disruptor-3.0.1.txt | 201 + lib/licenses/thrift-server-0.3.0.txt | 202 ++ 2 files changed, 403 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a8cbb3e/lib/licenses/disruptor-3.0.1.txt -- diff --git a/lib/licenses/disruptor-3.0.1.txt b/lib/licenses/disruptor-3.0.1.txt new file mode 100644 index 000..50086f8 --- /dev/null +++ b/lib/licenses/disruptor-3.0.1.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, r
[11/19] git commit: make scrubbing collection-aware
make scrubbing collection-aware
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/73b0ffba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/73b0ffba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/73b0ffba
Branch: refs/heads/cassandra-1.2
Commit: 73b0ffba82f431d6fca2a876504fe9f0631ef0a6
Parents: 9e0efa3
Author: Jonathan Ellis
Authored: Thu Aug 8 17:06:56 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:06:56 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java| 13 +++
.../cassandra/db/marshal/CompositeType.java | 24
2 files changed, 27 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 7b614f5..632392f 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -293,16 +293,9 @@ public class Column implements IColumn
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
ByteBuffer internalName;
-if (cfdef.isComposite && !cfdef.isCompact)
-{
-AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
-}
-else
-{
-internalName = name;
-}
+internalName = (cfdef.isComposite && !cfdef.isCompact)
+ ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
+ : name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 522be13..2a27617 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -124,6 +124,30 @@ public class CompositeType extends AbstractCompositeType
return build(serialized);
}
+// Extract component idx from bb. Return null if there is not enough
component.
+public static ByteBuffer extractComponent(ByteBuffer bb, int idx)
+{
+bb = bb.duplicate();
+int i = 0;
+while (bb.remaining() > 0)
+{
+ByteBuffer c = getWithShortLength(bb);
+if (i == idx)
+return c;
+
+bb.get(); // skip end-of-component
+++i;
+}
+return null;
+}
+
+// Extract CQL3 column name from the full column name.
+public ByteBuffer extractLastComponent(ByteBuffer bb)
+{
+int idx = types.get(types.size() - 1) instanceof
ColumnToCollectionType ? types.size() - 2 : types.size() - 1;
+return extractComponent(bb, idx);
+}
+
@Override
public boolean isCompatibleWith(AbstractType previous)
{
[10/19] git commit: make scrubbing collection-aware
make scrubbing collection-aware
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/73b0ffba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/73b0ffba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/73b0ffba
Branch: refs/heads/trunk
Commit: 73b0ffba82f431d6fca2a876504fe9f0631ef0a6
Parents: 9e0efa3
Author: Jonathan Ellis
Authored: Thu Aug 8 17:06:56 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:06:56 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java| 13 +++
.../cassandra/db/marshal/CompositeType.java | 24
2 files changed, 27 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 7b614f5..632392f 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -293,16 +293,9 @@ public class Column implements IColumn
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
ByteBuffer internalName;
-if (cfdef.isComposite && !cfdef.isCompact)
-{
-AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
-}
-else
-{
-internalName = name;
-}
+internalName = (cfdef.isComposite && !cfdef.isCompact)
+ ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
+ : name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 522be13..2a27617 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -124,6 +124,30 @@ public class CompositeType extends AbstractCompositeType
return build(serialized);
}
+// Extract component idx from bb. Return null if there is not enough
component.
+public static ByteBuffer extractComponent(ByteBuffer bb, int idx)
+{
+bb = bb.duplicate();
+int i = 0;
+while (bb.remaining() > 0)
+{
+ByteBuffer c = getWithShortLength(bb);
+if (i == idx)
+return c;
+
+bb.get(); // skip end-of-component
+++i;
+}
+return null;
+}
+
+// Extract CQL3 column name from the full column name.
+public ByteBuffer extractLastComponent(ByteBuffer bb)
+{
+int idx = types.get(types.size() - 1) instanceof
ColumnToCollectionType ? types.size() - 2 : types.size() - 1;
+return extractComponent(bb, idx);
+}
+
@Override
public boolean isCompatibleWith(AbstractType previous)
{
[08/19] git commit: Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0
Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e911b767 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e911b767 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e911b767 Branch: refs/heads/trunk Commit: e911b767e1ca90acbb4be65475a239b4a663308f Parents: b5f445b 5d9a33a Author: Jonathan Ellis Authored: Thu Aug 8 15:52:07 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:52:07 2013 -0500 -- .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++ .../org/apache/cassandra/service/StorageService.java | 5 + 2 files changed, 3 insertions(+), 16 deletions(-) --
[13/19] git commit: make scrubbing collection-aware
make scrubbing collection-aware
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/73b0ffba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/73b0ffba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/73b0ffba
Branch: refs/heads/cassandra-2.0.0
Commit: 73b0ffba82f431d6fca2a876504fe9f0631ef0a6
Parents: 9e0efa3
Author: Jonathan Ellis
Authored: Thu Aug 8 17:06:56 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 17:06:56 2013 -0500
--
src/java/org/apache/cassandra/db/Column.java| 13 +++
.../cassandra/db/marshal/CompositeType.java | 24
2 files changed, 27 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 7b614f5..632392f 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -293,16 +293,9 @@ public class Column implements IColumn
// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
// (Note that COMPACT composites are handled by validateName, above.)
ByteBuffer internalName;
-if (cfdef.isComposite && !cfdef.isCompact)
-{
-AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
-}
-else
-{
-internalName = name;
-}
+internalName = (cfdef.isComposite && !cfdef.isCompact)
+ ? ((CompositeType)
metadata.comparator).extractLastComponent(name)
+ : name;
AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/73b0ffba/src/java/org/apache/cassandra/db/marshal/CompositeType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
index 522be13..2a27617 100644
--- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java
@@ -124,6 +124,30 @@ public class CompositeType extends AbstractCompositeType
return build(serialized);
}
+// Extract component idx from bb. Return null if there is not enough
component.
+public static ByteBuffer extractComponent(ByteBuffer bb, int idx)
+{
+bb = bb.duplicate();
+int i = 0;
+while (bb.remaining() > 0)
+{
+ByteBuffer c = getWithShortLength(bb);
+if (i == idx)
+return c;
+
+bb.get(); // skip end-of-component
+++i;
+}
+return null;
+}
+
+// Extract CQL3 column name from the full column name.
+public ByteBuffer extractLastComponent(ByteBuffer bb)
+{
+int idx = types.get(types.size() - 1) instanceof
ColumnToCollectionType ? types.size() - 2 : types.size() - 1;
+return extractComponent(bb, idx);
+}
+
@Override
public boolean isCompatibleWith(AbstractType previous)
{
[09/19] git commit: Switch from crc32 to adler32 for compressed sstable checksumns and change to checksum the post-compressed data
Switch from crc32 to adler32 for compressed sstable checksumns
and change to checksum the post-compressed data
Patch by tjake; reviewed by jbellis for (CASSANDRA-5862)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/815b2382
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/815b2382
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/815b2382
Branch: refs/heads/trunk
Commit: 815b2382a34dc25f005c9cc1bde78e10bc5d2ae0
Parents: e911b76
Author: Jake Luciani
Authored: Thu Aug 8 17:21:40 2013 -0400
Committer: Jake Luciani
Committed: Thu Aug 8 17:23:57 2013 -0400
--
CHANGES.txt | 1 +
.../compress/CompressedRandomAccessReader.java | 14 --
.../io/compress/CompressedSequentialWriter.java | 7 ---
.../io/compress/CompressionMetadata.java | 6 --
.../apache/cassandra/io/sstable/Descriptor.java | 6 +-
.../compress/CompressedInputStream.java | 17 ++---
.../compress/CompressedStreamReader.java | 2 +-
test/data/serialization/2.0/db.RowMutation.bin | Bin 3599 -> 3599 bytes
.../CompressedRandomAccessReaderTest.java| 4 ++--
.../compress/CompressedInputStreamTest.java | 2 +-
10 files changed, 44 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 249c343..3379a80 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
2.0.1
* Notify indexer of columns shadowed by range tombstones (CASSANDRA-5614)
* Log Merkle tree stats (CASSANDRA-2698)
+ * Switch from crc32 to adler32 for compressed sstable checksums
(CASSANDRA-5862)
2.0.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 3269e4c..b6cffa2 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -19,6 +19,7 @@ package org.apache.cassandra.io.compress;
import java.io.*;
import java.nio.ByteBuffer;
+import java.util.zip.Adler32;
import java.util.zip.CRC32;
import java.util.zip.Checksum;
@@ -65,7 +66,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
private ByteBuffer compressed;
// re-use single crc object
-private final Checksum checksum = new CRC32();
+private final Checksum checksum;
// raw checksum bytes
private final ByteBuffer checksumBytes = ByteBuffer.wrap(new byte[4]);
@@ -74,6 +75,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
{
super(new File(dataFilePath), metadata.chunkLength(), owner);
this.metadata = metadata;
+checksum = metadata.hasPostCompressionAdlerChecksums ? new Adler32() :
new CRC32();
compressed = ByteBuffer.wrap(new
byte[metadata.compressor().initialCompressedBufferLength(metadata.chunkLength())]);
}
@@ -122,7 +124,15 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
-checksum.update(buffer, 0, validBufferBytes);
+
+if (metadata.hasPostCompressionAdlerChecksums)
+{
+checksum.update(compressed.array(), 0, chunk.length);
+}
+else
+{
+checksum.update(buffer, 0, validBufferBytes);
+}
if (checksum(chunk) != (int) checksum.getValue())
throw new CorruptBlockException(getPath(), chunk);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..386eca5 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.io.compress;
import java.io.EOFException;
import java.io.File;
import java.io.IOException;
+import java.util.zip.Adler32;
import java.util.zip.CRC32;
[01/19] git commit: add thrift-server and LMAX Disruptor licences
Updated Branches: refs/heads/cassandra-1.2 9e0efa3db -> 73b0ffba8 refs/heads/cassandra-2.0 815b2382a -> bb3f09c4b refs/heads/cassandra-2.0.0 5411e5ff8 -> d044fe6b1 refs/heads/trunk a1212c597 -> 70c0d30df add thrift-server and LMAX Disruptor licences Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7a8cbb3e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7a8cbb3e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7a8cbb3e Branch: refs/heads/cassandra-2.0 Commit: 7a8cbb3ea7e0898778ed41a6d19c40d67c065667 Parents: 50b40e9 Author: Pavel Yaskevich Authored: Thu Aug 8 11:30:02 2013 -0700 Committer: Pavel Yaskevich Committed: Thu Aug 8 11:30:02 2013 -0700 -- lib/licenses/disruptor-3.0.1.txt | 201 + lib/licenses/thrift-server-0.3.0.txt | 202 ++ 2 files changed, 403 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a8cbb3e/lib/licenses/disruptor-3.0.1.txt -- diff --git a/lib/licenses/disruptor-3.0.1.txt b/lib/licenses/disruptor-3.0.1.txt new file mode 100644 index 000..50086f8 --- /dev/null +++ b/lib/licenses/disruptor-3.0.1.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently i
[03/19] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e0efa3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e0efa3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e0efa3d
Branch: refs/heads/cassandra-2.0
Commit: 9e0efa3dbbeddf382c8e0f45cfdf2ebb91bc9db0
Parents: ccb32a8
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:51:34 2013 -0500
--
CHANGES.txt | 2 ++
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 859e7f9..30ce976 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,8 @@
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
* (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
(CASSANDRA-5718)
+Merged from 1.1:
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.2.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 616f3c0..7b614f5 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -26,6 +26,7 @@ import java.util.Collection;
import java.util.List;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -287,7 +288,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta
[
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734025#comment-13734025
]
Tyler Hobbs commented on CASSANDRA-5855:
+1 on 5585-followup.txt
> Scrub does not understand compound primary key created in CQL 3 beta
>
>
> Key: CASSANDRA-5855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: J.B. Langston
>Assignee: Tyler Hobbs
> Attachments: 0001-Correctly-validate-sparse-composite-columns.patch,
> 5855-followup.txt
>
>
> We have a customer who was using the beta version of CQL 3 in DSE 3.0 which
> includes Cassandra 1.1.9 plus patches backported from later versions.
> They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
> When restarting for the first time after running upgradesstables, they
> noticed the following error in the log:
> {noformat}
> Thread[SSTableBatchOpen:2,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
> at
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
> at
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
> at
> org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This error was also reported on CASSANDRA-5703. The comments suggested it
> was caused by an empty row key, so I had them run scrub on it. When they
> did, scrub reported the following warning almost 4 million times:
> {noformat}
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 52) Row at 530332255 is unreadable; skipping to next
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 57) Non-fatal error reading row (stacktrace follows)
> java.lang.RuntimeException: Error validating row
> DecoratedKey(139154446688383793922009760478335751546,
> 735fc9da503b11e2844b123140ff209f)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:98)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
> at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
> at
> org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
> at
> org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't
> validate.
> at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
> at org.apache.cassandra.db.Column.validateFields(Column.java:292)
> at
> org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:239)
> ... 15 mo
[jira] [Comment Edited] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734012#comment-13734012 ] T Jake Luciani edited comment on CASSANDRA-5862 at 8/8/13 9:27 PM: --- committed with suggested changes was (Author: tjake): committed. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Switch from crc32 to adler32 for compressed sstable checksumns and change to checksum the post-compressed data
Updated Branches:
refs/heads/cassandra-2.0 e911b767e -> 815b2382a
Switch from crc32 to adler32 for compressed sstable checksumns
and change to checksum the post-compressed data
Patch by tjake; reviewed by jbellis for (CASSANDRA-5862)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/815b2382
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/815b2382
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/815b2382
Branch: refs/heads/cassandra-2.0
Commit: 815b2382a34dc25f005c9cc1bde78e10bc5d2ae0
Parents: e911b76
Author: Jake Luciani
Authored: Thu Aug 8 17:21:40 2013 -0400
Committer: Jake Luciani
Committed: Thu Aug 8 17:23:57 2013 -0400
--
CHANGES.txt | 1 +
.../compress/CompressedRandomAccessReader.java | 14 --
.../io/compress/CompressedSequentialWriter.java | 7 ---
.../io/compress/CompressionMetadata.java | 6 --
.../apache/cassandra/io/sstable/Descriptor.java | 6 +-
.../compress/CompressedInputStream.java | 17 ++---
.../compress/CompressedStreamReader.java | 2 +-
test/data/serialization/2.0/db.RowMutation.bin | Bin 3599 -> 3599 bytes
.../CompressedRandomAccessReaderTest.java| 4 ++--
.../compress/CompressedInputStreamTest.java | 2 +-
10 files changed, 44 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 249c343..3379a80 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
2.0.1
* Notify indexer of columns shadowed by range tombstones (CASSANDRA-5614)
* Log Merkle tree stats (CASSANDRA-2698)
+ * Switch from crc32 to adler32 for compressed sstable checksums
(CASSANDRA-5862)
2.0.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 3269e4c..b6cffa2 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -19,6 +19,7 @@ package org.apache.cassandra.io.compress;
import java.io.*;
import java.nio.ByteBuffer;
+import java.util.zip.Adler32;
import java.util.zip.CRC32;
import java.util.zip.Checksum;
@@ -65,7 +66,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
private ByteBuffer compressed;
// re-use single crc object
-private final Checksum checksum = new CRC32();
+private final Checksum checksum;
// raw checksum bytes
private final ByteBuffer checksumBytes = ByteBuffer.wrap(new byte[4]);
@@ -74,6 +75,7 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
{
super(new File(dataFilePath), metadata.chunkLength(), owner);
this.metadata = metadata;
+checksum = metadata.hasPostCompressionAdlerChecksums ? new Adler32() :
new CRC32();
compressed = ByteBuffer.wrap(new
byte[metadata.compressor().initialCompressedBufferLength(metadata.chunkLength())]);
}
@@ -122,7 +124,15 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
if (metadata.parameters.getCrcCheckChance() >
FBUtilities.threadLocalRandom().nextDouble())
{
-checksum.update(buffer, 0, validBufferBytes);
+
+if (metadata.hasPostCompressionAdlerChecksums)
+{
+checksum.update(compressed.array(), 0, chunk.length);
+}
+else
+{
+checksum.update(buffer, 0, validBufferBytes);
+}
if (checksum(chunk) != (int) checksum.getValue())
throw new CorruptBlockException(getPath(), chunk);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815b2382/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 00eb5a7..386eca5 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.io.compress;
import java.io.EOFException;
import java.io.File;
import ja
[jira] [Updated] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta
[
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-5855:
--
Attachment: 5855-followup.txt
Attached followup patch to make it collection-aware.
> Scrub does not understand compound primary key created in CQL 3 beta
>
>
> Key: CASSANDRA-5855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: J.B. Langston
>Assignee: Tyler Hobbs
> Attachments: 0001-Correctly-validate-sparse-composite-columns.patch,
> 5855-followup.txt
>
>
> We have a customer who was using the beta version of CQL 3 in DSE 3.0 which
> includes Cassandra 1.1.9 plus patches backported from later versions.
> They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
> When restarting for the first time after running upgradesstables, they
> noticed the following error in the log:
> {noformat}
> Thread[SSTableBatchOpen:2,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
> at
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
> at
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
> at
> org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This error was also reported on CASSANDRA-5703. The comments suggested it
> was caused by an empty row key, so I had them run scrub on it. When they
> did, scrub reported the following warning almost 4 million times:
> {noformat}
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 52) Row at 530332255 is unreadable; skipping to next
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 57) Non-fatal error reading row (stacktrace follows)
> java.lang.RuntimeException: Error validating row
> DecoratedKey(139154446688383793922009760478335751546,
> 735fc9da503b11e2844b123140ff209f)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:98)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
> at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
> at
> org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
> at
> org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't
> validate.
> at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
> at org.apache.cassandra.db.Column.validateFields(Column.java:292)
> at
> org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:239)
[jira] [Updated] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-5862: Tester: alexzar > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733969#comment-13733969 ] Jonathan Ellis commented on CASSANDRA-5862: --- bq. My concern was the instance would be re-used across multiple sstables therefore should be chosen per buffer. Is that not the case? CRAR is always tied to a single SSTable. bq. I was trying to future proof us switching back to crc or something but not changing the post-compression. Think its safe to merge? Yeah, if we do that we'll end up adding new version tests anyway. I think YAGNI applies. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[03/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58
Branch: refs/heads/trunk
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:43:43 2013 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 19 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.next
* Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.1.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[12/16] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/670979d4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/670979d4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/670979d4 Branch: refs/heads/trunk Commit: 670979d4ef0c528c5c1fcc8f61ff05f8293387ed Parents: c7782f2 b5f445b Author: Jonathan Ellis Authored: Thu Aug 8 15:50:30 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:50:30 2013 -0500 -- CHANGES.txt | 4 src/java/org/apache/cassandra/db/Column.java | 18 +- 2 files changed, 21 insertions(+), 1 deletion(-) --
[10/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5f445b4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5f445b4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5f445b4 Branch: refs/heads/trunk Commit: b5f445b448e6047e717b17985c117843e66a2d40 Parents: c526131 fdfdd5c Author: Jonathan Ellis Authored: Thu Aug 8 15:50:16 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:50:16 2013 -0500 -- CHANGES.txt | 4 src/java/org/apache/cassandra/db/Column.java | 18 +- 2 files changed, 21 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5f445b4/CHANGES.txt --
[09/16] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1
Branch: refs/heads/cassandra-2.0
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:50:08 2013 -0500
--
CHANGES.txt | 4
src/java/org/apache/cassandra/db/Column.java | 18 +-
2 files changed, 21 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
-1.2.9
- * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
+2.0.0
+ * fix CAS contention timeout (CASSANDRA-5830)
+ * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
+
+
+2.0.0-rc1
+ * improve DecimalSerializer performance (CASSANDRA-5837)
+ * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
+ * fix schema-related trigger issues (CASSANDRA-5774)
+ * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
+ * Fix assertion error during repair (CASSANDRA-5801)
+ * Fix range tombstone bug (CASSANDRA-5805)
+ * DC-local CAS (CASSANDRA-5797)
+ * Add a native_protocol_version column to the system.local table
(CASSANRDA-5819)
+ * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
+ * Fix buffer underflow on socket close (CASSANDRA-5792)
+Merged from 1.2:
* fix bulk-loading compressed sstables (CASSANDRA-5820)
* (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter
(CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
(CASSANDRA-5752)
* add "all time blocked" to StatusLogger output (CASSANDRA-5825)
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+ * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
-Merged from 1.1:
- * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
-
-
-1.2.8
* Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
* cqlsh: add collections support to COPY (CASSANDRA-5698)
* retry important messages for any IOException (CASSANDRA-5804)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.util.ArrayList;
-import java.util.Collection;
+import java.util.Iterator;
import java.util.List;
+import com.google.common.collect.AbstractIterator;
+
import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.util.DataOutputBuffer;
+import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.utils.Allocator;
import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.HeapAllocator;
/**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
- AbstractType valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+
+ // If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite && !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+ else
+ {
+ internalName = name;
+ }
+
+
[13/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e0efa3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e0efa3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e0efa3d
Branch: refs/heads/cassandra-1.2
Commit: 9e0efa3dbbeddf382c8e0f45cfdf2ebb91bc9db0
Parents: ccb32a8
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:51:34 2013 -0500
--
CHANGES.txt | 2 ++
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 859e7f9..30ce976 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,8 @@
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
* (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
(CASSANDRA-5718)
+Merged from 1.1:
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.2.8
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 616f3c0..7b614f5 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -26,6 +26,7 @@ import java.util.Collection;
import java.util.List;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -287,7 +288,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[02/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58
Branch: refs/heads/cassandra-2.0.0
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:43:43 2013 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 19 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.next
* Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.1.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[16/16] git commit: Merge remote-tracking branch 'origin/trunk' into trunk
Merge remote-tracking branch 'origin/trunk' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a1212c59 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a1212c59 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a1212c59 Branch: refs/heads/trunk Commit: a1212c597482ba91dc41036f23199273e9177040 Parents: 670979d 28b827a Author: Jonathan Ellis Authored: Thu Aug 8 15:52:15 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:52:15 2013 -0500 -- .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++ .../org/apache/cassandra/service/StorageService.java | 5 + 2 files changed, 3 insertions(+), 16 deletions(-) --
[08/16] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1
Branch: refs/heads/cassandra-2.0.0
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:50:08 2013 -0500
--
CHANGES.txt | 4
src/java/org/apache/cassandra/db/Column.java | 18 +-
2 files changed, 21 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
-1.2.9
- * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
+2.0.0
+ * fix CAS contention timeout (CASSANDRA-5830)
+ * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
+
+
+2.0.0-rc1
+ * improve DecimalSerializer performance (CASSANDRA-5837)
+ * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
+ * fix schema-related trigger issues (CASSANDRA-5774)
+ * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
+ * Fix assertion error during repair (CASSANDRA-5801)
+ * Fix range tombstone bug (CASSANDRA-5805)
+ * DC-local CAS (CASSANDRA-5797)
+ * Add a native_protocol_version column to the system.local table
(CASSANRDA-5819)
+ * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
+ * Fix buffer underflow on socket close (CASSANDRA-5792)
+Merged from 1.2:
* fix bulk-loading compressed sstables (CASSANDRA-5820)
* (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter
(CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
(CASSANDRA-5752)
* add "all time blocked" to StatusLogger output (CASSANDRA-5825)
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+ * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
-Merged from 1.1:
- * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
-
-
-1.2.8
* Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
* cqlsh: add collections support to COPY (CASSANDRA-5698)
* retry important messages for any IOException (CASSANDRA-5804)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.util.ArrayList;
-import java.util.Collection;
+import java.util.Iterator;
import java.util.List;
+import com.google.common.collect.AbstractIterator;
+
import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.util.DataOutputBuffer;
+import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.utils.Allocator;
import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.HeapAllocator;
/**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
- AbstractType valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+
+ // If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite && !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+ else
+ {
+ internalName = name;
+ }
+
+
[01/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Updated Branches:
refs/heads/cassandra-1.2 ccb32a8c8 -> 9e0efa3db
refs/heads/cassandra-2.0 5d9a33a46 -> e911b767e
refs/heads/cassandra-2.0.0 7a8cbb3ea -> 5411e5ff8
refs/heads/trunk 28b827a85 -> a1212c597
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58
Branch: refs/heads/cassandra-2.0
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:43:43 2013 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 19 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.next
* Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.1.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[06/16] git commit: merge from 1.1
merge from 1.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a Branch: refs/heads/cassandra-2.0.0 Commit: 1143df1aa439618bff6005ca1cee34cb066a Parents: 23d7bd8 eb884a5 Author: Jonathan Ellis Authored: Thu Aug 8 15:46:52 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:46:52 2013 -0500 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/db/Column.java | 19 ++- 2 files changed, 20 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt -- diff --cc CHANGES.txt index 859e7f9,8790518..30ce976 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,214 -1,19 +1,216 @@@ -1.1.next - * Backport compaction exception handling from 1.2 +1.2.9 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831) + * fix bulk-loading compressed sstables (CASSANDRA-5820) + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter + (CASSANDRA-5824) + * update default LCS sstable size to 160MB (CASSANDRA-5727) + * Allow compacting 2Is via nodetool (CASSANDRA-5670) + * Hex-encode non-String keys in OPP (CASSANDRA-5793) + * nodetool history logging (CASSANDRA-5823) + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader + (CASSANDRA-5752) + * add "all time blocked" to StatusLogger output (CASSANDRA-5825) + * Future-proof inter-major-version schema migrations (CASSANDRA-5845) + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table + (CASSANDRA-5718) ++Merged from 1.1: + * Correctly validate sparse composite cells in scrub (CASSANDRA-5855) -1.1.12 +1.2.8 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814) + * cqlsh: add collections support to COPY (CASSANDRA-5698) + * retry important messages for any IOException (CASSANDRA-5804) + * Allow empty IN relations in SELECT/UPDATE/DELETE statements (CASSANDRA-5626) + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812) + + +1.2.7 + * if no seeds can be a reached a node won't start in a ring by itself (CASSANDRA-5768) + * add cassandra.unsafesystem property (CASSANDRA-5704) + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763) + * Add replace_node functionality for vnodes (CASSANDRA-5337) + * Add timeout events to query traces (CASSANDRA-5520) + * make starting native protocol server idempotent (CASSANDRA-5728) + * Fix loading key cache when a saved entry is no longer valid (CASSANDRA-5706) + * Fix serialization of the LEFT gossip value (CASSANDRA-5696) + * Pig: support for cql3 tables (CASSANDRA-5234) + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675) + * Race condition in detecting version on a mixed 1.1/1.2 cluster + (CASSANDRA-5692) + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712) + * Expire entries out of ThriftSessionManager (CASSANRDA-5719) + * Don't keep ancestor information in memory (CASSANDRA-5342) + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697) + * Expose native protocol server status in nodetool info (CASSANDRA-5735) + * Fix pathetic performance of range tombstones (CASSANDRA-5677) + * Fix querying with an empty (impossible) range (CASSANDRA-5573) + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760) + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771) + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766) + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769) + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781) + * Don't rely on row marker for queries in general to hide lost markers + after TTL expires (CASSANDRA-5762) + * Sort nodetool help output (CASSANDRA-5776) + * Fix column expiring during 2 phases compaction (CASSANDRA-5799) + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795) + + +1.2.6 + * Fix tracing when operation completes before all responses arrive + (CASSANDRA-5668) + * Fix cross-DC mutation forwarding (CASSANDRA-5632) + * Reduce SSTableLoader memory usage (CASSANDRA-) + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272) + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622) + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536) + * Fix dealing with ridiculously large max sstable sizes in LCS (CASSANDRA-5589) + * Ignore pre-truncate hints (CASSANDRA-4655) + * Move System.exit on OOM into a separate
[15/16] git commit: Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0
Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e911b767 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e911b767 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e911b767 Branch: refs/heads/cassandra-2.0 Commit: e911b767e1ca90acbb4be65475a239b4a663308f Parents: b5f445b 5d9a33a Author: Jonathan Ellis Authored: Thu Aug 8 15:52:07 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:52:07 2013 -0500 -- .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++ .../org/apache/cassandra/service/StorageService.java | 5 + 2 files changed, 3 insertions(+), 16 deletions(-) --
[14/16] git commit: Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0
Merge remote-tracking branch 'origin/cassandra-2.0.0' into cassandra-2.0.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5411e5ff Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5411e5ff Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5411e5ff Branch: refs/heads/cassandra-2.0.0 Commit: 5411e5ff850bd1bbbf8d9f0057ff4136402b2bff Parents: fdfdd5c 7a8cbb3 Author: Jonathan Ellis Authored: Thu Aug 8 15:51:58 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:51:58 2013 -0500 -- lib/licenses/disruptor-3.0.1.txt| 201 ++ lib/licenses/thrift-server-0.3.0.txt| 202 +++ .../cassandra/service/CassandraDaemon.java | 14 +- .../cassandra/service/StorageService.java | 5 +- 4 files changed, 406 insertions(+), 16 deletions(-) --
[07/16] git commit: merge from 1.2
merge from 1.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1
Branch: refs/heads/trunk
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:50:08 2013 -0500
--
CHANGES.txt | 4
src/java/org/apache/cassandra/db/Column.java | 18 +-
2 files changed, 21 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
-1.2.9
- * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
+2.0.0
+ * fix CAS contention timeout (CASSANDRA-5830)
+ * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
+
+
+2.0.0-rc1
+ * improve DecimalSerializer performance (CASSANDRA-5837)
+ * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
+ * fix schema-related trigger issues (CASSANDRA-5774)
+ * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
+ * Fix assertion error during repair (CASSANDRA-5801)
+ * Fix range tombstone bug (CASSANDRA-5805)
+ * DC-local CAS (CASSANDRA-5797)
+ * Add a native_protocol_version column to the system.local table
(CASSANRDA-5819)
+ * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
+ * Fix buffer underflow on socket close (CASSANDRA-5792)
+Merged from 1.2:
* fix bulk-loading compressed sstables (CASSANDRA-5820)
* (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter
(CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
(CASSANDRA-5752)
* add "all time blocked" to StatusLogger output (CASSANDRA-5825)
* Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+ * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
-Merged from 1.1:
- * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
-
-
-1.2.8
* Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
* cqlsh: add collections support to COPY (CASSANDRA-5698)
* retry important messages for any IOException (CASSANDRA-5804)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.util.ArrayList;
-import java.util.Collection;
+import java.util.Iterator;
import java.util.List;
+import com.google.common.collect.AbstractIterator;
+
import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.util.DataOutputBuffer;
+import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.utils.Allocator;
import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.HeapAllocator;
/**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
- AbstractType valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+
+ // If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite && !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
-List components =
comparator.deconstruct(name);
-internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+ else
+ {
+ internalName = name;
+ }
+
+ Abstrac
[11/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5f445b4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5f445b4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5f445b4 Branch: refs/heads/cassandra-2.0 Commit: b5f445b448e6047e717b17985c117843e66a2d40 Parents: c526131 fdfdd5c Author: Jonathan Ellis Authored: Thu Aug 8 15:50:16 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:50:16 2013 -0500 -- CHANGES.txt | 4 src/java/org/apache/cassandra/db/Column.java | 18 +- 2 files changed, 21 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5f445b4/CHANGES.txt --
[04/16] git commit: merge from 1.1
merge from 1.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a Branch: refs/heads/cassandra-2.0 Commit: 1143df1aa439618bff6005ca1cee34cb066a Parents: 23d7bd8 eb884a5 Author: Jonathan Ellis Authored: Thu Aug 8 15:46:52 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:46:52 2013 -0500 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/db/Column.java | 19 ++- 2 files changed, 20 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt -- diff --cc CHANGES.txt index 859e7f9,8790518..30ce976 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,214 -1,19 +1,216 @@@ -1.1.next - * Backport compaction exception handling from 1.2 +1.2.9 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831) + * fix bulk-loading compressed sstables (CASSANDRA-5820) + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter + (CASSANDRA-5824) + * update default LCS sstable size to 160MB (CASSANDRA-5727) + * Allow compacting 2Is via nodetool (CASSANDRA-5670) + * Hex-encode non-String keys in OPP (CASSANDRA-5793) + * nodetool history logging (CASSANDRA-5823) + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader + (CASSANDRA-5752) + * add "all time blocked" to StatusLogger output (CASSANDRA-5825) + * Future-proof inter-major-version schema migrations (CASSANDRA-5845) + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table + (CASSANDRA-5718) ++Merged from 1.1: + * Correctly validate sparse composite cells in scrub (CASSANDRA-5855) -1.1.12 +1.2.8 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814) + * cqlsh: add collections support to COPY (CASSANDRA-5698) + * retry important messages for any IOException (CASSANDRA-5804) + * Allow empty IN relations in SELECT/UPDATE/DELETE statements (CASSANDRA-5626) + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812) + + +1.2.7 + * if no seeds can be a reached a node won't start in a ring by itself (CASSANDRA-5768) + * add cassandra.unsafesystem property (CASSANDRA-5704) + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763) + * Add replace_node functionality for vnodes (CASSANDRA-5337) + * Add timeout events to query traces (CASSANDRA-5520) + * make starting native protocol server idempotent (CASSANDRA-5728) + * Fix loading key cache when a saved entry is no longer valid (CASSANDRA-5706) + * Fix serialization of the LEFT gossip value (CASSANDRA-5696) + * Pig: support for cql3 tables (CASSANDRA-5234) + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675) + * Race condition in detecting version on a mixed 1.1/1.2 cluster + (CASSANDRA-5692) + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712) + * Expire entries out of ThriftSessionManager (CASSANRDA-5719) + * Don't keep ancestor information in memory (CASSANDRA-5342) + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697) + * Expose native protocol server status in nodetool info (CASSANDRA-5735) + * Fix pathetic performance of range tombstones (CASSANDRA-5677) + * Fix querying with an empty (impossible) range (CASSANDRA-5573) + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760) + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771) + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766) + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769) + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781) + * Don't rely on row marker for queries in general to hide lost markers + after TTL expires (CASSANDRA-5762) + * Sort nodetool help output (CASSANDRA-5776) + * Fix column expiring during 2 phases compaction (CASSANDRA-5799) + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795) + + +1.2.6 + * Fix tracing when operation completes before all responses arrive + (CASSANDRA-5668) + * Fix cross-DC mutation forwarding (CASSANDRA-5632) + * Reduce SSTableLoader memory usage (CASSANDRA-) + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272) + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622) + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536) + * Fix dealing with ridiculously large max sstable sizes in LCS (CASSANDRA-5589) + * Ignore pre-truncate hints (CASSANDRA-4655) + * Move System.exit on OOM into a separate t
[05/16] git commit: merge from 1.1
merge from 1.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a Branch: refs/heads/trunk Commit: 1143df1aa439618bff6005ca1cee34cb066a Parents: 23d7bd8 eb884a5 Author: Jonathan Ellis Authored: Thu Aug 8 15:46:52 2013 -0500 Committer: Jonathan Ellis Committed: Thu Aug 8 15:46:52 2013 -0500 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/db/Column.java | 19 ++- 2 files changed, 20 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt -- diff --cc CHANGES.txt index 859e7f9,8790518..30ce976 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,214 -1,19 +1,216 @@@ -1.1.next - * Backport compaction exception handling from 1.2 +1.2.9 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831) + * fix bulk-loading compressed sstables (CASSANDRA-5820) + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter + (CASSANDRA-5824) + * update default LCS sstable size to 160MB (CASSANDRA-5727) + * Allow compacting 2Is via nodetool (CASSANDRA-5670) + * Hex-encode non-String keys in OPP (CASSANDRA-5793) + * nodetool history logging (CASSANDRA-5823) + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader + (CASSANDRA-5752) + * add "all time blocked" to StatusLogger output (CASSANDRA-5825) + * Future-proof inter-major-version schema migrations (CASSANDRA-5845) + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table + (CASSANDRA-5718) ++Merged from 1.1: + * Correctly validate sparse composite cells in scrub (CASSANDRA-5855) -1.1.12 +1.2.8 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814) + * cqlsh: add collections support to COPY (CASSANDRA-5698) + * retry important messages for any IOException (CASSANDRA-5804) + * Allow empty IN relations in SELECT/UPDATE/DELETE statements (CASSANDRA-5626) + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812) + + +1.2.7 + * if no seeds can be a reached a node won't start in a ring by itself (CASSANDRA-5768) + * add cassandra.unsafesystem property (CASSANDRA-5704) + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763) + * Add replace_node functionality for vnodes (CASSANDRA-5337) + * Add timeout events to query traces (CASSANDRA-5520) + * make starting native protocol server idempotent (CASSANDRA-5728) + * Fix loading key cache when a saved entry is no longer valid (CASSANDRA-5706) + * Fix serialization of the LEFT gossip value (CASSANDRA-5696) + * Pig: support for cql3 tables (CASSANDRA-5234) + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675) + * Race condition in detecting version on a mixed 1.1/1.2 cluster + (CASSANDRA-5692) + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712) + * Expire entries out of ThriftSessionManager (CASSANRDA-5719) + * Don't keep ancestor information in memory (CASSANDRA-5342) + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697) + * Expose native protocol server status in nodetool info (CASSANDRA-5735) + * Fix pathetic performance of range tombstones (CASSANDRA-5677) + * Fix querying with an empty (impossible) range (CASSANDRA-5573) + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760) + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771) + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766) + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769) + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781) + * Don't rely on row marker for queries in general to hide lost markers + after TTL expires (CASSANDRA-5762) + * Sort nodetool help output (CASSANDRA-5776) + * Fix column expiring during 2 phases compaction (CASSANDRA-5799) + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795) + + +1.2.6 + * Fix tracing when operation completes before all responses arrive + (CASSANDRA-5668) + * Fix cross-DC mutation forwarding (CASSANDRA-5632) + * Reduce SSTableLoader memory usage (CASSANDRA-) + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272) + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622) + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536) + * Fix dealing with ridiculously large max sstable sizes in LCS (CASSANDRA-5589) + * Ignore pre-truncate hints (CASSANDRA-4655) + * Move System.exit on OOM into a separate thread (C
git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Updated Branches:
refs/heads/cassandra-1.1 238139cda -> eb884a582
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58
Branch: refs/heads/cassandra-1.1
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis
Committed: Thu Aug 8 15:43:43 2013 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Column.java | 19 ++-
2 files changed, 19 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.1.next
* Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
1.1.12
http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
public void validateFields(CFMetaData metadata) throws MarshalException
{
validateName(metadata);
-AbstractType valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite && !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
+List components =
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType valueValidator =
metadata.getValueValidator(internalName);
if (valueValidator != null)
valueValidator.validate(value());
}
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733960#comment-13733960 ] T Jake Luciani commented on CASSANDRA-5862: --- bq. if we can come up with a single name to represent "adler and post-compression" then I'd prefer I was trying to future proof us switching back to crc or something but not changing the post-compression. Think its safe to merge? > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733955#comment-13733955 ] T Jake Luciani edited comment on CASSANDRA-5862 at 8/8/13 8:45 PM: --- bq. the dual checksum fields in CRAR are a bit clunky; the approach of instantiating the correct one in the constructor that you use in CIS is cleaner. My concern was the instance would be re-used across multiple sstables therefore should be chosen per buffer. Is that not the case? was (Author: tjake): .bq the dual checksum fields in CRAR are a bit clunky; the approach of instantiating the correct one in the constructor that you use in CIS is cleaner. My concern was the instance would be re-used across multiple sstables therefore should be chosen per buffer. Is that not the case? > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733955#comment-13733955 ] T Jake Luciani commented on CASSANDRA-5862: --- .bq the dual checksum fields in CRAR are a bit clunky; the approach of instantiating the correct one in the constructor that you use in CIS is cleaner. My concern was the instance would be re-used across multiple sstables therefore should be chosen per buffer. Is that not the case? > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733953#comment-13733953
]
T Jake Luciani commented on CASSANDRA-5862:
---
with current crc32
{code}
verages from the middle 80% of values:
interval_op_rate : 14556
interval_key_rate : 14556
latency median: 2.7
latency 95th percentile : 4.7
latency 99.9th percentile : 37.0
Total operation time : 00:01:15
END
{code}
with adler32
{code}
Averages from the middle 80% of values:
interval_op_rate : 17985
interval_key_rate : 17985
latency median: 2.0
latency 95th percentile : 3.9
latency 99.9th percentile : 73.7
Total operation time : 00:01:02
END
{code}
> Switch to adler checksum for sstables
> -
>
> Key: CASSANDRA-5862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Ellis
>Assignee: T Jake Luciani
> Fix For: 2.0.1
>
> Attachments: 5862.txt
>
>
> Adler is significantly faster than CRC32:
> http://java-performance.info/java-crc32-and-adler32/
> (Adler is weaker for short inputs, so we should leave the commitlog alone, as
> it checksums each mutation individually.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733952#comment-13733952 ] Jonathan Ellis commented on CASSANDRA-5862: --- Radim: hardware CRC32C is exactly what Jake tested above. Jake: the dual checksum fields in CRAR are a bit clunky; the approach of instantiating the correct one in the constructor that you use in CIS is cleaner. Nit: if we can come up with a single name to represent "adler and post-compression" then I'd prefer that to passing two booleans around that are always the same value. Otherwise LGTM. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta
[
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733932#comment-13733932
]
Jonathan Ellis commented on CASSANDRA-5855:
---
Actually, COMPACT case is handled by validateName so we're good. I'll fix the
nit and commit.
> Scrub does not understand compound primary key created in CQL 3 beta
>
>
> Key: CASSANDRA-5855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: J.B. Langston
>Assignee: Tyler Hobbs
> Attachments: 0001-Correctly-validate-sparse-composite-columns.patch
>
>
> We have a customer who was using the beta version of CQL 3 in DSE 3.0 which
> includes Cassandra 1.1.9 plus patches backported from later versions.
> They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
> When restarting for the first time after running upgradesstables, they
> noticed the following error in the log:
> {noformat}
> Thread[SSTableBatchOpen:2,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
> at
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
> at
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
> at
> org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This error was also reported on CASSANDRA-5703. The comments suggested it
> was caused by an empty row key, so I had them run scrub on it. When they
> did, scrub reported the following warning almost 4 million times:
> {noformat}
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 52) Row at 530332255 is unreadable; skipping to next
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 57) Non-fatal error reading row (stacktrace follows)
> java.lang.RuntimeException: Error validating row
> DecoratedKey(139154446688383793922009760478335751546,
> 735fc9da503b11e2844b123140ff209f)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:98)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
> at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
> at
> org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
> at
> org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't
> validate.
> at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
> at org.apache.cassandra.db.Column.validateFields(Column.java:292)
> at
> org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWi
[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta
[
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733928#comment-13733928
]
Jonathan Ellis commented on CASSANDRA-5855:
---
It looks to me like we actually need to handle 3 cases:
# classic one-cell-per-column, no composites; the existing code
# CQL-style one-cell-per-column; the code you added
# COMPACT style, multiple columns in one cell; need to split them up and check
each value
Nit: prefer assigning variables only once, e.g. we could write the
validationName code [without addressing case 3 above] as
{code}
ByteBuffer validationName;
if (cfdef.isComposite && !cfdef.isCompact)
{
AbstractCompositeType comparator = (AbstractCompositeType)
metadata.comparator;
List components =
comparator.deconstruct(name);
validationName = components.get(components.size() - 1).value;
}
else
{
validationName = name;
}
{code}
> Scrub does not understand compound primary key created in CQL 3 beta
>
>
> Key: CASSANDRA-5855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: J.B. Langston
>Assignee: Tyler Hobbs
> Attachments: 0001-Correctly-validate-sparse-composite-columns.patch
>
>
> We have a customer who was using the beta version of CQL 3 in DSE 3.0 which
> includes Cassandra 1.1.9 plus patches backported from later versions.
> They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
> When restarting for the first time after running upgradesstables, they
> noticed the following error in the log:
> {noformat}
> Thread[SSTableBatchOpen:2,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
> at
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
> at
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
> at
> org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This error was also reported on CASSANDRA-5703. The comments suggested it
> was caused by an empty row key, so I had them run scrub on it. When they
> did, scrub reported the following warning almost 4 million times:
> {noformat}
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 52) Row at 530332255 is unreadable; skipping to next
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 57) Non-fatal error reading row (stacktrace follows)
> java.lang.RuntimeException: Error validating row
> DecoratedKey(139154446688383793922009760478335751546,
> 735fc9da503b11e2844b123140ff209f)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:98)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
> at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
> at
> org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
> at
> org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
> at java.util.concurrent.FutureTask$Sync.innerRun(Fut
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733923#comment-13733923 ] Radim Kolar commented on CASSANDRA-5862: Today all new projects should use CRC32C, it can be done in cpu hardware SSE4 and polynomial is improved from crc32. In my test notes Adler32 has 5 times higher collision rate then CRC32, CRC32C is better then CRC32 but very little. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5859) Don't return internal StreamState objects from streaming mbeans
[ https://issues.apache.org/jira/browse/CASSANDRA-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5859: -- Reviewer: thobbs > Don't return internal StreamState objects from streaming mbeans > --- > > Key: CASSANDRA-5859 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5859 > Project: Cassandra > Issue Type: Bug >Reporter: Nick Bailey >Assignee: Yuki Morishita > Labels: streaming > Fix For: 2.0 > > Attachments: 5859.txt > > > The stream manager mbean returns StreamState objects. We want to avoid > returning internal C* objects over jmx. We should switch to a map or > something similar that can represent streaming state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-5862: -- Attachment: 5862.txt Patch attached, I switched to adler32 and changed checksumming to be on the compressed value. Going to run some benchmarks now. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > Attachments: 5862.txt > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1511991 - in /cassandra/site: publish/download/index.html src/settings.py
Author: eevans Date: Thu Aug 8 19:46:18 2013 New Revision: 1511991 URL: http://svn.apache.org/r1511991 Log: update versioning or 2.0.0 rc1 release Modified: cassandra/site/publish/download/index.html cassandra/site/src/settings.py Modified: cassandra/site/publish/download/index.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1511991&r1=1511990&r2=1511991&view=diff == --- cassandra/site/publish/download/index.html (original) +++ cassandra/site/publish/download/index.html Thu Aug 8 19:46:18 2013 @@ -107,16 +107,16 @@ Development Cassandra Server Releases (not production ready) - The latest development release is 2.0.0-beta2 (released on - 2013-07-25). + The latest development release is 2.0.0-rc1 (released on + 2013-08-08). -http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz";>apache-cassandra-2.0.0-beta2-bin.tar.gz -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.asc";>PGP] -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.md5";>MD5] -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.sha1";>SHA1] +http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz";>apache-cassandra-2.0.0-rc1-bin.tar.gz +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.asc";>PGP] +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.md5";>MD5] +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.sha1";>SHA1] @@ -195,10 +195,10 @@ -http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz";>apache-cassandra-2.0.0-beta2-src.tar.gz -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.asc";>PGP] -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.md5";>MD5] -[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.sha1";>SHA1] +http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz";>apache-cassandra-2.0.0-rc1-src.tar.gz +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.asc";>PGP] +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.md5";>MD5] +[http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.sha1";>SHA1] Modified: cassandra/site/src/settings.py URL: http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1511991&r1=1511990&r2=1511991&view=diff == --- cassandra/site/src/settings.py (original) +++ cassandra/site/src/settings.py Thu Aug 8 19:46:18 2013 @@ -100,8 +100,8 @@ class CassandraDef(object): veryoldstable_exists = True stable_version = '1.2.8' stable_release_date = '2013-07-28' -devel_version = '2.0.0-beta2' -devel_release_date = '2013-07-25' +devel_version = '2.0.0-rc1' +devel_release_date = '2013-08-08' devel_exists = True _apache_base_url = 'http://www.apache.org' _svn_base_url = 'https://svn.apache.org/repos/asf'
[jira] [Updated] (CASSANDRA-5859) Don't return internal StreamState objects from streaming mbeans
[ https://issues.apache.org/jira/browse/CASSANDRA-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5859: -- Attachment: 5859.txt Initial patch attached. (Also on github: https://github.com/yukim/cassandra/commits/5859) Since StreamState is complex object, I used JMX's [CompositeData|http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html] to convert StreamState to be transferred over JMX. Patch also adds support for exposing JMX notification support for stream events to StreamManagerMBean, so one can just monitor start/progress/completion of streaming. > Don't return internal StreamState objects from streaming mbeans > --- > > Key: CASSANDRA-5859 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5859 > Project: Cassandra > Issue Type: Bug >Reporter: Nick Bailey >Assignee: Yuki Morishita > Fix For: 2.0 > > Attachments: 5859.txt > > > The stream manager mbean returns StreamState objects. We want to avoid > returning internal C* objects over jmx. We should switch to a map or > something similar that can represent streaming state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta
[
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-5855:
---
Attachment: 0001-Correctly-validate-sparse-composite-columns.patch
Patch 0001 should apply to 1.1 or 1.2. It's not the most general technique, so
I welcome any suggestions, but it does fix this particular issue. If a sparse
composite schema is being used, only the last composite component is used to
try to look up the column validator.
I'll also note that this code path is currently only used by scrub, so it
should be safe to test.
> Scrub does not understand compound primary key created in CQL 3 beta
>
>
> Key: CASSANDRA-5855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: J.B. Langston
>Assignee: Tyler Hobbs
> Attachments: 0001-Correctly-validate-sparse-composite-columns.patch
>
>
> We have a customer who was using the beta version of CQL 3 in DSE 3.0 which
> includes Cassandra 1.1.9 plus patches backported from later versions.
> They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
> When restarting for the first time after running upgradesstables, they
> noticed the following error in the log:
> {noformat}
> Thread[SSTableBatchOpen:2,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
> at
> org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
> at
> org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
> at
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
> at
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
> at
> org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This error was also reported on CASSANDRA-5703. The comments suggested it
> was caused by an empty row key, so I had them run scrub on it. When they
> did, scrub reported the following warning almost 4 million times:
> {noformat}
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 52) Row at 530332255 is unreadable; skipping to next
> WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java
> (line 57) Non-fatal error reading row (stacktrace follows)
> java.lang.RuntimeException: Error validating row
> DecoratedKey(139154446688383793922009760478335751546,
> 735fc9da503b11e2844b123140ff209f)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
> at
> org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:98)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
> at
> org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
> at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
> at
> org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
> at
> org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
> at
> org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't
> validate.
> at org.apache.cassandra
Git Push Summary
Updated Tags: refs/tags/2.0.0-rc1-tentative [deleted] e8ae6720e
Git Push Summary
Updated Tags: refs/tags/cassandra-2.0.0-rc1 [created] 930e60a7a
[Cassandra Wiki] Update of "HowToPublishReleases" by EricEvans
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "HowToPublishReleases" page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diff&rev1=26&rev2=27
find . -type f -exec chmod 664 {} \;
}}}
1. Login to [[http://repository.apache.org|repository.apache.org]] and
release the staging repository.
- 1. ''TODO: Some stuff to do with updating the project site that I don't know
(Stephen)''
- 1. Marke the version as released in JIRA on the
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
Versions]] screen. ''TODO: Can somebody who has access please verify the link
- I've inferred it (Stephen)''
- 1. Wait for the sync
+ 1. Remove the tentative tag; Create the permanent tag (preferably signed /
annotated)
+ 1. Wait for the sync to http://www.apache.org/dist/cassandra before
continuing
+ 1. Update the website.
1. Publish the announce email to
[[mailto:u...@cassandra.apache.org|u...@cassandra.apache.org]]
{{{
Subject: [ANN] Apache Cassandra X.Y.Z released
[Cassandra Wiki] Update of "HowToPublishReleases" by EricEvans
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "HowToPublishReleases" page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diff&rev1=25&rev2=26
1. Send a message declaring the vote closed, and summarizing the results.
1. Mark the version as released in
[[https://issues.apache.org/jira/plugins/servlet/project-config/CASSANDRA/versions|JIRA]]
1. Login to people.apache.org and put the release artifacts into
{{{/www/www.apache.org/dist/cassandra/}}}, e.g.
- {{{
+ {{{
- cd /www/www.apache.org/dist/cassandra
- mkdir X.Y.Z
- cd X.Y.Z
+ STAGING_REPOID="orgapachecassandra-062"
+ RELEASE="2.0.0-rc1"
+
for type in bin src; do \
- for part in gz gz.md5 gz.sha1 gz.asc gz.asc.md5 gz.asc.sha1; do \
+ for part in gz gz.md5 gz.sha1 gz.asc; do \
- curl -O https://repository.apache.org/content/repositories/''staging
repository
id''/org/apache/cassandra/apache-cassandra/X.Y.Z/apache-cassandra-X.Y.Z-$type.tar.$part;
\
+ curl -O
https://repository.apache.org/content/repositories/$STAGING_REPOID/org/apache/cassandra/apache-cassandra/$RELEASE/apache-cassandra-$RELEASE-$type.tar.$part;
\
done \
done
- cd ..
+
- find X.Y.Z -type d -exec chmod a+rx,g+w {} \;
+ find . -exec chmod a+rx,g+w {} \;
- find X.Y.Z -type f -exec chmod 664 {} \;
+ find . -type f -exec chmod 664 {} \;
}}}
- After doing this, login to
[[http://repository.apache.org|repository.apache.org]] and release the staging
repository.
+ 1. Login to [[http://repository.apache.org|repository.apache.org]] and
release the staging repository.
1. ''TODO: Some stuff to do with updating the project site that I don't know
(Stephen)''
1. Marke the version as released in JIRA on the
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
Versions]] screen. ''TODO: Can somebody who has access please verify the link
- I've inferred it (Stephen)''
1. Wait for the sync
[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
[ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733830#comment-13733830 ] Ben Manes commented on CASSANDRA-5661: -- My benchmark had a bug and EBS may only be on par with LTQ performance wise. I need to investigate that again, though. I shifted focus to fixing the performance bottleneck in Guava's cache. The way we tracked usage history (e.g. LRU) was focused on common usage, but is a bottleneck on synthetic benchmarks. I made the fixes to CLHM (v1.4) and offered them upstream (issue 1487). I'll experiment with using CLHM instead to see if that removes the hotspot. > Discard pooled readers for cold data > > > Key: CASSANDRA-5661 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5661 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2.1 >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 2.0.1 > > Attachments: CASSANDRA-5661-global-multiway-cache.patch, > CASSANDRA-5661.patch, DominatorTree.png, Histogram.png > > > Reader pooling was introduced in CASSANDRA-4942 but pooled > RandomAccessReaders are never cleaned up until the SSTableReader is closed. > So memory use is "the worst case simultaneous RAR we had open for this file, > forever." > We should introduce a global limit on how much memory to use for RAR, and > evict old ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of "HowToPublishReleases" by EricEvans
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "HowToPublishReleases" page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diff&rev1=24&rev2=25
[2]: http://s.apache.org/link (NEWS.txt)
[3]: http://s.apache.org/link (Release notes from JIRA)
}}}
+
+ === If The Vote Passes ===
+ 1. Send a message declaring the vote closed, and summarizing the results.
+ 1. Mark the version as released in
[[https://issues.apache.org/jira/plugins/servlet/project-config/CASSANDRA/versions|JIRA]]
- 1. If the vote is a success, login to people.apache.org and put the release
artifacts into {{{/www/www.apache.org/dist/cassandra}}}, e.g.
+ 1. Login to people.apache.org and put the release artifacts into
{{{/www/www.apache.org/dist/cassandra/}}}, e.g.
{{{
cd /www/www.apache.org/dist/cassandra
mkdir X.Y.Z
@@ -105, +109 @@
find X.Y.Z -type f -exec chmod 664 {} \;
}}}
After doing this, login to
[[http://repository.apache.org|repository.apache.org]] and release the staging
repository.
-
- If the vote failed then login to
[[http://repository.apache.org|repository.apache.org]] and drop the staging
repository and start over again with ''Take N+1''
-
1. ''TODO: Some stuff to do with updating the project site that I don't know
(Stephen)''
1. Marke the version as released in JIRA on the
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
Versions]] screen. ''TODO: Can somebody who has access please verify the link
- I've inferred it (Stephen)''
1. Wait for the sync
@@ -139, +140 @@
1. Add the release to the PMC's next board report
1. Remove Old Versions from ASF Distribution Directory. Login to
people.apache.org and remove any superceded releases from
{{{/www/www.apache.org/dist/cassandra}}} (after confirming that they exist in
{{{/www/archive.apache.org/dist/cassandra}}}).
+ === If the vote fails ===
+ 1. Login to [[http://repository.apache.org|repository.apache.org]] and drop
the staging repository and start over again with ''Take N+1''
+
[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
[ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733815#comment-13733815 ] Pavel Yaskevich commented on CASSANDRA-5661: I was actually destructed with other things and kind of waiting until code is done so I can test the version of multipool with all of the changes. > Discard pooled readers for cold data > > > Key: CASSANDRA-5661 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5661 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2.1 >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 2.0.1 > > Attachments: CASSANDRA-5661-global-multiway-cache.patch, > CASSANDRA-5661.patch, DominatorTree.png, Histogram.png > > > Reader pooling was introduced in CASSANDRA-4942 but pooled > RandomAccessReaders are never cleaned up until the SSTableReader is closed. > So memory use is "the worst case simultaneous RAR we had open for this file, > forever." > We should introduce a global limit on how much memory to use for RAR, and > evict old ones. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733814#comment-13733814 ] Lior Golan commented on CASSANDRA-5862: --- What's the typical size of a compressed chunk (after compression)? If it's above a few 100 bytes then Adler should be ok. In addition, compressed data has higher entropy which improves the quality of the adler32 checksum > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4573) HSHA doesn't handle large messages gracefully
[
https://issues.apache.org/jira/browse/CASSANDRA-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733813#comment-13733813
]
Pavel Yaskevich commented on CASSANDRA-4573:
[~jbellis] Done, licences committed to cassandra-2.0.0 for thrift-server and
LMAX disruptor.
> HSHA doesn't handle large messages gracefully
> -
>
> Key: CASSANDRA-4573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4573
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Pavel Yaskevich
> Fix For: 2.0
>
> Attachments: CASSANDRA-4573.patch,
> disruptor-thrift-server-0.2.2-SNAPSHOT.jar, repro.py
>
>
> HSHA doesn't seem to enforce any kind of max message length, and when
> messages are too large, it doesn't fail gracefully.
> With debug logs enabled, you'll see this:
> {{DEBUG 13:13:31,805 Unexpected state 16}}
> Which seems to mean that there's a SelectionKey that's valid, but isn't ready
> for reading, writing, or accepting.
> Client-side, you'll get this thrift error (while trying to read a frame as
> part of {{recv_batch_mutate}}):
> {{TTransportException: TSocket read 0 bytes}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733811#comment-13733811 ] Jonathan Ellis commented on CASSANDRA-5862: --- bq. we do checksum the uncompressed data, but since adler32 is bad with small values we should prob keep it as is Even with 10x compression (2x as high as I've actually seen) that's still thousands of bytes to work with. I think we'd be okay. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: add thrift-server and LMAX Disruptor licences
Updated Branches: refs/heads/cassandra-2.0.0 50b40e91f -> 7a8cbb3ea add thrift-server and LMAX Disruptor licences Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7a8cbb3e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7a8cbb3e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7a8cbb3e Branch: refs/heads/cassandra-2.0.0 Commit: 7a8cbb3ea7e0898778ed41a6d19c40d67c065667 Parents: 50b40e9 Author: Pavel Yaskevich Authored: Thu Aug 8 11:30:02 2013 -0700 Committer: Pavel Yaskevich Committed: Thu Aug 8 11:30:02 2013 -0700 -- lib/licenses/disruptor-3.0.1.txt | 201 + lib/licenses/thrift-server-0.3.0.txt | 202 ++ 2 files changed, 403 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a8cbb3e/lib/licenses/disruptor-3.0.1.txt -- diff --git a/lib/licenses/disruptor-3.0.1.txt b/lib/licenses/disruptor-3.0.1.txt new file mode 100644 index 000..50086f8 --- /dev/null +++ b/lib/licenses/disruptor-3.0.1.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contribut
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733806#comment-13733806 ] T Jake Luciani commented on CASSANDRA-5862: --- [~hsn] ah ok, well as [~jbellis] mentioned it's a poor choice for small values. Compression chunks are a much larger value (default of 64kb) [~liorgo2] good point, we do checksum the uncompressed data, but since adler32 is bad with small values we should prob keep it as is. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733792#comment-13733792 ] Radim Kolar commented on CASSANDRA-5862: 60-80 bytes > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[4/6] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d9a33a4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d9a33a4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d9a33a4 Branch: refs/heads/trunk Commit: 5d9a33a46c24c5aff9fb3d3497edcd23d7a6c196 Parents: 988aa31 50b40e9 Author: Brandon Williams Authored: Thu Aug 8 13:03:17 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:03:17 2013 -0500 -- .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++ 1 file changed, 2 insertions(+), 12 deletions(-) --
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733784#comment-13733784 ] Lior Golan commented on CASSANDRA-5862: --- Are you computing the checksum on the data before or after compression? If you are computing the checksum of the uncompressed data, then it might be useful to switch to checksum the compressed data (less data to checksum) > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5686) During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS
[ https://issues.apache.org/jira/browse/CASSANDRA-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733779#comment-13733779 ] Robert Coli commented on CASSANDRA-5686: @OP : what version of cassandra on what platform, with which JVM? > During node repair operation, Cassandra throws exception on one of our nodes: > failed to uncompress the chunk: FAILED_TO_UNCOMPRESS > --- > > Key: CASSANDRA-5686 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5686 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.5 >Reporter: Rao > > During node repair operation, Cassandra throws exception on one of our nodes: > failed to uncompress the chunk: FAILED_TO_UNCOMPRESS > INFO [AntiEntropyStage:1] 2013-06-20 03:08:03,876 AntiEntropyService.java > (line 213) [repair #4ae2a970-d991-11e2-bd05-094c1105e54d] Received merkle > tree for tags from /135.163.214.175 > ERROR [Thread-40] 2013-06-20 03:08:03,903 CassandraDaemon.java (line 175) > Exception in thread Thread[Thread-40,5,main] > java.lang.RuntimeException: java.io.IOException: failed to uncompress the > chunk: FAILED_TO_UNCOMPRESS(5) > at > org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:438) > at > org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:421) > at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94) > at > org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:203) > at > org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135) > at > org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82) > Caused by: java.io.IOException: failed to uncompress the chunk: > FAILED_TO_UNCOMPRESS(5) > at > org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:361) > at > org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:383) > at java.io.DataInputStream.readInt(DataInputStream.java:387) > at > org.apache.cassandra.utils.MerkleTree$Leaf$LeafSerializer.deserialize(MerkleTree.java:793) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:920) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) > at > org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) > at > org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(Merk
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733778#comment-13733778 ] T Jake Luciani commented on CASSANDRA-5862: --- [~hsn] do you have any details? what size was the payload you were checksumming? > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/6] git commit: Remove jvm revision checks.. for now
Updated Branches:
refs/heads/cassandra-2.0 988aa3163 -> 5d9a33a46
refs/heads/cassandra-2.0.0 8ee09f7ab -> 50b40e91f
refs/heads/trunk ac6245364 -> 28b827a85
Remove jvm revision checks.. for now
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91
Branch: refs/heads/cassandra-2.0
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:03:08 2013 -0500
--
.../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
1 file changed, 2 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
{
logger.warn("Non-Oracle JVM detected. Some features, such as
immediate unmap of compacted SSTables, may not work as intended");
}
-else
+ /* else
{
String[] java_version = javaVersion.split("_");
String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
logger.info("Unable to parse java version {}",
Arrays.toString(java_version));
java_minor = 32;
}
-if (java_major.equals("1.6.0"))
-{
-// These need to be updated from time to time, but these are
currently valid (12.18.2012)
-if (java_minor < 29)
-// Seen to be a major contributing factor for heap and
load issues
-logger.error("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-else if (java_minor < 32)
-// Updates 32+ have been seen to work well enough in the
wild
-logger.warn("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-}
}
-
+ */
logger.info("Heap size: {}/{}", Runtime.getRuntime().totalMemory(),
Runtime.getRuntime().maxMemory());
logger.info("Classpath: {}", System.getProperty("java.class.path"));
CLibrary.tryMlockall();
[02/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c
Branch: refs/heads/cassandra-2.0.0
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 12:58:58 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return Long.parseLong(pieces[2]);
else
return 0L;
-} else
+}
+else
{
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
}
}
[2/6] git commit: Remove jvm revision checks.. for now
Remove jvm revision checks.. for now
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91
Branch: refs/heads/cassandra-2.0.0
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:03:08 2013 -0500
--
.../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
1 file changed, 2 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
{
logger.warn("Non-Oracle JVM detected. Some features, such as
immediate unmap of compacted SSTables, may not work as intended");
}
-else
+ /* else
{
String[] java_version = javaVersion.split("_");
String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
logger.info("Unable to parse java version {}",
Arrays.toString(java_version));
java_minor = 32;
}
-if (java_major.equals("1.6.0"))
-{
-// These need to be updated from time to time, but these are
currently valid (12.18.2012)
-if (java_minor < 29)
-// Seen to be a major contributing factor for heap and
load issues
-logger.error("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-else if (java_minor < 32)
-// Updates 32+ have been seen to work well enough in the
wild
-logger.warn("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-}
}
-
+ */
logger.info("Heap size: {}/{}", Runtime.getRuntime().totalMemory(),
Runtime.getRuntime().maxMemory());
logger.info("Classpath: {}", System.getProperty("java.class.path"));
CLibrary.tryMlockall();
[05/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0
Merge branch 'cassandra-1.2' into cassandra-2.0.0
Conflicts:
src/java/org/apache/cassandra/service/StorageService.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a
Branch: refs/heads/cassandra-2.0
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:01:40 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 5 +
1 file changed, 1 insertion(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
}
}
-protected long extractExpireTime(String[] pieces, int version)
+protected long extractExpireTime(String[] pieces)
{
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
-if (version < MessagingService.VERSION_12)
-{
-if (pieces.length >= 3)
-return Long.parseLong(pieces[2]);
-else
-return 0L;
-}
-else
-{
-return Long.parseLong(pieces[2]);
-}
++return Long.parseLong(pieces[2]);
}
/**
[5/6] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d9a33a4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d9a33a4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d9a33a4 Branch: refs/heads/cassandra-2.0 Commit: 5d9a33a46c24c5aff9fb3d3497edcd23d7a6c196 Parents: 988aa31 50b40e9 Author: Brandon Williams Authored: Thu Aug 8 13:03:17 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:03:17 2013 -0500 -- .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++ 1 file changed, 2 insertions(+), 12 deletions(-) --
[07/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0
Merge branch 'cassandra-1.2' into cassandra-2.0.0
Conflicts:
src/java/org/apache/cassandra/service/StorageService.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a
Branch: refs/heads/trunk
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:01:40 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 5 +
1 file changed, 1 insertion(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
}
}
-protected long extractExpireTime(String[] pieces, int version)
+protected long extractExpireTime(String[] pieces)
{
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
-if (version < MessagingService.VERSION_12)
-{
-if (pieces.length >= 3)
-return Long.parseLong(pieces[2]);
-else
-return 0L;
-}
-else
-{
-return Long.parseLong(pieces[2]);
-}
++return Long.parseLong(pieces[2]);
}
/**
[09/10] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/988aa316 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/988aa316 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/988aa316 Branch: refs/heads/cassandra-2.0 Commit: 988aa31631f59f2949cde68e4ffd9ddbe24fabe3 Parents: c526131 8ee09f7 Author: Brandon Williams Authored: Thu Aug 8 13:01:51 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:01:51 2013 -0500 -- src/java/org/apache/cassandra/service/StorageService.java | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) --
[08/10] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0
Merge branch 'cassandra-2.0.0' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/988aa316 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/988aa316 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/988aa316 Branch: refs/heads/trunk Commit: 988aa31631f59f2949cde68e4ffd9ddbe24fabe3 Parents: c526131 8ee09f7 Author: Brandon Williams Authored: Thu Aug 8 13:01:51 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:01:51 2013 -0500 -- src/java/org/apache/cassandra/service/StorageService.java | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) --
[01/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Updated Branches:
refs/heads/cassandra-1.2 23d7bd857 -> ccb32a8c8
refs/heads/cassandra-2.0 c52613124 -> 988aa3163
refs/heads/cassandra-2.0.0 f30629b83 -> 8ee09f7ab
refs/heads/trunk 93f838d18 -> ac6245364
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c
Branch: refs/heads/cassandra-1.2
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 12:58:58 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return Long.parseLong(pieces[2]);
else
return 0L;
-} else
+}
+else
{
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
}
}
[04/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c
Branch: refs/heads/trunk
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 12:58:58 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return Long.parseLong(pieces[2]);
else
return 0L;
-} else
+}
+else
{
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
}
}
[3/6] git commit: Remove jvm revision checks.. for now
Remove jvm revision checks.. for now
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91
Branch: refs/heads/trunk
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:03:08 2013 -0500
--
.../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
1 file changed, 2 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
{
logger.warn("Non-Oracle JVM detected. Some features, such as
immediate unmap of compacted SSTables, may not work as intended");
}
-else
+ /* else
{
String[] java_version = javaVersion.split("_");
String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
logger.info("Unable to parse java version {}",
Arrays.toString(java_version));
java_minor = 32;
}
-if (java_major.equals("1.6.0"))
-{
-// These need to be updated from time to time, but these are
currently valid (12.18.2012)
-if (java_minor < 29)
-// Seen to be a major contributing factor for heap and
load issues
-logger.error("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-else if (java_minor < 32)
-// Updates 32+ have been seen to work well enough in the
wild
-logger.warn("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-}
}
-
+ */
logger.info("Heap size: {}/{}", Runtime.getRuntime().totalMemory(),
Runtime.getRuntime().maxMemory());
logger.info("Classpath: {}", System.getProperty("java.class.path"));
CLibrary.tryMlockall();
[03/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c
Branch: refs/heads/cassandra-2.0
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 12:58:58 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return Long.parseLong(pieces[2]);
else
return 0L;
-} else
+}
+else
{
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
}
}
[06/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0
Merge branch 'cassandra-1.2' into cassandra-2.0.0
Conflicts:
src/java/org/apache/cassandra/service/StorageService.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a
Branch: refs/heads/cassandra-2.0.0
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 13:01:40 2013 -0500
--
src/java/org/apache/cassandra/service/StorageService.java | 5 +
1 file changed, 1 insertion(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
}
}
-protected long extractExpireTime(String[] pieces, int version)
+protected long extractExpireTime(String[] pieces)
{
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
-if (version < MessagingService.VERSION_12)
-{
-if (pieces.length >= 3)
-return Long.parseLong(pieces[2]);
-else
-return 0L;
-}
-else
-{
-return Long.parseLong(pieces[2]);
-}
++return Long.parseLong(pieces[2]);
}
/**
[6/6] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28b827a8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28b827a8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28b827a8 Branch: refs/heads/trunk Commit: 28b827a855e838c81c28bfef1746df48774b5935 Parents: ac62453 5d9a33a Author: Brandon Williams Authored: Thu Aug 8 13:03:30 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:03:30 2013 -0500 -- --
[10/10] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac624536 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac624536 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac624536 Branch: refs/heads/trunk Commit: ac62453649e1b47ec5a8c8d0a13687c9d7d2c481 Parents: 93f838d 988aa31 Author: Brandon Williams Authored: Thu Aug 8 13:02:01 2013 -0500 Committer: Brandon Williams Committed: Thu Aug 8 13:02:01 2013 -0500 -- src/java/org/apache/cassandra/service/StorageService.java | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) --
[jira] [Commented] (CASSANDRA-5857) NumberFormatException during decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733770#comment-13733770
]
Jason Brown commented on CASSANDRA-5857:
lgtm
> NumberFormatException during decommission
> -
>
> Key: CASSANDRA-5857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Minor
> Fix For: 1.2.9
>
> Attachments: 5857.txt
>
>
> We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is
> still looking at the token to get the expiretime in some cases.
> {noformat}
> java.lang.NumberFormatException: For input string:
> "113427455640312821154458202477256070484"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:444)
> at java.lang.Long.parseLong(Long.java:483)
> at
> org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
> at
> org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
> at
> org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
> at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Remove jvm revision checks.. for now
Updated Branches:
refs/heads/trunk c7782f2de -> 93f838d18
Remove jvm revision checks.. for now
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93f838d1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93f838d1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93f838d1
Branch: refs/heads/trunk
Commit: 93f838d18518b42427f6596041a7d7edecda4ff4
Parents: c7782f2
Author: Brandon Williams
Authored: Thu Aug 8 12:57:04 2013 -0500
Committer: Brandon Williams
Committed: Thu Aug 8 12:57:04 2013 -0500
--
.../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
1 file changed, 2 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/93f838d1/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
{
logger.warn("Non-Oracle JVM detected. Some features, such as
immediate unmap of compacted SSTables, may not work as intended");
}
-else
+ /* else
{
String[] java_version = javaVersion.split("_");
String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
logger.info("Unable to parse java version {}",
Arrays.toString(java_version));
java_minor = 32;
}
-if (java_major.equals("1.6.0"))
-{
-// These need to be updated from time to time, but these are
currently valid (12.18.2012)
-if (java_minor < 29)
-// Seen to be a major contributing factor for heap and
load issues
-logger.error("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-else if (java_minor < 32)
-// Updates 32+ have been seen to work well enough in the
wild
-logger.warn("Your JVM is out of date. Please upgrade to
the newest Oracle Java 6.");
-}
}
-
+ */
logger.info("Heap size: {}/{}", Runtime.getRuntime().totalMemory(),
Runtime.getRuntime().maxMemory());
logger.info("Classpath: {}", System.getProperty("java.class.path"));
CLibrary.tryMlockall();
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733753#comment-13733753
]
T Jake Luciani commented on CASSANDRA-5862:
---
Looks like Adler32 wins!
{code}
benchmark:
[java] 0% Scenario{vm=java, trial=0, benchmark=JavaCrc32} 58541.04 ns;
σ=334.60 ns @ 3 trials
[java] 33% Scenario{vm=java, trial=0, benchmark=SSECrc32} 15504.47 ns;
σ=46.66 ns @ 3 trials
[java] 67% Scenario{vm=java, trial=0, benchmark=JavaAdler32} 6610.37 ns;
σ=232.76 ns @ 10 trials
[java]
[java] benchmarkus linear runtime
[java] JavaCrc32 58.54 ==
[java]SSECrc32 15.50 ===
[java] JavaAdler32 6.61 ===
[java]
[java] vm: java
[java] trial: 0
{code}
> Switch to adler checksum for sstables
> -
>
> Key: CASSANDRA-5862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Ellis
>Assignee: T Jake Luciani
> Fix For: 2.0.1
>
>
> Adler is significantly faster than CRC32:
> http://java-performance.info/java-crc32-and-adler32/
> (Adler is weaker for short inputs, so we should leave the commitlog alone, as
> it checksums each mutation individually.)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733752#comment-13733752 ] Radim Kolar commented on CASSANDRA-5862: Adler is unreliable, i had bad experience with it, switched from crc32 to adler for a while. Use CRC32C. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5857) NumberFormatException during decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-5857:
Reviewer: jasobrown
Priority: Minor (was: Major)
Fix Version/s: 1.2.9
> NumberFormatException during decommission
> -
>
> Key: CASSANDRA-5857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Minor
> Fix For: 1.2.9
>
> Attachments: 5857.txt
>
>
> We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is
> still looking at the token to get the expiretime in some cases.
> {noformat}
> java.lang.NumberFormatException: For input string:
> "113427455640312821154458202477256070484"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:444)
> at java.lang.Long.parseLong(Long.java:483)
> at
> org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
> at
> org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
> at
> org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
> at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5857) NumberFormatException during decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-5857:
Attachment: 5857.txt
Patch to have SS look at the correct piece for the expire time.
> NumberFormatException during decommission
> -
>
> Key: CASSANDRA-5857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Brandon Williams
>Assignee: Brandon Williams
> Attachments: 5857.txt
>
>
> We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is
> still looking at the token to get the expiretime in some cases.
> {noformat}
> java.lang.NumberFormatException: For input string:
> "113427455640312821154458202477256070484"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:444)
> at java.lang.Long.parseLong(Long.java:483)
> at
> org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
> at
> org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
> at
> org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
> at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733697#comment-13733697 ] T Jake Luciani commented on CASSANDRA-5862: --- I have been working on a jni library to allow fast CRC checks using the newest SSE instruction set. this is what Hadoop and Impala are now using. I'll do some micro benchmarks of adler32 to this approach and see if it's worth it. The benefit of a fast CRC is the code could be used on commit log and thereby speed up writes as well. > Switch to adler checksum for sstables > - > > Key: CASSANDRA-5862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5862 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: T Jake Luciani > Fix For: 2.0.1 > > > Adler is significantly faster than CRC32: > http://java-performance.info/java-crc32-and-adler32/ > (Adler is weaker for short inputs, so we should leave the commitlog alone, as > it checksums each mutation individually.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5863) Create a Decompressed Chunk Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-5863: -- Description: Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. was: Currently, for every read the CRAR we reads compressed chunks into a byte[], check sends it to ICompressor and gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. > Create a Decompressed Chunk Cache > - > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: T Jake Luciani > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data. Initially this could be a off heap cache but it would > be great to put these decompressed chunks onto a SSD so the hot data lives on > a fast disk similar to https://github.com/facebook/flashcache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
