Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/cql3/QueryProcessor.java
src/java/org/apache/cassandra/metrics/CqlStatementMetrics.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/681c380b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/681c380b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/681c380b
Branch: refs/heads/cassandra-2.1
Commit: 681c380b5a9d0f57b85523a9c23c02e71417c799
Parents: e25d94e 19c6cc1
Author: Tyler Hobbs <[email protected]>
Authored: Tue Sep 16 13:21:36 2014 -0500
Committer: Tyler Hobbs <[email protected]>
Committed: Tue Sep 16 13:21:36 2014 -0500
----------------------------------------------------------------------
CHANGES.txt | 1 +
.../apache/cassandra/cql3/QueryProcessor.java | 53 +++++++++++++-------
.../cassandra/metrics/CqlStatementMetrics.java | 1 +
3 files changed, 36 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/681c380b/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index f89cc6d,cf7112c..0abc66a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,92 -1,10 +1,93 @@@
-2.0.11:
+2.1.1
+ * Fix saving caches when a table is dropped (CASSANDRA-7784)
+ * Add better error checking of new stress profile (CASSANDRA-7716)
+ * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom
(CASSANDRA-7934)
+ * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
+ * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
+ * GCInspector more closely tracks GC; cassandra-stress and nodetool report
it (CASSANDRA-7916)
+ * nodetool won't output bogus ownership info without a keyspace
(CASSANDRA-7173)
+ * Add human readable option to nodetool commands (CASSANDRA-5433)
+ * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
+ * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
+ * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
+ * (cqlsh) Support for query paging (CASSANDRA-7514)
+ * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
+ * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
+ * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
+ * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
+ * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
+ * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
+ * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
+ output (CASSANDRA-7659)
+ * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
+ * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
+ * Support list index operations with conditions (CASSANDRA-7499)
+ * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
+ * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
+ * (cqlsh) Error when tracing query (CASSANDRA-7613)
+ * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
+ * SSTableExport uses correct validator to create string representation of
partition
+ keys (CASSANDRA-7498)
+ * Avoid NPEs when receiving type changes for an unknown keyspace
(CASSANDRA-7689)
+ * Add support for custom 2i validation (CASSANDRA-7575)
+ * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
+ * Add listen_interface and rpc_interface options (CASSANDRA-7417)
+ * Improve schema merge performance (CASSANDRA-7444)
+ * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
+ * Optimise NativeCell comparisons (CASSANDRA-6755)
+ * Configurable client timeout for cqlsh (CASSANDRA-7516)
+ * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
+Merged from 2.0:
+ * Add metrics for prepared statement usage and eviction (CASSANDRA-7930)
* Make CQLSSTableWriter sync within partitions (CASSANDRA-7360)
* Potentially use non-local replicas in CqlConfigHelper (CASSANDRA-7906)
- * Explicitly disallowing mixing multi-column and single-column
+ * Explicitly disallow mixing multi-column and single-column
relations on clustering columns (CASSANDRA-7711)
* Better error message when condition is set on PK column (CASSANDRA-7804)
+ * Don't send schema change responses and events for no-op DDL
+ statements (CASSANDRA-7600)
+ * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774)
+ * Throw InvalidRequestException when queries contain relations on entire
+ collection columns (CASSANDRA-7506)
+ * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577)
+ * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252)
+ * (cqlsh) cqlsh should automatically disable tracing when selecting
+ from system_traces (CASSANDRA-7641)
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
+ * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
+ * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
+ * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229)
+ * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635)
+ * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS
(CASSANDRA-7611)
+ * Catch errors when the JVM pulls the rug out from GCInspector
(CASSANDRA-5345)
+ * cqlsh fails when version number parts are not int (CASSANDRA-7524)
+Merged from 1.2:
+ * Don't index tombstones (CASSANDRA-7828)
+ * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788)
+
+
+2.1.0
+ * (cqlsh) Removed "ALTER TYPE <name> RENAME TO <name>" from tab-completion
+ (CASSANDRA-7895)
+ * Fixed IllegalStateException in anticompaction (CASSANDRA-7892)
+ * cqlsh: DESCRIBE support for frozen UDTs, tuples (CASSANDRA-7863)
+ * Avoid exposing internal classes over JMX (CASSANDRA-7879)
+ * Add null check for keys when freezing collection (CASSANDRA-7869)
+ * Improve stress workload realism (CASSANDRA-7519)
+
+
+2.1.0-rc7
+ * Add frozen keyword and require UDT to be frozen (CASSANDRA-7857)
+ * Track added sstable size correctly (CASSANDRA-7239)
+ * (cqlsh) Fix case insensitivity (CASSANDRA-7834)
+ * Fix failure to stream ranges when moving (CASSANDRA-7836)
+ * Correctly remove tmplink files (CASSANDRA-7803)
+ * (cqlsh) Fix column name formatting for functions, CAS operations,
+ and UDT field selections (CASSANDRA-7806)
+ * (cqlsh) Fix COPY FROM handling of null/empty primary key
+ values (CASSANDRA-7792)
+ * Fix ordering of static cells (CASSANDRA-7763)
+Merged from 2.0:
* Forbid re-adding dropped counter columns (CASSANDRA-7831)
* Fix CFMetaData#isThriftCompatible() for PK-only tables (CASSANDRA-7832)
* Always reject inequality on the partition key without token()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/681c380b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/cql3/QueryProcessor.java
index 99972a2,ee188a3..c983d72
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@@ -19,10 -19,9 +19,12 @@@ package org.apache.cassandra.cql3
import java.nio.ByteBuffer;
import java.util.*;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentHashMap;
+ import java.util.concurrent.TimeUnit;
+ import java.util.concurrent.atomic.AtomicLong;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.primitives.Ints;
import com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap;
@@@ -36,13 -34,11 +38,15 @@@ import org.slf4j.LoggerFactory
import org.apache.cassandra.cql3.statements.*;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.composites.*;
+import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.exceptions.*;
+ import org.apache.cassandra.metrics.CqlStatementMetrics;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.service.pager.QueryPager;
+import org.apache.cassandra.service.pager.QueryPagers;
+ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.thrift.ThriftClientState;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.transport.messages.ResultMessage;
@@@ -78,72 -75,67 +82,87 @@@ public class QueryProcessor implements
}
};
- private static final ConcurrentLinkedHashMap<MD5Digest, CQLStatement>
preparedStatements;
+ private static final ConcurrentLinkedHashMap<MD5Digest,
ParsedStatement.Prepared> preparedStatements;
private static final ConcurrentLinkedHashMap<Integer, CQLStatement>
thriftPreparedStatements;
+ // A map for prepared statements used internally (which we don't want to
mix with user statement, in particular we don't
+ // bother with expiration on those.
+ private static final ConcurrentMap<String, ParsedStatement.Prepared>
internalStatements = new ConcurrentHashMap<>();
+
+ @VisibleForTesting
public static final CqlStatementMetrics metrics = new
CqlStatementMetrics();
+ private static AtomicLong evictionCount = new AtomicLong(0);
static
{
- if (MemoryMeter.isInitialized())
- {
- preparedStatements = new
ConcurrentLinkedHashMap.Builder<MD5Digest, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
- .weigher(cqlMemoryUsageWeigher)
- .listener(new EvictionListener<MD5Digest,
CQLStatement>()
- {
- @Override
- public void onEviction(MD5Digest
md5Digest, CQLStatement prepared)
- {
-
metrics.activePreparedStatements.dec();
-
metrics.evictedPreparedStatements.inc();
- evictionCount.incrementAndGet();
- }
- }).build();
- thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
- .weigher(thriftMemoryUsageWeigher)
- .listener(new
EvictionListener<Integer, CQLStatement>()
- {
- @Override
- public void onEviction(Integer i,
CQLStatement prepared)
- {
-
metrics.activePreparedStatements.dec();
-
metrics.evictedPreparedStatements.inc();
-
evictionCount.incrementAndGet();
- }
- }).build();
- }
- else
- {
- logger.error("Unable to initialize MemoryMeter (jamm not
specified as javaagent). This means "
- + "Cassandra will be unable to measure object sizes
accurately and may consequently OOM.");
- preparedStatements = new
ConcurrentLinkedHashMap.Builder<MD5Digest, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_COUNT)
- .build();
- thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_COUNT)
- .build();
- }
+ preparedStatements = new ConcurrentLinkedHashMap.Builder<MD5Digest,
ParsedStatement.Prepared>()
+
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
+ .weigher(cqlMemoryUsageWeigher)
+ .listener(new EvictionListener<MD5Digest,
ParsedStatement.Prepared>()
+ {
+ @Override
+ public void onEviction(MD5Digest md5Digest,
ParsedStatement.Prepared prepared)
+ {
+ metrics.activePreparedStatements.dec();
++ metrics.evictedPreparedStatements.inc();
++ evictionCount.incrementAndGet();
+ }
+ }).build();
+
+ thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
+
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
+ .weigher(thriftMemoryUsageWeigher)
+ .listener(new EvictionListener<Integer,
CQLStatement>()
+ {
+ @Override
+ public void onEviction(Integer
integer, CQLStatement cqlStatement)
+ {
+
metrics.activePreparedStatements.dec();
++
metrics.evictedPreparedStatements.inc();
++ evictionCount.incrementAndGet();
+ }
+ })
+ .build();
+ StorageService.scheduledTasks.scheduleAtFixedRate(new Runnable() {
+ @Override
+ public void run() {
+ long count = evictionCount.getAndSet(0);
+ if (count > 0)
+ {
+ logger.info("{} prepared statements discarded in the last
minute because cache limit reached (cache limit = {} bytes)", count,
MAX_CACHE_PREPARED_MEMORY);
+ }
+ }
+ }, 1, 1, TimeUnit.MINUTES);
}
+ // Work aound initialization dependency
+ private static enum InternalStateInstance
+ {
+ INSTANCE;
+
+ private final QueryState queryState;
+
+ InternalStateInstance()
+ {
+ ClientState state = ClientState.forInternalCalls();
+ try
+ {
+ state.setKeyspace(Keyspace.SYSTEM_KS);
+ }
+ catch (InvalidRequestException e)
+ {
+ throw new RuntimeException();
+ }
+ this.queryState = new QueryState(state);
+ }
+ }
+
+ private static QueryState internalQueryState()
+ {
+ return InternalStateInstance.INSTANCE.queryState;
+ }
+
private QueryProcessor()
{
}
@@@ -394,28 -306,26 +413,24 @@@
throw new InvalidRequestException(String.format("Prepared
statement of size %d bytes is larger than allowed maximum of %d bytes.",
statementSize,
MAX_CACHE_PREPARED_MEMORY));
- try
-
+ if (forThrift)
{
- if (forThrift)
- {
- int statementId = toHash.hashCode();
- thriftPreparedStatements.put(statementId, prepared.statement);
- logger.trace("Stored prepared statement #{} with {} bind
markers",
- statementId,
- prepared.statement.getBoundTerms());
- return ResultMessage.Prepared.forThrift(statementId,
prepared.boundNames);
- } else
- {
- MD5Digest statementId = MD5Digest.compute(toHash);
- preparedStatements.put(statementId, prepared);
- logger.trace("Stored prepared statement #{} with {} bind
markers",
- statementId,
- prepared.statement.getBoundTerms());
- return new ResultMessage.Prepared(statementId, prepared);
- }
- } finally
+ int statementId = toHash.hashCode();
+ thriftPreparedStatements.put(statementId, prepared.statement);
+ metrics.activePreparedStatements.inc();
- logger.trace(String.format("Stored prepared statement #%d with %d
bind markers",
- statementId,
- prepared.statement.getBoundTerms()));
++ logger.trace("Stored prepared statement #{} with {} bind markers",
++ statementId,
++ prepared.statement.getBoundTerms());
+ return ResultMessage.Prepared.forThrift(statementId,
prepared.boundNames);
- }
- else
++ } else
{
+ MD5Digest statementId = MD5Digest.compute(toHash);
- preparedStatements.put(statementId, prepared.statement);
++ preparedStatements.put(statementId, prepared);
metrics.activePreparedStatements.inc();
- logger.trace(String.format("Stored prepared statement %s with %d
bind markers",
- statementId,
- prepared.statement.getBoundTerms()));
++ logger.trace("Stored prepared statement #{} with {} bind markers",
++ statementId,
++ prepared.statement.getBoundTerms());
+ return new ResultMessage.Prepared(statementId, prepared);
}
}
