Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/cql3/QueryProcessor.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b319b406
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b319b406
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b319b406
Branch: refs/heads/cassandra-2.1
Commit: b319b406b9ff69fb64e696627f65b443431d1d64
Parents: 681c380 f19b4cb
Author: Tyler Hobbs <[email protected]>
Authored: Tue Sep 16 14:09:45 2014 -0500
Committer: Tyler Hobbs <[email protected]>
Committed: Tue Sep 16 14:09:45 2014 -0500
----------------------------------------------------------------------
CHANGES.txt | 1 -
.../apache/cassandra/cql3/QueryProcessor.java | 53 +++++++-------------
.../cassandra/metrics/CqlStatementMetrics.java | 1 -
3 files changed, 19 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b319b406/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 0abc66a,3ee938a..f89cc6d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,93 -1,9 +1,92 @@@
-2.0.11:
+2.1.1
+ * Fix saving caches when a table is dropped (CASSANDRA-7784)
+ * Add better error checking of new stress profile (CASSANDRA-7716)
+ * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom
(CASSANDRA-7934)
+ * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
+ * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
+ * GCInspector more closely tracks GC; cassandra-stress and nodetool report
it (CASSANDRA-7916)
+ * nodetool won't output bogus ownership info without a keyspace
(CASSANDRA-7173)
+ * Add human readable option to nodetool commands (CASSANDRA-5433)
+ * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
+ * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
+ * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
+ * (cqlsh) Support for query paging (CASSANDRA-7514)
+ * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
+ * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
+ * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
+ * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
+ * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
+ * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
+ * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
+ output (CASSANDRA-7659)
+ * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
+ * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
+ * Support list index operations with conditions (CASSANDRA-7499)
+ * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
+ * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
+ * (cqlsh) Error when tracing query (CASSANDRA-7613)
+ * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
+ * SSTableExport uses correct validator to create string representation of
partition
+ keys (CASSANDRA-7498)
+ * Avoid NPEs when receiving type changes for an unknown keyspace
(CASSANDRA-7689)
+ * Add support for custom 2i validation (CASSANDRA-7575)
+ * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
+ * Add listen_interface and rpc_interface options (CASSANDRA-7417)
+ * Improve schema merge performance (CASSANDRA-7444)
+ * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
+ * Optimise NativeCell comparisons (CASSANDRA-6755)
+ * Configurable client timeout for cqlsh (CASSANDRA-7516)
+ * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
+Merged from 2.0:
- * Add metrics for prepared statement usage and eviction (CASSANDRA-7930)
* Make CQLSSTableWriter sync within partitions (CASSANDRA-7360)
* Potentially use non-local replicas in CqlConfigHelper (CASSANDRA-7906)
- * Explicitly disallowing mixing multi-column and single-column
+ * Explicitly disallow mixing multi-column and single-column
relations on clustering columns (CASSANDRA-7711)
* Better error message when condition is set on PK column (CASSANDRA-7804)
+ * Don't send schema change responses and events for no-op DDL
+ statements (CASSANDRA-7600)
+ * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774)
+ * Throw InvalidRequestException when queries contain relations on entire
+ collection columns (CASSANDRA-7506)
+ * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577)
+ * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252)
+ * (cqlsh) cqlsh should automatically disable tracing when selecting
+ from system_traces (CASSANDRA-7641)
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
+ * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
+ * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
+ * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229)
+ * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635)
+ * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS
(CASSANDRA-7611)
+ * Catch errors when the JVM pulls the rug out from GCInspector
(CASSANDRA-5345)
+ * cqlsh fails when version number parts are not int (CASSANDRA-7524)
+Merged from 1.2:
+ * Don't index tombstones (CASSANDRA-7828)
+ * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788)
+
+
+2.1.0
+ * (cqlsh) Removed "ALTER TYPE <name> RENAME TO <name>" from tab-completion
+ (CASSANDRA-7895)
+ * Fixed IllegalStateException in anticompaction (CASSANDRA-7892)
+ * cqlsh: DESCRIBE support for frozen UDTs, tuples (CASSANDRA-7863)
+ * Avoid exposing internal classes over JMX (CASSANDRA-7879)
+ * Add null check for keys when freezing collection (CASSANDRA-7869)
+ * Improve stress workload realism (CASSANDRA-7519)
+
+
+2.1.0-rc7
+ * Add frozen keyword and require UDT to be frozen (CASSANDRA-7857)
+ * Track added sstable size correctly (CASSANDRA-7239)
+ * (cqlsh) Fix case insensitivity (CASSANDRA-7834)
+ * Fix failure to stream ranges when moving (CASSANDRA-7836)
+ * Correctly remove tmplink files (CASSANDRA-7803)
+ * (cqlsh) Fix column name formatting for functions, CAS operations,
+ and UDT field selections (CASSANDRA-7806)
+ * (cqlsh) Fix COPY FROM handling of null/empty primary key
+ values (CASSANDRA-7792)
+ * Fix ordering of static cells (CASSANDRA-7763)
+Merged from 2.0:
* Forbid re-adding dropped counter columns (CASSANDRA-7831)
* Fix CFMetaData#isThriftCompatible() for PK-only tables (CASSANDRA-7832)
* Always reject inequality on the partition key without token()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b319b406/src/java/org/apache/cassandra/cql3/QueryProcessor.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/cql3/QueryProcessor.java
index c983d72,a59fe9b..a5be108
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@@ -19,34 -19,21 +19,30 @@@ package org.apache.cassandra.cql3
import java.nio.ByteBuffer;
import java.util.*;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentHashMap;
- import java.util.concurrent.TimeUnit;
- import java.util.concurrent.atomic.AtomicLong;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.primitives.Ints;
import com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap;
import com.googlecode.concurrentlinkedhashmap.EntryWeigher;
+import com.googlecode.concurrentlinkedhashmap.EvictionListener;
import org.antlr.runtime.*;
- import org.apache.cassandra.metrics.CqlStatementMetrics;
import org.github.jamm.MemoryMeter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.cql3.statements.*;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.composites.*;
+import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.exceptions.*;
+import org.apache.cassandra.metrics.CqlStatementMetrics;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.service.pager.QueryPager;
+import org.apache.cassandra.service.pager.QueryPagers;
- import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.thrift.ThriftClientState;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.transport.messages.ResultMessage;
@@@ -82,79 -70,32 +78,64 @@@ public class QueryProcessor implements
}
};
- private static final ConcurrentLinkedHashMap<MD5Digest, CQLStatement>
preparedStatements;
+ private static final ConcurrentLinkedHashMap<MD5Digest,
ParsedStatement.Prepared> preparedStatements;
private static final ConcurrentLinkedHashMap<Integer, CQLStatement>
thriftPreparedStatements;
+ // A map for prepared statements used internally (which we don't want to
mix with user statement, in particular we don't
+ // bother with expiration on those.
+ private static final ConcurrentMap<String, ParsedStatement.Prepared>
internalStatements = new ConcurrentHashMap<>();
+
+ @VisibleForTesting
+ public static final CqlStatementMetrics metrics = new
CqlStatementMetrics();
- private static AtomicLong evictionCount = new AtomicLong(0);
+
static
{
- if (MemoryMeter.isInitialized())
- {
- preparedStatements = new
ConcurrentLinkedHashMap.Builder<MD5Digest, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
- .weigher(cqlMemoryUsageWeigher)
- .build();
- thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
- .weigher(thriftMemoryUsageWeigher)
- .build();
- }
- else
+ preparedStatements = new ConcurrentLinkedHashMap.Builder<MD5Digest,
ParsedStatement.Prepared>()
+
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
+ .weigher(cqlMemoryUsageWeigher)
+ .listener(new EvictionListener<MD5Digest,
ParsedStatement.Prepared>()
+ {
+ @Override
+ public void onEviction(MD5Digest md5Digest,
ParsedStatement.Prepared prepared)
+ {
+ metrics.activePreparedStatements.dec();
- metrics.evictedPreparedStatements.inc();
- evictionCount.incrementAndGet();
+ }
+ }).build();
+
+ thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
+
.maximumWeightedCapacity(MAX_CACHE_PREPARED_MEMORY)
+ .weigher(thriftMemoryUsageWeigher)
+ .listener(new EvictionListener<Integer,
CQLStatement>()
+ {
+ @Override
+ public void onEviction(Integer
integer, CQLStatement cqlStatement)
+ {
+
metrics.activePreparedStatements.dec();
-
metrics.evictedPreparedStatements.inc();
- evictionCount.incrementAndGet();
+ }
+ })
+ .build();
+
- StorageService.scheduledTasks.scheduleAtFixedRate(new Runnable() {
- @Override
- public void run() {
- long count = evictionCount.getAndSet(0);
- if (count > 0)
- {
- logger.info("{} prepared statements discarded in the last
minute because cache limit reached (cache limit = {} bytes)", count,
MAX_CACHE_PREPARED_MEMORY);
- }
- }
- }, 1, 1, TimeUnit.MINUTES);
+ }
+
+ // Work aound initialization dependency
+ private static enum InternalStateInstance
+ {
+ INSTANCE;
+
+ private final QueryState queryState;
+
+ InternalStateInstance()
{
- logger.error("Unable to initialize MemoryMeter (jamm not
specified as javaagent). This means "
- + "Cassandra will be unable to measure object sizes
accurately and may consequently OOM.");
- preparedStatements = new
ConcurrentLinkedHashMap.Builder<MD5Digest, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_COUNT)
- .build();
- thriftPreparedStatements = new
ConcurrentLinkedHashMap.Builder<Integer, CQLStatement>()
-
.maximumWeightedCapacity(MAX_CACHE_PREPARED_COUNT)
- .build();
+ ClientState state = ClientState.forInternalCalls();
+ try
+ {
+ state.setKeyspace(Keyspace.SYSTEM_KS);
+ }
+ catch (InvalidRequestException e)
+ {
+ throw new RuntimeException();
+ }
+ this.queryState = new QueryState(state);
}
}
@@@ -413,24 -266,24 +394,28 @@@
throw new InvalidRequestException(String.format("Prepared
statement of size %d bytes is larger than allowed maximum of %d bytes.",
statementSize,
MAX_CACHE_PREPARED_MEMORY));
- if (forThrift)
-
- if (forThrift)
++ try
{
-- int statementId = toHash.hashCode();
-- thriftPreparedStatements.put(statementId, prepared.statement);
- metrics.activePreparedStatements.inc();
- logger.trace("Stored prepared statement #{} with {} bind markers",
- statementId,
- prepared.statement.getBoundTerms());
- return ResultMessage.Prepared.forThrift(statementId,
prepared.boundNames);
- } else
- logger.trace(String.format("Stored prepared statement #%d with %d
bind markers",
- statementId,
- prepared.statement.getBoundTerms()));
- return ResultMessage.Prepared.forThrift(statementId,
prepared.boundNames);
- }
- else
++ if (forThrift)
++ {
++ int statementId = toHash.hashCode();
++ thriftPreparedStatements.put(statementId, prepared.statement);
++ logger.trace("Stored prepared statement #{} with {} bind
markers",
++ statementId,
++ prepared.statement.getBoundTerms());
++ return ResultMessage.Prepared.forThrift(statementId,
prepared.boundNames);
++ } else
++ {
++ MD5Digest statementId = MD5Digest.compute(toHash);
++ preparedStatements.put(statementId, prepared);
++ logger.trace("Stored prepared statement #{} with {} bind
markers",
++ statementId,
++ prepared.statement.getBoundTerms());
++ return new ResultMessage.Prepared(statementId, prepared);
++ }
++ } finally
{
-- MD5Digest statementId = MD5Digest.compute(toHash);
- preparedStatements.put(statementId, prepared);
- preparedStatements.put(statementId, prepared.statement);
- logger.trace(String.format("Stored prepared statement %s with %d
bind markers",
- statementId,
- prepared.statement.getBoundTerms()));
- return new ResultMessage.Prepared(statementId, prepared);
+ metrics.activePreparedStatements.inc();
- logger.trace("Stored prepared statement #{} with {} bind markers",
- statementId,
- prepared.statement.getBoundTerms());
- return new ResultMessage.Prepared(statementId, prepared);
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b319b406/src/java/org/apache/cassandra/metrics/CqlStatementMetrics.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/metrics/CqlStatementMetrics.java
index ba27d89,0000000..02b4ad0
mode 100644,000000..100644
--- a/src/java/org/apache/cassandra/metrics/CqlStatementMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/CqlStatementMetrics.java
@@@ -1,54 -1,0 +1,53 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.metrics;
+
+import com.yammer.metrics.Metrics;
+import com.yammer.metrics.core.Counter;
+import com.yammer.metrics.core.Gauge;
+import com.yammer.metrics.util.RatioGauge;
+
+
+public class CqlStatementMetrics
+{
+ private final MetricNameFactory factory = new
DefaultNameFactory("CqlStatement");
+ public final Counter activePreparedStatements =
Metrics.newCounter(factory.createMetricName("ActivePreparedStatements"));
- public final Counter evictedPreparedStatements =
Metrics.newCounter(factory.createMetricName("EvictedPreparedStatements"));
+ public final Counter executedPrepared =
Metrics.newCounter(factory.createMetricName("ExecutedPrepared"));
+ public final Counter executedUnprepared =
Metrics.newCounter(factory.createMetricName("ExecutedUnPrepared"));
+
+ public final Gauge<Double> preparedRatio =
Metrics.newGauge(factory.createMetricName("PreparedUnpreparedRatio"), new
RatioGauge()
+ {
+ protected double getNumerator()
+ {
+ long num = executedPrepared.count();
+ return num == 0 ? 1 : num;
+ }
+
+ protected double getDenominator()
+ {
+ long den = executedUnprepared.count();
+ return den == 0 ? 1 : den;
+ }
+ });
+
+ public void reset()
+ {
+ executedPrepared.clear();
+ executedUnprepared.clear();
+ }
+}
