Merge branch 'cassandra-3.11' into trunk


Branch: refs/heads/trunk
Commit: 4b4c05e0de3f49a29aae80b6b8f9537626d996b5
Parents: a7a36e7 a060698
Author: Paulo Motta <>
Authored: Tue Mar 6 11:21:38 2018 -0300
Committer: Paulo Motta <>
Committed: Tue Mar 6 11:24:14 2018 -0300

 CHANGES.txt                                             | 1 +
 NEWS.txt                                                | 8 ++++++++
 src/java/org/apache/cassandra/service/ | 7 +++++--
 3 files changed, 14 insertions(+), 2 deletions(-)
diff --cc CHANGES.txt
index 5fefc02,ff726a9..2b2baea
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,208 -1,8 +1,209 @@@
 + * Use Murmur3 for validation compactions (CASSANDRA-14002)
 + * Comma at the end of the seed list is interpretated as localhost 
 + * Refactor read executor and response resolver, abstract read repair 
 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993)
 + * Add a few options to nodetool verify (CASSANDRA-14201)
 + * CVE-2017-5929 Security vulnerability and redefine default log rotation 
policy (CASSANDRA-14183)
 + * Use JVM default SSL validation algorithm instead of custom default 
 + * Better document in code InetAddressAndPort usage post 7544, incorporate 
port into UUIDGen node (CASSANDRA-14226)
 + * Fix sstablemetadata date string for minLocalDeletionTime (CASSANDRA-14132)
 + * Make it possible to change neverPurgeTombstones during runtime 
 + * Remove GossipDigestSynVerbHandler#doSort() (CASSANDRA-14174)
 + * Add nodetool clientlist (CASSANDRA-13665)
 + * Revert ProtocolVersion changes from CASSANDRA-7544 (CASSANDRA-14211)
 + * Non-disruptive seed node list reload (CASSANDRA-14190)
 + * Nodetool tablehistograms to print statics for all the tables 
 + * Migrate dtests to use pytest and python3 (CASSANDRA-14134)
 + * Allow storage port to be configurable per node (CASSANDRA-7544)
 + * Make sub-range selection for non-frozen collections return null instead of 
empty (CASSANDRA-14182)
 + * BloomFilter serialization format should not change byte ordering 
 + * Remove unused on-heap BloomFilter implementation (CASSANDRA-14152)
 + * Delete temp test files on exit (CASSANDRA-14153)
 + * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
 + * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
 + * Fix cassandra-stress startup failure (CASSANDRA-14106)
 + * Remove initialDirectories from CFS (CASSANDRA-13928)
 + * Fix trivial log format error (CASSANDRA-14015)
 + * Allow sstabledump to do a json object per partition (CASSANDRA-13848)
 + * Add option to optimise merkle tree comparison across replicas 
 + * Remove unused and deprecated methods from AbstractCompactionStrategy 
 + * Fix Distribution.average in cassandra-stress (CASSANDRA-14090)
 + * Support a means of logging all queries as they were invoked 
 + * Presize collections (CASSANDRA-13760)
 + * Add GroupCommitLogService (CASSANDRA-13530)
 + * Parallelize initial materialized view build (CASSANDRA-12245)
 + * Fix flaky SecondaryIndexManagerTest.assert[Not]MarkedAsBuilt 
 + * Make LWTs send resultset metadata on every request (CASSANDRA-13992)
 + * Fix flaky indexWithFailedInitializationIsNotQueryableAfterPartialRebuild 
 + * Introduce leaf-only iterator (CASSANDRA-9988)
 + * Upgrade Guava to 23.3 and Airline to 0.8 (CASSANDRA-13997)
 + * Allow only one concurrent call to StatusLogger (CASSANDRA-12182)
 + * Refactoring to specialised functional interfaces (CASSANDRA-13982)
 + * Speculative retry should allow more friendly params (CASSANDRA-13876)
 + * Throw exception if we send/receive repair messages to incompatible nodes 
 + * Replace usages of MessageDigest with Guava's Hasher (CASSANDRA-13291)
 + * Add nodetool cmd to print hinted handoff window (CASSANDRA-13728)
 + * Fix some alerts raised by static analysis (CASSANDRA-13799)
 + * Checksum sstable metadata (CASSANDRA-13321, CASSANDRA-13593)
 + * Add result set metadata to prepared statement MD5 hash calculation 
 + * Refactor GcCompactionTest to avoid boxing (CASSANDRA-13941)
 + * Expose recent histograms in JmxHistograms (CASSANDRA-13642)
 + * Fix buffer length comparison when decompressing in netty-based streaming 
 + * Properly close StreamCompressionInputStream to release any ByteBuf 
 + * Add SERIAL and LOCAL_SERIAL support for cassandra-stress (CASSANDRA-13925)
 + * LCS needlessly checks for L0 STCS candidates multiple times 
 + * Correctly close netty channels when a stream session ends (CASSANDRA-13905)
 + * Update lz4 to 1.4.0 (CASSANDRA-13741)
 + * Optimize Paxos prepare and propose stage for local requests 
 + * Throttle base partitions during MV repair streaming to prevent OOM 
 + * Use compaction threshold for STCS in L0 (CASSANDRA-13861)
 + * Fix problem with min_compress_ratio: 1 and disallow ratio < 1 
 + * Add extra information to SASI timeout exception (CASSANDRA-13677)
 + * Add incremental repair support for --hosts, --force, and subrange repair 
 + * Rework CompactionStrategyManager.getScanners synchronization 
 + * Add additional unit tests for batch behavior, TTLs, Timestamps 
 + * Add keyspace and table name in schema validation exception 
 + * Emit metrics whenever we hit tombstone failures and warn thresholds 
 + * Make netty EventLoopGroups daemon threads (CASSANDRA-13837)
 + * Race condition when closing stream sessions (CASSANDRA-13852)
 + * NettyFactoryTest is failing in trunk on macOS (CASSANDRA-13831)
 + * Allow changing log levels via nodetool for related classes 
 + * Add stress profile yaml with LWT (CASSANDRA-7960)
 + * Reduce memory copies and object creations when acting on ByteBufs 
 + * Simplify mx4j configuration (Cassandra-13578)
 + * Fix trigger example on 4.0 (CASSANDRA-13796)
 + * Force minumum timeout value (CASSANDRA-9375)
 + * Use netty for streaming (CASSANDRA-12229)
 + * Use netty for internode messaging (CASSANDRA-8457)
 + * Add bytes repaired/unrepaired to nodetool tablestats (CASSANDRA-13774)
 + * Don't delete incremental repair sessions if they still have sstables 
 + * Fix pending repair manager index out of bounds check (CASSANDRA-13769)
 + * Don't use RangeFetchMapCalculator when RF=1 (CASSANDRA-13576)
 + * Don't optimise trivial ranges in RangeFetchMapCalculator (CASSANDRA-13664)
 + * Use an ExecutorService for repair commands instead of new 
Thread(..).start() (CASSANDRA-13594)
 + * Fix race / ref leak in anticompaction (CASSANDRA-13688)
 + * Expose tasks queue length via JMX (CASSANDRA-12758)
 + * Fix race / ref leak in PendingRepairManager (CASSANDRA-13751)
 + * Enable ppc64le runtime as unsupported architecture (CASSANDRA-13615)
 + * Improve sstablemetadata output (CASSANDRA-11483)
 + * Support for migrating legacy users to roles has been dropped 
 + * Introduce error metrics for repair (CASSANDRA-13387)
 + * Refactoring to primitive functional interfaces in AuthCache 
 + * Update metrics to 3.1.5 (CASSANDRA-13648)
 + * batch_size_warn_threshold_in_kb can now be set at runtime (CASSANDRA-13699)
 + * Avoid always rebuilding secondary indexes at startup (CASSANDRA-13725)
 + * Upgrade JMH from 1.13 to 1.19 (CASSANDRA-13727)
 + * Upgrade SLF4J from 1.7.7 to 1.7.25 (CASSANDRA-12996)
 + * Default for start_native_transport now true if not set in config 
 + * Don't add localhost to the graph when calculating where to stream from 
 + * Make CDC availability more deterministic via hard-linking (CASSANDRA-12148)
 + * Allow skipping equality-restricted clustering columns in ORDER BY clause 
 + * Use common nowInSec for validation compactions (CASSANDRA-13671)
 + * Improve handling of IR prepare failures (CASSANDRA-13672)
 + * Send IR coordinator messages synchronously (CASSANDRA-13673)
 + * Flush table before IR finalize promise (CASSANDRA-13660)
 + * Fix column filter creation for wildcard queries (CASSANDRA-13650)
 + * Add 'nodetool getbatchlogreplaythrottle' and 'nodetool 
setbatchlogreplaythrottle' (CASSANDRA-13614)
 + * fix race condition in PendingRepairManager (CASSANDRA-13659)
 + * Allow noop incremental repair state transitions (CASSANDRA-13658)
 + * Run repair with down replicas (CASSANDRA-10446)
 + * Added started & completed repair metrics (CASSANDRA-13598)
 + * Added started & completed repair metrics (CASSANDRA-13598)
 + * Improve secondary index (re)build failure and concurrency handling 
 + * Improve calculation of available disk space for compaction 
 + * Change the accessibility of RowCacheSerializer for third party row cache 
plugins (CASSANDRA-13579)
 + * Allow sub-range repairs for a preview of repaired data (CASSANDRA-13570)
 + * NPE in IR cleanup when columnfamily has no sstables (CASSANDRA-13585)
 + * Fix Randomness of stress values (CASSANDRA-12744)
 + * Allow selecting Map values and Set elements (CASSANDRA-7396)
 + * Fast and garbage-free Streaming Histogram (CASSANDRA-13444)
 + * Update repairTime for keyspaces on completion (CASSANDRA-13539)
 + * Add configurable upper bound for validation executor threads 
 + * Bring back maxHintTTL propery (CASSANDRA-12982)
 + * Add testing guidelines (CASSANDRA-13497)
 + * Add more repair metrics (CASSANDRA-13531)
 + * RangeStreamer should be smarter when picking endpoints for streaming 
 + * Avoid rewrapping an exception thrown for cache load functions 
 + * Log time elapsed for each incremental repair phase (CASSANDRA-13498)
 + * Add multiple table operation support to cassandra-stress (CASSANDRA-8780)
 + * Fix incorrect cqlsh results when selecting same columns multiple times 
 + * Fix WriteResponseHandlerTest is sensitive to test execution order 
 + * Improve incremental repair logging (CASSANDRA-13468)
 + * Start compaction when incremental repair finishes (CASSANDRA-13454)
 + * Add repair streaming preview (CASSANDRA-13257)
 + * Cleanup isIncremental/repairedAt usage (CASSANDRA-13430)
 + * Change protocol to allow sending key space independent of query string 
 + * Make gc_log and gc_warn settable at runtime (CASSANDRA-12661)
 + * Take number of files in L0 in account when estimating remaining compaction 
tasks (CASSANDRA-13354)
 + * Skip building views during base table streams on range movements 
 + * Improve error messages for +/- operations on maps and tuples 
 + * Remove deprecated repair JMX APIs (CASSANDRA-11530)
 + * Fix version check to enable streaming keep-alive (CASSANDRA-12929)
 + * Make it possible to monitor an ideal consistency level separate from 
actual consistency level (CASSANDRA-13289)
 + * Outbound TCP connections ignore internode authenticator (CASSANDRA-13324)
 + * Upgrade junit from 4.6 to 4.12 (CASSANDRA-13360)
 + * Cleanup ParentRepairSession after repairs (CASSANDRA-13359)
 + * Upgrade snappy-java to (CASSANDRA-13336)
 + * Incremental repair not streaming correct sstables (CASSANDRA-13328)
 + * Upgrade the jna version to 4.3.0 (CASSANDRA-13300)
 + * Add the currentTimestamp, currentDate, currentTime and currentTimeUUID 
functions (CASSANDRA-13132)
 + * Remove config option index_interval (CASSANDRA-10671)
 + * Reduce lock contention for collection types and serializers 
 + * Make it possible to override MessagingService.Verb ids (CASSANDRA-13283)
 + * Avoid synchronized on prepareForRepair in ActiveRepairService 
 + * Adds the ability to use uncompressed chunks in compressed files 
 + * Don't flush sstables when streaming for incremental repair 
 + * Remove unused method (CASSANDRA-13227)
 + * Fix minor bugs related to #9143 (CASSANDRA-13217)
 + * Output warning if user increases RF (CASSANDRA-13079)
 + * Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
 + * Add support for + and - operations on dates (CASSANDRA-11936)
 + * Fix consistency of incrementally repaired data (CASSANDRA-9143)
 + * Increase commitlog version (CASSANDRA-13161)
 + * Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
 + * Refactor ColumnCondition (CASSANDRA-12981)
 + * Parallelize streaming of different keyspaces (CASSANDRA-4663)
 + * Improved compactions metrics (CASSANDRA-13015)
 + * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
 + * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
 + * Thrift removal (CASSANDRA-11115)
 + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool Documentation (CASSANDRA-12672)
 + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
 + * Reject invalid replication settings when creating or altering a keyspace 
 + * Clean up the SSTableReader#getScanner API wrt removal of RateLimiter 
 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
 + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
 + * Require forceful decommission if number of nodes is less than replication 
factor (CASSANDRA-12510)
 + * Allow IN restrictions on column families with collections (CASSANDRA-12654)
 + * Log message size in trace message in OutboundTcpConnection 
 + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
 + * Add mutation size and batch metrics (CASSANDRA-12649)
 + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
 + * Expose time spent waiting in thread pool queue (CASSANDRA-8398)
 + * Conditionally update index built status to avoid unnecessary flushes 
 + * cqlsh auto completion: refactor definition of compaction strategy options 
 + * Add support for arithmetic operators (CASSANDRA-11935)
 + * Add histogram for delay to deliver hints (CASSANDRA-13234)
 + * Fix cqlsh automatic protocol downgrade regression (CASSANDRA-13307)
 + * Changing `max_hint_window_in_ms` at runtime (CASSANDRA-11720)
 + * Trivial format error in StorageProxy (CASSANDRA-13551)
 + * Nodetool repair can hang forever if we lose the notification for the 
repair completing/failing (CASSANDRA-13480)
 + * Anticompaction can cause noisy log messages (CASSANDRA-13684)
 + * Switch to client init for sstabledump (CASSANDRA-13683)
 + * CQLSH: Don't pause when capturing data (CASSANDRA-13743)
 + * nodetool clearsnapshot requires --all to clear all snapshots 
 + * Correctly count range tombstones in traces and tombstone thresholds 
 + * cqlshrc.sample uses incorrect option for time formatting (CASSANDRA-14243)
 - * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter  (CASSANDRA-14163)
   * Fix wildcard GROUP BY queries (CASSANDRA-14209)
  Merged from 3.0:
+  * Write to pending endpoint when view replica is also base replica 
   * Chain commit log marker potential performance regression in batch commit 
mode (CASSANDRA-14194)
   * Fully utilise specified compaction threads (CASSANDRA-14210)
   * Pre-create deletion log records to finish compactions quicker 
diff --cc NEWS.txt
index 4c73b89,745dad2..5826cf2
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -33,121 -42,14 +33,129 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
 +New features
 +   - An experimental option to compare all merkle trees together has been 
added - for example, in
 +     a 3 node cluster with 2 replicas identical and 1 out-of-date, with this 
option enabled, the
 +     out-of-date replica will only stream a single copy from up-to-date 
replica. Enable it by adding
 +     "-os" to nodetool repair. See CASSANDRA-3200.
 +   - The currentTimestamp, currentDate, currentTime and currentTimeUUID 
functions have been added.
 +     See CASSANDRA-13132
 +   - Support for arithmetic operations between `timestamp`/`date` and 
`duration` has been added.
 +     See CASSANDRA-11936
 +   - Support for arithmetic operations on number has been added. See 
 +   - Preview expected streaming required for a repair (nodetool repair 
--preview), and validate the
 +     consistency of repaired data between nodes (nodetool repair --validate). 
 +   - Support for selecting Map values and Set elements has been added for 
SELECT queries. See CASSANDRA-7396
 +   - Change-Data-Capture has been modified to make CommitLogSegments available
 +     immediately upon creation via hard-linking the files. This means that 
 +     segments will be available in cdc_raw rather than fully flushed. See 
 +     and CASSANDRA-12148 for more detail.
 +   - The initial build of materialized views can be parallelized. The number 
of concurrent builder
 +     threads is specified by the property 
 +     This property can be modified at runtime through both JMX and the new 
 +     and `getconcurrentviewbuilders` nodetool commands. See CASSANDRA-12245 
for more details.
 +   - There is now a binary full query log based on Chronicle Queue that can 
be controlled using
 +     nodetool enablefullquerylog, disablefullquerylog, and resetfullquerylog. 
The log
 +     contains all queries invoked, approximate time they were invoked, any 
parameters necessary
 +     to bind wildcard values, and all query options. A human readable version 
of the log can be
 +     dumped or tailed using the new bin/fqltool utility. The full query log 
is designed to be safe
 +     to use in production and limits utilization of heap memory and disk 
space with limits
 +     you can specify when enabling the log.
 +     See nodetool and fqltool help text for more information.
 +   - SSTableDump now supports the -l option to output each partition as it's 
own json object
 +     See CASSANDRA-13848 for more detail
 +    - Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact 
 +      have to be migrated using `ALTER ... DROP COMPACT STORAGE` statement in 
 +      Cassandra starting 4.0 will not start if flags indicate that the table 
is non-CQL.
 +      Syntax for creating compact tables is also deprecated.
 +    - Support for legacy auth tables in the system_auth keyspace (users,
 +      permissions, credentials) and the migration code has been removed. 
 +      of these legacy auth tables must have been completed before the upgrade 
 +      4.0 and the legacy tables must have been removed. See the 'Upgrading' 
 +      for version 2.2 for migration instructions.
 +    - Cassandra 4.0 removed support for the deprecated Thrift interface. 
 +      other things, this implies the removal of all yaml options related to 
 +      ('start_rpc', rpc_port, ...).
 +    - Cassandra 4.0 removed support for any pre-3.0 format. This means you
 +      cannot upgrade from a 2.x version to 4.0 directly, you have to upgrade 
 +      a 3.0.x/3.x version first (and run upgradesstable). In particular, this
 +      mean Cassandra 4.0 cannot load or read pre-3.0 sstables in any way: you
 +      will need to upgrade those sstable in 3.0.x/3.x first.
 +    - Upgrades from 3.0.x or 3.x are supported since 3.0.13 or 3.11.0, 
 +      versions will causes issues during rolling upgrades (CASSANDRA-13274).
 +    - Cassandra will no longer allow invalid keyspace replication options, 
 +      as invalid datacenter names for NetworkTopologyStrategy. Operators MUST
 +      add new nodes to a datacenter before they can set set ALTER or CREATE
 +      keyspace replication policies using that datacenter. Existing keyspaces
 +      will continue to operate, but CREATE and ALTER will validate that all
 +      datacenters specified exist in the cluster.
 +    - Cassandra 4.0 fixes a problem with incremental repair which caused 
 +      data to be inconsistent between nodes. The fix changes the behavior of 
 +      full and incremental repairs. For full repairs, data is no longer marked
 +      repaired. For incremental repairs, anticompaction is run at the 
 +      of the repair, instead of at the end. If incremental repair was being 
 +      prior to upgrading, a full repair should be run after upgrading to 
 +      any inconsistencies.
 +    - Config option index_interval has been removed (it was deprecated since 
 +    - Deprecated repair JMX APIs are removed.
 +    - The version of snappy-java has been upgraded to
 +      - the miniumum value for internode message timeouts is 10ms. 
Previously, any
 +        positive value was allowed. See cassandra.yaml entries like
 +        read_request_timeout_in_ms for more details.
 +      - Cassandra 4.0 allows a single port to be used for both secure and 
 +        connections between cassandra nodes (CASSANDRA-10404). See the yaml 
 +        specific property changes, and see the security doc for full details.
 +    - Due to the parallelization of the initial build of materialized views,
 +      the per token range view building status is stored in the new table
 +      `system.view_builds_in_progress`. The old table 
 +      is no longer used and can be removed. See CASSANDRA-12245 for more 
 +      - Config option commitlog_sync_batch_window_in_ms has been deprecated 
as it's
 +        documentation has been incorrect and the setting itself near useless.
 +        Batch mode remains a valid commit log mode, however.
 +      - There is a new commit log mode, group, which is similar to batch mode
 +        but blocks for up to a configurable number of milliseconds between 
disk flushes.
 +      - nodetool clearsnapshot now required the --all flag to remove all 
 +        Previous behavior would delete all snapshots by default.
 +    - Nodes are now identified by a combination of IP, and storage port.
 +      Existing JMX APIs, nodetool, and system tables continue to work
 +      and accept/return just an IP, but there is a new
 +      version of each that works with the full unambiguous identifier.
 +      You should prefer these over the deprecated ambiguous versions that only
 +      work with an IP. This was done to support multiple instances per IP.
 +      Additionally we are moving to only using a single port for encrypted and
 +      unencrypted traffic and if you want multiple instances per IP you must
 +      first switch encrypted traffic to the storage port and not a separate
 +      encrypted port. If you want to use multiple instances per IP
 +      with SSL you will need to use StartTLS on storage_port and set
 +      outgoing_encrypted_port_source to gossip outbound connections
 +      know what port to connect to for each instance. Before changing
 +      storage port or native port at nodes you must first upgrade the entire 
 +      and clients to 4.0 so they can handle the port not being consistent 
 +      the cluster.
 +Materialized Views
 +   - Following a discussion regarding concerns about the design and safety of 
Materialized Views, the C* development
 +     community no longer recommends them for production use, and considers 
them experimental. Warnings messages will
 +     now be logged when they are created. (See
 +   - An 'enable_materialized_views' flag has been added to cassandra.yaml to 
allow operators to prevent creation of
 +     views
+ 3.11.3
+ =====
+ Upgrading
+ ---------
+     - Materialized view users upgrading from 3.0.15 (3.0.X series) or 3.11.1 
(3.11.X series) and  later that have performed range movements (join, 
decommission, move, etc),
+       should run repair on the base tables, and subsequently on the views to 
ensure data affected by CASSANDRA-14251 is correctly propagated to all replicas.
diff --cc src/java/org/apache/cassandra/service/
index 7d4e34c,4dc05e3..b512722
--- a/src/java/org/apache/cassandra/service/
+++ b/src/java/org/apache/cassandra/service/
@@@ -866,8 -795,11 +866,11 @@@ public class StorageProxy implements St
-                     // When local node is the paired endpoint just apply the 
mutation locally.
-                     if 
(pairedEndpoint.get().equals(FBUtilities.getBroadcastAddressAndPort()) && 
+                     // When local node is the endpoint we can just apply the 
mutation locally,
+                     // unless there are pending endpoints, in which case we 
want to do an ordinary
+                     // write so the view mutation is sent to the pending 
 -                    if 
(pairedEndpoint.get().equals(FBUtilities.getBroadcastAddress()) && 
++                    if 
(pairedEndpoint.get().equals(FBUtilities.getBroadcastAddressAndPort()) && 
+                         && pendingEndpoints.isEmpty())

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to