[jira] [Updated] (CASSANDRA-9666) Provide an alternative to DTCS

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-9666:
---
   Resolution: Fixed
Fix Version/s: (was: 3.x)
   3.8
   3.0.x
   Status: Resolved  (was: Patch Available)

committed, thanks! \o/

I'll update the 3.0 fixver to 3.0.8 once it is available in jira

> Provide an alternative to DTCS
> --
>
> Key: CASSANDRA-9666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9666
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.8
>
> Attachments: compactomatic.py, dashboard-DTCS_to_TWCS.png, 
> dtcs-twcs-io.png, dtcs-twcs-load.png
>
>
> DTCS is great for time series data, but it comes with caveats that make it 
> difficult to use in production (typical operator behaviors such as bootstrap, 
> removenode, and repair have MAJOR caveats as they relate to 
> max_sstable_age_days, and hints/read repair break the selection algorithm).
> I'm proposing an alternative, TimeWindowCompactionStrategy, that sacrifices 
> the tiered nature of DTCS in order to address some of DTCS' operational 
> shortcomings. I believe it is necessary to propose an alternative rather than 
> simply adjusting DTCS, because it fundamentally removes the tiered nature in 
> order to remove the parameter max_sstable_age_days - the result is very very 
> different, even if it is heavily inspired by DTCS. 
> Specifically, rather than creating a number of windows of ever increasing 
> sizes, this strategy allows an operator to choose the window size, compact 
> with STCS within the first window of that size, and aggressive compact down 
> to a single sstable once that window is no longer current. The window size is 
> a combination of unit (minutes, hours, days) and size (1, etc), such that an 
> operator can expect all data using a block of that size to be compacted 
> together (that is, if your unit is hours, and size is 6, you will create 
> roughly 4 sstables per day, each one containing roughly 6 hours of data). 
> The result addresses a number of the problems with 
> DateTieredCompactionStrategy:
> - At the present time, DTCS’s first window is compacted using an unusual 
> selection criteria, which prefers files with earlier timestamps, but ignores 
> sizes. In TimeWindowCompactionStrategy, the first window data will be 
> compacted with the well tested, fast, reliable STCS. All STCS options can be 
> passed to TimeWindowCompactionStrategy to configure the first window’s 
> compaction behavior.
> - HintedHandoff may put old data in new sstables, but it will have little 
> impact other than slightly reduced efficiency (sstables will cover a wider 
> range, but the old timestamps will not impact sstable selection criteria 
> during compaction)
> - ReadRepair may put old data in new sstables, but it will have little impact 
> other than slightly reduced efficiency (sstables will cover a wider range, 
> but the old timestamps will not impact sstable selection criteria during 
> compaction)
> - Small, old sstables resulting from streams of any kind will be swiftly and 
> aggressively compacted with the other sstables matching their similar 
> maxTimestamp, without causing sstables in neighboring windows to grow in size.
> - The configuration options are explicit and straightforward - the tuning 
> parameters leave little room for error. The window is set in common, easily 
> understandable terms such as “12 hours”, “1 Day”, “30 days”. The 
> minute/hour/day options are granular enough for users keeping data for hours, 
> and users keeping data for years. 
> - There is no explicitly configurable max sstable age, though sstables will 
> naturally stop compacting once new data is written in that window. 
> - Streaming operations can create sstables with old timestamps, and they'll 
> naturally be joined together with sstables in the same time bucket. This is 
> true for bootstrap/repair/sstableloader/removenode. 
> - It remains true that if old data and new data is written into the memtable 
> at the same time, the resulting sstables will be treated as if they were new 
> sstables, however, that no longer negatively impacts the compaction 
> strategy’s selection criteria for older windows. 
> Patch provided for : 
> - 2.1: https://github.com/jeffjirsa/cassandra/commits/twcs-2.1 
> - 2.2: https://github.com/jeffjirsa/cassandra/commits/twcs-2.2
> - trunk (post-8099):  https://github.com/jeffjirsa/cassandra/commits/twcs 
> Rebased, force-pushed July 18, with bug fixes for estimated pending 
> compactions and potential starvation if more than min_threshold tables 
> existed in current window but STCS did not consider them viable 

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[marcuse]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ef1e2ce
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ef1e2ce
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ef1e2ce

Branch: refs/heads/trunk
Commit: 8ef1e2ce252217a8a945b32087b5ae612568f908
Parents: 5e913ab 6c867f0
Author: Marcus Eriksson 
Authored: Tue Jun 7 07:45:10 2016 +0200
Committer: Marcus Eriksson 
Committed: Tue Jun 7 07:45:10 2016 +0200

--
 CHANGES.txt |   2 +
 NEWS.txt|  10 +
 doc/cql3/CQL.textile|   5 +-
 pylib/cqlshlib/cql3handling.py  |   7 +
 pylib/cqlshlib/cqlhandling.py   |   3 +-
 pylib/cqlshlib/test/test_cqlsh_completion.py|  11 +-
 .../DateTieredCompactionStrategy.java   |   4 +
 .../TimeWindowCompactionStrategy.java   | 380 +++
 .../TimeWindowCompactionStrategyOptions.java| 148 
 .../db/compaction/CompactionsCQLTest.java   |  13 +
 .../TimeWindowCompactionStrategyTest.java   | 274 +
 11 files changed, 854 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ef1e2ce/CHANGES.txt
--
diff --cc CHANGES.txt
index 40ffdf3,cdbaebb..d18fc9d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,8 +1,22 @@@
 -3.0.8
 +3.8
 + * Switch counter shards' clock to timestamps (CASSANDRA-9811)
 + * Introduce HdrHistogram and response/service/wait separation to stress tool 
(CASSANDRA-11853)
 + * entry-weighers in QueryProcessor should respect partitionKeyBindIndexes 
field (CASSANDRA-11718)
 + * Support older ant versions (CASSANDRA-11807)
 + * Estimate compressed on disk size when deciding if sstable size limit 
reached (CASSANDRA-11623)
 + * cassandra-stress profiles should support case sensitive schemas 
(CASSANDRA-11546)
 + * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
 + * Faster streaming (CASSANDRA-9766)
 + * Add prepared query parameter to trace for "Execute CQL3 prepared query" 
session (CASSANDRA-11425)
 + * Add repaired percentage metric (CASSANDRA-11503)
++Merged from 3.0:
+  * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
  
  
 -3.0.7
 +3.7
 + * Support multiple folders for user defined compaction tasks 
(CASSANDRA-11765)
 + * Fix race in CompactionStrategyManager's pause/resume (CASSANDRA-11922)
 +Merged from 3.0:
   * Fix legacy serialization of Thrift-generated non-compound range tombstones
 when communicating with 2.x nodes (CASSANDRA-11930)
   * Fix Directories instantiations where CFS.initialDirectories should be used 
(CASSANDRA-11849)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ef1e2ce/NEWS.txt
--
diff --cc NEWS.txt
index f8b589f,dbaece1..076d024
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,33 -13,31 +13,43 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
  
 -3.0.7
 -=
  
 -Upgrading
 --
 -   - A maximum size for SSTables values has been introduced, to prevent out 
of memory
 - exceptions when reading corrupt SSTables. This maximum size can be set 
via
 - max_value_size_in_mb in cassandra.yaml. The default is 256MB, which 
matches the default
 - value of native_transport_max_frame_size_in_mb. SSTables will be 
considered corrupt if
 - they contain values whose size exceeds this limit. See CASSANDRA-9530 
for more details.
 +3.8
 +===
 +
 +New features
 +
 +   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
 + that forces a scheduled rate of operations/sec over time. Using this, 
stress can
 + accurately account for coordinated ommission from the stress process.
 +   - The cassandra-stress "-rate limit=" option has been renamed to "-rate 
throttle="
 +   - hdr histograms have been added to stress runs, it's output can be saved 
to disk using:
 + "-log hdrfile=" option. This histogram includes response/service/wait 
times when used with the
 + fixed or throttle rate options.  The histogram file can be plotted on
 + http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
++   - TimeWindowCompactionStrategy has been added. This has proven to be a 
better approach
++ to time series compaction and new tables should use this instead of 
DTCS. See
++ CASSANDRA-9666 for details.
+ 
+ Deprecation
+ ---
+- DateTieredCompactionStrategy has been deprecated - new tables should use
+  

[2/3] cassandra git commit: Add TimeWindowCompactionStrategy

[marcuse]

Add TimeWindowCompactionStrategy

Patch by Jeff Jirsa; reviewed by marcuse for CASSANDRA-9666


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c867f00
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c867f00
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c867f00

Branch: refs/heads/trunk
Commit: 6c867f00309a61af12fa452020c45dc0f2748aa1
Parents: 040ac66
Author: Jeff Jirsa 
Authored: Tue May 24 20:21:22 2016 -0700
Committer: Marcus Eriksson 
Committed: Tue Jun 7 07:38:22 2016 +0200

--
 CHANGES.txt |   4 +
 NEWS.txt|  13 +
 doc/cql3/CQL.textile|   5 +-
 pylib/cqlshlib/cql3handling.py  |   7 +
 pylib/cqlshlib/cqlhandling.py   |   3 +-
 pylib/cqlshlib/test/test_cqlsh_completion.py|  11 +-
 .../DateTieredCompactionStrategy.java   |   4 +
 .../TimeWindowCompactionStrategy.java   | 380 +++
 .../TimeWindowCompactionStrategyOptions.java| 148 
 .../db/compaction/CompactionsCQLTest.java   |  13 +
 .../TimeWindowCompactionStrategyTest.java   | 274 +
 11 files changed, 859 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 201a36c..cdbaebb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+3.0.8
+ * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
+
+
 3.0.7
  * Fix legacy serialization of Thrift-generated non-compound range tombstones
when communicating with 2.x nodes (CASSANDRA-11930)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index ac1ef17..dbaece1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -24,6 +24,19 @@ Upgrading
  value of native_transport_max_frame_size_in_mb. SSTables will be 
considered corrupt if
  they contain values whose size exceeds this limit. See CASSANDRA-9530 for 
more details.
 
+Deprecation
+---
+   - DateTieredCompactionStrategy has been deprecated - new tables should use
+ TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table 
to TWCS might
+ cause increased compaction load for a while after the migration so make 
sure you run
+ tests before migrating. Read CASSANDRA-9666 for background on this.
+
+New features
+
+   - TimeWindowCompactionStrategy has been added. This has proven to be a 
better approach
+ to time series compaction and new tables should use this instead of DTCS. 
See
+ CASSANDRA-9666 for details.
+
 3.0.6
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 059b195..2a37452 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -346,7 +346,7 @@ Table creation supports the following other @@:
 
 h4(#compactionOptions). Compaction options
 
-The @compaction@ property must at least define the @'class'@ sub-option, that 
defines the compaction strategy class to use. The default supported class are 
@'SizeTieredCompactionStrategy'@, @'LeveledCompactionStrategy'@ and 
@'DateTieredCompactionStrategy'@. Custom strategy can be provided by specifying 
the full class name as a "string constant":#constants. The rest of the 
sub-options depends on the chosen class. The sub-options supported by the 
default classes are:
+The @compaction@ property must at least define the @'class'@ sub-option, that 
defines the compaction strategy class to use. The default supported class are 
@'SizeTieredCompactionStrategy'@, @'LeveledCompactionStrategy'@, 
@'DateTieredCompactionStrategy'@ and @'TimeWindowCompactionStrategy'@. Custom 
strategy can be provided by specifying the full class name as a "string 
constant":#constants. The rest of the sub-options depends on the chosen class. 
The sub-options supported by the default classes are:
 
 |_. option |_. supported compaction strategy |_. 
default|_. description |
 | @enabled@| _all_   | true
 | A boolean denoting whether compaction should be enabled or not.|
@@ -362,6 +362,9 @@ The @compaction@ property must at least define the 
@'class'@ sub-option, that de
 | @timestamp_resolution@   | DateTieredCompactionStrategy| 
MICROSECONDS | The timestamp resolution used when inserting data, could be 
MILLISECONDS, MICROSECONDS etc 

[1/3] cassandra git commit: Add TimeWindowCompactionStrategy

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 040ac666a -> 6c867f003
  refs/heads/trunk 5e913ab76 -> 8ef1e2ce2


Add TimeWindowCompactionStrategy

Patch by Jeff Jirsa; reviewed by marcuse for CASSANDRA-9666


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c867f00
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c867f00
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c867f00

Branch: refs/heads/cassandra-3.0
Commit: 6c867f00309a61af12fa452020c45dc0f2748aa1
Parents: 040ac66
Author: Jeff Jirsa 
Authored: Tue May 24 20:21:22 2016 -0700
Committer: Marcus Eriksson 
Committed: Tue Jun 7 07:38:22 2016 +0200

--
 CHANGES.txt |   4 +
 NEWS.txt|  13 +
 doc/cql3/CQL.textile|   5 +-
 pylib/cqlshlib/cql3handling.py  |   7 +
 pylib/cqlshlib/cqlhandling.py   |   3 +-
 pylib/cqlshlib/test/test_cqlsh_completion.py|  11 +-
 .../DateTieredCompactionStrategy.java   |   4 +
 .../TimeWindowCompactionStrategy.java   | 380 +++
 .../TimeWindowCompactionStrategyOptions.java| 148 
 .../db/compaction/CompactionsCQLTest.java   |  13 +
 .../TimeWindowCompactionStrategyTest.java   | 274 +
 11 files changed, 859 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 201a36c..cdbaebb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+3.0.8
+ * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
+
+
 3.0.7
  * Fix legacy serialization of Thrift-generated non-compound range tombstones
when communicating with 2.x nodes (CASSANDRA-11930)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index ac1ef17..dbaece1 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -24,6 +24,19 @@ Upgrading
  value of native_transport_max_frame_size_in_mb. SSTables will be 
considered corrupt if
  they contain values whose size exceeds this limit. See CASSANDRA-9530 for 
more details.
 
+Deprecation
+---
+   - DateTieredCompactionStrategy has been deprecated - new tables should use
+ TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table 
to TWCS might
+ cause increased compaction load for a while after the migration so make 
sure you run
+ tests before migrating. Read CASSANDRA-9666 for background on this.
+
+New features
+
+   - TimeWindowCompactionStrategy has been added. This has proven to be a 
better approach
+ to time series compaction and new tables should use this instead of DTCS. 
See
+ CASSANDRA-9666 for details.
+
 3.0.6
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6c867f00/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 059b195..2a37452 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -346,7 +346,7 @@ Table creation supports the following other @@:
 
 h4(#compactionOptions). Compaction options
 
-The @compaction@ property must at least define the @'class'@ sub-option, that 
defines the compaction strategy class to use. The default supported class are 
@'SizeTieredCompactionStrategy'@, @'LeveledCompactionStrategy'@ and 
@'DateTieredCompactionStrategy'@. Custom strategy can be provided by specifying 
the full class name as a "string constant":#constants. The rest of the 
sub-options depends on the chosen class. The sub-options supported by the 
default classes are:
+The @compaction@ property must at least define the @'class'@ sub-option, that 
defines the compaction strategy class to use. The default supported class are 
@'SizeTieredCompactionStrategy'@, @'LeveledCompactionStrategy'@, 
@'DateTieredCompactionStrategy'@ and @'TimeWindowCompactionStrategy'@. Custom 
strategy can be provided by specifying the full class name as a "string 
constant":#constants. The rest of the sub-options depends on the chosen class. 
The sub-options supported by the default classes are:
 
 |_. option |_. supported compaction strategy |_. 
default|_. description |
 | @enabled@| _all_   | true
 | A boolean denoting whether compaction should be enabled or not.|
@@ -362,6 +362,9 @@ The @compaction@ property must at least define the 
@'class'@ sub-option, that de
 | @timestamp_resolution@   | 

[jira] [Commented] (CASSANDRA-11580) remove DatabaseDescriptor dependency from SegmentedFile

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317744#comment-15317744
 ] 

Yuki Morishita commented on CASSANDRA-11580:


I ended up renaming {{SegmentedFile}} to {{FileHandle}}. If anyone has better 
name, I'm open to suggestion.
I think I fixed he points in the review, except: The EMPTY BufferHolder in 
Rebuffered.java at line 62 can still be static final. variable in interface is 
implicitly declared as public static final, and my IntelliJ gives me warning if 
I left those.

||branch||testall||dtest||
|[11580|https://github.com/yukim/cassandra/tree/11580]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11580-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11580-dtest/lastCompletedBuild/testReport/]|


> remove DatabaseDescriptor dependency from SegmentedFile
> ---
>
> Key: CASSANDRA-11580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11580
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 3.x
>
>
> Several configurable parameters are pulled from {{DatabaseDescriptor}} from 
> {{SegmentedFile}} and its subclasses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11580) remove DatabaseDescriptor dependency from SegmentedFile

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11580:
---
Status: Patch Available  (was: In Progress)

> remove DatabaseDescriptor dependency from SegmentedFile
> ---
>
> Key: CASSANDRA-11580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11580
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 3.x
>
>
> Several configurable parameters are pulled from {{DatabaseDescriptor}} from 
> {{SegmentedFile}} and its subclasses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11873) Add duration type

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317719#comment-15317719
 ] 

Sylvain Lebresne commented on CASSANDRA-11873:
--

bq.  use {{µs}} instead of {{us}}

Why _instead_? let's just support both, it's trivial. I agree that in a number 
of situations it's easier to stick to ascii, but no reason to also be modern.

> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10961) Not enough bytes error when add nodes to cluster

[Akshay Kulkarni (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317529#comment-15317529
 ] 

Akshay Kulkarni commented on CASSANDRA-10961:
-

Our cluster also had a similar issue and we are on Cassandra 2.1.12 version. 
However, we were not adding any new nodes to our cluster but we faced this 
issue when suddenly our node went down and we when try to restart them again 
they are not coming up. Does this bug come during bootstrapping as well? 

> Not enough bytes error when add nodes to cluster
> 
>
> Key: CASSANDRA-10961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10961
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: xiaost
>Assignee: Paulo Motta
> Fix For: 2.1.13, 2.2.5, 3.0.3, 3.3
>
> Attachments: apache-cassandra-2.2.4-SNAPSHOT.jar, debug.1.log, 
> debug.logs.zip, netstats.1.log
>
>
> we got the same problem all the time when we add nodes to cluster.
> netstats:
> on HostA
> {noformat}
> /la-38395-big-Data.db 14792091851/14792091851 bytes(100%) sent to idx:0/HostB
> {noformat}
> on HostB
> {noformat}
> tmp-la-4-big-Data.db 2667087450/14792091851 bytes(18%) received from 
> idx:0/HostA
> {noformat}
> After a while, Error on HostB
> {noformat}
> WARN  [STREAM-IN-/HostA] 2016-01-02 12:08:14,737 StreamSession.java:644 - 
> [Stream #b91a4e90-b105-11e5-bd57-dd0cc3b4634c] Retrying for following error
> java.lang.IllegalArgumentException: Not enough bytes
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:362)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:98)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:365)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.appendFromStream(BigTableWriter.java:243)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:173) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:95)
>  ~[apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:49)
>  [apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38)
>  [apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
>  [apache-cassandra-2.2.4.jar:2.2.4]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
>  [apache-cassandra-2.2.4.jar:2.2.4]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66-internal]
> ERROR [Thread-28] 2016-01-02 12:08:14,737 CassandraDaemon.java:185 - 
> Exception in thread Thread[Thread-28,5,main]
> java.lang.RuntimeException: java.lang.InterruptedException
> at com.google.common.base.Throwables.propagate(Throwables.java:160) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) 
> ~[apache-cassandra-2.2.4.jar:2.2.4]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
> Caused by: java.lang.InterruptedException: null
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
>  ~[na:1.8.0_66-internal]
> at 
> java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
>  ~[na:1.8.0_66-internal]
> at 
> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350) 
> 

[jira] [Updated] (CASSANDRA-11854) Remove finished streaming connections from MessagingService

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11854:

Reviewer: Yuki Morishita

> Remove finished streaming connections from MessagingService
> ---
>
> Key: CASSANDRA-11854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11854
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Attachments: oom.png
>
>
> When a new {{IncomingStreamingConnection}} is created, [we register it in the 
> connections 
> map|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/MessagingService.java#L1109]
>  of {{MessagingService}}, but we [only remove it if there is an 
> exception|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/IncomingStreamingConnection.java#L83]
>  while attaching the socket to the stream session.
> On nodes with SSL and large number of vnodes, after many repair sessions 
> these old connections can accumulate and cause OOM (heap dump attached).
> The connection should be removed from the connections map after if it's 
> finished in order to be garbage collected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11868) unused imports and generic types

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11868:

Reviewer: Alex Petrov

> unused imports and generic types
> 
>
> Key: CASSANDRA-11868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11868
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 3.8
>
>
> I was going through Cassandra source and for busy work I started looking at 
> all the .java files eclipse flags as warning. They are broken roughly into a 
> few cases. 
> 1) unused imports 
> 2) raw types missing <> 
> 3) case statements without defaults 
> 4) @resource annotation 
> My IDE claims item 4 is not needed (it looks like we have done this to 
> signify methods that return objects that need to be closed) I can guess 4 was 
> done intentionally and short of making out own annotation I will ignore these 
> for now. 
> I would like to tackle this busy work before I get started. I have some 
> questions: 
> 1) Do this only on trunk? or multiple branches 
> 2) should I tackle 1,2,3 in separate branches/patches



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11913) BufferUnderFlowException in CompressorTest

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11913:

Reviewer: Branimir Lambov

> BufferUnderFlowException in CompressorTest
> --
>
> Key: CASSANDRA-11913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11913
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Non-x86 environments
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11913-2.2.txt
>
>
> org.apache.cassandra.io.compress.CompressorTest causes 
> java.nio.BufferUnderflowException on environments where FastByteOperations 
> uses PureJavaOperations. The root cause is that 
> CompressorTest.testArrayUncompress() copies data from a ByteBuffer to a byte 
> array beyond the limit of the ByteBuffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11345) Assertion Errors "Memory was freed" during streaming

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11345:

Reviewer: Yuki Morishita

> Assertion Errors "Memory was freed" during streaming
> 
>
> Key: CASSANDRA-11345
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11345
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Jean-Francois Gosselin
>Assignee: Paulo Motta
>
> We encountered the following AssertionError (twice on the same node) during a 
> repair :
> On node /172.16.63.41
> {noformat}
> INFO  [STREAM-IN-/10.174.216.160] 2016-03-09 02:38:13,900 
> StreamResultFuture.java:180 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Session with /10.174.216.160 is complete  
>   
> WARN  [STREAM-IN-/10.174.216.160] 2016-03-09 02:38:13,900 
> StreamResultFuture.java:207 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Stream failed   
> ERROR [STREAM-OUT-/10.174.216.160] 2016-03-09 02:38:13,906 
> StreamSession.java:505 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Streaming error occurred
> java.lang.AssertionError: Memory was freed
>   
>
> at 
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
>   
> at org.apache.cassandra.io.util.Memory.getLong(Memory.java:249) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
>  
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.getTotalSizeForSections(CompressionMetadata.java:247)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.messages.FileMessageHeader.size(FileMessageHeader.java:112)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.StreamSession.fileSent(StreamSession.java:546) 
> ~[apache-cassandra-2.1.13.jar:2.1.13] 
> 
> at 
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:50)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]  
> at 
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:41)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]  
> at 
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:45)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]
> at 
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:351)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]   
> at 
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:331)
>  ~[apache-cassandra-2.1.13.jar:2.1.13]   
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
>   
>
> {noformat} 
> On node /10.174.216.160
>  
> {noformat}   
> ERROR [STREAM-OUT-/172.16.63.41] 2016-03-09 02:38:14,140 
> StreamSession.java:505 - [Stream #f6980580-e55f-11e5-8f08-ef9e099ce99e] 
> Streaming error occurred  
> java.io.IOException: Connection reset by peer 
>   
>
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[na:1.7.0_65] 
>   
>
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) 
> ~[na:1.7.0_65]
>   
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) 
> ~[na:1.7.0_65]
>   
> at sun.nio.ch.IOUtil.write(IOUtil.java:65) ~[na:1.7.0_65] 
>   
>
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487) 
> ~[na:1.7.0_65]
>
> at 
> 

[jira] [Created] (CASSANDRA-11963) Paged queries limited to Integer.MAX_VALUE total rows

[Adam Holmberg (JIRA)]

Adam Holmberg created CASSANDRA-11963:
-

 Summary: Paged queries limited to Integer.MAX_VALUE total rows
 Key: CASSANDRA-11963
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11963
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Adam Holmberg
Priority: Minor
 Fix For: 2.1.x, 2.2.x


Paged queries are artificially limited to Integer.MAX_INT rows in total. This 
appears to be related to PagingState.remaining, which decrements monotonically 
as pages are consumed. 

I don't think this is intentional behavior, and haven't found any mention of it 
in the docs.

Issue observed in latest 2.1 and 2.2 releases. Does not occur in 3.x



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1747074 - /cassandra/site/publish/index.html

[jake]

Author: jake
Date: Mon Jun  6 20:38:59 2016
New Revision: 1747074

URL: http://svn.apache.org/viewvc?rev=1747074=rev
Log:
switch archive ml links - v2

Modified:
cassandra/site/publish/index.html

Modified: cassandra/site/publish/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/index.html?rev=1747074=1747073=1747074=diff
==
--- cassandra/site/publish/index.html (original)
+++ cassandra/site/publish/index.html Mon Jun  6 20:38:59 2016
@@ -186,17 +186,17 @@
   Users list  General mailing list for users.
   mailto:user-subscr...@cassandra.apache.org;>Subscribe
   | mailto:user-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/user@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?u...@cassandra.apache.org;>Archives
   
   Developers list  Development related mailing list.
   mailto:dev-subscr...@cassandra.apache.org;>Subscribe
   | mailto:dev-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/dev@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?d...@cassandra.apache.org;>Archives
   
   Commits list  Commit notifications.
   mailto:commits-subscr...@cassandra.apache.org;>Subscribe
   | mailto:commits-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/commits@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?commits@cassandra.apache.org;>Archives
   
 
   

svn commit: r1747073 - /cassandra/site/src/content/index.html

[jake]

Author: jake
Date: Mon Jun  6 20:38:21 2016
New Revision: 1747073

URL: http://svn.apache.org/viewvc?rev=1747073=rev
Log:
switch archive ml links

Modified:
cassandra/site/src/content/index.html

Modified: cassandra/site/src/content/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/content/index.html?rev=1747073=1747072=1747073=diff
==
--- cassandra/site/src/content/index.html (original)
+++ cassandra/site/src/content/index.html Mon Jun  6 20:38:21 2016
@@ -124,17 +124,17 @@
   Users list  General mailing list for users.
   mailto:user-subscr...@cassandra.apache.org;>Subscribe
   | mailto:user-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/user@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?u...@cassandra.apache.org;>Archives
   
   Developers list  Development related mailing list.
   mailto:dev-subscr...@cassandra.apache.org;>Subscribe
   | mailto:dev-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/dev@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?d...@cassandra.apache.org;>Archives
   
   Commits list  Commit notifications.
   mailto:commits-subscr...@cassandra.apache.org;>Subscribe
   | mailto:commits-unsubscr...@cassandra.apache.org;>Unsubscribe
-  | http://www.mail-archive.com/commits@cassandra.apache.org/;>Archives
+  | https://lists.apache.org/list.html?commits@cassandra.apache.org;>Archives
   
 
   

svn commit: r1747066 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py

[jake]

Author: jake
Date: Mon Jun  6 20:12:38 2016
New Revision: 1747066

URL: http://svn.apache.org/viewvc?rev=1747066=rev
Log:
3.6 release

Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1747066=1747065=1747066=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Mon Jun  6 20:12:38 2016
@@ -49,7 +49,25 @@
 
 Cassandra is moving to a monthly release process called Tick-Tock.  
Even-numbered releases (e.g. 3.2) contain new features; 
odd-numbered releases (e.g. 3.3) contain bug fixes only.  If a critical 
bug is found, a patch will be released against the most recent bug fix release. 
 http://www.planetcassandra.org/blog/cassandra-2-2-3-0-and-beyond/;>Read 
more about tick-tock here.
 
-The latest tick-tock release is 3.5, released on
+The latest tick-tock release is 3.6, released on
+2016-06-06.
+
+
+
+  
+  http://www.apache.org/dyn/closer.lua/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz;>apache-cassandra-3.6-bin.tar.gz
+  [http://www.apache.org/dist/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc;>PGP]
+  [http://www.apache.org/dist/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.md5;>MD5]
+  [http://www.apache.org/dist/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.sha1;>SHA1]
+  
+  
+   http://wiki.apache.org/cassandra/DebianPackaging;>Debian 
installation instructions
+  
+
+
+
+
+The previous tick-tock bugfix release is 3.5, released on
 2016-04-13.
 
 

Modified: cassandra/site/publish/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/index.html?rev=1747066=1747065=1747066=diff
==
--- cassandra/site/publish/index.html (original)
+++ cassandra/site/publish/index.html Mon Jun  6 20:12:38 2016
@@ -77,7 +77,7 @@
   
   
   
-  http://www.planetcassandra.org/blog/cassandra-2-2-3-0-and-beyond/;>Tick-Tock
 release 3.5 (http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-3.5;>Changes)
+  http://www.planetcassandra.org/blog/cassandra-2-2-3-0-and-beyond/;>Tick-Tock
 release 3.6 (http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-3.6;>Changes)
   
   
 

Modified: cassandra/site/src/settings.py
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1747066=1747065=1747066=diff
==
--- cassandra/site/src/settings.py (original)
+++ cassandra/site/src/settings.py Mon Jun  6 20:12:38 2016
@@ -92,11 +92,11 @@ SITE_POST_PROCESSORS = {
 }
 
 class CassandraDef(object):
-ticktock_both_exist = False
-ticktock_version = '3.5'
-ticktock_version_date = '2016-04-13'
-ticktock_odd_version = '3.3'
-ticktock_odd_version_date = '2016-02-09'
+ticktock_both_exist = True
+ticktock_version = '3.6'
+ticktock_version_date = '2016-06-06'
+ticktock_odd_version = '3.5'
+ticktock_odd_version_date = '2016-04-13'
 stable_version = '3.0.6'
 stable_release_date = '2016-05-13'
 oldstable_version = '2.2.6'

[jira] [Commented] (CASSANDRA-9666) Provide an alternative to DTCS

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317120#comment-15317120
 ] 

Jeff Jirsa commented on CASSANDRA-9666:
---

nits + NEWS entry look good to me. Thanks Marcus!


> Provide an alternative to DTCS
> --
>
> Key: CASSANDRA-9666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9666
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
> Attachments: compactomatic.py, dashboard-DTCS_to_TWCS.png, 
> dtcs-twcs-io.png, dtcs-twcs-load.png
>
>
> DTCS is great for time series data, but it comes with caveats that make it 
> difficult to use in production (typical operator behaviors such as bootstrap, 
> removenode, and repair have MAJOR caveats as they relate to 
> max_sstable_age_days, and hints/read repair break the selection algorithm).
> I'm proposing an alternative, TimeWindowCompactionStrategy, that sacrifices 
> the tiered nature of DTCS in order to address some of DTCS' operational 
> shortcomings. I believe it is necessary to propose an alternative rather than 
> simply adjusting DTCS, because it fundamentally removes the tiered nature in 
> order to remove the parameter max_sstable_age_days - the result is very very 
> different, even if it is heavily inspired by DTCS. 
> Specifically, rather than creating a number of windows of ever increasing 
> sizes, this strategy allows an operator to choose the window size, compact 
> with STCS within the first window of that size, and aggressive compact down 
> to a single sstable once that window is no longer current. The window size is 
> a combination of unit (minutes, hours, days) and size (1, etc), such that an 
> operator can expect all data using a block of that size to be compacted 
> together (that is, if your unit is hours, and size is 6, you will create 
> roughly 4 sstables per day, each one containing roughly 6 hours of data). 
> The result addresses a number of the problems with 
> DateTieredCompactionStrategy:
> - At the present time, DTCS’s first window is compacted using an unusual 
> selection criteria, which prefers files with earlier timestamps, but ignores 
> sizes. In TimeWindowCompactionStrategy, the first window data will be 
> compacted with the well tested, fast, reliable STCS. All STCS options can be 
> passed to TimeWindowCompactionStrategy to configure the first window’s 
> compaction behavior.
> - HintedHandoff may put old data in new sstables, but it will have little 
> impact other than slightly reduced efficiency (sstables will cover a wider 
> range, but the old timestamps will not impact sstable selection criteria 
> during compaction)
> - ReadRepair may put old data in new sstables, but it will have little impact 
> other than slightly reduced efficiency (sstables will cover a wider range, 
> but the old timestamps will not impact sstable selection criteria during 
> compaction)
> - Small, old sstables resulting from streams of any kind will be swiftly and 
> aggressively compacted with the other sstables matching their similar 
> maxTimestamp, without causing sstables in neighboring windows to grow in size.
> - The configuration options are explicit and straightforward - the tuning 
> parameters leave little room for error. The window is set in common, easily 
> understandable terms such as “12 hours”, “1 Day”, “30 days”. The 
> minute/hour/day options are granular enough for users keeping data for hours, 
> and users keeping data for years. 
> - There is no explicitly configurable max sstable age, though sstables will 
> naturally stop compacting once new data is written in that window. 
> - Streaming operations can create sstables with old timestamps, and they'll 
> naturally be joined together with sstables in the same time bucket. This is 
> true for bootstrap/repair/sstableloader/removenode. 
> - It remains true that if old data and new data is written into the memtable 
> at the same time, the resulting sstables will be treated as if they were new 
> sstables, however, that no longer negatively impacts the compaction 
> strategy’s selection criteria for older windows. 
> Patch provided for : 
> - 2.1: https://github.com/jeffjirsa/cassandra/commits/twcs-2.1 
> - 2.2: https://github.com/jeffjirsa/cassandra/commits/twcs-2.2
> - trunk (post-8099):  https://github.com/jeffjirsa/cassandra/commits/twcs 
> Rebased, force-pushed July 18, with bug fixes for estimated pending 
> compactions and potential starvation if more than min_threshold tables 
> existed in current window but STCS did not consider them viable candidates
> Rebased, force-pushed Aug 20 to bring in relevant logic from CASSANDRA-9882



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10783) Allow literal value as parameter of UDF & UDA

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317088#comment-15317088
 ] 

Benjamin Lerer edited comment on CASSANDRA-10783 at 6/6/16 7:50 PM:


I have the following remarks:
* The fact that {{Term.Raw}} implements {{Selectable}} looks confusing to me 
from a hierachy point of view. I think it would be better if there was a new 
{{Selectable}} implementation for Litterals that hold a reference to the 
{{Term.Raw}} (I have pushed an example 
[here|https://github.com/apache/cassandra/commit/cdde2447b86611a158357b757cb9eb4153fe2dc2]).
* The fact that {{ColumnIdentifier.Raw::prepare}} produce a 
{{ColumnDefinition}} does not look really logical. I would be in favor of 
moving {{ColumnIdentifier::Raw}} and its implementations to 
{{ColumnDefinition}}.
* {{UnrecognizedEntityException}} is not used anymore and can be remove as well 
as the {{try-catch}} block in {{SelectStatement::prepareRestrictions}} and the 
{{containsAlias}} method.
* In {{WithFieldSelection}} switching from {{ColumnIdentifier}} to 
{{ByteBuffer}} makes the column name unreadable in the error message. As it did 
not break any test it means that we are probably not testing those checks and 
that we should add some tests for it.
* {{Relation::toColumnDefinition}} could be inlined
* {{SelectorFactories::asList}} is not used and could be removed
* {{Selector}} contains some unused imports

I think it would also be good to add some extra tests to {{TermSelectionTest}}:
* in {{testSelectLiteral}}:
{code}
assertColumnNames(execute("SELECT ck, t, (int)42, (int)43 FROM %s"), "ck", "t", 
"(int)42", "(int)43");
assertRows(execute("SELECT ck, t, (int) 42, (int) 43 FROM %s"),
   row(1, "one", 42, 43),
   row(2, "two", 42, 43),
   row(3, "three", 42, 43));
{code}
* in {{testSelectPrepared}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset())
{code}
* in {{testPreparedFunctionArgs}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset());
{code}
In fact, the latest query fail with a {{NPE}}.


was (Author: blerer):
I have the following remarks:
* The fact that {{Term.Raw}} implements {{Selectable}} looks confusing to me 
from a hierachy point of view. I think it would be better if there was a new 
{{Selectable}} implementation for Litterals that hold a reference to the 
{{Term.Raw}} (I have pushed an example 
[here|https://github.com/apache/cassandra/commit/cdde2447b86611a158357b757cb9eb4153fe2dc2]).
* The fact that {{ColumnIdentifier.Raw::prepare}} produce a 
{{ColumnDefinition}} does not look really logical. I would be in favor of 
moving {{ColumnIdentifier::Raw}} and its implementations to 
{{ColumnDefinition}}.
* {{UnrecognizedEntityException}} is not used anymore and can be remove as well 
as the {{try-catch}} block in {{SelectStatement::prepareRestrictions}} and the 
{{containsAlias}} method.
* In {{WithFieldSelection}} switching from {{ColumnIdentifier}} to 
{{ByteBuffer}} makes the column name unreadable in the error message. As it did 
not break any test it means that we are probably not testing those checks and 
that we should add some tests for it.
* {{Relation::toColumnDefinition}} could be inlined
* {{SelectorFactories::asList}} is not used and could be removed
* {{Selector}} contains some unused imports

I think it would also be good to add some extra tests to {{TermSelectionTest}}:
* to {{testSelectLiteral}}:
{code}
assertColumnNames(execute("SELECT ck, t, (int)42, (int)43 FROM %s"), "ck", "t", 
"(int)42", "(int)43");
assertRows(execute("SELECT ck, t, (int) 42, (int) 43 FROM %s"),
   row(1, "one", 42, 43),
   row(2, "two", 42, 43),
   row(3, "three", 42, 43));
{code}
* to {{testSelectPrepared}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset())
{code}
* to {{testPreparedFunctionArgs}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset());
{code}
In fact, the latest query fail with a {{NPE}}.

> Allow literal value as parameter of UDF & UDA
> -
>
> Key: CASSANDRA-10783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10783
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: DOAN DuyHai
>Assignee: Robert Stupp
>Priority: Minor
>  Labels: CQL3, UDF, client-impacting, doc-impacting
> Fix For: 3.x
>
>
> I have defined the following UDF
> {code:sql}
> CREATE OR REPLACE FUNCTION  maxOf(current int, testValue int) RETURNS NULL ON 
> NULL INPUT 
> RETURNS int 
> LANGUAGE java 
> AS  'return 

[jira] [Commented] (CASSANDRA-10783) Allow literal value as parameter of UDF & UDA

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317088#comment-15317088
 ] 

Benjamin Lerer commented on CASSANDRA-10783:


I have the following remarks:
* The fact that {{Term.Raw}} implements {{Selectable}} looks confusing to me 
from a hierachy point of view. I think it would be better if there was a new 
{{Selectable}} implementation for Litterals that hold a reference to the 
{{Term.Raw}} (I have pushed an example 
[here|https://github.com/apache/cassandra/commit/cdde2447b86611a158357b757cb9eb4153fe2dc2]).
* The fact that {{ColumnIdentifier.Raw::prepare}} produce a 
{{ColumnDefinition}} does not look really logical. I would be in favor of 
moving {{ColumnIdentifier::Raw}} and its implementations to 
{{ColumnDefinition}}.
* {{UnrecognizedEntityException}} is not used anymore and can be remove as well 
as the {{try-catch}} block in {{SelectStatement::prepareRestrictions}} and the 
{{containsAlias}} method.
* In {{WithFieldSelection}} switching from {{ColumnIdentifier}} to 
{{ByteBuffer}} makes the column name unreadable in the error message. As it did 
not break any test it means that we are probably not testing those checks and 
that we should add some tests for it.
* {{Relation::toColumnDefinition}} could be inlined
* {{SelectorFactories::asList}} is not used and could be removed
* {{Selector}} contains some unused imports

I think it would also be good to add some extra tests to {{TermSelectionTest}}:
* to {{testSelectLiteral}}:
{code}
assertColumnNames(execute("SELECT ck, t, (int)42, (int)43 FROM %s"), "ck", "t", 
"(int)42", "(int)43");
assertRows(execute("SELECT ck, t, (int) 42, (int) 43 FROM %s"),
   row(1, "one", 42, 43),
   row(2, "two", 42, 43),
   row(3, "three", 42, 43));
{code}
* to {{testSelectPrepared}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset())
{code}
* to {{testPreparedFunctionArgs}}:
{code}
execute("SELECT pk, ck, " + fIntMax + "(i, (int)?) FROM %s WHERE pk = " + 
fIntMax + "((int)1,(int)1)", unset());
{code}
In fact, the latest query fail with a {{NPE}}.

> Allow literal value as parameter of UDF & UDA
> -
>
> Key: CASSANDRA-10783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10783
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: DOAN DuyHai
>Assignee: Robert Stupp
>Priority: Minor
>  Labels: CQL3, UDF, client-impacting, doc-impacting
> Fix For: 3.x
>
>
> I have defined the following UDF
> {code:sql}
> CREATE OR REPLACE FUNCTION  maxOf(current int, testValue int) RETURNS NULL ON 
> NULL INPUT 
> RETURNS int 
> LANGUAGE java 
> AS  'return Math.max(current,testValue);'
> CREATE TABLE maxValue(id int primary key, val int);
> INSERT INTO maxValue(id, val) VALUES(1, 100);
> SELECT maxOf(val, 101) FROM maxValue WHERE id=1;
> {code}
> I got the following error message:
> {code}
> SyntaxException:  message="line 1:19 no viable alternative at input '101' (SELECT maxOf(val1, 
> [101]...)">
> {code}
>  It would be nice to allow literal value as parameter of UDF and UDA too.
>  I was thinking about an use-case for an UDA groupBy() function where the end 
> user can *inject* at runtime a literal value to select which aggregation he 
> want to display, something similar to GROUP BY ... HAVING 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9666) Provide an alternative to DTCS

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317011#comment-15317011
 ] 

Marcus Eriksson commented on CASSANDRA-9666:


I've pushed a few nits 
[here|https://github.com/krummas/cassandra/commits/jjirsa/twcs-3.0] - mostly 
using TimeUnit to convert timestamps and moving a few log lines to trace 
instead of debug. Also added the same timestamp_resolution warning we have in 
DTCS.

I also want some feedback on the NEWS.txt entry I added in the commit above 
([~jjirsa] [~jbellis]) 

If the code changes look good you you [~jjirsa], I'll get this committed 
tomorrow.

> Provide an alternative to DTCS
> --
>
> Key: CASSANDRA-9666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9666
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
> Attachments: compactomatic.py, dashboard-DTCS_to_TWCS.png, 
> dtcs-twcs-io.png, dtcs-twcs-load.png
>
>
> DTCS is great for time series data, but it comes with caveats that make it 
> difficult to use in production (typical operator behaviors such as bootstrap, 
> removenode, and repair have MAJOR caveats as they relate to 
> max_sstable_age_days, and hints/read repair break the selection algorithm).
> I'm proposing an alternative, TimeWindowCompactionStrategy, that sacrifices 
> the tiered nature of DTCS in order to address some of DTCS' operational 
> shortcomings. I believe it is necessary to propose an alternative rather than 
> simply adjusting DTCS, because it fundamentally removes the tiered nature in 
> order to remove the parameter max_sstable_age_days - the result is very very 
> different, even if it is heavily inspired by DTCS. 
> Specifically, rather than creating a number of windows of ever increasing 
> sizes, this strategy allows an operator to choose the window size, compact 
> with STCS within the first window of that size, and aggressive compact down 
> to a single sstable once that window is no longer current. The window size is 
> a combination of unit (minutes, hours, days) and size (1, etc), such that an 
> operator can expect all data using a block of that size to be compacted 
> together (that is, if your unit is hours, and size is 6, you will create 
> roughly 4 sstables per day, each one containing roughly 6 hours of data). 
> The result addresses a number of the problems with 
> DateTieredCompactionStrategy:
> - At the present time, DTCS’s first window is compacted using an unusual 
> selection criteria, which prefers files with earlier timestamps, but ignores 
> sizes. In TimeWindowCompactionStrategy, the first window data will be 
> compacted with the well tested, fast, reliable STCS. All STCS options can be 
> passed to TimeWindowCompactionStrategy to configure the first window’s 
> compaction behavior.
> - HintedHandoff may put old data in new sstables, but it will have little 
> impact other than slightly reduced efficiency (sstables will cover a wider 
> range, but the old timestamps will not impact sstable selection criteria 
> during compaction)
> - ReadRepair may put old data in new sstables, but it will have little impact 
> other than slightly reduced efficiency (sstables will cover a wider range, 
> but the old timestamps will not impact sstable selection criteria during 
> compaction)
> - Small, old sstables resulting from streams of any kind will be swiftly and 
> aggressively compacted with the other sstables matching their similar 
> maxTimestamp, without causing sstables in neighboring windows to grow in size.
> - The configuration options are explicit and straightforward - the tuning 
> parameters leave little room for error. The window is set in common, easily 
> understandable terms such as “12 hours”, “1 Day”, “30 days”. The 
> minute/hour/day options are granular enough for users keeping data for hours, 
> and users keeping data for years. 
> - There is no explicitly configurable max sstable age, though sstables will 
> naturally stop compacting once new data is written in that window. 
> - Streaming operations can create sstables with old timestamps, and they'll 
> naturally be joined together with sstables in the same time bucket. This is 
> true for bootstrap/repair/sstableloader/removenode. 
> - It remains true that if old data and new data is written into the memtable 
> at the same time, the resulting sstables will be treated as if they were new 
> sstables, however, that no longer negatively impacts the compaction 
> strategy’s selection criteria for older windows. 
> Patch provided for : 
> - 2.1: https://github.com/jeffjirsa/cassandra/commits/twcs-2.1 
> - 2.2: https://github.com/jeffjirsa/cassandra/commits/twcs-2.2
> - trunk (post-8099):  

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.7-tentative [created] 6815dc970

[7/8] cassandra git commit: 3.7 debian version

[jake]

3.7 debian version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6815dc97
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6815dc97
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6815dc97

Branch: refs/heads/cassandra-3.7
Commit: 6815dc970565e6cd1e0169b5379f37da7a5a8a32
Parents: 30243a7
Author: T Jake Luciani 
Authored: Mon Jun 6 14:30:44 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:30:44 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6815dc97/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 44db441..aa4f0c4 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.7) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 06 Jun 2016 14:29:19 -0400
+
 cassandra (3.6) unstable; urgency=medium
 
   * New release

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.0.7-tentative [created] 040ac666a

[4/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[jake]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30243a7b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30243a7b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30243a7b

Branch: refs/heads/cassandra-3.7
Commit: 30243a7b05752e09c75c4837d79b6fee3f855d0b
Parents: 40e4b0b 040ac66
Author: T Jake Luciani 
Authored: Mon Jun 6 14:29:05 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:29:05 2016 -0400

--

--

[3/8] cassandra git commit: bump debian to 3.0.7

[jake]

bump debian to 3.0.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/040ac666
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/040ac666
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/040ac666

Branch: refs/heads/trunk
Commit: 040ac666ac5cdf9cd0a01a845f2ea0af3a81a08b
Parents: c98b648
Author: T Jake Luciani 
Authored: Mon Jun 6 14:28:41 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:28:41 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/040ac666/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 9f02a88..0fbb787 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.7) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 06 Jun 2016 14:27:28 -0400
+
 cassandra (3.0.6) unstable; urgency=medium
 
   * New release

[1/8] cassandra git commit: bump debian to 3.0.7

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 c98b6484e -> 040ac666a
  refs/heads/cassandra-3.7 40e4b0b08 -> 6815dc970
  refs/heads/trunk fed476f9c -> 5e913ab76


bump debian to 3.0.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/040ac666
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/040ac666
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/040ac666

Branch: refs/heads/cassandra-3.0
Commit: 040ac666ac5cdf9cd0a01a845f2ea0af3a81a08b
Parents: c98b648
Author: T Jake Luciani 
Authored: Mon Jun 6 14:28:41 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:28:41 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/040ac666/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 9f02a88..0fbb787 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.7) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 06 Jun 2016 14:27:28 -0400
+
 cassandra (3.0.6) unstable; urgency=medium
 
   * New release

[6/8] cassandra git commit: 3.7 debian version

[jake]

3.7 debian version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6815dc97
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6815dc97
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6815dc97

Branch: refs/heads/trunk
Commit: 6815dc970565e6cd1e0169b5379f37da7a5a8a32
Parents: 30243a7
Author: T Jake Luciani 
Authored: Mon Jun 6 14:30:44 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:30:44 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6815dc97/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 44db441..aa4f0c4 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.7) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 06 Jun 2016 14:29:19 -0400
+
 cassandra (3.6) unstable; urgency=medium
 
   * New release

[8/8] cassandra git commit: Merge branch 'cassandra-3.7' into trunk

[jake]

Merge branch 'cassandra-3.7' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5e913ab7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5e913ab7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5e913ab7

Branch: refs/heads/trunk
Commit: 5e913ab76c39eada650965e16619dbf75bf1ea12
Parents: fed476f 6815dc9
Author: T Jake Luciani 
Authored: Mon Jun 6 14:31:28 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:31:28 2016 -0400

--

--

[5/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7

[jake]

Merge branch 'cassandra-3.0' into cassandra-3.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30243a7b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30243a7b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30243a7b

Branch: refs/heads/trunk
Commit: 30243a7b05752e09c75c4837d79b6fee3f855d0b
Parents: 40e4b0b 040ac66
Author: T Jake Luciani 
Authored: Mon Jun 6 14:29:05 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:29:05 2016 -0400

--

--

[2/8] cassandra git commit: bump debian to 3.0.7

[jake]

bump debian to 3.0.7


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/040ac666
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/040ac666
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/040ac666

Branch: refs/heads/cassandra-3.7
Commit: 040ac666ac5cdf9cd0a01a845f2ea0af3a81a08b
Parents: c98b648
Author: T Jake Luciani 
Authored: Mon Jun 6 14:28:41 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 6 14:28:41 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/040ac666/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 9f02a88..0fbb787 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.0.7) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 06 Jun 2016 14:27:28 -0400
+
 cassandra (3.0.6) unstable; urgency=medium
 
   * New release

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.6 [deleted] 8d22d9fd1

[jira] [Comment Edited] (CASSANDRA-7622) Implement virtual tables

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316878#comment-15316878
 ] 

Jeremiah Jordan edited comment on CASSANDRA-7622 at 6/6/16 5:58 PM:


The issue I see is that the "replication" of virtual tables is tied to the 
implementation class.  So you have to be very careful about the replication of 
the keyspace you put the table in.

Things I would want to do with virtual tables:
Should apply to the node the query is run on, or be able to specify the node:
1. Get/Set current configuration values
2. Get metrics
3. Pretty much anything we currently expose in JMX

For those three things I would either want queries to be local to the current 
node, or have the routing based on the IP someone queried.  Like "node" would 
be the PK and when someone queries for "select * from config_table;" it goes to 
all nodes, but "select * from config_table where node=1.2.3.4;" it goes only to 
node 1.2.3.4

Could be node local or not, completely depends on what you are doing:
1. Run application code to do something.  Stuff I would do here would probably 
be similar to things one might do in UDF/UDA but you would have inherently 
lower level/more access to the rest of C* as you wouldn't be confined by the 
security models those impose.

So at least for the things I can think of the "nicest" interface would be that 
virtual tables have some kind of "where should I get routed" function, so you 
could do custom routing.  But just having choices of "local/not local" can 
work.  But I see a lot of gotcha if you are going to allow them into any random 
keyspace and trying to follow the replication strategy of that keyspace when 
querying them.


was (Author: jjordan):
The issue I see is that the "replication" of virtual tables is tied to the 
implementation class.  So you have to be very careful about the replication of 
the keyspace you put the table in.

Things I would want to do with virtual tables:
Should apply to the node the query is run on, or be able to specify the node:
1. Get/Set current configuration values
2. Get metrics
3. Pretty much anything we currently expose in JMX

For those three things I would either want queries to be local to the current 
node, or have the routing based on the IP someone queried.  Like "node" would 
be the PK and when someone queries for "select * from config_table;" it goes to 
all nodes, but "select * from config_table where node=1.2.3.4;" it goes only to 
node 1.2.3.4

Could be node local or not, completely depends on what you are doing:
1. Run application code to do something.  Stuff I would do here would probably 
be similar to things one might do in UDF/UDA but you would have inherently 
lower level/more access to the rest of C* as you wouldn't be confined by the 
security models those impose.

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7622) Implement virtual tables

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316878#comment-15316878
 ] 

Jeremiah Jordan edited comment on CASSANDRA-7622 at 6/6/16 5:56 PM:


The issue I see is that the "replication" of virtual tables is tied to the 
implementation class.  So you have to be very careful about the replication of 
the keyspace you put the table in.

Things I would want to do with virtual tables:
Should apply to the node the query is run on, or be able to specify the node:
1. Get/Set current configuration values
2. Get metrics
3. Pretty much anything we currently expose in JMX

For those three things I would either want queries to be local to the current 
node, or have the routing based on the IP someone queried.  Like "node" would 
be the PK and when someone queries for "select * from config_table;" it goes to 
all nodes, but "select * from config_table where node=1.2.3.4;" it goes only to 
node 1.2.3.4

Could be node local or not, completely depends on what you are doing:
1. Run application code to do something.  Stuff I would do here would probably 
be similar to things one might do in UDF/UDA but you would have inherently 
lower level/more access to the rest of C* as you wouldn't be confined by the 
security models those impose.


was (Author: jjordan):
The issue I see is that the "replication" of virtual tables is tied to the 
implementation class.  So you have to be very careful about the replication of 
the keyspace you put the table in.

Things I would want to do with virtual tables:
Should apply to the node the query is run on, or be able to specify the node:
1. Get/Set current configuration values
2. Get metrics
3. Pretty much anything we currently expose in JMX

For those three things I would either want queries to be local to the current 
node, or have the routing based on the IP someone queried.  Like "node" would 
be the PK and when someone queries for select * from config_table;"

Could be node local or not, completely depends on what you are doing:
1. Run application code to do something.  Stuff I would do here would probably 
be similar to things one might do in UDF/UDA but you would have inherently 
lower level/more access to the rest of C* as you wouldn't be confined by the 
security models those impose.

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316878#comment-15316878
 ] 

Jeremiah Jordan commented on CASSANDRA-7622:


The issue I see is that the "replication" of virtual tables is tied to the 
implementation class.  So you have to be very careful about the replication of 
the keyspace you put the table in.

Things I would want to do with virtual tables:
Should apply to the node the query is run on, or be able to specify the node:
1. Get/Set current configuration values
2. Get metrics
3. Pretty much anything we currently expose in JMX

For those three things I would either want queries to be local to the current 
node, or have the routing based on the IP someone queried.  Like "node" would 
be the PK and when someone queries for select * from config_table;"

Could be node local or not, completely depends on what you are doing:
1. Run application code to do something.  Stuff I would do here would probably 
be similar to things one might do in UDF/UDA but you would have inherently 
lower level/more access to the rest of C* as you wouldn't be confined by the 
security models those impose.

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Brian Hess (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316875#comment-15316875
 ] 

 Brian Hess commented on CASSANDRA-7622:


It appears there are a few different things that could be accomplished with 
Virtual Tables, and I'm curious which one this ticket will focus on - and I'd 
suggest that we consider breaking out the full set of things below into 
separate tickets as I think some deserve more discussion/debate than others.
1. CQL access to internal metrics and management.  This would be access to 
metrics (such as JMX) and management settings (such as cassandra.yaml 
settings).  These could be covered by read-only access to built-in virtual 
tables (update/write access is below).
2. CQL access to updating management settings.  This is different than 
management actions (such as `nodetool flush` or repair).
3. CQL access to management actions.  This is the category of things such as 
`nodetool flush`, repair, drain, etc.
4. CQL access to arbitrary classes that implement user-defined behavior.

I think that 1 is the least controversial, and clearly of high value.  My 
suggestion is that this be built-in Virtual Tables, perhaps in system._vt_*.  
For example system._vt_metrics as SELECT access to the JMX metrics, or 
system._vt_settings as SELECT access to yaml/command-line/etc settings.  These 
would be predefined virtual tables.  I am concerned about how we will indicate 
how we specify which node's metrics we are querying, and how we route the query 
to that node (more on this below - it applies to all 4 (and more if we come up 
with more) scenarios).

I also think that 2 is not very controversial.  This would allow updating 
settings.  That could be, for example via a CQL UPDATE command (e.g., `UPDATE 
system._vt_settings SET max_hint_window_in_ms = 1080 WHERE node='1.2.3.4'` 
- for illustrative purposes).  The reason why this is different than 3 is that 
after doing the update, a read will return the same value, since it was 
changed.  Would we want to support `UPDATE system._vt_settings SET 
max_hint_window_in_ms = 1080` and update all nodes, which is not valid CQL 
since we don't specify a partition key?  

I'm not sure about 3, and I think it should be discussed further.  The thing is 
that something like `ALTER system_virtual.actions SET drain=true` or really 
`UPDATE system_virtual.actions SET drain=true` what would the value be if we 
SELECT it after the ALTER/UPDATE?  It feels like it would be set back to FALSE 
after the drain, which semantically feels strange.  I like [~snazy]'s 
suggestion of completely separate CQL syntax for these actions.  Maybe 
something like `EXECUTE ACTION drain ON '1.2.3.4'` - again, syntax just for 
illustrative purposes.  The semantics here is that the system is being altered, 
but outside a table, so a SELECT doesn't have the confusion as above.  As 
[~snazy] pointed out, we would also need to make sure we are careful about 
access to this capability (perhaps user/role permissions is sufficient, and we 
just need to add new permission types).

4 is probably the one that causes the most pause, IMO, as there would be 
arbitrary user code running in the read and write path.  We need to take care 
with that - as we have with UDFs.  It isn't exactly clear to me what the goal 
is for arbitrary virtual tables.  What are we trying to accomplish with this 
extension point?

As I mentioned above, I'm curious how we will route the query (SELECT, UPDATE, 
EXECUTE, whatever) to the right node.  In the simple case of SELECT of JMX 
metrics, how would we query the JMX metrics of node 1.2.3.4?  These 
queries/operations are per-node, not cluster-wide, so we need to think about 
that.  Is that the partition key of the virtual table (` PARTITION BY 
((node), ...)`)?  How would we route that internally in CQL (and with awareness 
on the driver)?  

It sounds like it would require a special partitioner (like system tables have 
- or maybe ByteOrderedPartitioner could work with a customer replication 
strategy, but with Murmur3Partitioner things are hard (it's essentially hash 
inversion)) and possibly replication strategy (I don't think that a replication 
strategy alone would cover this, right [~jjordan]).  If these are fully 
user-defined (as [~jjirsa] suggests), then we need to have a way to specify 
partitioners at the table (or at least keyspace) level, as opposed to cluster 
wide (which would actually be a nice change, IMO).  If they are built-in, then 
we can cover this case like we do with system keyspace tables.

Another approach may be to have the drivers deal with the routing.  The Virtual 
Tables could be special cases for routing purposes and have an API call to 
specify which node to query - or the LoadBalancingPolicy could inspect the 
Statement and route appropriately, which could work for any/all partitioners 
and/or replication 

[jira] [Updated] (CASSANDRA-11464) C* doesn't respond to OPTIONS request containing low protocol number

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-11464:
---
Fix Version/s: (was: 3.x)
   3.6

> C* doesn't respond to OPTIONS request containing low protocol number
> 
>
> Key: CASSANDRA-11464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11464
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Sandeep Tamhankar
>Assignee: Tyler Hobbs
> Fix For: 3.6, 3.0.6
>
>
> Observed in Ruby and Python drivers: if you send an OPTIONS message to C* 3.4 
> (though I believe this goes back to 3.0) with a protocol version of 1, C* 
> doesn't send a response to the client. If you specify a high protocol version 
> (e.g. 5), C* does correctly respond with a protocol error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316844#comment-15316844
 ] 

Jeff Jirsa commented on CASSANDRA-7622:
---

[~jjordan] - Is the KS replication strategy not sufficient (virtual tables 
still have a KS, which has a replication strategy)? What about putting options 
in the map and letting the underlying class setup custom replication 
accordingly? Or are you suggesting a more explicit virtual table option {{WITH 
virtual_table_replication=}} ?


> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11464) C* doesn't respond to OPTIONS request containing low protocol number

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-11464:
---
Fix Version/s: (was: 3.6)
   3.x

> C* doesn't respond to OPTIONS request containing low protocol number
> 
>
> Key: CASSANDRA-11464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11464
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Sandeep Tamhankar
>Assignee: Tyler Hobbs
> Fix For: 3.x, 3.0.6
>
>
> Observed in Ruby and Python drivers: if you send an OPTIONS message to C* 3.4 
> (though I believe this goes back to 3.0) with a protocol version of 1, C* 
> doesn't send a response to the client. If you specify a high protocol version 
> (e.g. 5), C* does correctly respond with a protocol error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11706) Tracing payload not passed through newSession(..)

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-11706:
---
Fix Version/s: (was: 3.6)
   3.x

> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
>  Labels: tracing
> Fix For: 3.x
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map 
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough 
> workaround it by also overriding the {{newSession(UUID sessionId, 
> Map customPayload)}} method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11663) dtest failure in upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test and upgrade_with_wide_partition_reversed_test

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita resolved CASSANDRA-11663.

Resolution: Duplicate

> dtest failure in 
> upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test
>  and upgrade_with_wide_partition_reversed_test
> --
>
> Key: CASSANDRA-11663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11663
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Yuki Morishita
>  Labels: dtest
> Attachments: node1.log, node1_debug.log
>
>
> including two tests here, look to be failing for the same reason, example 
> failures:
> http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_test
> http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_reversed_test/
> Failed on CassCI build trunk_dtest #1152



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11182) Enable SASI index for collections

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-11182:

Description: 
This is a follow up ticket for post Cassandra 3.4 SASI integration.

Right now it is possible with standard Cassandra 2nd index to:

1. index list and set elements ( {{WHERE list CONTAINS xxx}})
2. index map keys ( {{WHERE map CONTAINS KEYS 'abc'}} )
3. index map entries ( {{WHERE map\['key'\]=value}})

 It would be nice to enable these features in SASI too.

 With regard to tokenizing, we might want to allow wildcards ({{%}}) with the 
CONTAINS syntax as well as with index map entries. Ex:

* {{WHERE list CONTAINS 'John%'}}
* {{WHERE map CONTAINS KEY '%an%'}}
* {{WHERE map\['key'\] LIKE '%val%'}}


/cc [~xedin] [~rustyrazorblade] [~jkrupan]

  was:
This is a follow up ticket for post Cassandra 3.4 SASI integration.

Right now it is possible with standard Cassandra 2nd index to:

1. index list and set elements ( {{WHERE list CONTAINS xxx}})
2. index map keys ( {{WHERE map CONTAINS KEYS 'abc' }} )
3. index map entries ( {{WHERE map\['key'\]=value}})

 It would be nice to enable these features in SASI too.

 With regard to tokenizing, we might want to allow wildcards ({{%}}) with the 
CONTAINS syntax as well as with index map entries. Ex:

* {{WHERE list CONTAINS 'John%'}}
* {{WHERE map CONTAINS KEY '%an%'}}
* {{WHERE map\['key'\] LIKE '%val%'}}


/cc [~xedin] [~rustyrazorblade] [~jkrupan]


> Enable SASI index for collections
> -
>
> Key: CASSANDRA-11182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11182
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: DOAN DuyHai
>Assignee: Alex Petrov
>Priority: Minor
>
> This is a follow up ticket for post Cassandra 3.4 SASI integration.
> Right now it is possible with standard Cassandra 2nd index to:
> 1. index list and set elements ( {{WHERE list CONTAINS xxx}})
> 2. index map keys ( {{WHERE map CONTAINS KEYS 'abc'}} )
> 3. index map entries ( {{WHERE map\['key'\]=value}})
>  It would be nice to enable these features in SASI too.
>  With regard to tokenizing, we might want to allow wildcards ({{%}}) with the 
> CONTAINS syntax as well as with index map entries. Ex:
> * {{WHERE list CONTAINS 'John%'}}
> * {{WHERE map CONTAINS KEY '%an%'}}
> * {{WHERE map\['key'\] LIKE '%val%'}}
> /cc [~xedin] [~rustyrazorblade] [~jkrupan]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r13907 - in /release/cassandra: 3.0.5/ 3.6/ debian/dists/36x/ debian/dists/36x/main/ debian/dists/36x/main/binary-amd64/ debian/dists/36x/main/binary-i386/ debian/dists/36x/main/source/ de

[jake]

Author: jake
Date: Mon Jun  6 16:40:14 2016
New Revision: 13907

Log:
3.6 rel

Added:
release/cassandra/3.6/
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz   (with props)
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.md5
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.sha1
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.md5
release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.sha1
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz   (with props)
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz.asc
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz.asc.md5
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz.asc.sha1
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz.md5
release/cassandra/3.6/apache-cassandra-3.6-src.tar.gz.sha1
release/cassandra/debian/dists/36x/
release/cassandra/debian/dists/36x/InRelease
release/cassandra/debian/dists/36x/Release
release/cassandra/debian/dists/36x/Release.gpg
release/cassandra/debian/dists/36x/main/
release/cassandra/debian/dists/36x/main/binary-amd64/
release/cassandra/debian/dists/36x/main/binary-amd64/Packages
release/cassandra/debian/dists/36x/main/binary-amd64/Packages.gz   (with 
props)
release/cassandra/debian/dists/36x/main/binary-amd64/Release
release/cassandra/debian/dists/36x/main/binary-i386/
release/cassandra/debian/dists/36x/main/binary-i386/Packages
release/cassandra/debian/dists/36x/main/binary-i386/Packages.gz   (with 
props)
release/cassandra/debian/dists/36x/main/binary-i386/Release
release/cassandra/debian/dists/36x/main/source/
release/cassandra/debian/dists/36x/main/source/Release
release/cassandra/debian/dists/36x/main/source/Sources.gz   (with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_3.6_all.deb  
 (with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.6.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.6.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.6.orig.tar.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.6.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_3.6_all.deb   
(with props)
Removed:
release/cassandra/3.0.5/

Added: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc
==
--- release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc (added)
+++ release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc Mon Jun  6 
16:40:14 2016
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAABAgAGBQJXTdATAAoJEHSdbuwDU7Esp84QAISatICbn1eR4LSt4bzSfhO9
+XcucYt/Y2i5kpvr5EE+IApQgYV2nTxERUT7XU8VhcxFqNW5It7r+sMD5N/Yej9j1
+ygsgbeigOC49MoDqAqNgP6er05HVyummq9rRJxCzwNx+fpRaTIoM3DmzZe17vHJd
+c7wXer8TB180dEfNm6quJP39S53g2CmsJQqxkxBubO/GMADEyEJVOhweJj/Yj3l3
+wQ7JL8gvuBk3t18zXpM3uIbj5cXKyrTPxSEEdNsX6vVYWpp2Lv6wWQumOr6NlC1z
+El2RAR3aehw7CA4FanCLYBnKkE4iSdDoGoSCzi4eRQWJciFzXXDOOU9JwT19D2fW
+zmz2hM8uzsqWWqqNiuHCpvEght6MABB7sLuZYr+uI7BdJaiC5m9HppJRUakECTAw
+wmxYmuJZ3FRNukiF7WbyEjaMBiB1MCfWH7uUPogPTOcDxNMHoYJHCB/gSSVnHZwF
+/qtFj7ve5IxlCDwGq6JJ8Ikp9ycqwu2i5lwElcTVDJTH4Ll53fwGOq0x1IkBlY0C
+ntp9RfmuiDIG1EwXsZyOtCza2RVw3DrRunN8FgRhjq+fRGHAeQLixvi6WbzapkdO
+gTMIaBDKUATimaKkfC6QSMkmE6rEgU/2P3bz+s8smcOEVg5TmF1eo49VvBUsO2Fz
+6zqXFjsR2pAEyXDXufPQ
+=8MQb
+-END PGP SIGNATURE-

Added: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.md5
==
--- release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.md5 (added)
+++ release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.md5 Mon Jun  6 
16:40:14 2016
@@ -0,0 +1 @@
+a73213bbfcdfcd0f4c5abce9207f07d7
\ No newline at end of file

Added: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.sha1
==
--- release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.sha1 (added)
+++ release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.asc.sha1 Mon Jun  6 
16:40:14 2016
@@ -0,0 +1 @@
+63548d836504ed82166e90b4c81c0a1d3892e182
\ No newline at end of file

Added: release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.md5
==
--- release/cassandra/3.6/apache-cassandra-3.6-bin.tar.gz.md5 (added)
+++ 

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.6-tentative [deleted] 8d22d9fd1

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/cassandra-3.6 [created] dd4c2ef58

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316722#comment-15316722
 ] 

Jeremiah Jordan commented on CASSANDRA-7622:


The replication of the virtual tables needs to be exposed as well.  Some 
virtual tables would be local only, but I can definitely see others that would 
possibly get stuff from other nodes/queries should be forwarded to other nodes. 
 So they need some kind of interaction with a custom replication strategy which 
they can setup.

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316717#comment-15316717
 ] 

Jeff Jirsa commented on CASSANDRA-7622:
---

Thanks [~blerer] - {{CREATE VIRTUAL TABLE}} with {{class}} in 
{{virtual_table_options}} seems consistent with compaction and easy to 
understand. I do believe it's possible for the virtual table to accomplish this 
with {{QueryState state, QueryOptions options}} arguments, but it would have a 
lot of non-trivial, duplicate code in each virtual table, and I agree there's a 
better way. Will refine the proposal.



> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11875) Create sstableconvert tool with support to ma format

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316642#comment-15316642
 ] 

Paulo Motta commented on CASSANDRA-11875:
-

Thanks for the update! See follow-up comments below:

bq. Changed isCompatibleForWriting(String version) -> isCompatibleForWriting() 
since we are dealing directly with versions so it would be redundant to pass it.

Good idea, that's probably better.

bq. 
DatabaseDescriptor.getSSTableFormat().info.getVersion(options.version).isCompatibleForWriting(),
 or maybe there's some way to obtain it?

Maybe you can add a {{BigVersion.supportsWritingversion(version)}} that could 
wrap that.

bq. I modified Options access level to protected and parseArgs now varies on 
boolean isLatestVersion, so now StandaloneUpgrader only contains the execution 
code which is identical than StandaloneConverter, do you think we should 
abstract it into a common method SSTableConverter.execute(Option options)?

The idea is to abstract only the identical part, leaving specific parsing to 
each class (so you can probably extract the {{parseArgs}} code from inside 
{{Options}}). The main class of StandaloneUpgrader would look more or less like:

{code:java}
public static void main(String args[])
{
Options opts = parseArgs(args);
StandaloneConverter converter = new StandaloneConverter(opts)
converter.convert()
}
{code}

With this approach {{StandaloneUpgrader}} doesn't need to be a subclass of 
{{StandaloneConverter}}.

On tests, it seems you're using {{assertNotSame}} to check objects are not 
equals, but {{assertNotSame}} only compare object equality ({{obj1 != obj2}}), 
not if objects are effectively not equal ({{!obj1.equals(obj2)}}). You should 
probably use {{assertFalse(object.equals(other))}} to assert non-equality;

The {{testUnsupportedVersionShouldFail}} is failing, you should generally use 
this format to assert that exceptions are thrown while making the test pass:

{code:java}
try {
methodThatShouldThrow();
fail( "My method didn't throw when I expected it to" );
} catch (MyException expectedException) {
}
{code}

> Create sstableconvert tool with support to ma format
> 
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Paulo Motta
>Assignee: Kaide Mu
>Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes 
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it 
> receive a {{target}} version and probably also rename it to 
> {{SSTableConverter}}. 
> Based on this we can create an {{StandaloneDowngrader}} tool which will 
> perform downgrade of specified sstables to a target version. To start with, 
> we should support only downgrading to {{ma}} format (from current format 
> {{mb}}), downgrade to any other version should be forbidden. Since we already 
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the 
> current format, perform the downgrade, and verify data in the new format is 
> correct. This will be the base tests suite for more advanced conversions in 
> the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11961) Nonfatal NPE in CompactionMetrics

[Mahdi Mohammadi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahdi Mohammadi updated CASSANDRA-11961:

Labels: lhf  (was: )

> Nonfatal NPE in CompactionMetrics
> -
>
> Key: CASSANDRA-11961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11961
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Minor
>  Labels: lhf
>
> Just saw the following NPE on trunk. Means, that {{metaData}} from 
> {{CFMetaData metaData = compaction.getCompactionInfo().getCFMetaData();}} is 
> {{null}}. A simple {{if (metaData == null) continue;}} should fix this.
> {code}
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:103)
>  ~[main/:na]
>   at 
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:78)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11886) Streaming will miss sections for early opened sstables during compaction

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316488#comment-15316488
 ] 

Benedict commented on CASSANDRA-11886:
--

That makes the cardinality of the set arbitrarily large, so this probably is a 
good change to make now.

LGTM, +1

> Streaming will miss sections for early opened sstables during compaction
> 
>
> Key: CASSANDRA-11886
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11886
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefan Podkowinski
>Assignee: Marcus Eriksson
>Priority: Critical
>  Labels: correctness, repair, streaming
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 9700-test-2_1.patch
>
>
> Once validation compaction has been finished, all mismatching sstable 
> sections for a token range will be used for streaming as return by 
> {{StreamSession.getSSTableSectionsForRanges}}. Currently 2.1 will try to 
> restrict the sstable candidates by checking if they can be found in 
> {{CANONICAL_SSTABLES}} and will ignore them otherwise. At the same time 
> {{IntervalTree}} in the {{DataTracker}} will be build based on replaced 
> non-canonical sstables as well. In case of early opened sstables this becomes 
> a problem, as the tree will be update with {{OpenReason.EARLY}} replacements 
> that cannot be found in canonical. But whenever 
> {{getSSTableSectionsForRanges}} will get a early instance from the view, it 
> will fail to retrieve the corresponding canonical version from the map, as 
> the different generation will cause a hashcode mismatch. Please find a test 
> attached.
> As a consequence not all sections for a range are streamed. In our case this 
> has caused deleted data to reappear, as sections holding tombstones were left 
> out due to this behavior.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10993) Make read and write requests paths fully non-blocking, eliminate related stages

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316423#comment-15316423
 ] 

Joshua McKenzie edited comment on CASSANDRA-10993 at 6/6/16 12:33 PM:
--

Just skimming, and with total acknowledgement that it's a PoC and super rough, 
but 
[this|https://github.com/thobbs/cassandra/blob/CASSANDRA-10993-WIP/src/java/org/apache/cassandra/poc/WriteTask.java#L264]
 caught my eye.

I'd expect that to return true if success, else false.

edit: I say "super rough", but that's not to mean the code doesn't look clean 
and well organized.


was (Author: joshuamckenzie):
Just skimming, and with total acknowledgement that it's a PoC and super rough, 
but 
[this|https://github.com/thobbs/cassandra/blob/CASSANDRA-10993-WIP/src/java/org/apache/cassandra/poc/WriteTask.java#L264]
 caught my eye.

I'd expect that to return true if success, else false.

> Make read and write requests paths fully non-blocking, eliminate related 
> stages
> ---
>
> Key: CASSANDRA-10993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10993
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.x
>
>
> Building on work done by [~tjake] (CASSANDRA-10528), [~slebresne] 
> (CASSANDRA-5239), and others, convert read and write request paths to be 
> fully non-blocking, to enable the eventual transition from SEDA to TPC 
> (CASSANDRA-10989)
> Eliminate {{MUTATION}}, {{COUNTER_MUTATION}}, {{VIEW_MUTATION}}, {{READ}}, 
> and {{READ_REPAIR}} stages, move read and write execution directly to Netty 
> context.
> For lack of decent async I/O options on Linux, we’ll still have to retain an 
> extra thread pool for serving read requests for data not residing in our page 
> cache (CASSANDRA-5863), however.
> Implementation-wise, we only have two options available to us: explicit FSMs 
> and chained futures. Fibers would be the third, and easiest option, but 
> aren’t feasible in Java without resorting to direct bytecode manipulation 
> (ourselves or using [quasar|https://github.com/puniverse/quasar]).
> I have seen 4 implementations bases on chained futures/promises now - three 
> in Java and one in C++ - and I’m not convinced that it’s the optimal (or 
> sane) choice for representing our complex logic - think 2i quorum read 
> requests with timeouts at all levels, read repair (blocking and 
> non-blocking), and speculative retries in the mix, {{SERIAL}} reads and 
> writes.
> I’m currently leaning towards an implementation based on explicit FSMs, and 
> intend to provide a prototype - soonish - for comparison with 
> {{CompletableFuture}}-like variants.
> Either way the transition is a relatively boring straightforward refactoring.
> There are, however, some extension points on both write and read paths that 
> we do not control:
> - authorisation implementations will have to be non-blocking. We have control 
> over built-in ones, but for any custom implementation we will have to execute 
> them in a separate thread pool
> - 2i hooks on the write path will need to be non-blocking
> - any trigger implementations will not be allowed to block
> - UDFs and UDAs
> We are further limited by API compatibility restrictions in the 3.x line, 
> forbidding us to alter, or add any non-{{default}} interface methods to those 
> extension points, so these pose a problem.
> Depending on logistics, expecting to get this done in time for 3.4 or 3.6 
> feature release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10993) Make read and write requests paths fully non-blocking, eliminate related stages

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316423#comment-15316423
 ] 

Joshua McKenzie commented on CASSANDRA-10993:
-

Just skimming, and with total acknowledgement that it's a PoC and super rough, 
but 
[this|https://github.com/thobbs/cassandra/blob/CASSANDRA-10993-WIP/src/java/org/apache/cassandra/poc/WriteTask.java#L264]
 caught my eye.

I'd expect that to return true if success, else false.

> Make read and write requests paths fully non-blocking, eliminate related 
> stages
> ---
>
> Key: CASSANDRA-10993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10993
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Local Write-Read Paths
>Reporter: Aleksey Yeschenko
>Assignee: Tyler Hobbs
> Fix For: 3.x
>
>
> Building on work done by [~tjake] (CASSANDRA-10528), [~slebresne] 
> (CASSANDRA-5239), and others, convert read and write request paths to be 
> fully non-blocking, to enable the eventual transition from SEDA to TPC 
> (CASSANDRA-10989)
> Eliminate {{MUTATION}}, {{COUNTER_MUTATION}}, {{VIEW_MUTATION}}, {{READ}}, 
> and {{READ_REPAIR}} stages, move read and write execution directly to Netty 
> context.
> For lack of decent async I/O options on Linux, we’ll still have to retain an 
> extra thread pool for serving read requests for data not residing in our page 
> cache (CASSANDRA-5863), however.
> Implementation-wise, we only have two options available to us: explicit FSMs 
> and chained futures. Fibers would be the third, and easiest option, but 
> aren’t feasible in Java without resorting to direct bytecode manipulation 
> (ourselves or using [quasar|https://github.com/puniverse/quasar]).
> I have seen 4 implementations bases on chained futures/promises now - three 
> in Java and one in C++ - and I’m not convinced that it’s the optimal (or 
> sane) choice for representing our complex logic - think 2i quorum read 
> requests with timeouts at all levels, read repair (blocking and 
> non-blocking), and speculative retries in the mix, {{SERIAL}} reads and 
> writes.
> I’m currently leaning towards an implementation based on explicit FSMs, and 
> intend to provide a prototype - soonish - for comparison with 
> {{CompletableFuture}}-like variants.
> Either way the transition is a relatively boring straightforward refactoring.
> There are, however, some extension points on both write and read paths that 
> we do not control:
> - authorisation implementations will have to be non-blocking. We have control 
> over built-in ones, but for any custom implementation we will have to execute 
> them in a separate thread pool
> - 2i hooks on the write path will need to be non-blocking
> - any trigger implementations will not be allowed to block
> - UDFs and UDAs
> We are further limited by API compatibility restrictions in the 3.x line, 
> forbidding us to alter, or add any non-{{default}} interface methods to those 
> extension points, so these pose a problem.
> Depending on logistics, expecting to get this done in time for 3.4 or 3.6 
> feature release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316329#comment-15316329
 ] 

Benjamin Lerer commented on CASSANDRA-7622:
---

Thanks for the proposal. I think It would be nice to have one with more 
details. For example, some description of how it would be implemented for 
metrics will help.

For the moment, I do not see how the interface that you proposed can handle 
{{SELECT}} queries. What about: paging, restrictions, ordering, limit, ... We 
might not want to support everything but a minimum has to be supported. It 
might be worth to specify it. 

It will also be interesting to know what should be provided by Cassandra and 
what must be implemented by the virtual table provider.

>From the syntax point of view I think that we should use {{CREATE VIRTUAL 
>TABLE ...}}. It is clearer and it will allow us to handle it separatly in the 
>ANTLR file. 

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11962) Non-fatal NPE during concurrent repair check

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-11962:


 Summary: Non-fatal NPE during concurrent repair check
 Key: CASSANDRA-11962
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11962
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Priority: Minor


"Usual" checks for multiple repairs result in this exception in the log file:
{code}
ERROR [ValidationExecutor:6] 2016-06-06 07:56:23,530 CassandraDaemon.java:222 - 
Exception in thread Thread[ValidationExecutor:6,1,main]
java.lang.RuntimeException: Cannot start multiple repair sessions over the same 
sstables
at 
org.apache.cassandra.db.compaction.CompactionManager.getSSTablesToValidate(CompactionManager.java:1325)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1215)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:81)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager$11.call(CompactionManager.java:844)
 ~[main/:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_91]
{code}

However, I saw this one:

{code}
ERROR [ValidationExecutor:6] 2016-06-06 07:56:25,002 CassandraDaemon.java:222 - 
Exception in thread Thread[ValidationExecutor:6,1,main]
java.lang.NullPointerException: null
at 
org.apache.cassandra.service.ActiveRepairService$ParentRepairSession.getActiveSSTables(ActiveRepairService.java:495)
 ~[main/:na]
at 
org.apache.cassandra.service.ActiveRepairService$ParentRepairSession.access$300(ActiveRepairService.java:451)
 ~[main/:na]
at 
org.apache.cassandra.service.ActiveRepairService.currentlyRepairing(ActiveRepairService.java:338)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager.getSSTablesToValidate(CompactionManager.java:1320)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1215)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:81)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager$11.call(CompactionManager.java:844)
 ~[main/:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_91]
{code}

Looks like there is no entry for {{cfId}} in {{getActiveSStables}} at {{for 
(SSTableReader sstable : columnFamilyStores.get(cfId).getSSTables())}}.

(against trunk)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11749) CQLSH gets SSL exception following a COPY FROM

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316239#comment-15316239
 ] 

Stefania commented on CASSANDRA-11749:
--

Thanks for the review. You're correct regarding Windows dtests: I aborted them 
before the {{PlainTextAuthProvider}} revert, and then forgot to restart them.

I've just started them now, I will commit if the results on Windows are also 
good.

> CQLSH gets SSL exception following a COPY FROM
> --
>
> Key: CASSANDRA-11749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11749
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: driver_debug.txt, stdout.txt.zip, 
> stdout_single_process.txt.zip
>
>
> When running Cassandra and cqlsh with SSL, the following command occasionally 
> results in the exception below:
> {code}
> cqlsh --ssl -f kv.cql
> {code}
> {code}
> ERROR [SharedPool-Worker-2] 2016-05-11 12:41:03,583 Message.java:538 - 
> Unexpected exception during request; channel = [id: 0xeb75e05d, 
> /127.0.0.1:51083 => /127.0.0.1:9042]
> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: bad 
> record MAC
> at 
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) 
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) 
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: javax.net.ssl.SSLException: bad record MAC
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) 
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) 
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:981) 
> ~[na:1.8.0_91]
> at 
> sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) 
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) 
> ~[na:1.8.0_91]
> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_91]
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:982) 
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:908) 
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854) 
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at 
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
>  ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> ... 10 common frames omitted
> Caused by: javax.crypto.BadPaddingException: bad record MAC
> at sun.security.ssl.InputRecord.decrypt(InputRecord.java:219) 
> ~[na:1.8.0_91]
> at 
> sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:177) 
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:974) 
> ~[na:1.8.0_91]
> ... 17 common frames omitted
> {code}
> where
> {code}
> cat kv.cql 
> create keyspace if not exists cvs_copy_ks with replication = {'class': 
> 'SimpleStrategy', 'replication_factor':1};
> create table if not exists cvs_copy_ks.kv (key int primary key, value text);
> truncate cvs_copy_ks.kv;
> copy cvs_copy_ks.kv (key, value) from 'kv.csv' with header='true';
> select * from cvs_copy_ks.kv;
> drop keyspace cvs_copy_ks;
> 

[jira] [Commented] (CASSANDRA-11765) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316236#comment-15316236
 ] 

Stefania commented on CASSANDRA-11765:
--

dtest pull request is 
[here|https://github.com/riptano/cassandra-dtest/pull/1022].

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11765
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Philip Thompson
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.7
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> Example failure:
> http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/
> The test is seeing a corrupt hint sstable after upgrade from 2.2.5 to 3.6. 
> Relevant stack trace is
> {code}
> ERROR [main] 2016-05-11 16:22:25,180 CassandraDaemon.java:727 - Exception 
> encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) 
> [main/:na]
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_51]
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
> ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115)
>  ~[main/:na]
>   ... 5 common frames omitted
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: 
> Corrupted: 
> /mnt/tmp/dtest-X7IReF/test/node1/data2/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-3-big-Data.db
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:463)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:805)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)