[jira] [Commented] (CASSANDRA-13127) Materialized Views: View row expires too soon

2017-01-24 Thread Vincent White (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837348#comment-15837348
 ] 

Vincent White commented on CASSANDRA-13127:
---

I think I pretty much understand what's happening here. It basically all stems 
from the base upsert behaviour  (creating a row via {{UPDATE}} so the primary 
key columns don't exist on their own vs {{INSERT}}). I'm still not sure it 
matches the MV docs though and the comments in the code say things like:
{code}1) either the columns for the base and view PKs are exactly the same: in 
that case, the view entry should live as long as the base row lives. This means 
the view entry should only expire once *everything* in the base row has 
expired. Which means the row TTL should be the max of any other TTL.{code} I 
think the logic in {{computeLivenessInfoForEntry}} doesn't make sense for 
updates because it only ever expected inserts. It leads to some funky behaviour 
if you're mixing updates, inserts and TTL's. I didn't test with deletes but I 
guees they could cause similar results.

Simply patching computeLivenessInfoForEntry like:
{code:title=ViewUpdateGenerator.java#computeLivenessInfoForEntry}
int expirationTime = baseLiveness.localExpirationTime();
for (Cell cell : baseRow.cells())
{

-if (cell.ttl() > ttl)
+if (cell.localDeletionTime() > expirationTime)
{
ttl = cell.ttl();
expirationTime = cell.localDeletionTime();
}
}
-return ttl == baseLiveness.ttl()
+return expirationTime == baseLiveness.localExpirationTime()
 ? baseLiveness
 : LivenessInfo.withExpirationTime(baseLiveness.timestamp(), 
ttl, expirationTime);
}
{code} isn't enough because it leads to further unexpected behaviour where 
update statements will resurrect previously TTL'd MV entries in some cases. If 
an update statement sets a column that could cause the update of _any_ view in 
that keyspace it will resurrect entries in views that have PK's made up of only 
columns from the base PK, regardless of whether the statement updates non-PK 
columns in that view. If the update statement only sets values of columns that 
don't appear in the keyspace's MV's then no MV TTL'd entries for that PK will 
be resurrected. If there was never an entry in the MV for that MV PK then it 
won't create a a new one. This is because upserts don't create new MV entries 
unless they set the value of a non-PK column in that view (with or without this 
patch).

I don't think I've seen it referenced anywhere but is that intended behaviour 
when using upserts and materialized views? That an {{UPDATE}} to a column not 
in a view will not create an entry in an MV if the veiw's PK is only made up of 
columns from the base table's PK, but the matching {{INSERT}} statement will?

> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Duarte Nunes
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" | 
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT 
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
>  p | c | ttl(v)
> ---+---+
>  0 | 0 |  7
> (1 rows)
>  c | p
> ---+---
>  0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
>  p | c | ttl(v)
> ---+---+
>  0 | 0 |  3
> (1 rows)
>  c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I 
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry 
> the TTLs are compared instead of the expiration times, but I'm not sure I'm 
> getting that far ahead in the code when updating a column that's not in the 
> view.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12664) GCCompactionTest is flaky

2017-01-24 Thread Stefania (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12664:
-
Reviewer: Stefania  (was: Stefania Alborghetti)

> GCCompactionTest is flaky
> -
>
> Key: CASSANDRA-12664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12664
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Branimir Lambov
>Priority: Minor
> Fix For: 3.x
>
>
> {{GCCompactionTest}} was introduced by CASSANDRA-7019 and appears to be 
> flaky, see for example 
> [here|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3/GcCompactionTest/testGcCompactionStatic/].
>  
> I think it's the same root cause as CASSANDRA-12282: the tables in the test 
> keyspace are dropped asynchronously after each test, and this might cause 
> additional flush operations for all dirty tables in the keyspace. See the 
> [callstack|https://issues.apache.org/jira/browse/CASSANDRA-12282?focusedCommentId=15399098=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15399098]
>  in 12282. 
> A possible solution is to use KEYSPACE_PER_TEST, which is instead dropped 
> synchronously.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12664) GCCompactionTest is flaky

2017-01-24 Thread Stefania (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837328#comment-15837328
 ] 

Stefania commented on CASSANDRA-12664:
--

+1 !

> GCCompactionTest is flaky
> -
>
> Key: CASSANDRA-12664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12664
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Branimir Lambov
>Priority: Minor
> Fix For: 3.x
>
>
> {{GCCompactionTest}} was introduced by CASSANDRA-7019 and appears to be 
> flaky, see for example 
> [here|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3/GcCompactionTest/testGcCompactionStatic/].
>  
> I think it's the same root cause as CASSANDRA-12282: the tables in the test 
> keyspace are dropped asynchronously after each test, and this might cause 
> additional flush operations for all dirty tables in the keyspace. See the 
> [callstack|https://issues.apache.org/jira/browse/CASSANDRA-12282?focusedCommentId=15399098=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15399098]
>  in 12282. 
> A possible solution is to use KEYSPACE_PER_TEST, which is instead dropped 
> synchronously.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12664) GCCompactionTest is flaky

2017-01-24 Thread Stefania (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12664:
-
Status: Ready to Commit  (was: Patch Available)

> GCCompactionTest is flaky
> -
>
> Key: CASSANDRA-12664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12664
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Branimir Lambov
>Priority: Minor
> Fix For: 3.x
>
>
> {{GCCompactionTest}} was introduced by CASSANDRA-7019 and appears to be 
> flaky, see for example 
> [here|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3/GcCompactionTest/testGcCompactionStatic/].
>  
> I think it's the same root cause as CASSANDRA-12282: the tables in the test 
> keyspace are dropped asynchronously after each test, and this might cause 
> additional flush operations for all dirty tables in the keyspace. See the 
> [callstack|https://issues.apache.org/jira/browse/CASSANDRA-12282?focusedCommentId=15399098=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15399098]
>  in 12282. 
> A possible solution is to use KEYSPACE_PER_TEST, which is instead dropped 
> synchronously.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12664) GCCompactionTest is flaky

2017-01-24 Thread Branimir Lambov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837316#comment-15837316
 ] 

Branimir Lambov commented on CASSANDRA-12664:
-

Multiplexed run was also clean.

> GCCompactionTest is flaky
> -
>
> Key: CASSANDRA-12664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12664
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Branimir Lambov
>Priority: Minor
> Fix For: 3.x
>
>
> {{GCCompactionTest}} was introduced by CASSANDRA-7019 and appears to be 
> flaky, see for example 
> [here|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3/GcCompactionTest/testGcCompactionStatic/].
>  
> I think it's the same root cause as CASSANDRA-12282: the tables in the test 
> keyspace are dropped asynchronously after each test, and this might cause 
> additional flush operations for all dirty tables in the keyspace. See the 
> [callstack|https://issues.apache.org/jira/browse/CASSANDRA-12282?focusedCommentId=15399098=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15399098]
>  in 12282. 
> A possible solution is to use KEYSPACE_PER_TEST, which is instead dropped 
> synchronously.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13151) Unicode unittest fail

2017-01-24 Thread Jay Zhuang (JIRA) 
Jay Zhuang created CASSANDRA-13151:
--

 Summary: Unicode unittest fail
 Key: CASSANDRA-13151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13151
 Project: Cassandra
  Issue Type: Bug
Reporter: Jay Zhuang


Following unittests are failed in 3.11 and trunk(4.0) branch
{noformat}
SASIIndexTest.testUnicodeSupport
StandardAnalyzerTest.testTokenizationJaJp1
StandardAnalyzerTest.testTokenizationJaJp2
StandardAnalyzerTest.testTokenizationRuRu1
StandardAnalyzerTest.testTokenizationZnTw1
{noformat}
It works fine on my local mac, but not linux server. I guess it's related to 
Unicode setting. Does anyone have any idea on that? (could it be related to 
CASSANDRA-11077, CASSANDRA-11431?)

Here are the failure details
{noformat}
$ ant testsome -Dtest.name=org.apache.cassandra.index.sasi.SASIIndexTest 
-Dtest.methods=testUnicodeSupport
...
[junit] Testcase: 
testUnicodeSupport(org.apache.cassandra.index.sasi.SASIIndexTest):FAILED
[junit] []
[junit] junit.framework.AssertionFailedError: []
[junit] at 
org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1159)
[junit] at 
org.apache.cassandra.index.sasi.SASIIndexTest.testUnicodeSupport(SASIIndexTest.java:1122)
{noformat}
{noformat}
$ ant test -Dtest.name=StandardAnalyzerTest
...
[junit] Testcase: 
testTokenizationJaJp1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
 FAILED
[junit] expected:<210> but was:<0>
[junit] junit.framework.AssertionFailedError: expected:<210> but was:<0>
[junit] at 
org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp1(StandardAnalyzerTest.java:85)
[junit]
[junit]
[junit] Testcase: 
testTokenizationJaJp2(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
 FAILED
[junit] expected:<57> but was:<9>
[junit] junit.framework.AssertionFailedError: expected:<57> but was:<9>
[junit] at 
org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationJaJp2(StandardAnalyzerTest.java:104)
[junit]
[junit]
[junit] Testcase: 
testTokenizationRuRu1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
 FAILED
[junit] expected:<456> but was:<0>
[junit] junit.framework.AssertionFailedError: expected:<456> but was:<0>
[junit] at 
org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationRuRu1(StandardAnalyzerTest.java:120)
[junit]
[junit]
[junit] Testcase: 
testTokenizationZnTw1(org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest):
 FAILED
[junit] expected:<963> but was:<0>
[junit] junit.framework.AssertionFailedError: expected:<963> but was:<0>
[junit] at 
org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest.testTokenizationZnTw1(StandardAnalyzerTest.java:136)
[junit]
[junit]
[junit] Test org.apache.cassandra.index.sasi.analyzer.StandardAnalyzerTest 
FAILED
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10689) java.lang.OutOfMemoryError: Direct buffer memory

2017-01-24 Thread Jacob Willoughby (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837162#comment-15837162
 ] 

Jacob Willoughby commented on CASSANDRA-10689:
--

This also affected us.

We are running Cassandra 2.2.8 with java 8.  Setting maxCachedBufferSize fixed 
the issue.

It seems like the cassandra-env.sh script could be updated to detect and set 
this if needed?

> java.lang.OutOfMemoryError: Direct buffer memory
> 
>
> Key: CASSANDRA-10689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10689
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mlowicki
>
> {code}
> ERROR [SharedPool-Worker-63] 2015-11-11 17:53:16,161 
> JVMStabilityInspector.java:117 - JVM state determined to be unstable.  
> Exiting forcefully due to:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_80]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123) 
> ~[na:1.7.0_80]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306) 
> ~[na:1.7.0_80]
> at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174) 
> ~[na:1.7.0_80]
> at sun.nio.ch.IOUtil.read(IOUtil.java:195) ~[na:1.7.0_80]
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:149) 
> ~[na:1.7.0_80]
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:104)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]  
> at 
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:310)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:64)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1894)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.columniterator.IndexedSliceReader.setToRowStart(IndexedSliceReader.java:107)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.columniterator.IndexedSliceReader.(IndexedSliceReader.java:83)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:42)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:246)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:270)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1994)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1837)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_80]
> at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-13150) Error in DDL documentation clustering columns example

2017-01-24 Thread Adrian Hempel (JIRA) 
Adrian Hempel created CASSANDRA-13150:
-

 Summary: Error in DDL documentation clustering columns example
 Key: CASSANDRA-13150
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13150
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation and Website
Reporter: Adrian Hempel


The following line:

https://git1-us-west.apache.org/repos/asf?p=cassandra.git;a=blob;f=doc/source/cql/ddl.rst;h=fb97e54388b4cea5b84b9eb533904e43f0b18ec7;hb=HEAD#l362

Should read:

{{PRIMARY KEY (a, b, c)}}

rather than:

{{PRIMARY KEY (a, c, d)}}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

2017-01-24 Thread Kurt Greaves (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-13142:
-
Description: 
Since at least 1.2 upgradesstables will cancel any compactions bar validations 
when run. This was originally determined as a non-issue in CASSANDRA-3430 
however can be quite annoying (especially with STCS) as a compaction will 
output the new version anyway. Furthermore, as per CASSANDRA-12243 it also 
stops things like view builds and I assume secondary index builds as well which 
is not ideal.

We should avoid cancelling compactions unnecessarily.

  was:
Since at least 1.2 upgradesstables will cancel any compactions bar validations 
when run. This was originally determined as a non-issue in CASSANDRA-3430 
however can be quite annoying (especially with STCS) as a compaction will 
output the new version anyway. Furthermore, as per CASSANDRA-12243 it also 
stops things like view builds and I assume secondary index builds as well which 
is not ideal.

We should avoid cancelling compactions unnecessarily.

I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
for debate. The main negative effect is annoying operators (which I tend to 
rate pretty highly), but other than that there are no serious consequences.


> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

2017-01-24 Thread Kurt Greaves (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-13142:
-
Priority: Major  (was: Minor)

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily

2017-01-24 Thread Kurt Greaves (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837117#comment-15837117
 ] 

Kurt Greaves commented on CASSANDRA-13142:
--

+1000, I didn't consider the case where builds/compactions are not re-triggered 
automatically. I think that's sufficient to bump it up to major.

> Upgradesstables cancels compactions unnecessarily
> -
>
> Key: CASSANDRA-13142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13142
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Minor
>
> Since at least 1.2 upgradesstables will cancel any compactions bar 
> validations when run. This was originally determined as a non-issue in 
> CASSANDRA-3430 however can be quite annoying (especially with STCS) as a 
> compaction will output the new version anyway. Furthermore, as per 
> CASSANDRA-12243 it also stops things like view builds and I assume secondary 
> index builds as well which is not ideal.
> We should avoid cancelling compactions unnecessarily.
> I'd class this as a very minor bug, should at least be fixed from 3.0, but up 
> for debate. The main negative effect is annoying operators (which I tend to 
> rate pretty highly), but other than that there are no serious consequences.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-13135) Forced termination of repair session leaves repair jobs running

2017-01-24 Thread Yuki Morishita (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836964#comment-15836964
 ] 

Yuki Morishita commented on CASSANDRA-13135:


Patch to remove queued repair jobs at the end of session:

||branch||testall||dtest||
|[13135-2.2|https://github.com/yukim/cassandra/tree/13135-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-2.2-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-2.2-dtest/lastCompletedBuild/testReport/]|
|[13135-3.0|https://github.com/yukim/cassandra/tree/13135-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-3.0-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-3.0-dtest/lastCompletedBuild/testReport/]|
|[13135-3.11|https://github.com/yukim/cassandra/tree/13135-3.11]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-3.11-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-3.11-dtest/lastCompletedBuild/testReport/]|
|[13135-trunk|https://github.com/yukim/cassandra/tree/13135-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-trunk-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-13135-trunk-dtest/lastCompletedBuild/testReport/]|


> Forced termination of repair session leaves repair jobs running
> ---
>
> Key: CASSANDRA-13135
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13135
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Forced termination of repair session (by failure detector or jmx) keeps 
> repair jobs running that the session created after session is terminated.
> This can cause increase in repair time by those unnecessary works left in 
> repair job queue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13135) Forced termination of repair session leaves repair jobs running

2017-01-24 Thread Yuki Morishita (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-13135:
---
Status: Patch Available  (was: Open)

> Forced termination of repair session leaves repair jobs running
> ---
>
> Key: CASSANDRA-13135
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13135
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Forced termination of repair session (by failure detector or jmx) keeps 
> repair jobs running that the session created after session is terminated.
> This can cause increase in repair time by those unnecessary works left in 
> repair job queue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-13149) AssertionError prepending to a list

2017-01-24 Thread Jeff Jirsa (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13149:
---
Description: 
Prepending to a list produces the following AssertionError randomly. Changing 
the update to append (and sort in the client) works around the issue.

{code}
java.lang.AssertionError: null
at 
org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
 [apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
 [apache-cassandra-3.0.8.jar:3.0.8]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_101]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
 [apache-cassandra-3.0.8.jar:3.0.8]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[apache-cassandra-3.0.8.jar:3.0.8]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
{code}

  was:
Prepending to a list produces the following AssertionError randomly. Changing 
the update to append (and sort in the client) works around the issue.

java.lang.AssertionError: null
at 
org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at

[jira] [Created] (CASSANDRA-13149) AssertionError prepending to a list

2017-01-24 Thread Steven Warren (JIRA) 
Steven Warren created CASSANDRA-13149:
-

 Summary: AssertionError prepending to a list
 Key: CASSANDRA-13149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: 3.0.8
Reporter: Steven Warren


Prepending to a list produces the following AssertionError randomly. Changing 
the update to append (and sort in the client) works around the issue.

java.lang.AssertionError: null
at 
org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
 ~[apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
 [apache-cassandra-3.0.8.jar:3.0.8]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
 [apache-cassandra-3.0.8.jar:3.0.8]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_101]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
 [apache-cassandra-3.0.8.jar:3.0.8]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[apache-cassandra-3.0.8.jar:3.0.8]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2017-01-24 Thread samt 
Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60160671
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60160671
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60160671

Branch: refs/heads/cassandra-3.11
Commit: 6016067197f9bf7f557609a927520c94fdd384de
Parents: 1a56dd0 4bbf993
Author: Sam Tunnicliffe 
Authored: Tue Jan 24 08:05:08 2017 -0800
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 08:05:08 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60160671/CHANGES.txt
--
diff --cc CHANGES.txt
index 224ef5d,396fa3f..2f0e8f1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,122 -1,13 +1,123 @@@
 -3.0.11
 +3.10
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

2017-01-24 Thread samt 
Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3465799a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3465799a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3465799a

Branch: refs/heads/trunk
Commit: 3465799a387046e476bad01e684066b1800cd7d5
Parents: 065d87a 6016067
Author: Sam Tunnicliffe 
Authored: Tue Jan 24 08:07:59 2017 -0800
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 08:07:59 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3465799a/CHANGES.txt
--
diff --cc CHANGES.txt
index 86ecbc4,2f0e8f1..dd25cac
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -140,13 -109,15 +140,14 @@@
   * Remove pre-startup check for open JMX port (CASSANDRA-12074)
   * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
   * Restore resumable hints delivery (CASSANDRA-11960)
 - * Properly report LWT contention (CASSANDRA-12626)
 + * Properly record CAS contention (CASSANDRA-12626)
  Merged from 3.0:
+  * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
   * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
 + * Stress daemon help is incorrect (CASSANDRA-12563)
   * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
 - * Stress daemon help is incorrect(CASSANDRA-12563)
 - * Remove ALTER TYPE support (CASSANDRA-12443)
 - * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203)
   * Replace empty strings with null values if they cannot be converted 
(CASSANDRA-12794)
 + * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
   * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620)
   * Add parent repair session id to anticompaction log message 
(CASSANDRA-12186)
   * Improve contention handling on failure to acquire MV lock for streaming 
and hints (CASSANDRA-12905)

[2/6] cassandra git commit: Better error for unqualified functions in authz statements

2017-01-24 Thread samt 
Better error for unqualified functions in authz statements

Patch by Sam Tunnicliffe; reviewed by Carl Yeksigian for CASSANDRA-12925


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bbf9937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bbf9937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bbf9937

Branch: refs/heads/cassandra-3.11
Commit: 4bbf99372e677979c46de432d3b849895fb433ee
Parents: f3b452c
Author: Sam Tunnicliffe 
Authored: Thu Nov 17 11:37:39 2016 +
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 07:56:08 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a85386b..396fa3f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
  * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
  * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
  * Stress daemon help is incorrect (CASSANDRA-12563)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/src/java/org/apache/cassandra/auth/FunctionResource.java
--
diff --git a/src/java/org/apache/cassandra/auth/FunctionResource.java 
b/src/java/org/apache/cassandra/auth/FunctionResource.java
index 2c5b8a1..01a4de5 100644
--- a/src/java/org/apache/cassandra/auth/FunctionResource.java
+++ b/src/java/org/apache/cassandra/auth/FunctionResource.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.functions.Function;
 import org.apache.cassandra.cql3.functions.FunctionName;
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
  * IResource implementation representing functions.
@@ -146,6 +147,9 @@ public class FunctionResource implements IResource
  */
 public static FunctionResource functionFromCql(String keyspace, String 
name, List argTypes)
 {
+if (keyspace == null)
+throw new InvalidRequestException("In this context function name 
must be " +
+  "explictly qualified by a 
keyspace");
 List abstractTypes = new ArrayList<>();
 for (CQL3Type.Raw cqlType : argTypes)
 abstractTypes.add(cqlType.prepare(keyspace).getType());

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
index 6993bec..e5ecc72 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
@@ -449,6 +449,31 @@ public class UFAuthTest extends CQLTester
 getStatement(cql).checkAccess(clientState);
 }
 
+@Test
+public void grantAndRevokeSyntaxRequiresExplicitKeyspace() throws Throwable
+{
+setupTable("CREATE TABLE %s (k int, s int STATIC, v1 int, v2 int, 
PRIMARY KEY(k, v1))");
+String functionName = shortFunctionName(createSimpleFunction());
+assertRequiresKeyspace(String.format("GRANT EXECUTE ON FUNCTION %s() 
TO %s",
+ functionName,
+ role.getRoleName()));
+assertRequiresKeyspace(String.format("REVOKE EXECUTE ON FUNCTION %s() 
FROM %s",
+ functionName,
+ role.getRoleName()));
+}
+
+private void assertRequiresKeyspace(String cql) throws Throwable
+{
+try
+{
+getStatement(cql);
+}
+catch (InvalidRequestException e)
+{
+assertEquals("In this context function name must be explictly 
qualified by a keyspace", e.getMessage());
+}
+}
+
 private void assertPermissionsOnNestedFunctions(String innerFunction, 
String outerFunction) throws Throwable
 {
 String cql = String.format("SELECT k, %s FROM %s WHERE k=0",

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2017-01-24 Thread samt 
Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60160671
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60160671
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60160671

Branch: refs/heads/trunk
Commit: 6016067197f9bf7f557609a927520c94fdd384de
Parents: 1a56dd0 4bbf993
Author: Sam Tunnicliffe 
Authored: Tue Jan 24 08:05:08 2017 -0800
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 08:05:08 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60160671/CHANGES.txt
--
diff --cc CHANGES.txt
index 224ef5d,396fa3f..2f0e8f1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,122 -1,13 +1,123 @@@
 -3.0.11
 +3.10
 + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed 
(CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy 
(CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier 
customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and 
introduce ProtocolVersion enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL 
(CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are 
overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata 
(CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster 
(CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable 
(CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on

[3/6] cassandra git commit: Better error for unqualified functions in authz statements

2017-01-24 Thread samt 
Better error for unqualified functions in authz statements

Patch by Sam Tunnicliffe; reviewed by Carl Yeksigian for CASSANDRA-12925


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bbf9937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bbf9937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bbf9937

Branch: refs/heads/trunk
Commit: 4bbf99372e677979c46de432d3b849895fb433ee
Parents: f3b452c
Author: Sam Tunnicliffe 
Authored: Thu Nov 17 11:37:39 2016 +
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 07:56:08 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a85386b..396fa3f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
  * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
  * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
  * Stress daemon help is incorrect (CASSANDRA-12563)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/src/java/org/apache/cassandra/auth/FunctionResource.java
--
diff --git a/src/java/org/apache/cassandra/auth/FunctionResource.java 
b/src/java/org/apache/cassandra/auth/FunctionResource.java
index 2c5b8a1..01a4de5 100644
--- a/src/java/org/apache/cassandra/auth/FunctionResource.java
+++ b/src/java/org/apache/cassandra/auth/FunctionResource.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.functions.Function;
 import org.apache.cassandra.cql3.functions.FunctionName;
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
  * IResource implementation representing functions.
@@ -146,6 +147,9 @@ public class FunctionResource implements IResource
  */
 public static FunctionResource functionFromCql(String keyspace, String 
name, List argTypes)
 {
+if (keyspace == null)
+throw new InvalidRequestException("In this context function name 
must be " +
+  "explictly qualified by a 
keyspace");
 List abstractTypes = new ArrayList<>();
 for (CQL3Type.Raw cqlType : argTypes)
 abstractTypes.add(cqlType.prepare(keyspace).getType());

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
index 6993bec..e5ecc72 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
@@ -449,6 +449,31 @@ public class UFAuthTest extends CQLTester
 getStatement(cql).checkAccess(clientState);
 }
 
+@Test
+public void grantAndRevokeSyntaxRequiresExplicitKeyspace() throws Throwable
+{
+setupTable("CREATE TABLE %s (k int, s int STATIC, v1 int, v2 int, 
PRIMARY KEY(k, v1))");
+String functionName = shortFunctionName(createSimpleFunction());
+assertRequiresKeyspace(String.format("GRANT EXECUTE ON FUNCTION %s() 
TO %s",
+ functionName,
+ role.getRoleName()));
+assertRequiresKeyspace(String.format("REVOKE EXECUTE ON FUNCTION %s() 
FROM %s",
+ functionName,
+ role.getRoleName()));
+}
+
+private void assertRequiresKeyspace(String cql) throws Throwable
+{
+try
+{
+getStatement(cql);
+}
+catch (InvalidRequestException e)
+{
+assertEquals("In this context function name must be explictly 
qualified by a keyspace", e.getMessage());
+}
+}
+
 private void assertPermissionsOnNestedFunctions(String innerFunction, 
String outerFunction) throws Throwable
 {
 String cql = String.format("SELECT k, %s FROM %s WHERE k=0",

[1/6] cassandra git commit: Better error for unqualified functions in authz statements

2017-01-24 Thread samt 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 f3b452c54 -> 4bbf99372
  refs/heads/cassandra-3.11 1a56dd063 -> 601606719
  refs/heads/trunk 065d87a3e -> 3465799a3


Better error for unqualified functions in authz statements

Patch by Sam Tunnicliffe; reviewed by Carl Yeksigian for CASSANDRA-12925


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bbf9937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bbf9937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bbf9937

Branch: refs/heads/cassandra-3.0
Commit: 4bbf99372e677979c46de432d3b849895fb433ee
Parents: f3b452c
Author: Sam Tunnicliffe 
Authored: Thu Nov 17 11:37:39 2016 +
Committer: Sam Tunnicliffe 
Committed: Tue Jan 24 07:56:08 2017 -0800

--
 CHANGES.txt |  1 +
 .../apache/cassandra/auth/FunctionResource.java |  4 
 .../cql3/validation/entities/UFAuthTest.java| 25 
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a85386b..396fa3f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.11
+ * Better error when modifying function permissions without explicit keyspace 
(CASSANDRA-12925)
  * Indexer is not correctly invoked when building indexes over sstables 
(CASSANDRA-13075)
  * Read repair is not blocking repair to finish in foreground repair 
(CASSANDRA-13115)
  * Stress daemon help is incorrect (CASSANDRA-12563)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/src/java/org/apache/cassandra/auth/FunctionResource.java
--
diff --git a/src/java/org/apache/cassandra/auth/FunctionResource.java 
b/src/java/org/apache/cassandra/auth/FunctionResource.java
index 2c5b8a1..01a4de5 100644
--- a/src/java/org/apache/cassandra/auth/FunctionResource.java
+++ b/src/java/org/apache/cassandra/auth/FunctionResource.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.cql3.functions.Function;
 import org.apache.cassandra.cql3.functions.FunctionName;
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
  * IResource implementation representing functions.
@@ -146,6 +147,9 @@ public class FunctionResource implements IResource
  */
 public static FunctionResource functionFromCql(String keyspace, String 
name, List argTypes)
 {
+if (keyspace == null)
+throw new InvalidRequestException("In this context function name 
must be " +
+  "explictly qualified by a 
keyspace");
 List abstractTypes = new ArrayList<>();
 for (CQL3Type.Raw cqlType : argTypes)
 abstractTypes.add(cqlType.prepare(keyspace).getType());

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4bbf9937/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java 
b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
index 6993bec..e5ecc72 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/UFAuthTest.java
@@ -449,6 +449,31 @@ public class UFAuthTest extends CQLTester
 getStatement(cql).checkAccess(clientState);
 }
 
+@Test
+public void grantAndRevokeSyntaxRequiresExplicitKeyspace() throws Throwable
+{
+setupTable("CREATE TABLE %s (k int, s int STATIC, v1 int, v2 int, 
PRIMARY KEY(k, v1))");
+String functionName = shortFunctionName(createSimpleFunction());
+assertRequiresKeyspace(String.format("GRANT EXECUTE ON FUNCTION %s() 
TO %s",
+ functionName,
+ role.getRoleName()));
+assertRequiresKeyspace(String.format("REVOKE EXECUTE ON FUNCTION %s() 
FROM %s",
+ functionName,
+ role.getRoleName()));
+}
+
+private void assertRequiresKeyspace(String cql) throws Throwable
+{
+try
+{
+getStatement(cql);
+}
+catch (InvalidRequestException e)
+{
+assertEquals("In this context function name must be explictly 
qualified by a keyspace", e.getMessage());
+}
+}
+
 private