[jira] [Commented] (CASSANDRA-13253) include a search on the doc home

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881677#comment-15881677
 ] 

Jon Haddad commented on CASSANDRA-13253:


I built the docs using "make html" in the docs directory.  If you click into 
one of the links, you should see the search form on the left side.  It's the 
same form HTML.  

I opened the page using {{open build/html/index.html}} when in the docs 
directory.  

> include a search on the doc home
> 
>
> Key: CASSANDRA-13253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13253
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>
> Current doc homepage doesn't have a search.  
> http://cassandra.apache.org/doc/latest/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13261) Improve speculative retry to avoid being overloaded

[Simon Zhou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Zhou updated CASSANDRA-13261:
---
Status: Patch Available  (was: Open)

> Improve speculative retry to avoid being overloaded
> ---
>
> Key: CASSANDRA-13261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13261
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Attachments: CASSANDRA-13261-v1.patch
>
>
> In CASSANDRA-13009, I was suggested to separate the 2nd part of my patch as 
> an improvement.
> This is to avoid Cassandra being overloaded when using CUSTOM speculative 
> retry parameter. Steps to reason/repro this with 3.0.10:
> 1. Use custom speculative retry threshold like this:
> cqlsh> alter TABLE to_repair1.users0 with speculative_retry='10ms';
> 2. SpeculatingReadExecutor will be used, according to this piece of code in 
> AbstractReadExecutor:
> {code}
> if (retry.equals(SpeculativeRetryParam.ALWAYS))
> return new AlwaysSpeculatingReadExecutor(keyspace, cfs, command, 
> consistencyLevel, targetReplicas);
> else // PERCENTILE or CUSTOM.
> return new SpeculatingReadExecutor(keyspace, cfs, command, 
> consistencyLevel, targetReplicas);
> {code}
> 3. When RF=3 and LOCAL_QUORUM is used, the below code (from 
> SpeculatingReadExecutor#maybeTryAdditionalReplicas) won't be able to protect 
> Cassandra from being overloaded, even though the inline comment suggests such 
> intention:
> {code}
> // no latency information, or we're overloaded
> if (cfs.sampleLatencyNanos > 
> TimeUnit.MILLISECONDS.toNanos(command.getTimeout()))
> return;
> {code}
> The reason is that cfs.sampleLatencyNanos is assigned as 
> retryPolicy.threshold() which is 10ms in step #1 above, at line 405 of 
> ColumnFamilyStore. However pretty often the timeout is the default one 5000ms.
> As the name suggests, sampleLatencyNanos should be used to keep sampled 
> latency, not something configured "statically". My proposal:
> a. Introduce option -Dcassandra.overload.threshold to allow customizing 
> overload threshold. The default threshold would be 
> DatabaseDescriptor.getRangeRpcTimeout().
> b. Assign sampled P99 latency to cfs.sampleLatencyNanos. For overload 
> detection, we just compare cfs.sampleLatencyNanos with the customizable 
> threshold above.
> c. Use retryDelayNanos (instead of cfs.sampleLatencyNanos) for waiting time 
> before retry (see line 282 of AbstractReadExecutor). This is the value from 
> table setting (PERCENTILE or CUSTOM).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13261) Improve speculative retry to avoid being overloaded

[Simon Zhou (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Zhou updated CASSANDRA-13261:
---
Attachment: CASSANDRA-13261-v1.patch

I'm not sure what's the next release for 3.0.* and 3.0.11 was just merged to 
trunk. The attached patch is for trunk but I'd like to have this improvement 
included in the next release for 3.0.*.

[~tjake] Maybe you can help review this patch since you have some context from 
CASSANDRA-13009?

> Improve speculative retry to avoid being overloaded
> ---
>
> Key: CASSANDRA-13261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13261
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Attachments: CASSANDRA-13261-v1.patch
>
>
> In CASSANDRA-13009, I was suggested to separate the 2nd part of my patch as 
> an improvement.
> This is to avoid Cassandra being overloaded when using CUSTOM speculative 
> retry parameter. Steps to reason/repro this with 3.0.10:
> 1. Use custom speculative retry threshold like this:
> cqlsh> alter TABLE to_repair1.users0 with speculative_retry='10ms';
> 2. SpeculatingReadExecutor will be used, according to this piece of code in 
> AbstractReadExecutor:
> {code}
> if (retry.equals(SpeculativeRetryParam.ALWAYS))
> return new AlwaysSpeculatingReadExecutor(keyspace, cfs, command, 
> consistencyLevel, targetReplicas);
> else // PERCENTILE or CUSTOM.
> return new SpeculatingReadExecutor(keyspace, cfs, command, 
> consistencyLevel, targetReplicas);
> {code}
> 3. When RF=3 and LOCAL_QUORUM is used, the below code (from 
> SpeculatingReadExecutor#maybeTryAdditionalReplicas) won't be able to protect 
> Cassandra from being overloaded, even though the inline comment suggests such 
> intention:
> {code}
> // no latency information, or we're overloaded
> if (cfs.sampleLatencyNanos > 
> TimeUnit.MILLISECONDS.toNanos(command.getTimeout()))
> return;
> {code}
> The reason is that cfs.sampleLatencyNanos is assigned as 
> retryPolicy.threshold() which is 10ms in step #1 above, at line 405 of 
> ColumnFamilyStore. However pretty often the timeout is the default one 5000ms.
> As the name suggests, sampleLatencyNanos should be used to keep sampled 
> latency, not something configured "statically". My proposal:
> a. Introduce option -Dcassandra.overload.threshold to allow customizing 
> overload threshold. The default threshold would be 
> DatabaseDescriptor.getRangeRpcTimeout().
> b. Assign sampled P99 latency to cfs.sampleLatencyNanos. For overload 
> detection, we just compare cfs.sampleLatencyNanos with the customizable 
> threshold above.
> c. Use retryDelayNanos (instead of cfs.sampleLatencyNanos) for waiting time 
> before retry (see line 282 of AbstractReadExecutor). This is the value from 
> table setting (PERCENTILE or CUSTOM).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13261) Improve speculative retry to avoid being overloaded

[Simon Zhou (JIRA)]

Simon Zhou created CASSANDRA-13261:
--

 Summary: Improve speculative retry to avoid being overloaded
 Key: CASSANDRA-13261
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13261
 Project: Cassandra
  Issue Type: Improvement
Reporter: Simon Zhou
Assignee: Simon Zhou


In CASSANDRA-13009, I was suggested to separate the 2nd part of my patch as an 
improvement.

This is to avoid Cassandra being overloaded when using CUSTOM speculative retry 
parameter. Steps to reason/repro this with 3.0.10:
1. Use custom speculative retry threshold like this:
cqlsh> alter TABLE to_repair1.users0 with speculative_retry='10ms';

2. SpeculatingReadExecutor will be used, according to this piece of code in 
AbstractReadExecutor:
{code}
if (retry.equals(SpeculativeRetryParam.ALWAYS))
return new AlwaysSpeculatingReadExecutor(keyspace, cfs, command, 
consistencyLevel, targetReplicas);
else // PERCENTILE or CUSTOM.
return new SpeculatingReadExecutor(keyspace, cfs, command, 
consistencyLevel, targetReplicas);
{code}

3. When RF=3 and LOCAL_QUORUM is used, the below code (from 
SpeculatingReadExecutor#maybeTryAdditionalReplicas) won't be able to protect 
Cassandra from being overloaded, even though the inline comment suggests such 
intention:

{code}
// no latency information, or we're overloaded
if (cfs.sampleLatencyNanos > 
TimeUnit.MILLISECONDS.toNanos(command.getTimeout()))
return;
{code}

The reason is that cfs.sampleLatencyNanos is assigned as 
retryPolicy.threshold() which is 10ms in step #1 above, at line 405 of 
ColumnFamilyStore. However pretty often the timeout is the default one 5000ms.

As the name suggests, sampleLatencyNanos should be used to keep sampled 
latency, not something configured "statically". My proposal:
a. Introduce option -Dcassandra.overload.threshold to allow customizing 
overload threshold. The default threshold would be 
DatabaseDescriptor.getRangeRpcTimeout().
b. Assign sampled P99 latency to cfs.sampleLatencyNanos. For overload 
detection, we just compare cfs.sampleLatencyNanos with the customizable 
threshold above.
c. Use retryDelayNanos (instead of cfs.sampleLatencyNanos) for waiting time 
before retry (see line 282 of AbstractReadExecutor). This is the value from 
table setting (PERCENTILE or CUSTOM).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13253) include a search on the doc home

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881518#comment-15881518
 ] 

Blake Eggleston commented on CASSANDRA-13253:
-

Does this need any additional doc build steps to make it work? This never 
returns search results for me (and I'm using SimpleHTTPServer to avoid any 
browser script blocking issues)

> include a search on the doc home
> 
>
> Key: CASSANDRA-13253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13253
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>
> Current doc homepage doesn't have a search.  
> http://cassandra.apache.org/doc/latest/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-9292) ParentRepairSession potentially block ANTI_ENTROPY stage

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-9292:
---
Status: Ready to Commit  (was: Patch Available)

> ParentRepairSession potentially block ANTI_ENTROPY stage
> 
>
> Key: CASSANDRA-9292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9292
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Yuki Morishita
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 4.x
>
>
> Follow up from CASSANDRA-9151,
> {quote}
> potentially block this stage again since many methods are synchronized in 
> ActiveRepairService.
> Methods prepareForRepair(could block for 1 hour for prepare message response) 
> and finishParentSession(this one block for anticompaction to finish) are 
> synchronized and could hold on to the lock for a long time.
> In RepairMessageVerbHandler.doVerb, if there is an exception for another 
> repair, removeParentRepairSession(also synchronized) will block.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-9292) ParentRepairSession potentially block ANTI_ENTROPY stage

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881334#comment-15881334
 ] 

Blake Eggleston commented on CASSANDRA-9292:


+1

> ParentRepairSession potentially block ANTI_ENTROPY stage
> 
>
> Key: CASSANDRA-9292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9292
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Yuki Morishita
>Assignee: Marcus Eriksson
>Priority: Minor
> Fix For: 4.x
>
>
> Follow up from CASSANDRA-9151,
> {quote}
> potentially block this stage again since many methods are synchronized in 
> ActiveRepairService.
> Methods prepareForRepair(could block for 1 hour for prepare message response) 
> and finishParentSession(this one block for anticompaction to finish) are 
> synchronized and could hold on to the lock for a long time.
> In RepairMessageVerbHandler.doVerb, if there is an exception for another 
> repair, removeParentRepairSession(also synchronized) will block.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13257) Add repair streaming preview

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880947#comment-15880947
 ] 

Blake Eggleston commented on CASSANDRA-13257:
-

[~pauloricardomg], [~yukim] - would one of you be interested in reviewing?

> Add repair streaming preview
> 
>
> Key: CASSANDRA-13257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13257
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> It would be useful to be able to estimate the amount of repair streaming that 
> needs to be done, without actually doing any streaming. Our main motivation 
> for this having something this is validating CASSANDRA-9143 in production, 
> but I’d imagine it could also be a useful tool in troubleshooting.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13257) Add repair streaming preview

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13257:

Status: Patch Available  (was: Open)

|[branch|https://github.com/bdeggleston/cassandra/tree/13257]|[dtest|http://cassci.datastax.com/view/Dev/view/bdeggleston/job/bdeggleston-13257-dtest/]|[testall|http://cassci.datastax.com/view/Dev/view/bdeggleston/job/bdeggleston-13257-testall/]|

> Add repair streaming preview
> 
>
> Key: CASSANDRA-13257
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13257
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> It would be useful to be able to estimate the amount of repair streaming that 
> needs to be done, without actually doing any streaming. Our main motivation 
> for this having something this is validating CASSANDRA-9143 in production, 
> but I’d imagine it could also be a useful tool in troubleshooting.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Sotirios Delimanolis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880853#comment-15880853
 ] 

Sotirios Delimanolis commented on CASSANDRA-12100:
--

Thank you!

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 2.2.10, 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12513) IOException (No such file or directory) closing MessagingService's server socket (locally)

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-12513:
-
Fix Version/s: 3.0.x
   Status: Patch Available  (was: Open)

||cassandra-3.0|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...snazy:12513-ms-close-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-3.0-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-3.0-dtest/lastSuccessfulBuild/]
||cassandra-3.11|[branch|https://github.com/apache/cassandra/compare/cassandra-3.11...snazy:12513-ms-close-3.11]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-3.11-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-3.11-dtest/lastSuccessfulBuild/]
||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:12513-ms-close-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-trunk-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12513-ms-close-trunk-dtest/lastSuccessfulBuild/]

CI triggered

Before I really ask myself again  [~mkjellman], could you review?

> IOException (No such file or directory) closing MessagingService's server 
> socket (locally)
> --
>
> Key: CASSANDRA-12513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12513
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.x
>
>
> _Sometimes_ the {{RemoveTest}} fails with the following exception. It's not 
> related to the test itself.
> The exception is risen in 
> {{ServerSocketChannelImpl.implCloseSelectableChannel}} where it checks that a 
> thread ID is non-zero. The {{thread}} instance field is set inside its accept 
> and poll methods. It looks like this is caused by some race condition - i.e. 
> stopping in debugger at certain points prevents it from being triggered.
> I could not find any misuse in the code base - but want to document this 
> issue.
> No difference between 8u92 and 8u102
> {code}
> INFO  [ACCEPT-/127.0.0.1] 2016-08-22 08:35:16,606 ?:? - MessagingService has 
> terminated the accept() thread
> java.io.IOError: java.io.IOException: No such file or directory
>   at 
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:914)
>   at org.apache.cassandra.service.RemoveTest.tearDown(RemoveTest.java:103)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
>   at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
> Caused by: java.io.IOException: No such file or directory
>   at sun.nio.ch.NativeThread.signal(Native Method)
>   at 
> sun.nio.ch.ServerSocketChannelImpl.implCloseSelectableChannel(ServerSocketChannelImpl.java:292)
>   at 
> java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:234)
> 

[jira] [Commented] (CASSANDRA-9556) Add newer data types to cassandra stress

[Jeremy Hanna (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880748#comment-15880748
 ] 

Jeremy Hanna commented on CASSANDRA-9556:
-

Sorry for the delay in responding.  There wasn't a separate ticket opened for 
UDT support in Cassandra stress.  I created CASSANDRA-13260.

> Add newer data types to cassandra stress
> 
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: stress
> Fix For: 2.2.5, 3.0.3
>
> Attachments: CASSANDRA-9556-2.2.patch
>
>
> Currently you can't define a data model with decimal types and use Cassandra 
> stress with it.  Also, I imagine that holds true with other newer data types 
> such as the new date and time types.  Besides that, now that data models are 
> including user defined types, we should allow users to create those 
> structures with stress as well.  Perhaps we could split out the UDTs into a 
> different ticket if it holds the other types up.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13260) Add UDT support to Cassandra stress

[Jeremy Hanna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-13260:
-
Labels: lhf stress  (was: )

> Add UDT support to Cassandra stress
> ---
>
> Key: CASSANDRA-13260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13260
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>  Labels: lhf, stress
>
> Splitting out UDT support in cassandra stress from CASSANDRA-9556.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13260) Add UDT support to Cassandra stress

[Jeremy Hanna (JIRA)]

Jeremy Hanna created CASSANDRA-13260:


 Summary: Add UDT support to Cassandra stress
 Key: CASSANDRA-13260
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13260
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jeremy Hanna


Splitting out UDT support in cassandra stress from CASSANDRA-9556.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13053) GRANT/REVOKE on table without keyspace performs permissions check incorrectly

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie reassigned CASSANDRA-13053:
---

Assignee: Aleksey Yeschenko

> GRANT/REVOKE on table without keyspace performs permissions check incorrectly
> -
>
> Key: CASSANDRA-13053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13053
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sam Tunnicliffe
>Assignee: Aleksey Yeschenko
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> When a {{GRANT}} or {{REVOKE}} statement is executed on a table without 
> specifying the keyspace, we attempt to use the client session's keyspace to 
> qualify the resource. 
> This is done when validating the statement, which occurs after checking that 
> the user executing the statement has sufficient permissions. This means that 
> the permissions checking uses an incorrect resource, namely a table with a 
> null keyspace. If that user is a superuser, then no error is encountered as 
> superuser privs implicitly grants *all* permissions. If the user is not a 
> superuser, then the {{GRANT}} or {{REVOKE}} fails with an ugly error, 
> regardless of which keyspace the client session is bound to:
> {code}
> Unauthorized: Error from server: code=2100 [Unauthorized] message="User admin 
> has no AUTHORIZE permission on  or any of its parents"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[1/2] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 831c05b1c -> 2d8be34a2


Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Legacy deserializer can create unexpected boundary range tombstones


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1dc1aa19
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1dc1aa19
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1dc1aa19

Branch: refs/heads/trunk
Commit: 1dc1aa1982a7ab84034c95fa6ce6b3e4e2346fd2
Parents: 6487876 ab71748
Author: Sylvain Lebresne 
Authored: Thu Feb 23 14:37:35 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 14:37:35 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/UnfilteredDeserializer.java| 334 ++-
 .../cassandra/db/rows/RangeTombstoneMarker.java |   1 -
 .../apache/cassandra/service/DataResolver.java  |  31 +-
 .../cassandra/db/OldFormatDeserializerTest.java | 110 ++
 .../cassandra/service/DataResolverTest.java | 125 ++-
 6 files changed, 432 insertions(+), 170 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dc1aa19/CHANGES.txt
--
diff --cc CHANGES.txt
index 233898f,386029e..f5b9d28
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,19 +1,19 @@@
 -3.0.12
 +3.11.0
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 +Merged from 3.0:
+  * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
   * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
   * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 -Merged from 2.2
 - * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 - * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 - * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 - * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 -Merged from 2.1:
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 -
 -3.0.11
   * Use keyspace replication settings on system.size_estimates table 
(CASSANDRA-9639)
   * Add vm.max_map_count StartupCheck (CASSANDRA-13008)
 - * Hint related logging should include the IP address of the destination in 
addition to 
 + * Hint related logging should include the IP address of the destination in 
addition to
 host ID (CASSANDRA-13205)
   * Reloading logback.xml does not work (CASSANDRA-13173)
   * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 
(CASSANDRA-13109)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dc1aa19/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --cc src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 2c3bc1b,42a806a..79b8636
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@@ -317,16 -331,7 +331,7 @@@ public abstract class UnfilteredDeseria
  }
  }
  
- private boolean isRow(LegacyLayout.LegacyAtom atom)
- {
- if (atom.isCell())
- return true;
- 
- LegacyLayout.LegacyRangeTombstone tombstone = 
atom.asRangeTombstone();
- return tombstone.isCollectionTombstone() || 
tombstone.isRowDeletion(metadata);
- }
- 
 -public int compareNextTo(Slice.Bound bound) throws IOException
 +public int compareNextTo(ClusteringBound bound) throws IOException
  {
  if (!hasNext())
  throw new IllegalStateException();
@@@ -397,13 -405,27 +405,28 @@@
  
  private Unfiltered next;
  
- private UnfilteredIterator(DeletionTime partitionDeletion)
+ UnfilteredIterator(CFMetaData metadata,
+DeletionTime partitionDeletion,
+SerializationHelper helper,
+Supplier atomReader)
  {
+ this.metadata = metadata;
+ this.helper = helper;
  this.grouper = new 

[2/2] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[slebresne]

Merge branch 'cassandra-3.11' into trunk

* cassandra-3.11:


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d8be34a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d8be34a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d8be34a

Branch: refs/heads/trunk
Commit: 2d8be34a240db84d4368a4d30a2258ca6e67043f
Parents: 831c05b 1dc1aa1
Author: Sylvain Lebresne 
Authored: Thu Feb 23 15:48:25 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 15:48:25 2017 +0100

--

--

[jira] [Updated] (CASSANDRA-13237) Legacy deserializer can create unexpected boundary range tombstones

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-13237:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Alright, committed, thanks.

> Legacy deserializer can create unexpected boundary range tombstones
> ---
>
> Key: CASSANDRA-13237
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13237
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.11.x
>
>
> Most of the code don't generate a range tombstone boundary with the same 
> deletion time on both side as this is basically useless, and there is some 
> assertion in {{DataResolver}} that actually expect this. However, the 
> deserializer for legacy sstable doesn't always properly avoid their creation 
> and we can thus generate them (and break the {{DataResolver}} assertion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[2/5] cassandra git commit: Legacy deserializer can create unexpected boundary range tombstones

[slebresne]

Legacy deserializer can create unexpected boundary range tombstones

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13237


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab717484
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab717484
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab717484

Branch: refs/heads/cassandra-3.11
Commit: ab7174849599c62f4bef3cb719c644bae13e9321
Parents: 42977db
Author: Sylvain Lebresne 
Authored: Thu Feb 23 14:32:03 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 14:32:34 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/UnfilteredDeserializer.java| 343 ++-
 .../cassandra/db/rows/RangeTombstoneMarker.java |   2 +-
 .../apache/cassandra/service/DataResolver.java  |  31 +-
 .../cassandra/db/OldFormatDeserializerTest.java | 110 ++
 .../cassandra/service/DataResolverTest.java | 129 ++-
 6 files changed, 436 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e978a5c..386029e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.12
+ * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
  * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
  * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 Merged from 2.2

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index a2d51e13..42a806a 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -20,7 +20,9 @@ package org.apache.cassandra.db;
 import java.io.IOException;
 import java.io.IOError;
 import java.util.*;
+import java.util.function.Supplier;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.PeekingIterator;
 
@@ -265,11 +267,23 @@ public abstract class UnfilteredDeserializer
   boolean readAllAsDynamic)
 {
 super(metadata, in, helper);
-this.iterator = new UnfilteredIterator(partitionDeletion);
+this.iterator = new UnfilteredIterator(metadata, 
partitionDeletion, helper, this::readAtom);
 this.readAllAsDynamic = readAllAsDynamic;
 this.lastConsumedPosition = currentPosition();
 }
 
+private LegacyLayout.LegacyAtom readAtom()
+{
+try
+{
+return LegacyLayout.readLegacyAtom(metadata, in, 
readAllAsDynamic);
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+}
+
 public void setSkipStatic()
 {
 this.skipStatic = true;
@@ -317,15 +331,6 @@ public abstract class UnfilteredDeserializer
 }
 }
 
-private boolean isRow(LegacyLayout.LegacyAtom atom)
-{
-if (atom.isCell())
-return true;
-
-LegacyLayout.LegacyRangeTombstone tombstone = 
atom.asRangeTombstone();
-return tombstone.isCollectionTombstone() || 
tombstone.isRowDeletion(metadata);
-}
-
 public int compareNextTo(Slice.Bound bound) throws IOException
 {
 if (!hasNext())
@@ -389,19 +394,36 @@ public abstract class UnfilteredDeserializer
 // Groups atoms from the input into proper Unfiltered.
 // Note: this could use guava AbstractIterator except that we want to 
be able to clear
 // the internal state of the iterator so it's cleaner to do it 
ourselves.
-private class UnfilteredIterator implements PeekingIterator
+@VisibleForTesting
+static class UnfilteredIterator implements PeekingIterator
 {
 private final AtomIterator atoms;
 private final LegacyLayout.CellGrouper grouper;
 private final TombstoneTracker tombstoneTracker;
+private final CFMetaData metadata;
+private final SerializationHelper helper;
 
 private Unfiltered next;
 
-private UnfilteredIterator(DeletionTime partitionDeletion)
+UnfilteredIterator(CFMetaData metadata,
+   

[4/5] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[slebresne]

Merge branch 'cassandra-3.0' into cassandra-3.11

* cassandra-3.0:
  Legacy deserializer can create unexpected boundary range tombstones


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1dc1aa19
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1dc1aa19
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1dc1aa19

Branch: refs/heads/cassandra-3.11
Commit: 1dc1aa1982a7ab84034c95fa6ce6b3e4e2346fd2
Parents: 6487876 ab71748
Author: Sylvain Lebresne 
Authored: Thu Feb 23 14:37:35 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 14:37:35 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/UnfilteredDeserializer.java| 334 ++-
 .../cassandra/db/rows/RangeTombstoneMarker.java |   1 -
 .../apache/cassandra/service/DataResolver.java  |  31 +-
 .../cassandra/db/OldFormatDeserializerTest.java | 110 ++
 .../cassandra/service/DataResolverTest.java | 125 ++-
 6 files changed, 432 insertions(+), 170 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dc1aa19/CHANGES.txt
--
diff --cc CHANGES.txt
index 233898f,386029e..f5b9d28
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,19 +1,19 @@@
 -3.0.12
 +3.11.0
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 +Merged from 3.0:
+  * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
   * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
   * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 -Merged from 2.2
 - * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 - * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 - * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 - * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 -Merged from 2.1:
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 -
 -3.0.11
   * Use keyspace replication settings on system.size_estimates table 
(CASSANDRA-9639)
   * Add vm.max_map_count StartupCheck (CASSANDRA-13008)
 - * Hint related logging should include the IP address of the destination in 
addition to 
 + * Hint related logging should include the IP address of the destination in 
addition to
 host ID (CASSANDRA-13205)
   * Reloading logback.xml does not work (CASSANDRA-13173)
   * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 
(CASSANDRA-13109)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dc1aa19/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --cc src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 2c3bc1b,42a806a..79b8636
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@@ -317,16 -331,7 +331,7 @@@ public abstract class UnfilteredDeseria
  }
  }
  
- private boolean isRow(LegacyLayout.LegacyAtom atom)
- {
- if (atom.isCell())
- return true;
- 
- LegacyLayout.LegacyRangeTombstone tombstone = 
atom.asRangeTombstone();
- return tombstone.isCollectionTombstone() || 
tombstone.isRowDeletion(metadata);
- }
- 
 -public int compareNextTo(Slice.Bound bound) throws IOException
 +public int compareNextTo(ClusteringBound bound) throws IOException
  {
  if (!hasNext())
  throw new IllegalStateException();
@@@ -397,13 -405,27 +405,28 @@@
  
  private Unfiltered next;
  
- private UnfilteredIterator(DeletionTime partitionDeletion)
+ UnfilteredIterator(CFMetaData metadata,
+DeletionTime partitionDeletion,
+SerializationHelper helper,
+Supplier atomReader)
  {
+ this.metadata = metadata;
+ this.helper = helper;
  this.grouper = new LegacyLayout.CellGrouper(metadata, helper);
  this.tombstoneTracker = new 

[1/5] cassandra git commit: Legacy deserializer can create unexpected boundary range tombstones

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 42977dbce -> ab7174849
  refs/heads/cassandra-3.11 6487876dd -> 1dc1aa198
  refs/heads/trunk cd29d44df -> 831c05b1c


Legacy deserializer can create unexpected boundary range tombstones

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13237


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab717484
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab717484
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab717484

Branch: refs/heads/cassandra-3.0
Commit: ab7174849599c62f4bef3cb719c644bae13e9321
Parents: 42977db
Author: Sylvain Lebresne 
Authored: Thu Feb 23 14:32:03 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 14:32:34 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/UnfilteredDeserializer.java| 343 ++-
 .../cassandra/db/rows/RangeTombstoneMarker.java |   2 +-
 .../apache/cassandra/service/DataResolver.java  |  31 +-
 .../cassandra/db/OldFormatDeserializerTest.java | 110 ++
 .../cassandra/service/DataResolverTest.java | 129 ++-
 6 files changed, 436 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e978a5c..386029e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.12
+ * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
  * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
  * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 Merged from 2.2

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index a2d51e13..42a806a 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -20,7 +20,9 @@ package org.apache.cassandra.db;
 import java.io.IOException;
 import java.io.IOError;
 import java.util.*;
+import java.util.function.Supplier;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.PeekingIterator;
 
@@ -265,11 +267,23 @@ public abstract class UnfilteredDeserializer
   boolean readAllAsDynamic)
 {
 super(metadata, in, helper);
-this.iterator = new UnfilteredIterator(partitionDeletion);
+this.iterator = new UnfilteredIterator(metadata, 
partitionDeletion, helper, this::readAtom);
 this.readAllAsDynamic = readAllAsDynamic;
 this.lastConsumedPosition = currentPosition();
 }
 
+private LegacyLayout.LegacyAtom readAtom()
+{
+try
+{
+return LegacyLayout.readLegacyAtom(metadata, in, 
readAllAsDynamic);
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+}
+
 public void setSkipStatic()
 {
 this.skipStatic = true;
@@ -317,15 +331,6 @@ public abstract class UnfilteredDeserializer
 }
 }
 
-private boolean isRow(LegacyLayout.LegacyAtom atom)
-{
-if (atom.isCell())
-return true;
-
-LegacyLayout.LegacyRangeTombstone tombstone = 
atom.asRangeTombstone();
-return tombstone.isCollectionTombstone() || 
tombstone.isRowDeletion(metadata);
-}
-
 public int compareNextTo(Slice.Bound bound) throws IOException
 {
 if (!hasNext())
@@ -389,19 +394,36 @@ public abstract class UnfilteredDeserializer
 // Groups atoms from the input into proper Unfiltered.
 // Note: this could use guava AbstractIterator except that we want to 
be able to clear
 // the internal state of the iterator so it's cleaner to do it 
ourselves.
-private class UnfilteredIterator implements PeekingIterator
+@VisibleForTesting
+static class UnfilteredIterator implements PeekingIterator
 {
 private final AtomIterator atoms;
 private final LegacyLayout.CellGrouper grouper;
 private final TombstoneTracker tombstoneTracker;
+private final CFMetaData metadata;
+private final SerializationHelper helper;
 
 

[3/5] cassandra git commit: Legacy deserializer can create unexpected boundary range tombstones

[slebresne]

Legacy deserializer can create unexpected boundary range tombstones

patch by Sylvain Lebresne; reviewed by Branimir Lambov for CASSANDRA-13237


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab717484
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab717484
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab717484

Branch: refs/heads/trunk
Commit: ab7174849599c62f4bef3cb719c644bae13e9321
Parents: 42977db
Author: Sylvain Lebresne 
Authored: Thu Feb 23 14:32:03 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 14:32:34 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/UnfilteredDeserializer.java| 343 ++-
 .../cassandra/db/rows/RangeTombstoneMarker.java |   2 +-
 .../apache/cassandra/service/DataResolver.java  |  31 +-
 .../cassandra/db/OldFormatDeserializerTest.java | 110 ++
 .../cassandra/service/DataResolverTest.java | 129 ++-
 6 files changed, 436 insertions(+), 180 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e978a5c..386029e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.12
+ * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
  * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
  * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 Merged from 2.2

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab717484/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index a2d51e13..42a806a 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -20,7 +20,9 @@ package org.apache.cassandra.db;
 import java.io.IOException;
 import java.io.IOError;
 import java.util.*;
+import java.util.function.Supplier;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.Iterables;
 import com.google.common.collect.PeekingIterator;
 
@@ -265,11 +267,23 @@ public abstract class UnfilteredDeserializer
   boolean readAllAsDynamic)
 {
 super(metadata, in, helper);
-this.iterator = new UnfilteredIterator(partitionDeletion);
+this.iterator = new UnfilteredIterator(metadata, 
partitionDeletion, helper, this::readAtom);
 this.readAllAsDynamic = readAllAsDynamic;
 this.lastConsumedPosition = currentPosition();
 }
 
+private LegacyLayout.LegacyAtom readAtom()
+{
+try
+{
+return LegacyLayout.readLegacyAtom(metadata, in, 
readAllAsDynamic);
+}
+catch (IOException e)
+{
+throw new IOError(e);
+}
+}
+
 public void setSkipStatic()
 {
 this.skipStatic = true;
@@ -317,15 +331,6 @@ public abstract class UnfilteredDeserializer
 }
 }
 
-private boolean isRow(LegacyLayout.LegacyAtom atom)
-{
-if (atom.isCell())
-return true;
-
-LegacyLayout.LegacyRangeTombstone tombstone = 
atom.asRangeTombstone();
-return tombstone.isCollectionTombstone() || 
tombstone.isRowDeletion(metadata);
-}
-
 public int compareNextTo(Slice.Bound bound) throws IOException
 {
 if (!hasNext())
@@ -389,19 +394,36 @@ public abstract class UnfilteredDeserializer
 // Groups atoms from the input into proper Unfiltered.
 // Note: this could use guava AbstractIterator except that we want to 
be able to clear
 // the internal state of the iterator so it's cleaner to do it 
ourselves.
-private class UnfilteredIterator implements PeekingIterator
+@VisibleForTesting
+static class UnfilteredIterator implements PeekingIterator
 {
 private final AtomIterator atoms;
 private final LegacyLayout.CellGrouper grouper;
 private final TombstoneTracker tombstoneTracker;
+private final CFMetaData metadata;
+private final SerializationHelper helper;
 
 private Unfiltered next;
 
-private UnfilteredIterator(DeletionTime partitionDeletion)
+UnfilteredIterator(CFMetaData metadata,
+   DeletionTime 

[5/5] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk

* cassandra-3.0:
  Legacy deserializer can create unexpected boundary range tombstones


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/831c05b1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/831c05b1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/831c05b1

Branch: refs/heads/trunk
Commit: 831c05b1c047cdccbc6a82fe0d3f72302fdfb18a
Parents: cd29d44 ab71748
Author: Sylvain Lebresne 
Authored: Thu Feb 23 15:21:47 2017 +0100
Committer: Sylvain Lebresne 
Committed: Thu Feb 23 15:21:47 2017 +0100

--
 CHANGES.txt |   1 +
 .../cassandra/db/rows/RangeTombstoneMarker.java |   1 -
 .../apache/cassandra/service/DataResolver.java  |  31 +++--
 .../cassandra/service/DataResolverTest.java | 129 ++-
 4 files changed, 142 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/831c05b1/CHANGES.txt
--
diff --cc CHANGES.txt
index 0641011,386029e..f541c9a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,55 -1,19 +1,56 @@@
 -3.0.12
 +4.0
 + * Adds the ability to use uncompressed chunks in compressed files 
(CASSANDRA-10520)
 + * Don't flush sstables when streaming for incremental repair 
(CASSANDRA-13226)
 + * Remove unused method (CASSANDRA-13227)
 + * Fix minor bugs related to #9143 (CASSANDRA-13217)
 + * Output warning if user increases RF (CASSANDRA-13079)
 + * Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
 + * Add support for + and - operations on dates (CASSANDRA-11936)
 + * Fix consistency of incrementally repaired data (CASSANDRA-9143)
 + * Increase commitlog version (CASSANDRA-13161)
 + * Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
 + * Refactor ColumnCondition (CASSANDRA-12981)
 + * Parallelize streaming of different keyspaces (CASSANDRA-4663)
 + * Improved compactions metrics (CASSANDRA-13015)
 + * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
 + * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
 + * Thrift removal (CASSANDRA-5)
 + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool Documentation (CASSANDRA-12672)
 + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
 + * Reject invalid replication settings when creating or altering a keyspace 
(CASSANDRA-12681)
 + * Clean up the SSTableReader#getScanner API wrt removal of RateLimiter 
(CASSANDRA-12422)
 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
 + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
 + * Require forceful decommission if number of nodes is less than replication 
factor (CASSANDRA-12510)
 + * Allow IN restrictions on column families with collections (CASSANDRA-12654)
 + * Log message size in trace message in OutboundTcpConnection 
(CASSANDRA-13028)
 + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
 + * Add mutation size and batch metrics (CASSANDRA-12649)
 + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
 + * Expose time spent waiting in thread pool queue (CASSANDRA-8398)
 + * Conditionally update index built status to avoid unnecessary flushes 
(CASSANDRA-12969)
 + * cqlsh auto completion: refactor definition of compaction strategy options 
(CASSANDRA-12946)
 + * Add support for arithmetic operators (CASSANDRA-11935)
 +
 +
 +3.11.0
 + * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor 
(CASSANDRA-12983)
 +Merged from 3.0:
+  * Legacy deserializer can create unexpected boundary range tombstones 
(CASSANDRA-13237)
   * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
   * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
 -Merged from 2.2
 - * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
 - * Fix failing COPY TO STDOUT (CASSANDRA-12497)
 - * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
 - * Exceptions encountered calling getSeeds() breaks OTC thread 
(CASSANDRA-13018)
 -Merged from 2.1:
 - * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
 -
 -3.0.11
   * Use keyspace 

[jira] [Updated] (CASSANDRA-13174) Indexing is allowed on Duration type when it should not be

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13174:
---
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Committed into 3.11 at 6487876dde14c46d5753f972909e5acec854cb53 and merged into 
trunk

> Indexing is allowed on Duration type when it should not be
> --
>
> Key: CASSANDRA-13174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13174
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.10
>Reporter: Kishan Karunaratne
>Assignee: Benjamin Lerer
> Fix For: 3.11.x, 4.x
>
>
> Looks like secondary indexing is allowed on duration type columns. Since 
> comparisons are not possible for the duration type, indexing on it also 
> should be invalid.
> 1) 
> {noformat}
> CREATE TABLE duration_table (k int PRIMARY KEY, d duration);
> INSERT INTO duration_table (k, d) VALUES (0, 1s);
> SELECT * from duration_table WHERE d=1s ALLOW FILTERING;
> {noformat}
> The above throws an error: 
> {noformat}
> WARN  [ReadStage-2] 2017-01-31 17:09:57,821 
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread 
> Thread[ReadStage-2,10,main]: {}
> java.lang.RuntimeException: java.lang.UnsupportedOperationException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2591)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  ~[main/:na]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
>  [main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: java.lang.UnsupportedOperationException: null
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compareCustom(AbstractType.java:174)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:160) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compareForCQL(AbstractType.java:204)
>  ~[main/:na]
>   at org.apache.cassandra.cql3.Operator.isSatisfiedBy(Operator.java:201) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:719)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:324)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:120) 
> ~[main/:na]
>   at org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:110) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:44) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.Transformation.add(Transformation.java:174) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.Transformation.apply(Transformation.java:140)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:307)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:292)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:310)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:145)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:138)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:134)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:333) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1884)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2587)
>  ~[main/:na]
>   ... 5 common frames omitted
> {noformat}
> 2)
> Similarly, if an index is created on the duration column:
> {noformat}
> CREATE INDEX d_index ON simplex.duration_table (d);
> SELECT * from duration_table WHERE d=1s;
> {noformat}
> 

[jira] [Commented] (CASSANDRA-13174) Indexing is allowed on Duration type when it should not be

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880433#comment-15880433
 ] 

Benjamin Lerer commented on CASSANDRA-13174:


Thanks for the review :-)

> Indexing is allowed on Duration type when it should not be
> --
>
> Key: CASSANDRA-13174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13174
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: C* 3.10
>Reporter: Kishan Karunaratne
>Assignee: Benjamin Lerer
> Fix For: 3.11.x, 4.x
>
>
> Looks like secondary indexing is allowed on duration type columns. Since 
> comparisons are not possible for the duration type, indexing on it also 
> should be invalid.
> 1) 
> {noformat}
> CREATE TABLE duration_table (k int PRIMARY KEY, d duration);
> INSERT INTO duration_table (k, d) VALUES (0, 1s);
> SELECT * from duration_table WHERE d=1s ALLOW FILTERING;
> {noformat}
> The above throws an error: 
> {noformat}
> WARN  [ReadStage-2] 2017-01-31 17:09:57,821 
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread 
> Thread[ReadStage-2,10,main]: {}
> java.lang.RuntimeException: java.lang.UnsupportedOperationException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2591)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_91]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  ~[main/:na]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
>  [main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: java.lang.UnsupportedOperationException: null
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compareCustom(AbstractType.java:174)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:160) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.compareForCQL(AbstractType.java:204)
>  ~[main/:na]
>   at org.apache.cassandra.cql3.Operator.isSatisfiedBy(Operator.java:201) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:719)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:324)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:120) 
> ~[main/:na]
>   at org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:110) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:44) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.Transformation.add(Transformation.java:174) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.Transformation.apply(Transformation.java:140)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:307)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:292)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:310)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:145)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:138)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:134)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:333) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1884)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2587)
>  ~[main/:na]
>   ... 5 common frames omitted
> {noformat}
> 2)
> Similarly, if an index is created on the duration column:
> {noformat}
> CREATE INDEX d_index ON simplex.duration_table (d);
> SELECT * from duration_table WHERE d=1s;
> {noformat}
> results in:
> {noformat}
> WARN  [ReadStage-2] 2017-01-31 17:12:00,623 
> 

[1/2] cassandra git commit: Fix equality comparisons of columns using the duration type

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6e17d65f2 -> cd29d44df


Fix equality comparisons of columns using the duration type

patch by Benjamin Lerer; reviewed by Tyler Hobbs for CASSANDRA-13174


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6487876d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6487876d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6487876d

Branch: refs/heads/trunk
Commit: 6487876dde14c46d5753f972909e5acec854cb53
Parents: 0fe76da
Author: Benjamin Lerer 
Authored: Thu Feb 23 14:05:30 2017 +0100
Committer: Benjamin Lerer 
Committed: Thu Feb 23 14:05:30 2017 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  |  19 ++
 .../org/apache/cassandra/cql3/Operator.java |   9 +
 .../cassandra/cql3/SingleColumnRelation.java|  10 +-
 .../cql3/statements/CreateIndexStatement.java   |  12 +
 .../cassandra/db/marshal/AbstractType.java  |   5 +
 .../cassandra/db/marshal/DurationType.java  |   3 +-
 .../apache/cassandra/db/marshal/TupleType.java  |   6 +
 .../apache/cassandra/db/marshal/UserType.java   |   6 +
 .../validation/entities/SecondaryIndexTest.java |  27 ++
 .../cql3/validation/operations/CreateTest.java  |   4 +-
 .../operations/InsertUpdateIfConditionTest.java | 250 +++
 .../cql3/validation/operations/SelectTest.java  | 165 
 13 files changed, 512 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6487876d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3e38844..233898f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.0
+ * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
  * Obfuscate password in stress-graphs (CASSANDRA-12233)
  * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
  * nodetool stopdaemon errors out (CASSANDRA-13030)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6487876d/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 5395a9b..acb95a4 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -26,12 +26,16 @@ import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.Term.Terminal;
 import org.apache.cassandra.cql3.functions.Function;
+import org.apache.cassandra.cql3.statements.RequestValidations;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 import org.apache.cassandra.transport.ProtocolVersion;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
+import static 
org.apache.cassandra.cql3.statements.RequestValidations.checkFalse;
+import static 
org.apache.cassandra.cql3.statements.RequestValidations.invalidRequest;
+
 /**
  * A CQL3 condition on the value of a column or collection element.  For 
example, "UPDATE .. IF a = 0".
  */
@@ -1037,6 +1041,7 @@ public class ColumnCondition
 default:
 throw new AssertionError();
 }
+
 if (operator == Operator.IN)
 {
 if (inValues == null)
@@ -1048,6 +1053,7 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(valueSpec.type);
 return ColumnCondition.condition(receiver, 
collectionElement.prepare(keyspace, elementSpec), value.prepare(keyspace, 
valueSpec), operator);
 }
 }
@@ -1071,6 +1077,7 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(fieldReceiver.type);
 return ColumnCondition.condition(receiver, udtField, 
value.prepare(keyspace, fieldReceiver), operator);
 }
 }
@@ -1087,9 +1094,21 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(receiver.type);
 return ColumnCondition.condition(receiver, 
value.prepare(keyspace, receiver), operator);
 }
 }
 }
+
+private void validateOperationOnDurations(AbstractType type)
+{
+if 

[2/2] cassandra git commit: Merge branch cassandra-3.11 into trunk

[blerer]

Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cd29d44d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cd29d44d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cd29d44d

Branch: refs/heads/trunk
Commit: cd29d44dfeb2a9a4a4bf2b63bedb71503bda8760
Parents: 6e17d65 6487876
Author: Benjamin Lerer 
Authored: Thu Feb 23 14:26:29 2017 +0100
Committer: Benjamin Lerer 
Committed: Thu Feb 23 14:36:36 2017 +0100

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/cql3/Operator.java |   9 +
 .../cassandra/cql3/SingleColumnRelation.java|  10 +-
 .../cql3/conditions/ColumnCondition.java|  14 ++
 .../cql3/statements/CreateIndexStatement.java   |  11 +
 .../cassandra/db/marshal/DurationType.java  |   3 +-
 .../validation/entities/SecondaryIndexTest.java |  26 ++
 .../cql3/validation/operations/CreateTest.java  |   4 +-
 .../operations/InsertUpdateIfConditionTest.java | 250 +++
 .../cql3/validation/operations/SelectTest.java  | 165 
 10 files changed, 488 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cd29d44d/CHANGES.txt
--
diff --cc CHANGES.txt
index 28ca6d9,233898f..0641011
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,41 -1,6 +1,42 @@@
 +4.0
 + * Adds the ability to use uncompressed chunks in compressed files 
(CASSANDRA-10520)
 + * Don't flush sstables when streaming for incremental repair 
(CASSANDRA-13226)
 + * Remove unused method (CASSANDRA-13227)
 + * Fix minor bugs related to #9143 (CASSANDRA-13217)
 + * Output warning if user increases RF (CASSANDRA-13079)
 + * Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
 + * Add support for + and - operations on dates (CASSANDRA-11936)
 + * Fix consistency of incrementally repaired data (CASSANDRA-9143)
 + * Increase commitlog version (CASSANDRA-13161)
 + * Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
 + * Refactor ColumnCondition (CASSANDRA-12981)
 + * Parallelize streaming of different keyspaces (CASSANDRA-4663)
 + * Improved compactions metrics (CASSANDRA-13015)
 + * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
 + * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
 + * Thrift removal (CASSANDRA-5)
 + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool Documentation (CASSANDRA-12672)
 + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
 + * Reject invalid replication settings when creating or altering a keyspace 
(CASSANDRA-12681)
 + * Clean up the SSTableReader#getScanner API wrt removal of RateLimiter 
(CASSANDRA-12422)
 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
 + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
 + * Require forceful decommission if number of nodes is less than replication 
factor (CASSANDRA-12510)
 + * Allow IN restrictions on column families with collections (CASSANDRA-12654)
 + * Log message size in trace message in OutboundTcpConnection 
(CASSANDRA-13028)
 + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
 + * Add mutation size and batch metrics (CASSANDRA-12649)
 + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
 + * Expose time spent waiting in thread pool queue (CASSANDRA-8398)
 + * Conditionally update index built status to avoid unnecessary flushes 
(CASSANDRA-12969)
 + * cqlsh auto completion: refactor definition of compaction strategy options 
(CASSANDRA-12946)
 + * Add support for arithmetic operators (CASSANDRA-11935)
 +
 +
  3.11.0
+  * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
 - * Obfuscate password in stress-graphs (CASSANDRA-12233)
   * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
   * nodetool stopdaemon errors out (CASSANDRA-13030)
   * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cd29d44d/src/java/org/apache/cassandra/cql3/Operator.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cd29d44d/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java
--
diff --cc src/java/org/apache/cassandra/cql3/SingleColumnRelation.java
index 412eb16,ae07f56..bbeb560
--- a/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java
+++ 

cassandra git commit: Fix equality comparisons of columns using the duration type

[blerer]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 0fe76da17 -> 6487876dd


Fix equality comparisons of columns using the duration type

patch by Benjamin Lerer; reviewed by Tyler Hobbs for CASSANDRA-13174


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6487876d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6487876d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6487876d

Branch: refs/heads/cassandra-3.11
Commit: 6487876dde14c46d5753f972909e5acec854cb53
Parents: 0fe76da
Author: Benjamin Lerer 
Authored: Thu Feb 23 14:05:30 2017 +0100
Committer: Benjamin Lerer 
Committed: Thu Feb 23 14:05:30 2017 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnCondition.java  |  19 ++
 .../org/apache/cassandra/cql3/Operator.java |   9 +
 .../cassandra/cql3/SingleColumnRelation.java|  10 +-
 .../cql3/statements/CreateIndexStatement.java   |  12 +
 .../cassandra/db/marshal/AbstractType.java  |   5 +
 .../cassandra/db/marshal/DurationType.java  |   3 +-
 .../apache/cassandra/db/marshal/TupleType.java  |   6 +
 .../apache/cassandra/db/marshal/UserType.java   |   6 +
 .../validation/entities/SecondaryIndexTest.java |  27 ++
 .../cql3/validation/operations/CreateTest.java  |   4 +-
 .../operations/InsertUpdateIfConditionTest.java | 250 +++
 .../cql3/validation/operations/SelectTest.java  | 165 
 13 files changed, 512 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6487876d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3e38844..233898f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.0
+ * Fix equality comparisons of columns using the duration type 
(CASSANDRA-13174)
  * Obfuscate password in stress-graphs (CASSANDRA-12233)
  * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
  * nodetool stopdaemon errors out (CASSANDRA-13030)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6487876d/src/java/org/apache/cassandra/cql3/ColumnCondition.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnCondition.java 
b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
index 5395a9b..acb95a4 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnCondition.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnCondition.java
@@ -26,12 +26,16 @@ import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
 import org.apache.cassandra.cql3.Term.Terminal;
 import org.apache.cassandra.cql3.functions.Function;
+import org.apache.cassandra.cql3.statements.RequestValidations;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 import org.apache.cassandra.transport.ProtocolVersion;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
+import static 
org.apache.cassandra.cql3.statements.RequestValidations.checkFalse;
+import static 
org.apache.cassandra.cql3.statements.RequestValidations.invalidRequest;
+
 /**
  * A CQL3 condition on the value of a column or collection element.  For 
example, "UPDATE .. IF a = 0".
  */
@@ -1037,6 +1041,7 @@ public class ColumnCondition
 default:
 throw new AssertionError();
 }
+
 if (operator == Operator.IN)
 {
 if (inValues == null)
@@ -1048,6 +1053,7 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(valueSpec.type);
 return ColumnCondition.condition(receiver, 
collectionElement.prepare(keyspace, elementSpec), value.prepare(keyspace, 
valueSpec), operator);
 }
 }
@@ -1071,6 +1077,7 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(fieldReceiver.type);
 return ColumnCondition.condition(receiver, udtField, 
value.prepare(keyspace, fieldReceiver), operator);
 }
 }
@@ -1087,9 +1094,21 @@ public class ColumnCondition
 }
 else
 {
+validateOperationOnDurations(receiver.type);
 return ColumnCondition.condition(receiver, 
value.prepare(keyspace, receiver), operator);
 }
 }
 }
+
+private void validateOperationOnDurations(AbstractType type)
+{
+  

[jira] [Created] (CASSANDRA-13259) Use platform specific X.509 default algorithm

[Stefan Podkowinski (JIRA)]

Stefan Podkowinski created CASSANDRA-13259:
--

 Summary: Use platform specific X.509 default algorithm
 Key: CASSANDRA-13259
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13259
 Project: Cassandra
  Issue Type: Improvement
Reporter: Stefan Podkowinski
Assignee: Stefan Podkowinski
Priority: Minor


We should replace the hardcoded "SunX509" default algorithm and use the JRE 
default instead. This implementation will currently not work on less popular 
platforms (e.g. IBM) and won't get any further updates.

See also:
https://bugs.openjdk.java.net/browse/JDK-8169745



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13071) cqlsh copy-from should error out when csv contains invalid data for collections

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-13071:
-
Status: Patch Available  (was: In Progress)

> cqlsh copy-from should error out when csv contains invalid data for 
> collections
> ---
>
> Key: CASSANDRA-13071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
>
> If the csv file contains invalid data for collection types, at the moment the 
> data is imported incorrectly, an error would be a better behavior.
> For example this table:
> {code}
> CREATE TABLE test.test (key text, value frozen, PRIMARY KEY 
> (key)); 
> {code}
> with this data:
> {code}
> "key1","{'test1', 'test2'}"
> "Key2","not_a_set"
> {code}
> will be imported by {{COPY test.test FROM 'test.csv';}} without errors but 
> will result in the following data:
> {code}
> cqlsh> select * from test.test;
>  key  | value
> --+
>  key1 | {'test1', 'test2'}
>  Key2 |{'ot_a_se'}
> (2 rows)
> {code}
> The second row should have been rejected. The reason is that the [{{split}} 
> function|https://github.com/stef1927/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L1898]
>  assumes that the first and last characters of the string passed in are 
> parentheses, without actually checking it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13071) cqlsh copy-from should error out when csv contains invalid data for collections

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880271#comment-15880271
 ] 

Stefania commented on CASSANDRA-13071:
--

Patch is ready [~pauloricardomg]:

||3.0||3.11||trunk||
|[patch|https://github.com/stef1927/cassandra/tree/13071-cqlsh-3.0]|[patch|https://github.com/stef1927/cassandra/tree/13071-cqlsh-3.11]|[patch|https://github.com/stef1927/cassandra/tree/13071-cqlsh]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-13071-cqlsh-3.0-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-13071-cqlsh-3.11-cqlsh-tests/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-13071-cqlsh-cqlsh-tests/]|

Tests are [here|https://github.com/riptano/cassandra-dtest/pull/1448].

> cqlsh copy-from should error out when csv contains invalid data for 
> collections
> ---
>
> Key: CASSANDRA-13071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
>
> If the csv file contains invalid data for collection types, at the moment the 
> data is imported incorrectly, an error would be a better behavior.
> For example this table:
> {code}
> CREATE TABLE test.test (key text, value frozen, PRIMARY KEY 
> (key)); 
> {code}
> with this data:
> {code}
> "key1","{'test1', 'test2'}"
> "Key2","not_a_set"
> {code}
> will be imported by {{COPY test.test FROM 'test.csv';}} without errors but 
> will result in the following data:
> {code}
> cqlsh> select * from test.test;
>  key  | value
> --+
>  key1 | {'test1', 'test2'}
>  Key2 |{'ot_a_se'}
> (2 rows)
> {code}
> The second row should have been rejected. The reason is that the [{{split}} 
> function|https://github.com/stef1927/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L1898]
>  assumes that the first and last characters of the string passed in are 
> parentheses, without actually checking it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12653) In-flight shadow round requests

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880207#comment-15880207
 ] 

Stefan Podkowinski commented on CASSANDRA-12653:


The goal of introducing the firstSynSendAt timestamp was to prevent processing 
ACKs (lagged over from shadow gossip round) before any regular SYN has been 
send. I still think checking this value against the local message construction 
time is a good idea, as messages will be queued before getting dispatched and 
it's not that unlikely that the scheduler will make this happen for older 
messages just after we send the syn. As [mentioned 
before|https://issues.apache.org/jira/browse/CASSANDRA-12653?focusedCommentId=15818621=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15818621]
 the proper way to do this would be to be able to correlate between messages or 
do the shadow round conversation based on new dedicated message types.


> In-flight shadow round requests
> ---
>
> Key: CASSANDRA-12653
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12653
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Bootstrapping or replacing a node in the cluster requires to gather and check 
> some host IDs or tokens by doing a gossip "shadow round" once before joining 
> the cluster. This is done by sending a gossip SYN to all seeds until we 
> receive a response with the cluster state, from where we can move on in the 
> bootstrap process. Receiving a response will call the shadow round done and 
> calls {{Gossiper.resetEndpointStateMap}} for cleaning up the received state 
> again.
> The issue here is that at this point there might be other in-flight requests 
> and it's very likely that shadow round responses from other seeds will be 
> received afterwards, while the current state of the bootstrap process doesn't 
> expect this to happen (e.g. gossiper may or may not be enabled). 
> One side effect will be that MigrationTasks are spawned for each shadow round 
> reply except the first. Tasks might or might not execute based on whether at 
> execution time {{Gossiper.resetEndpointStateMap}} had been called, which 
> effects the outcome of {{FailureDetector.instance.isAlive(endpoint))}} at 
> start of the task. You'll see error log messages such as follows when this 
> happend:
> {noformat}
> INFO  [SharedPool-Worker-1] 2016-09-08 08:36:39,255 Gossiper.java:993 - 
> InetAddress /xx.xx.xx.xx is now UP
> ERROR [MigrationStage:1]2016-09-08 08:36:39,255 FailureDetector.java:223 
> - unknown endpoint /xx.xx.xx.xx
> {noformat}
> Although is isn't pretty, I currently don't see any serious harm from this, 
> but it would be good to get a second opinion (feel free to close as "wont 
> fix").
> /cc [~Stefania] [~thobbs]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13245) Unable to match traces to queries

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-13245:
--

Assignee: (was: Benjamin Lerer)

> Unable to match traces to queries
> -
>
> Key: CASSANDRA-13245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13245
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Eric Evans
>
> Tracing queries node-wide ala {{nodetool settraceprobability}} is of limited 
> utility when you are using prepared statements; I cannot find any way of 
> associating a trace session to an application query (it's not even possible 
> to make out the keyspace name).
> https://gist.github.com/eevans/81650261c2b1b5b99f83112865fef24b



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12213:
-
Component/s: Distributed Metadata

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.x, 3.11.x
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880179#comment-15880179
 ] 

Stefania commented on CASSANDRA-12213:
--

Both multiplexed runs completed without failures. CI results have failures but 
they are all present on unpatched branches except for this 
[one|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12213-3.0-dtest/lastCompletedBuild/testReport/repair_tests.repair_test/TestRepair/simple_sequential_repair_test/],
 which passes locally and seems unrelated to this patch.

[~thobbs] are you OK to review?

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.x, 3.11.x
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12213) dtest failure in write_failures_test.TestWriteFailures.test_paxos_any

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12213:
-
Fix Version/s: 3.0.x
   Status: Patch Available  (was: In Progress)

> dtest failure in write_failures_test.TestWriteFailures.test_paxos_any
> -
>
> Key: CASSANDRA-12213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12213
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Craig Kodman
>Assignee: Stefania
>  Labels: dtest
> Fix For: 3.0.x, 3.11.x
>
> Attachments: jenkins-stef1927-12014-dtest-2_logs.001.tar.gz, 
> node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, 
> node2.log, node3_debug.log, node3_gc.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_paxos_any
> and:
> http://cassci.datastax.com/job/cassandra-3.9_dtest/10/testReport/write_failures_test/TestWriteFailures/test_mutation_v3/
> Failed on CassCI build cassandra-3.9_dtest #10



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880149#comment-15880149
 ] 

Stefania commented on CASSANDRA-12100:
--

Thanks, committed to 2.2 as dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74.

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 2.2.10, 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12100:
-
Fix Version/s: 2.2.10

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 2.2.10, 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-12100) Compactions are stuck after TRUNCATE

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880149#comment-15880149
 ] 

Stefania edited comment on CASSANDRA-12100 at 2/23/17 9:07 AM:
---

Thanks, committed to 2.2 as {{dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74}} and 
merged upwards with {{-s ours}}.


was (Author: stefania):
Thanks, committed to 2.2 as dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74.

> Compactions are stuck after TRUNCATE
> 
>
> Key: CASSANDRA-12100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Stefano Ortolani
>Assignee: Stefania
> Fix For: 2.2.10, 3.0.9, 3.8
>
> Attachments: node3_jstack.log
>
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when 
> truncating the column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>  id   compaction type  
> keyspacetable   completed  totalunit   progress
>24e1ad30-3cac-11e6-870d-5de740693258Compaction  
> schema  table_1   0   57558382   bytes  0.00%
>2be2e3b0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_2   0   65063705   bytes  0.00%
>54de38f0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_3   0 187031   bytes  0.00%
>31926ce0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_4   0   42951119   bytes  0.00%
>3911ad00-3cac-11e6-870d-5de740693258Compaction  
> schema  table_5   0   25918949   bytes  0.00%
>3e6a8ab0-3cac-11e6-870d-5de740693258Compaction  
> schema  table_6   0   65466210   bytes  0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[stefania]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e17d65f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e17d65f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e17d65f

Branch: refs/heads/trunk
Commit: 6e17d65f26fc37257c62a49de575b3da5f447a6d
Parents: ac21013 0fe76da
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:05:26 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:05:26 2017 +

--

--

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/42977dbc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/42977dbc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/42977dbc

Branch: refs/heads/cassandra-3.0
Commit: 42977dbcea5b81ef3cd679808e06a0846ce9b167
Parents: 61f0c98 dffb1a6
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:04:42 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:04:42 2017 +

--

--

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/42977dbc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/42977dbc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/42977dbc

Branch: refs/heads/cassandra-3.11
Commit: 42977dbcea5b81ef3cd679808e06a0846ce9b167
Parents: 61f0c98 dffb1a6
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:04:42 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:04:42 2017 +

--

--

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[stefania]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/42977dbc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/42977dbc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/42977dbc

Branch: refs/heads/trunk
Commit: 42977dbcea5b81ef3cd679808e06a0846ce9b167
Parents: 61f0c98 dffb1a6
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:04:42 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:04:42 2017 +

--

--

[01/10] cassandra git commit: Make sure compaction stats are updated when compaction is interrupted (back-port from 3.0)

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 6ffd5cc5d -> dffb1a6da
  refs/heads/cassandra-3.0 61f0c988f -> 42977dbce
  refs/heads/cassandra-3.11 b887ae944 -> 0fe76da17
  refs/heads/trunk ac2101305 -> 6e17d65f2


Make sure compaction stats are updated when compaction is interrupted 
(back-port from 3.0)

patch by Stefania Alborghetti; reviewed by Marcus Eriksson for CASSANDRA-12100


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dffb1a6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dffb1a6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dffb1a6d

Branch: refs/heads/cassandra-2.2
Commit: dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74
Parents: 6ffd5cc
Author: Stefania Alborghetti 
Authored: Thu Aug 4 14:20:38 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 08:58:11 2017 +

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 033b366..b565acb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
  * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
  * Fix failing COPY TO STDOUT (CASSANDRA-12497)
  * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 7489b3d..006c8ff 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -165,13 +165,14 @@ public class CompactionTask extends AbstractCompactionTask
 ci = new CompactionIterable(compactionType, scanners.scanners, 
controller, sstableFormat, taskId);
 try (CloseableIterator iter = 
ci.iterator())
 {
-if (collector != null)
-collector.beginCompaction(ci);
 long lastCheckObsoletion = start;
 
 if (!controller.cfs.getCompactionStrategy().isActive)
 throw new 
CompactionInterruptedException(ci.getCompactionInfo());
 
+if (collector != null)
+collector.beginCompaction(ci);
+
 try (CompactionAwareWriter writer = 
getCompactionAwareWriter(cfs, transaction, actuallyCompact))
 {
 estimatedKeys = writer.estimatedKeys();

[03/10] cassandra git commit: Make sure compaction stats are updated when compaction is interrupted (back-port from 3.0)

[stefania]

Make sure compaction stats are updated when compaction is interrupted 
(back-port from 3.0)

patch by Stefania Alborghetti; reviewed by Marcus Eriksson for CASSANDRA-12100


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dffb1a6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dffb1a6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dffb1a6d

Branch: refs/heads/cassandra-3.11
Commit: dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74
Parents: 6ffd5cc
Author: Stefania Alborghetti 
Authored: Thu Aug 4 14:20:38 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 08:58:11 2017 +

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 033b366..b565acb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
  * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
  * Fix failing COPY TO STDOUT (CASSANDRA-12497)
  * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 7489b3d..006c8ff 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -165,13 +165,14 @@ public class CompactionTask extends AbstractCompactionTask
 ci = new CompactionIterable(compactionType, scanners.scanners, 
controller, sstableFormat, taskId);
 try (CloseableIterator iter = 
ci.iterator())
 {
-if (collector != null)
-collector.beginCompaction(ci);
 long lastCheckObsoletion = start;
 
 if (!controller.cfs.getCompactionStrategy().isActive)
 throw new 
CompactionInterruptedException(ci.getCompactionInfo());
 
+if (collector != null)
+collector.beginCompaction(ci);
+
 try (CompactionAwareWriter writer = 
getCompactionAwareWriter(cfs, transaction, actuallyCompact))
 {
 estimatedKeys = writer.estimatedKeys();

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe76da1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe76da1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe76da1

Branch: refs/heads/trunk
Commit: 0fe76da17ea21237a623cf88121b3d98f30b97fd
Parents: b887ae9 42977db
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:05:07 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:05:07 2017 +

--

--

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0fe76da1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0fe76da1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0fe76da1

Branch: refs/heads/cassandra-3.11
Commit: 0fe76da17ea21237a623cf88121b3d98f30b97fd
Parents: b887ae9 42977db
Author: Stefania Alborghetti 
Authored: Thu Feb 23 09:05:07 2017 +
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 09:05:07 2017 +

--

--

[02/10] cassandra git commit: Make sure compaction stats are updated when compaction is interrupted (back-port from 3.0)

[stefania]

Make sure compaction stats are updated when compaction is interrupted 
(back-port from 3.0)

patch by Stefania Alborghetti; reviewed by Marcus Eriksson for CASSANDRA-12100


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dffb1a6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dffb1a6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dffb1a6d

Branch: refs/heads/cassandra-3.0
Commit: dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74
Parents: 6ffd5cc
Author: Stefania Alborghetti 
Authored: Thu Aug 4 14:20:38 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 08:58:11 2017 +

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 033b366..b565acb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
  * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
  * Fix failing COPY TO STDOUT (CASSANDRA-12497)
  * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 7489b3d..006c8ff 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -165,13 +165,14 @@ public class CompactionTask extends AbstractCompactionTask
 ci = new CompactionIterable(compactionType, scanners.scanners, 
controller, sstableFormat, taskId);
 try (CloseableIterator iter = 
ci.iterator())
 {
-if (collector != null)
-collector.beginCompaction(ci);
 long lastCheckObsoletion = start;
 
 if (!controller.cfs.getCompactionStrategy().isActive)
 throw new 
CompactionInterruptedException(ci.getCompactionInfo());
 
+if (collector != null)
+collector.beginCompaction(ci);
+
 try (CompactionAwareWriter writer = 
getCompactionAwareWriter(cfs, transaction, actuallyCompact))
 {
 estimatedKeys = writer.estimatedKeys();

[04/10] cassandra git commit: Make sure compaction stats are updated when compaction is interrupted (back-port from 3.0)

[stefania]

Make sure compaction stats are updated when compaction is interrupted 
(back-port from 3.0)

patch by Stefania Alborghetti; reviewed by Marcus Eriksson for CASSANDRA-12100


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dffb1a6d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dffb1a6d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dffb1a6d

Branch: refs/heads/trunk
Commit: dffb1a6da8e2c3a9c08bb94bfd12130b9ddded74
Parents: 6ffd5cc
Author: Stefania Alborghetti 
Authored: Thu Aug 4 14:20:38 2016 +0800
Committer: Stefania Alborghetti 
Committed: Thu Feb 23 08:58:11 2017 +

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 5 +++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 033b366..b565acb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.10
+ * Make sure compaction stats are updated when compaction is interrupted 
(Backport from 3.0, CASSANDRA-12100)
  * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
  * Fix failing COPY TO STDOUT (CASSANDRA-12497)
  * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dffb1a6d/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 7489b3d..006c8ff 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -165,13 +165,14 @@ public class CompactionTask extends AbstractCompactionTask
 ci = new CompactionIterable(compactionType, scanners.scanners, 
controller, sstableFormat, taskId);
 try (CloseableIterator iter = 
ci.iterator())
 {
-if (collector != null)
-collector.beginCompaction(ci);
 long lastCheckObsoletion = start;
 
 if (!controller.cfs.getCompactionStrategy().isActive)
 throw new 
CompactionInterruptedException(ci.getCompactionInfo());
 
+if (collector != null)
+collector.beginCompaction(ci);
+
 try (CompactionAwareWriter writer = 
getCompactionAwareWriter(cfs, transaction, actuallyCompact))
 {
 estimatedKeys = writer.estimatedKeys();

[jira] [Commented] (CASSANDRA-13006) Disable automatic heap dumps on OOM error

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880141#comment-15880141
 ] 

Benjamin Lerer commented on CASSANDRA-13006:


Sorry, for the misunderstanding.
Your patch fix only one part of the problem and I would rather fix all the 
problems at once than solving them in multiple tickets.

Basically, the fix need to take into account the JVM being used and handle the 
processing of {{OOM}} errors accordingly. It also need to support options like 
{{-XX:OnOutOfMemoryError}}.
Apparently, {{Oracle}} and {{Zing}} JVMs use the same options names. The 
{{IBM}} one seems to use different options.
The patch should also log a clear error message if the JVM is not supported.

If you are still interested in working on this issue, feel free to reasign it 
to yourself. 

> Disable automatic heap dumps on OOM error
> -
>
> Key: CASSANDRA-13006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13006
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: anmols
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 3.0.9
>
> Attachments: 13006-3.0.9.txt
>
>
> With CASSANDRA-9861, a change was added to enable collecting heap dumps by 
> default if the process encountered an OOM error. These heap dumps are stored 
> in the Apache Cassandra home directory unless configured otherwise (see 
> [Cassandra Support 
> Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps]
>  for this feature).
>  
> The creation and storage of heap dumps aides debugging and investigative 
> workflows, but is not be desirable for a production environment where these 
> heap dumps may occupy a large amount of disk space and require manual 
> intervention for cleanups. 
>  
> Managing heap dumps on out of memory errors and configuring the paths for 
> these heap dumps are available as JVM options in JVM. The current behavior 
> conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. 
>  
> A patch can be proposed here that would make the heap dump on OOM error honor 
> the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate 
> heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM 
> option.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13252) Include RPM packages in releases

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880123#comment-15880123
 ] 

Stefan Podkowinski commented on CASSANDRA-13252:


Snapshot packages seems like a good way to get started to work on production 
ready RPMs. There are probably still some issues to work out and to improve 
(systemd support?) that could use feedback from the community, without having 
to wait a full release cycle before being able to address the issue in another 
published RPM.

> Include RPM packages in releases
> 
>
> Key: CASSANDRA-13252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13252
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Packaging
>Reporter: Michael Shuler
>Assignee: Michael Shuler
>
> CASSANDRA-13230 sets up docker builds for deb and rpm packages out of the 
> {{cassandra-builds}} git repo. We should now be able to do builds in CI, at 
> least for build testing of snapshot packages.
> In order to include in releases, we may wish to set up bintray for RPMs in a 
> {{redhat/}} dir, similar to the current {{debian/}} dir redirect from 
> cassandra-dist, or start with direct downloads from cassandra-dist and see 
> how it goes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-9556) Add newer data types to cassandra stress

[Andrei Pavel (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880112#comment-15880112
 ] 

Andrei Pavel commented on CASSANDRA-9556:
-

Has UDT support been logged under a separate ticket yet?

> Add newer data types to cassandra stress
> 
>
> Key: CASSANDRA-9556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9556
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jeremy Hanna
>Assignee: ZhaoYang
>Priority: Minor
>  Labels: stress
> Fix For: 2.2.5, 3.0.3
>
> Attachments: CASSANDRA-9556-2.2.patch
>
>
> Currently you can't define a data model with decimal types and use Cassandra 
> stress with it.  Also, I imagine that holds true with other newer data types 
> such as the new date and time types.  Besides that, now that data models are 
> including user defined types, we should allow users to create those 
> structures with stress as well.  Perhaps we could split out the UDTs into a 
> different ticket if it holds the other types up.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12760) SELECT JSON "firstName" FROM ... results in {"\"firstName\"": "Bill"}

[Niek Bartholomeus (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880078#comment-15880078
 ] 

Niek Bartholomeus commented on CASSANDRA-12760:
---

Hi Paul, indeed you're right this seems to be the intended behaviour, so I 
changed this jira ticket from bug to improvement. I still think the optimal 
solution would not include the double quotes in the json though as json is 
intended for communication with the outside world it should ideally not contain 
any cassandra-specific implementation details like this double quoting.

> SELECT JSON "firstName" FROM ... results in {"\"firstName\"": "Bill"}
> -
>
> Key: CASSANDRA-12760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12760
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Cassandra 3.7
>Reporter: Niek Bartholomeus
>Assignee: Shivang Nagaria
>  Labels: lhf
>
> I'm using Cassandra to store data coming from Spark and intended for being 
> consumed by a javascript front end.
> To avoid unnecessary field name mappings I have decided to use mixed case 
> fields in Cassandra. I also happily leave it to Cassandra to jsonify the data 
> (using SELECT JSON ...) so my scala/play web server can send the results from 
> Cassandra straight through to the front end.
> I noticed however that all mixed case fields (that were created with quotes 
> as Cassandra demands) end up having a double set of quotes
> {code}
> create table user(id text PRIMARY KEY, "firstName" text);
> insert into user(id, "firstName") values ('b', 'Bill');
> select json * from user;
>  [json]
> --
>  {"id": "b", "\"firstName\"": "Bill"}
> {code}
> Ideally that would be:
> {code}
>  [json]
> --
>  {"id": "b", "firstName": "Bill"}
> {code}
> I worked around it for now by removing all "\""'s before sending the json to 
> the front end.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880077#comment-15880077
 ] 

Michael Kjellman commented on CASSANDRA-9754:
-

Just wanted to give a quick update:

# I'm really sorry for the delay getting this finished for trunk. I've started 
a trunk based/post 8099 version 3 times now -- the holidays happened -- more 
pressing things stole my attention -- big commits like removing Thrift, 
CFMetadata, etc, etc kept getting committed before I was done, and well -- 
enough excuses from me...
# A belated thanks for your initial comments Branimir -- I did read them and 
I'll be addressing them with my trunk rebased changes.
# I'm almost done with the refactoring to move all the current array based 
index logic into a IndexEntry implementation. I have all unit tests passing 
(finally) with the exception of KeyCacheCqlTest (which I'm working on right 
now).
# Assuming I get the post 8099 Indexed Iterator based abstractions/changes 
correct it should be a matter of just dropping in the Birch package/classes I 
had for 2.1 and switching the default serializer to use the Birch IndexedEntry 
implementation.


> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 0f8e28c220fd5af6c7b5dd2d3dab6936c4aa4b6b.patch, 
> gc_collection_times_with_birch.png, gc_collection_times_without_birch.png, 
> gc_counts_with_birch.png, gc_counts_without_birch.png, 
> perf_cluster_1_with_birch_read_latency_and_counts.png, 
> perf_cluster_1_with_birch_write_latency_and_counts.png, 
> perf_cluster_2_with_birch_read_latency_and_counts.png, 
> perf_cluster_2_with_birch_write_latency_and_counts.png, 
> perf_cluster_3_without_birch_read_latency_and_counts.png, 
> perf_cluster_3_without_birch_write_latency_and_counts.png
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)