[jira] [Updated] (CASSANDRA-13226) StreamPlan for incremental repairs flushing memtables unnecessarily

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13226:

Status: Ready to Commit  (was: Patch Available)

> StreamPlan for incremental repairs flushing memtables unnecessarily
> ---
>
> Key: CASSANDRA-13226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13226
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> Since incremental repairs are run against a fixed dataset, there's no need to 
> flush memtables when streaming for them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13226) StreamPlan for incremental repairs flushing memtables unnecessarily

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871301#comment-15871301
 ] 

Marcus Eriksson commented on CASSANDRA-13226:
-

+1 - maybe just add a comment why we don't need to flush on incremental repairs?

> StreamPlan for incremental repairs flushing memtables unnecessarily
> ---
>
> Key: CASSANDRA-13226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13226
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> Since incremental repairs are run against a fixed dataset, there's no need to 
> flush memtables when streaming for them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13226) StreamPlan for incremental repairs flushing memtables unnecessarily

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13226:

Reviewer: Marcus Eriksson

> StreamPlan for incremental repairs flushing memtables unnecessarily
> ---
>
> Key: CASSANDRA-13226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13226
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Minor
> Fix For: 4.0
>
>
> Since incremental repairs are run against a fixed dataset, there's no need to 
> flush memtables when streaming for them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-11983) Migration task failed to complete

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-11983:
---
Reproduced In: 3.0.10

> Migration task failed to complete
> -
>
> Key: CASSANDRA-11983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11983
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle
> Environment: Docker / Kubernetes running
> Linux cassandra-21 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-1 (2016-03-06) 
> x86_64 GNU/Linux
> openjdk version "1.8.0_91"
> OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-1~bpo8+1-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassnadra 3.5 installed from 
> deb-src http://www.apache.org/dist/cassandra/debian 35x main
>Reporter: Chris Love
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x
>
> Attachments: cass.log
>
>
> When nodes are boostrapping I am getting mulitple errors: "Migration task 
> failed to complete", from MigrationManager.java
> The errors increase as more nodes are added to the ring, as I am creating a 
> ring of 1k nodes.
> Cassandra yaml i here 
> https://github.com/k8s-for-greeks/gpmr/blob/3d50ff91a139b9c4a7a26eda0fb4dcf9a008fbed/pet-race-devops/docker/cassandra-debian/files/cassandra.yaml



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13232) "multiple versions of ant detected in path for junit" printed for every junit test case spawned by "ant test"

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-13232:
--

Assignee: Michael Kjellman

> "multiple versions of ant detected in path for junit" printed for every junit 
> test case spawned by "ant test"
> -
>
> Key: CASSANDRA-13232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13232
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
> Attachments: 673.diff
>
>
> There is a super annoying junit warning logged before every junit test case 
> when you run "ant test". This is due to the fact that the ant junit task that 
> we have configured in our build.xml sources the system class path and most 
> importantly what's in ant.library.dir.
> [junit] WARNING: multiple versions of ant detected in path for junit 
> [junit]  
> jar:file:/usr/local/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
> [junit]  and 
> jar:file:/Users/mkjellman/Documents/mkjellman-cie-cassandra-trunk/build/lib/jars/ant-1.9.6.jar!/org/apache/tools/ant/Project.class
> The fix here is to explicitly exclude the ant jar downloaded from the maven 
> tasks that ends up in ${build.lib} and ${build.dir.lib} so only the ant 
> libraries from the system class path are used.
> I played around with excluding the ant classes/jars from the system class 
> path in favor of using the ones we copy into ${build.lib} and 
> ${build.dir.lib} with no success. After reading the documentation it seems 
> you always want to use the libs that shipped with whatever is in $ANT_HOME so 
> i believe excluding the jars from the build lib directories is the correct 
> change anyways.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13234) Add histogram for delay to deliver hints

[Jeff Jirsa (JIRA)]

Jeff Jirsa created CASSANDRA-13234:
--

 Summary: Add histogram for delay to deliver hints
 Key: CASSANDRA-13234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13234
 Project: Cassandra
  Issue Type: Improvement
  Components: Observability
Reporter: Jeff Jirsa
Assignee: Jeff Jirsa
Priority: Minor
 Fix For: 3.0.x, 3.11.x


There is very little visibility into hint delivery in general - having 
histograms available to understand how long it takes to deliver hints is useful 
for operators to better identify problems. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11303) New inbound throughput parameters for streaming

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870872#comment-15870872
 ] 

Jason Brown commented on CASSANDRA-11303:
-

OK, read through the patch and I understand what's going on here. In my current 
work for CASSANDRA-12229 (which I'm trying to post ASAP), I'm taking into 
account *some* degree of inbound throughput management for streaming (I'm using 
netty High/Low water marks and disabling auto-read). I haven't accounted for 
things at a global level, but that's a task specific to that ticket.

Nevertheless, I will make sure to think again about the inbound throughout 
limits again as I finish up CASSANDRA-12229.

> New inbound throughput parameters for streaming
> ---
>
> Key: CASSANDRA-11303
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11303
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Configuration
>Reporter: Satoshi Konno
>Assignee: Satoshi Konno
>Priority: Minor
> Attachments: 11303_inbound_limit_debug_20160419.log, 
> 11303_inbound_nolimit_debug_20160419.log, 
> 11303_inbound_patch_for_trunk_20160419.diff, 
> 11303_inbound_patch_for_trunk_20160525.diff, 
> 11303_inbound_patch_for_trunk_20160704.diff, 
> 200vs40inboundstreamthroughput.png, cassandra_inbound_stream.diff
>
>
> Hi,
> To specify stream throughputs of a node more clearly, I would like to add the 
> following new inbound parameters like existing outbound parameters in the 
> cassandra.yaml.
> - stream_throughput_inbound_megabits_per_sec
> - inter_dc_stream_throughput_outbound_megabits_per_sec  
> We use only the existing outbound parameters now, but it is difficult to 
> control the total throughputs of a node.　In our production network, some 
> critical alerts occurs when a node exceed the specified total throughput 
> which is the sum of the input and output throughputs.
> In our operation of Cassandra, the alerts occurs during the bootstrap or 
> repair processing when a new node is added. In the worst case, we have to 
> stop the operation of the exceed node.
> I have attached the patch under consideration. I would like to add a new 
> limiter class, StreamInboundRateLimiter, and use the limiter class in 
> StreamDeserializer class. I use Row::dataSize( ）to get the input throughput 
> in StreamDeserializer::newPartition(), but I am not sure whether the 
> dataSize() returns the correct data size.
> Can someone please tell me how to do it ?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13233) no libsigar-universal64-macosx.dylib in java.library.path

[Michael Kjellman (JIRA)]

Michael Kjellman created CASSANDRA-13233:


 Summary: no libsigar-universal64-macosx.dylib in java.library.path
 Key: CASSANDRA-13233
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13233
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Kjellman
 Attachments: 28827709.diff

The changes introduced in https://issues.apache.org/jira/browse/CASSANDRA-7838 
(Resolved; Fixed; 2.2.0 beta 1): "Warn user when OS settings are poor / 
integrate sigar" are not Mac friendly.

{code}

INFO  [main] 2016-10-18T11:20:10,330 SigarLibrary.java:44 - Initializing SIGAR 
library
DEBUG [main] 2016-10-18T11:20:10,342 SigarLog.java:60 - no 
libsigar-universal64-macosx.dylib in java.library.path
org.hyperic.sigar.SigarException: no libsigar-universal64-macosx.dylib in 
java.library.path
at org.hyperic.sigar.Sigar.loadLibrary(Sigar.java:172) 
~[sigar-1.6.4.jar:?]
at org.hyperic.sigar.Sigar.(Sigar.java:100) [sigar-1.6.4.jar:?]
at org.apache.cassandra.utils.SigarLibrary.(SigarLibrary.java:47) 
[main/:?]
at 
org.apache.cassandra.utils.SigarLibrary.(SigarLibrary.java:28) [main/:?]
at org.apache.cassandra.utils.UUIDGen.hash(UUIDGen.java:363) [main/:?]
at org.apache.cassandra.utils.UUIDGen.makeNode(UUIDGen.java:342) 
[main/:?]
at 
org.apache.cassandra.utils.UUIDGen.makeClockSeqAndNode(UUIDGen.java:291) 
[main/:?]
at org.apache.cassandra.utils.UUIDGen.(UUIDGen.java:42) 
[main/:?]
at 
org.apache.cassandra.config.CFMetaData$Builder.build(CFMetaData.java:1278) 
[main/:?]
at 
org.apache.cassandra.SchemaLoader.standardCFMD(SchemaLoader.java:369) 
[classes/:?]
at 
org.apache.cassandra.SchemaLoader.standardCFMD(SchemaLoader.java:356) 
[classes/:?]
at 
org.apache.cassandra.SchemaLoader.standardCFMD(SchemaLoader.java:351) 
[classes/:?]
at 
org.apache.cassandra.batchlog.BatchTest.defineSchema(BatchTest.java:59) 
[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_66]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_66]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_66]
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
 [junit-4.6.jar:?]
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
 [junit-4.6.jar:?]
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
 [junit-4.6.jar:?]
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) 
[junit-4.6.jar:?]
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) 
[junit-4.6.jar:?]
at org.junit.runners.ParentRunner.run(ParentRunner.java:220) 
[junit-4.6.jar:?]
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) 
[junit-4.6.jar:?]
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535)
 [ant-junit.jar:?]
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182)
 [ant-junit.jar:?]
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033)
 [ant-junit.jar:?]
INFO  [main] 2016-10-18T11:20:10,350 SigarLibrary.java:57 - Could not 
initialize SIGAR library 
org.hyperic.sigar.Sigar.getFileSystemListNative()[Lorg/hyperic/sigar/FileSystem;
{code}

There are 2 issues addressed by the attached patch:
# Create platform aware (windows, Darwin, linux) implementations of CLibrary 
(for instance CLibrary today assumes all platforms have support for 
posix_fadvise but this doesn't exist in the Darwin kernel). If methods are 
defined with the "native" JNI keyword in java when the class is loaded it will 
cause our jna check to fail incorrectly making all of CLibrary "disabled" even 
though because jnaAvailable = false even though on a platform like Darwin all 
of the native methods except posix_fadvise are supported.
# Replace sigar usage to get current pid with calls to CLibrary/native 
equivalent -- and fall back to Sigar for platforms like Windows who don't have 
that support with JDK8 (and without a CLibrary equivalent)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13232) "multiple versions of ant detected in path for junit" printed for every junit test case spawned by "ant test"

[Michael Kjellman (JIRA)]

Michael Kjellman created CASSANDRA-13232:


 Summary: "multiple versions of ant detected in path for junit" 
printed for every junit test case spawned by "ant test"
 Key: CASSANDRA-13232
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13232
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Kjellman
 Attachments: 673.diff

There is a super annoying junit warning logged before every junit test case 
when you run "ant test". This is due to the fact that the ant junit task that 
we have configured in our build.xml sources the system class path and most 
importantly what's in ant.library.dir.

[junit] WARNING: multiple versions of ant detected in path for junit 
[junit]  
jar:file:/usr/local/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit]  and 
jar:file:/Users/mkjellman/Documents/mkjellman-cie-cassandra-trunk/build/lib/jars/ant-1.9.6.jar!/org/apache/tools/ant/Project.class

The fix here is to explicitly exclude the ant jar downloaded from the maven 
tasks that ends up in ${build.lib} and ${build.dir.lib} so only the ant 
libraries from the system class path are used.

I played around with excluding the ant classes/jars from the system class path 
in favor of using the ones we copy into ${build.lib} and ${build.dir.lib} with 
no success. After reading the documentation it seems you always want to use the 
libs that shipped with whatever is in $ANT_HOME so i believe excluding the jars 
from the build lib directories is the correct change anyways.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13231) org.apache.cassandra.db.DirectoriesTest(testStandardDirs) unit test failing

[Michael Kjellman (JIRA)]

Michael Kjellman created CASSANDRA-13231:


 Summary: org.apache.cassandra.db.DirectoriesTest(testStandardDirs) 
unit test failing
 Key: CASSANDRA-13231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13231
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Kjellman
 Attachments: 674.diff

The testStandardDirs(org.apache.cassandra.db.DirectoriesTest) unit test always 
fails. This appears to be due to a commit by Yuki for CASSANDRA-10587 which 
switched the SSTable descriptor to use the canonical path.

>From one of Yuki's comments in CASSANDRA-10587:
"I ended up fixing Descriptor object to always have canonical path as its 
directory.
This way we don't need to think about given directory is relative or absolute.
In fact, right now Desctiptor (and corresponding SSTable) is not considered 
equal between Descriptor's directory being relative and absolute. (Added simple 
unit test to DescriptorTest)."

The issue here is that canonical path will expand out differently than even 
absolute path. In this case /var/folders -> /private/var/folders. The unit test 
is looking for /var/folders/... but the Descriptor expands out to 
/private/var/folders and the unit test fails.

Descriptor#L88 seems to be the real root cause.

   [junit] Testcase: testStandardDirs(org.apache.cassandra.db.DirectoriesTest): 
FAILED
[junit] 
expected:
 but 
was:
[junit] junit.framework.AssertionFailedError: 
expected:
 but 
was:
[junit] at 
org.apache.cassandra.db.DirectoriesTest.testStandardDirs(DirectoriesTest.java:159)
[junit] 
[junit] 
[junit] Test org.apache.cassandra.db.DirectoriesTest FAILED

I'm guessing given we went to canonicalPath() on purpose the "fix" here is to 
call .getCanonicalFile() on both expected Files generated (snapshotDir and 
backupsDir) for the junit assert.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13196) test failure in snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address

[Aleksandr Sorokoumov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870729#comment-15870729
 ] 

Aleksandr Sorokoumov commented on CASSANDRA-13196:
--

Hello,

I have been checking out what happened with this test and found out that the 
keyspace1 does not exist on the node2 because all 3 submitted migration tasks 
failed to complete.
After there were no more migration tasks in progress 
{{MigrationManager.waitUntilReadyForBootstrap}} returned and the node 
bootstrapped.

>From the discussion in CASSANDRA-10731 I understood that bootstrapping a node 
>with out of sync schema is undesired behavior.
In case the {{MigrationTask.inflightTasks}} queue is empty and the schema 
wasn't pulled yet, does it make sense to schedule more MigrationTasks one by 
one until at least one of them succeeds?

What do you think?

> test failure in 
> snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
> -
>
> Key: CASSANDRA-13196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13196
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>  Labels: dtest, test-failure
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1487/testReport/snitch_test/TestGossipingPropertyFileSnitch/test_prefer_local_reconnect_on_listen_address
> {code}
> {novnode}
> Error Message
> Error from server: code=2200 [Invalid query] message="keyspace keyspace1 does 
> not exist"
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-k6b0iF
> dtest: DEBUG: Done setting configuration options:
> {   'initial_token': None,
> 'num_tokens': '32',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'dc1' for DCAwareRoundRobinPolicy 
> (via host '127.0.0.1'); if incorrect, please specify a local_dc to the 
> constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/snitch_test.py", line 87, in 
> test_prefer_local_reconnect_on_listen_address
> new_rows = list(session.execute("SELECT * FROM {}".format(stress_table)))
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 1998, in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state).result()
>   File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 
> 3784, in result
> raise self._final_exception
> 'Error from server: code=2200 [Invalid query] message="keyspace keyspace1 
> does not exist"\n >> begin captured logging << 
> \ndtest: DEBUG: cluster ccm directory: 
> /tmp/dtest-k6b0iF\ndtest: DEBUG: Done setting configuration options:\n{   
> \'initial_token\': None,\n\'num_tokens\': \'32\',\n
> \'phi_convict_threshold\': 5,\n\'range_request_timeout_in_ms\': 1,\n  
>   \'read_request_timeout_in_ms\': 1,\n\'request_timeout_in_ms\': 
> 1,\n\'truncate_request_timeout_in_ms\': 1,\n
> \'write_request_timeout_in_ms\': 1}\ncassandra.policies: INFO: Using 
> datacenter \'dc1\' for DCAwareRoundRobinPolicy (via host \'127.0.0.1\'); if 
> incorrect, please specify a local_dc to the constructor, or limit contact 
> points to local cluster nodes\ncassandra.cluster: INFO: New Cassandra host 
>  discovered\n- >> end captured 
> logging << -'
> {novnode}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13225) Best Consistency Level

[Connor Warrington (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870607#comment-15870607
 ] 

Connor Warrington commented on CASSANDRA-13225:
---

I was looking at the statistics from the 9318 JIRA and that's pretty much what 
I was hoping to accomplish. Less mutations dropped and less hints written.

Yes, I get we aren't waiting for all replicas but we kinda are because of the 
rate limiting feedback loop.

> Best Consistency Level
> --
>
> Key: CASSANDRA-13225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13225
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Connor Warrington
>Priority: Minor
>
> When writing data into a cluster there are a few consistency levels to choose 
> from. When choosing the consistency level to write with you are making a 
> tradeoff between consistency and failover availability. If you choose 
> consistency level ALL then all replicas have to be up and when a write 
> succeeds all replicas received the write. If you choose consistency level 
> QUORUM then a quorum number of replicas have to be up and when a write 
> succeeds at quorum number of replicas received the write. The tradeoff comes 
> in when there are more then quorum nodes available for the write. We would 
> like a write to succeed only when all replicas that are up have received the 
> write. Hence the suggestion of best as a consistency level. This would be 
> available for the existing consistency levels. The main idea behind this 
> feature request is that we are okay with a replica going down (fault 
> tolerance) but when the cluster is in a good state we don't mind waiting for 
> all nodes to get the write. This would enable the writer to operate at speed 
> of the slowest node instead of potentially getting into a state where that 
> slow node gets even further behind. This would also enable back pressure to 
> be better propagated through the system as the slowest node likely has back 
> pressure which is trying to tell the client about but if we don't wait for 
> that node the writer loses that information.
> Example scenarios:
> If we have replication factor of 3: 
> ALL consistency means 3 replicas have to be up and 3 replicas have to 
> successfully get the write. 
> QUORUM consistency means 2 replicas have to be up and 2 replicas have to 
> successfully get the write. 
> BEST_QUORUM consistency means 2 replicas have be up and all up replicas have 
> to successfully get the write.
> If 3 replicas are up with replication factor of 3: 
> ALL would succeed as all 3 replicas are up and would return success when all 
> 3 replicas get the write 
> QUORUM would succeed as all 3 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as all 3 replicas are up and would return success 
> when all 3 replicas get the write
> If 2 replicas are up with replication factor of 3: 
> ALL would fail as only 2 replicas are up 
> QUORUM would succeed as 2 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as 2 replicas are up and would return success when 
> 2 replicas get the write



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13225) Best Consistency Level

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870593#comment-15870593
 ] 

Sergio Bossa commented on CASSANDRA-13225:
--

bq. Does the back-pressure strategy set at SLOW and a consistency level of 
Quorum mean that while we will only wait for 3 replicas out of 5 for the write 
to succeed we will overall be limiting the number of write requests to all 
replicas based on the slowest responses we are getting from the cluster?

Yes. Specifically, the rate limiting is computed based on the ratio between 
sent messages and received responses.

bq. Does this then mean that a large set of individual writes will be rate 
limited by the slowest write responses and we will in effect be waiting for all 
replicas even though each individual write is only waiting for Quorum replicas?

Kinda. Keep in mind we're still not actually *waiting* for all replicas, but 
just rate limiting (so pausing) based on the feedback loop produced by messages 
sent and responses received.

> Best Consistency Level
> --
>
> Key: CASSANDRA-13225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13225
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Connor Warrington
>Priority: Minor
>
> When writing data into a cluster there are a few consistency levels to choose 
> from. When choosing the consistency level to write with you are making a 
> tradeoff between consistency and failover availability. If you choose 
> consistency level ALL then all replicas have to be up and when a write 
> succeeds all replicas received the write. If you choose consistency level 
> QUORUM then a quorum number of replicas have to be up and when a write 
> succeeds at quorum number of replicas received the write. The tradeoff comes 
> in when there are more then quorum nodes available for the write. We would 
> like a write to succeed only when all replicas that are up have received the 
> write. Hence the suggestion of best as a consistency level. This would be 
> available for the existing consistency levels. The main idea behind this 
> feature request is that we are okay with a replica going down (fault 
> tolerance) but when the cluster is in a good state we don't mind waiting for 
> all nodes to get the write. This would enable the writer to operate at speed 
> of the slowest node instead of potentially getting into a state where that 
> slow node gets even further behind. This would also enable back pressure to 
> be better propagated through the system as the slowest node likely has back 
> pressure which is trying to tell the client about but if we don't wait for 
> that node the writer loses that information.
> Example scenarios:
> If we have replication factor of 3: 
> ALL consistency means 3 replicas have to be up and 3 replicas have to 
> successfully get the write. 
> QUORUM consistency means 2 replicas have to be up and 2 replicas have to 
> successfully get the write. 
> BEST_QUORUM consistency means 2 replicas have be up and all up replicas have 
> to successfully get the write.
> If 3 replicas are up with replication factor of 3: 
> ALL would succeed as all 3 replicas are up and would return success when all 
> 3 replicas get the write 
> QUORUM would succeed as all 3 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as all 3 replicas are up and would return success 
> when all 3 replicas get the write
> If 2 replicas are up with replication factor of 3: 
> ALL would fail as only 2 replicas are up 
> QUORUM would succeed as 2 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as 2 replicas are up and would return success when 
> 2 replicas get the write



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (CASSANDRA-13130) Strange result of several list updates in a single request

[Mikhail Krupitskiy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870591#comment-15870591
 ] 

Mikhail Krupitskiy edited comment on CASSANDRA-13130 at 2/16/17 7:46 PM:
-

{quote}When you issue a single query with two updates, the two updates are 
actually having the same timestamp which means that the update with the higher 
value will win.{quote}
Could you please clarify what does "the higher value will win" mean?
Does it mean that it's not defined which of two updates will be actually 
applied?


was (Author: mkrupits_jb):
>>  When you issue a single query with two updates, the two updates are 
>> actually having the same timestamp which means that the update with the 
>> higher value will win.
Could you please clarify what does "the higher value will win" mean?
Does it mean that it's not defined which of two updates will be actually 
applied?

> Strange result of several list updates in a single request
> --
>
> Key: CASSANDRA-13130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13130
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mikhail Krupitskiy
>Assignee: Benjamin Lerer
>Priority: Trivial
>
> Let's assume that we have a row with the 'listColumn' column and value 
> \{1,2,3,4\}.
> For me it looks logical to expect that the following two pieces of code will 
> ends up with the same result but it isn't so.
> Code1:
> {code}
> UPDATE t SET listColumn[2] = 7, listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,7,8,4\}
> Code2:
> {code}
> UPDATE t SET listColumn[2] = 7  WHERE id = 1;
> UPDATE t SET listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,8,4\}
> So the question is why Code1 and Code2 give different results?
> Looks like Code1 should give the same result as Code2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13130) Strange result of several list updates in a single request

[Mikhail Krupitskiy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870591#comment-15870591
 ] 

Mikhail Krupitskiy commented on CASSANDRA-13130:


>>  When you issue a single query with two updates, the two updates are 
>> actually having the same timestamp which means that the update with the 
>> higher value will win.
Could you please clarify what does "the higher value will win" mean?
Does it mean that it's not defined which of two updates will be actually 
applied?

> Strange result of several list updates in a single request
> --
>
> Key: CASSANDRA-13130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13130
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mikhail Krupitskiy
>Assignee: Benjamin Lerer
>Priority: Trivial
>
> Let's assume that we have a row with the 'listColumn' column and value 
> \{1,2,3,4\}.
> For me it looks logical to expect that the following two pieces of code will 
> ends up with the same result but it isn't so.
> Code1:
> {code}
> UPDATE t SET listColumn[2] = 7, listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,7,8,4\}
> Code2:
> {code}
> UPDATE t SET listColumn[2] = 7  WHERE id = 1;
> UPDATE t SET listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,8,4\}
> So the question is why Code1 and Code2 give different results?
> Looks like Code1 should give the same result as Code2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13225) Best Consistency Level

[Connor Warrington (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870573#comment-15870573
 ] 

Connor Warrington commented on CASSANDRA-13225:
---

Looks like this JIRA ticket is what I should be looking at: 
https://issues.apache.org/jira/browse/CASSANDRA-9318

> Best Consistency Level
> --
>
> Key: CASSANDRA-13225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13225
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Connor Warrington
>Priority: Minor
>
> When writing data into a cluster there are a few consistency levels to choose 
> from. When choosing the consistency level to write with you are making a 
> tradeoff between consistency and failover availability. If you choose 
> consistency level ALL then all replicas have to be up and when a write 
> succeeds all replicas received the write. If you choose consistency level 
> QUORUM then a quorum number of replicas have to be up and when a write 
> succeeds at quorum number of replicas received the write. The tradeoff comes 
> in when there are more then quorum nodes available for the write. We would 
> like a write to succeed only when all replicas that are up have received the 
> write. Hence the suggestion of best as a consistency level. This would be 
> available for the existing consistency levels. The main idea behind this 
> feature request is that we are okay with a replica going down (fault 
> tolerance) but when the cluster is in a good state we don't mind waiting for 
> all nodes to get the write. This would enable the writer to operate at speed 
> of the slowest node instead of potentially getting into a state where that 
> slow node gets even further behind. This would also enable back pressure to 
> be better propagated through the system as the slowest node likely has back 
> pressure which is trying to tell the client about but if we don't wait for 
> that node the writer loses that information.
> Example scenarios:
> If we have replication factor of 3: 
> ALL consistency means 3 replicas have to be up and 3 replicas have to 
> successfully get the write. 
> QUORUM consistency means 2 replicas have to be up and 2 replicas have to 
> successfully get the write. 
> BEST_QUORUM consistency means 2 replicas have be up and all up replicas have 
> to successfully get the write.
> If 3 replicas are up with replication factor of 3: 
> ALL would succeed as all 3 replicas are up and would return success when all 
> 3 replicas get the write 
> QUORUM would succeed as all 3 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as all 3 replicas are up and would return success 
> when all 3 replicas get the write
> If 2 replicas are up with replication factor of 3: 
> ALL would fail as only 2 replicas are up 
> QUORUM would succeed as 2 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as 2 replicas are up and would return success when 
> 2 replicas get the write



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-11471) Add SASL mechanism negotiation to the native protocol

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870562#comment-15870562
 ] 

Ariel Weisberg commented on CASSANDRA-11471:


I can review this in a little bit. I don't know anything about SASL, but no 
time like the present to learn.

> Add SASL mechanism negotiation to the native protocol
> -
>
> Key: CASSANDRA-11471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11471
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Sam Tunnicliffe
>Assignee: Ben Bromhead
>  Labels: client-impacting
> Attachments: CASSANDRA-11471
>
>
> Introducing an additional message exchange into the authentication sequence 
> would allow us to support multiple authentication schemes and [negotiation of 
> SASL mechanisms|https://tools.ietf.org/html/rfc4422#section-3.2]. 
> The current {{AUTHENTICATE}} message sent from Client to Server includes the 
> java classname of the configured {{IAuthenticator}}. This could be superceded 
> by a new message which lists the SASL mechanisms supported by the server. The 
> client would then respond with a new message which indicates it's choice of 
> mechanism.  This would allow the server to support multiple mechanisms, for 
> example enabling both {{PLAIN}} for username/password authentication and 
> {{EXTERNAL}} for a mechanism for extracting credentials from SSL 
> certificates\* (see the example in 
> [RFC-4422|https://tools.ietf.org/html/rfc4422#appendix-A]). Furthermore, the 
> server could tailor the list of supported mechanisms on a per-connection 
> basis, e.g. only offering certificate based auth to encrypted clients. 
> The client's response should include the selected mechanism and any initial 
> response data. This is mechanism-specific; the {{PLAIN}} mechanism consists 
> of a single round in which the client sends encoded credentials as the 
> initial response data and the server response indicates either success or 
> failure with no futher challenges required.
> From a protocol perspective, after the mechanism negotiation the exchange 
> would continue as in protocol v4, with one or more rounds of 
> {{AUTH_CHALLENGE}} and {{AUTH_RESPONSE}} messages, terminated by an 
> {{AUTH_SUCCESS}} sent from Server to Client upon successful authentication or 
> an {{ERROR}} on auth failure. 
> XMPP performs mechanism negotiation in this way, 
> [RFC-3920|http://tools.ietf.org/html/rfc3920#section-6] includes a good 
> overview.
> \* Note: this would require some a priori agreement between client and server 
> over the implementation of the {{EXTERNAL}} mechanism.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13225) Best Consistency Level

[Connor Warrington (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870527#comment-15870527
 ] 

Connor Warrington commented on CASSANDRA-13225:
---

The back-pressure strategy looks like it was added in a newer version of 
Cassandra then I am on. It does look like it would help keep the cluster 
operating at it's peak performance but not allow it to go past peak 
performance. The intent behind the feature request was to limit the dropped 
mutations, and ultimately limit the hints written.

If I modify the scenario as follows:
If 4 replicas are up with replication factor of 5: 
ALL would fail as only 4 replicas are up 
QUORUM would succeed as 4 replicas are up and would return success when 3 
replicas get the write 
BEST_QUORUM would succeed as 4 replicas are up and would return success when 4 
replicas get the write

Does the back-pressure strategy set at SLOW and a consistency level of Quorum 
mean that while we will only wait for 3 replicas out of 5 for the write to 
succeed we will overall be limiting the number of write requests to all 
replicas based on the slowest responses we are getting from the cluster? Does 
this then mean that a large set of individual writes will be rate limited by 
the slowest write responses and we will in effect be waiting for all replicas 
even though each individual write is only waiting for Quorum replicas?

> Best Consistency Level
> --
>
> Key: CASSANDRA-13225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13225
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Connor Warrington
>Priority: Minor
>
> When writing data into a cluster there are a few consistency levels to choose 
> from. When choosing the consistency level to write with you are making a 
> tradeoff between consistency and failover availability. If you choose 
> consistency level ALL then all replicas have to be up and when a write 
> succeeds all replicas received the write. If you choose consistency level 
> QUORUM then a quorum number of replicas have to be up and when a write 
> succeeds at quorum number of replicas received the write. The tradeoff comes 
> in when there are more then quorum nodes available for the write. We would 
> like a write to succeed only when all replicas that are up have received the 
> write. Hence the suggestion of best as a consistency level. This would be 
> available for the existing consistency levels. The main idea behind this 
> feature request is that we are okay with a replica going down (fault 
> tolerance) but when the cluster is in a good state we don't mind waiting for 
> all nodes to get the write. This would enable the writer to operate at speed 
> of the slowest node instead of potentially getting into a state where that 
> slow node gets even further behind. This would also enable back pressure to 
> be better propagated through the system as the slowest node likely has back 
> pressure which is trying to tell the client about but if we don't wait for 
> that node the writer loses that information.
> Example scenarios:
> If we have replication factor of 3: 
> ALL consistency means 3 replicas have to be up and 3 replicas have to 
> successfully get the write. 
> QUORUM consistency means 2 replicas have to be up and 2 replicas have to 
> successfully get the write. 
> BEST_QUORUM consistency means 2 replicas have be up and all up replicas have 
> to successfully get the write.
> If 3 replicas are up with replication factor of 3: 
> ALL would succeed as all 3 replicas are up and would return success when all 
> 3 replicas get the write 
> QUORUM would succeed as all 3 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as all 3 replicas are up and would return success 
> when all 3 replicas get the write
> If 2 replicas are up with replication factor of 3: 
> ALL would fail as only 2 replicas are up 
> QUORUM would succeed as 2 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as 2 replicas are up and would return success when 
> 2 replicas get the write



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-13229:

Fix Version/s: (was: 4.x)
   4.0

> dtest failure in topology_test.TestTopology.size_estimates_multidc_test
> ---
>
> Key: CASSANDRA-13229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>  Labels: dtest, test-failure
> Fix For: 4.0
>
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 
> - Exception in thread Thread[MemtablePostFlush:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
>   at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
>  ~[main/:na]
>   at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
> ~[main/:na]
>   at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Unexpected error in node1 log, error: 
> ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
>  ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
>  ~[main/:na]
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
>   at 
> org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> 

[jira] [Updated] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-13229:

Fix Version/s: 4.x

> dtest failure in topology_test.TestTopology.size_estimates_multidc_test
> ---
>
> Key: CASSANDRA-13229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>  Labels: dtest, test-failure
> Fix For: 4.0
>
> Attachments: node1_debug.log, node1_gc.log, node1.log, 
> node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
> node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 
> - Exception in thread Thread[MemtablePostFlush:1,5,main]
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
>   at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
>  ~[main/:na]
>   at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
> ~[main/:na]
>   at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Unexpected error in node1 log, error: 
> ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at 
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
>  ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
>  Source) ~[na:na]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
>  ~[main/:na]
>   at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
>   at 
> org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
> ~[main/:na]
>   at 
> org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
>  [main/:na]
>   at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
>  Source) [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
> ~[na:1.8.0_45]
>   at 

[jira] [Updated] (CASSANDRA-13230) Build RPM packages for release

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13230:
---
Description: 
Currently, releases are built locally on Debian/Ubuntu based machines, without 
native support for building RPM packages. This can be done with a docker image.

The current cassandra-release scripts are here (please, do not randomly run and 
push tags..):
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=cassandra-release
A couple incomplete docker run scripts are here:
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=docker-wip

{code}
git clone https://git-wip-us.apache.org/repos/asf/cassandra-builds.git
{code}

Patches for build infra improvements are welcome!
/cc [~spo...@gmail.com] if you want to assign to yourself, I'd be happy to 
review :)

  was:
Currently, releases are built locally on Debian/Ubuntu based machines, without 
native support for building RPM packages. This can be done with a docker image.

The current cassandra-release scripts are here (please, do not randomly run and 
push tags..):
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=cassandra-release
A couple incomplete docker run scripts are here:
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=docker-wip

Patches for build infra improvements are welcome!
/cc [~spo...@gmail.com] if you want to assign to yourself, I'd be happy to 
review :)


> Build RPM packages for release
> --
>
> Key: CASSANDRA-13230
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13230
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Packaging
>Reporter: Michael Shuler
>
> Currently, releases are built locally on Debian/Ubuntu based machines, 
> without native support for building RPM packages. This can be done with a 
> docker image.
> The current cassandra-release scripts are here (please, do not randomly run 
> and push tags..):
> https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=cassandra-release
> A couple incomplete docker run scripts are here:
> https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=docker-wip
> {code}
> git clone https://git-wip-us.apache.org/repos/asf/cassandra-builds.git
> {code}
> Patches for build infra improvements are welcome!
> /cc [~spo...@gmail.com] if you want to assign to yourself, I'd be happy to 
> review :)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13230) Build RPM packages for release

[Michael Shuler (JIRA)]

Michael Shuler created CASSANDRA-13230:
--

 Summary: Build RPM packages for release
 Key: CASSANDRA-13230
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13230
 Project: Cassandra
  Issue Type: New Feature
  Components: Packaging
Reporter: Michael Shuler


Currently, releases are built locally on Debian/Ubuntu based machines, without 
native support for building RPM packages. This can be done with a docker image.

The current cassandra-release scripts are here (please, do not randomly run and 
push tags..):
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=cassandra-release
A couple incomplete docker run scripts are here:
https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git;a=tree;f=docker-wip

Patches for build infra improvements are welcome!
/cc [~spo...@gmail.com] if you want to assign to yourself, I'd be happy to 
review :)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra-builds git commit: Add cassandra-release scripts to repo

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master 145fa130a -> ac05c7153


Add cassandra-release scripts to repo


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/ac05c715
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/ac05c715
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/ac05c715

Branch: refs/heads/master
Commit: ac05c7153bb290572cf1f7a98ea9fc652ca965e4
Parents: 145fa13
Author: Michael Shuler 
Authored: Thu Feb 16 11:50:42 2017 -0600
Committer: Michael Shuler 
Committed: Thu Feb 16 11:50:42 2017 -0600

--
 cassandra-release/README.md  |   5 +
 cassandra-release/finish_release.sh  | 221 
 cassandra-release/prepare_release.sh | 229 ++
 cassandra-release/upload_bintray.sh  |  22 +++
 4 files changed, 477 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/ac05c715/cassandra-release/README.md
--
diff --git a/cassandra-release/README.md b/cassandra-release/README.md
new file mode 100644
index 000..d443cf8
--- /dev/null
+++ b/cassandra-release/README.md
@@ -0,0 +1,5 @@
+Cassandra release scripts
+=
+
+For release managers of cassandra.
+

http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/ac05c715/cassandra-release/finish_release.sh
--
diff --git a/cassandra-release/finish_release.sh 
b/cassandra-release/finish_release.sh
new file mode 100755
index 000..99655c1
--- /dev/null
+++ b/cassandra-release/finish_release.sh
@@ -0,0 +1,221 @@
+#!/bin/bash
+
+# TO EDIT #
+
+asf_username="$USER"
+
+# The name of remote for the asf remote in your git repo
+git_asf_remote="origin"
+
+# Same as for .prepare_release.sh
+mail_dir="$HOME/Mail"
+debian_package_dir="$HOME/tmp/debian"
+
+# The directory for reprepro
+reprepro_dir="$debian_package_dir/packages"
+artifacts_svn_dir="$HOME/svn/cassandra-dist"
+
+###
+
+asf_git_repo="http://git-wip-us.apache.org/repos/asf;
+apache_host="people.apache.org"
+
+# Reset getopts in case it has been used previously in the shell.
+OPTIND=1
+
+# Initialize our own variables:
+verbose=0
+fake_mode=0
+
+show_help()
+{
+local name=`basename $0`
+echo "$name [options]  "
+echo ""
+echo "where [options] are:"
+echo "  -h: print this help"
+echo "  -v: verbose mode (show everything that is going on)"
+echo "  -f: fake mode, print any output but don't do anything (for 
debugging)"
+echo ""
+echo "Example: $name 2.0.3 1024"
+}
+
+while getopts ":hvf" opt; do
+case "$opt" in
+h)
+show_help
+exit 0
+;;
+v)  verbose=1
+;;
+f)  fake_mode=1
+;;
+\?)
+echo "Invalid option: -$OPTARG" >&2
+show_help
+exit 1
+;;
+esac
+done
+
+shift $(($OPTIND-1))
+
+release=$1
+staging_number=$2
+deb_release=${release/-/\~}
+
+if [ -z "$release" ]
+then
+echo "Missing argument "
+show_help
+exit 1
+fi
+if [ -z "$staging_number" ]
+then
+echo "Missing argument "
+show_help
+exit 1
+fi
+
+if [ "$#" -gt 2 ]
+then
+shift
+echo "Too many arguments. Don't know what to do with '$@'"
+show_help
+exit 1
+fi
+
+# Somewhat lame way to check we're in a git repo but that will do
+git log -1 &> /dev/null
+if [ $? -ne 0 ]
+then
+echo "The current directory does not appear to be a git repository."
+echo "You must run this from the Cassandra git source repository."
+exit 1
+fi
+
+if [ "$release" == "$deb_release" ]
+then
+echo "Publishing release $release using staging number $staging_number"
+else
+echo "Publishing release $release (debian uses $deb_release) using staging 
number $staging_number"
+fi
+
+# "Saves" stdout to other descriptor since we might redirect them below
+exec 3>&1 4>&2
+
+if [ $verbose -eq 0 ]
+then
+# Not verbose, redirect all ouptut to a logfile 
+logfile="release-${release}.log"
+[ ! -e "$logfile" ] || rm $logfile
+touch $logfile
+exec > $logfile
+exec 2> $logfile
+fi
+
+execute()
+{
+local cmd=$1
+
+echo ">> $cmd"
+[ $fake_mode -eq 1 ] || $cmd
+if [ $? -ne 0 ]
+then
+echo "Error running $cmd" 1>&3 2>&4
+exit $?
+fi
+}
+
+idx=`expr index "$release" -`
+if [ $idx -eq 0 ]
+then
+release_short=${release}
+else
+release_short=${release:0:$((idx-1))}
+fi
+
+echo "Deploying artifacts ..." 1>&3 2>&4
+start_dir=$PWD
+cd $artifacts_svn_dir
+mkdir $release_short
+cd $release_short
+for type in bin src; do
+for part in gz gz.md5 gz.sha1 gz.asc 

[jira] [Assigned] (CASSANDRA-13224) testall failure in org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston reassigned CASSANDRA-13224:
---

Assignee: Blake Eggleston

> testall failure in 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized
> ---
>
> Key: CASSANDRA-13224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13224
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Sean McCarthy
>Assignee: Blake Eggleston
>  Labels: test-failure, testall
> Attachments: 
> TEST-org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_testall/1407/testReport/org.apache.cassandra.db.compaction/CompactionStrategyManagerPendingRepairTest/cleanupCompactionFinalized
> {code}
> Stacktrace
> junit.framework.AssertionFailedError: 
>   at 
> org.apache.cassandra.db.compaction.CompactionStrategyManagerPendingRepairTest.cleanupCompactionFinalized(CompactionStrategyManagerPendingRepairTest.java:235)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13130) Strange result of several list updates in a single request

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870279#comment-15870279
 ] 

Benjamin Lerer commented on CASSANDRA-13130:


There is clearly a bug. The {{7}} should not appear.
At the same time you should be carefull with those type of queries. When you 
issue a single query with two updates, the two updates are  actually having the 
same timestamp which means that the update with the higher value will win. As 
if you had send 2 {{UPDATE}} queries with exactly the same timestamp.

So, {code}UPDATE t SET listColumn[2] = 8, listColumn[2] = 7  WHERE id = 
1;{code} should also return {{listColumn=(1,2,8,4)}}.

> Strange result of several list updates in a single request
> --
>
> Key: CASSANDRA-13130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13130
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mikhail Krupitskiy
>Assignee: Benjamin Lerer
>Priority: Trivial
>
> Let's assume that we have a row with the 'listColumn' column and value 
> \{1,2,3,4\}.
> For me it looks logical to expect that the following two pieces of code will 
> ends up with the same result but it isn't so.
> Code1:
> {code}
> UPDATE t SET listColumn[2] = 7, listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,7,8,4\}
> Code2:
> {code}
> UPDATE t SET listColumn[2] = 7  WHERE id = 1;
> UPDATE t SET listColumn[2] = 8  WHERE id = 1;
> {code}
> Expected result: listColumn=\{1,2,8,4\} 
> Actual result: listColumn=\{1,2,8,4\}
> So the question is why Code1 and Code2 give different results?
> Looks like Code1 should give the same result as Code2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13227) remove CompactionManager#submitAntiCompaction

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13227:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

committed as aea8bbff3b3a9fade3358c56e2c3f32a5f64862f

> remove CompactionManager#submitAntiCompaction
> -
>
> Key: CASSANDRA-13227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13227
> Project: Cassandra
>  Issue Type: Task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Trivial
> Fix For: 4.0
>
>
> Method is no longer used



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

cassandra git commit: remove unused method

[bdeggleston]

Repository: cassandra
Updated Branches:
  refs/heads/trunk b6fdaba6e -> aea8bbff3


remove unused method


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aea8bbff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aea8bbff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aea8bbff

Branch: refs/heads/trunk
Commit: aea8bbff3b3a9fade3358c56e2c3f32a5f64862f
Parents: b6fdaba
Author: Blake Eggleston 
Authored: Wed Feb 15 11:22:45 2017 -0800
Committer: Blake Eggleston 
Committed: Thu Feb 16 08:43:03 2017 -0800

--
 .../db/compaction/CompactionManager.java| 42 
 1 file changed, 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aea8bbff/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 5a1313b..6d868c9 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -565,48 +565,6 @@ public class CompactionManager implements 
CompactionManagerMBean
 }, jobs, OperationType.RELOCATE);
 }
 
-public ListenableFuture submitAntiCompaction(final ColumnFamilyStore 
cfs,
-final 
Collection ranges,
-final Refs 
sstables,
-final long repairedAt,
-final UUID pendingRepair,
-final UUID 
parentRepairSession)
-{
-Runnable runnable = new WrappedRunnable()
-{
-@Override
-@SuppressWarnings("resource")
-public void runMayThrow() throws Exception
-{
-LifecycleTransaction modifier = null;
-while (modifier == null)
-{
-for (SSTableReader compactingSSTable : 
cfs.getTracker().getCompacting())
-sstables.releaseIfHolds(compactingSSTable);
-Set compactedSSTables = new HashSet<>();
-for (SSTableReader sstable : sstables)
-if (sstable.isMarkedCompacted())
-compactedSSTables.add(sstable);
-sstables.release(compactedSSTables);
-modifier = cfs.getTracker().tryModify(sstables, 
OperationType.ANTICOMPACTION);
-}
-performAnticompaction(cfs, ranges, sstables, modifier, 
repairedAt, pendingRepair, parentRepairSession);
-}
-};
-
-ListenableFuture ret = null;
-try
-{
-ret = executor.submitIfRunning(runnable, "anticompaction");
-return ret;
-}
-finally
-{
-if (ret == null || ret.isCancelled())
-sstables.release();
-}
-}
-
 /**
  * Splits the given token ranges of the given sstables into a pending 
repair silo
  */

cassandra-builds git commit: Add WIP docker run scripts for deb/rpm build

[mshuler]

Repository: cassandra-builds
Updated Branches:
  refs/heads/master 96d70572a -> 145fa130a


Add WIP docker run scripts for deb/rpm build


Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/145fa130
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/145fa130
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/145fa130

Branch: refs/heads/master
Commit: 145fa130aeb37024d33a45154cab4575e6f0df84
Parents: 96d7057
Author: Michael Shuler 
Authored: Thu Feb 16 10:40:42 2017 -0600
Committer: Michael Shuler 
Committed: Thu Feb 16 10:40:42 2017 -0600

--
 docker-wip/centos7-rpmbuild.docker | 15 +++
 docker-wip/jessie-debbuild.docker  | 31 +++
 2 files changed, 46 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/145fa130/docker-wip/centos7-rpmbuild.docker
--
diff --git a/docker-wip/centos7-rpmbuild.docker 
b/docker-wip/centos7-rpmbuild.docker
new file mode 100644
index 000..22c9429
--- /dev/null
+++ b/docker-wip/centos7-rpmbuild.docker
@@ -0,0 +1,15 @@
+FROM centos:7.0.1406
+
+# install deps
+RUN yum -y install ant epel-release git java-1.7.0-openjdk-devel 
java-1.8.0-openjdk-devel make rpm-build
+RUN yum -y install python2-pip
+RUN pip install Sphinx sphinx_rtd_theme
+RUN mkdir -p /root/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
+
+RUN git clone https://git.apache.org/cassandra.git
+# javadoc target is broken in docker without this mkdir
+RUN mkdir -p cassandra/build/javadoc
+RUN cd cassandra/ ; ant artifacts -Drelease=true
+RUN cp cassandra/build/apache-cassandra-*-src.tar.gz /root/rpmbuild/SOURCES/
+RUN rpmbuild --define="version 4.0" -ba cassandra/redhat/cassandra.spec
+RUN ls -lR /root/rpmbuild/*RPMS/

http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/145fa130/docker-wip/jessie-debbuild.docker
--
diff --git a/docker-wip/jessie-debbuild.docker 
b/docker-wip/jessie-debbuild.docker
new file mode 100644
index 000..40c7b28
--- /dev/null
+++ b/docker-wip/jessie-debbuild.docker
@@ -0,0 +1,31 @@
+FROM debian:jessie-backports
+
+# install deps
+RUN apt-get update
+RUN apt-get -y install ant build-essential curl devscripts git
+RUN apt-get -y --no-install-recommends install openjdk-7-jdk openjdk-8-jdk
+RUN apt-get -y -t jessie-backports install python-sphinx 
python-sphinx-rtd-theme
+RUN update-java-alternatives --set java-1.8.0-openjdk-amd64
+
+RUN curl -fL --retry 3 --retry-delay 3 -O --header 'Cookie: 
oraclelicense=accept-securebackup-cookie' 
http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz
+RUN curl -fL --retry 3 --retry-delay 3 -O --header 'Cookie: 
oraclelicense=accept-securebackup-cookie' 
http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip
+RUN echo "777bd7d5268408a5a94f5e366c2e43e720c6ce4fe8c59d9a71e2961e50d774a5  
jdk-8u112-linux-x64.tar.gz" | sha256sum -c
+RUN echo "f3020a3922efd6626c2fff45695d527f34a8020e938a49292561f18ad1320b59  
jce_policy-8.zip" | sha256sum -c
+RUN curl -fLC - --retry 3 --retry-delay 3 -O --header 'Cookie: 
oraclelicense=accept-securebackup-cookie' 
http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
+RUN curl -fL --retry 3 --retry-delay 3 -O --header 'Cookie: 
oraclelicense=accept-securebackup-cookie' 
http://download.oracle.com/otn-pub/java/jce/7/UnlimitedJCEPolicyJDK7.zip
+RUN echo "bad9a731639655118740bee119139c1ed019737ec802a630dd7ad7aab4309623  
jdk-7u80-linux-x64.tar.gz" | sha256sum -c
+RUN echo "7a8d790e7bd9c2f82a83baddfae765797a4a56ea603c9150c87b7cdb7800194d  
UnlimitedJCEPolicyJDK7.zip" | sha256sum -c
+
+# untar/zip, replace JCE bits
+# symlink to something generic
+# set JAVA_HOMEs for upgrade
+
+
+
+### test deps setup above
+# can we clone and build?
+RUN git clone https://git.apache.org/cassandra.git
+# javadoc target is broken in docker without this mkdir
+RUN mkdir -p cassandra/build/javadoc
+RUN cd cassandra/ ; ant artifacts
+RUN cd cassandra/ ; echo "y" | mk-build-deps --install ; dpkg-buildpackage -uc 
-us

[jira] [Commented] (CASSANDRA-13227) remove CompactionManager#submitAntiCompaction

[Blake Eggleston (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870229#comment-15870229
 ] 

Blake Eggleston commented on CASSANDRA-13227:
-

I'd figured as much, but felt like ninja'ing my second commit might be getting 
ahead of myself a little :)

> remove CompactionManager#submitAntiCompaction
> -
>
> Key: CASSANDRA-13227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13227
> Project: Cassandra
>  Issue Type: Task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Trivial
> Fix For: 4.0
>
>
> Method is no longer used



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (CASSANDRA-13181) test failure in paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown resolved CASSANDRA-13181.
-
Resolution: Fixed
  Reviewer: Philip Thompson

> test failure in 
> paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys
> 
>
> Key: CASSANDRA-13181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13181
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Jason Brown
>  Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/352/testReport/paging_test/TestPagingData/test_select_in_clause_with_duplicate_keys
> {noformat}
> Error Message
> 'TestPagingData' object has no attribute 'create_ks'
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-I7zCAw
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/paging_test.py", line 1721, in 
> test_select_in_clause_with_duplicate_keys
> self.create_ks(session, 'test_paging_static_cols', 2)
> "'TestPagingData' object has no attribute 'create_ks'\n 
> >> begin captured logging << \ndtest: DEBUG: cluster ccm 
> directory: /tmp/dtest-I7zCAw\ndtest: DEBUG: Done setting configuration 
> options:\n{   'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ncassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster 
> nodes\ncassandra.cluster: INFO: New Cassandra host  datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host  127.0.0.2 datacenter1> discovered\n- >> end captured 
> logging << -"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13181) test failure in paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870159#comment-15870159
 ] 

Jason Brown commented on CASSANDRA-13181:
-

PR was merged. closing ticket

> test failure in 
> paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys
> 
>
> Key: CASSANDRA-13181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13181
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Jason Brown
>  Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/352/testReport/paging_test/TestPagingData/test_select_in_clause_with_duplicate_keys
> {noformat}
> Error Message
> 'TestPagingData' object has no attribute 'create_ks'
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-I7zCAw
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/paging_test.py", line 1721, in 
> test_select_in_clause_with_duplicate_keys
> self.create_ks(session, 'test_paging_static_cols', 2)
> "'TestPagingData' object has no attribute 'create_ks'\n 
> >> begin captured logging << \ndtest: DEBUG: cluster ccm 
> directory: /tmp/dtest-I7zCAw\ndtest: DEBUG: Done setting configuration 
> options:\n{   'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ncassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster 
> nodes\ncassandra.cluster: INFO: New Cassandra host  datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host  127.0.0.2 datacenter1> discovered\n- >> end captured 
> logging << -"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (CASSANDRA-13229) dtest failure in topology_test.TestTopology.size_estimates_multidc_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-13229:
-

 Summary: dtest failure in 
topology_test.TestTopology.size_estimates_multidc_test
 Key: CASSANDRA-13229
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13229
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Sean McCarthy
 Attachments: node1_debug.log, node1_gc.log, node1.log, 
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
node3.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/508/testReport/topology_test/TestTopology/size_estimates_multidc_test

{code}
Standard Output

Unexpected error in node1 log, error: 
ERROR [MemtablePostFlush:1] 2017-02-15 16:07:33,837 CassandraDaemon.java:211 - 
Exception in thread Thread[MemtablePostFlush:1,5,main]
java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
at 
org.apache.cassandra.dht.Splitter.splitOwnedRangesNoPartialRanges(Splitter.java:92)
 ~[main/:na]
at org.apache.cassandra.dht.Splitter.splitOwnedRanges(Splitter.java:59) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.getDiskBoundaries(StorageService.java:5180)
 ~[main/:na]
at 
org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:312) 
~[main/:na]
at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:304) 
~[main/:na]
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1150)
 ~[main/:na]
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1115)
 ~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
 [main/:na]
at 
org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
 Source) [main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Unexpected error in node1 log, error: 
ERROR [MigrationStage:1] 2017-02-15 16:07:33,853 CassandraDaemon.java:211 - 
Exception in thread Thread[MigrationStage:1,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
at 
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:401) 
~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.lambda$flush$496(SchemaKeyspace.java:284)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace$$Lambda$222/1949434065.accept(Unknown
 Source) ~[na:na]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_45]
at 
org.apache.cassandra.schema.SchemaKeyspace.flush(SchemaKeyspace.java:284) 
~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.applyChanges(SchemaKeyspace.java:1265)
 ~[main/:na]
at org.apache.cassandra.schema.Schema.merge(Schema.java:577) ~[main/:na]
at 
org.apache.cassandra.schema.Schema.mergeAndAnnounceVersion(Schema.java:564) 
~[main/:na]
at 
org.apache.cassandra.schema.MigrationManager$1.runMayThrow(MigrationManager.java:402)
 ~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$290(NamedThreadFactory.java:81)
 [main/:na]
at 
org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$5/1321203216.run(Unknown
 Source) [main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: java.util.concurrent.ExecutionException: 
java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
~[na:1.8.0_45]
at 
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:397) 
~[main/:na]
... 16 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45]
at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45]
at 

[jira] [Commented] (CASSANDRA-12497) COPY ... TO STDOUT regression in 2.2.7

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870080#comment-15870080
 ] 

Paulo Motta commented on CASSANDRA-12497:
-

+1, submitted dtests:

||2.2||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-12497]|
|[cqlsh|http://cassci.datastax.com/job/pauloricardomg-2.2-12497-cqlsh-cqlsh-tests/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-12497-dtest/lastCompletedBuild/testReport/]|

> COPY ... TO STDOUT regression in 2.2.7
> --
>
> Key: CASSANDRA-12497
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12497
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Max Bowsher
>Assignee: Márton Salomváry
>
> Cassandra 2.2.7 introduces a regression over 2.2.6 breaking COPY ... TO 
> STDOUT.
> In pylib/cqlshlib/copyutil.py, in CopyTask.__init__, self.printmsg is 
> conditionally defined to EITHER a module level function accepting arguments 
> (msg, eol=, encoding=), OR a lambda accepting arguments only (_, eol=).
> Consequently, when the lambda is in use (which requires COPY ... TO STDOUT 
> without --debug), any attempt to call CopyTask.printmsg with an encoding 
> parameter causes an exception.
> This occurs in ExportTask.run, thus rendering all COPY ... TO STDOUT without 
> --debug broken.
> The fix is to update the lambda's arguments to include encoding, or better, 
> replace it with a module-level function defined next to printmsg, so that 
> people realize the two argument lists must be kept in sync.
> The regression was introduced in this commit:
> commit 5de9de1f5832f2a0e92783e2f4412874423e6e15
> Author: Tyler Hobbs 
> Date:   Thu May 5 11:33:35 2016 -0500
> cqlsh: Handle non-ascii chars in error messages
> 
> Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-12497) COPY ... TO STDOUT regression in 2.2.7

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-12497:

Status: Patch Available  (was: Open)

> COPY ... TO STDOUT regression in 2.2.7
> --
>
> Key: CASSANDRA-12497
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12497
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Max Bowsher
>Assignee: Márton Salomváry
>
> Cassandra 2.2.7 introduces a regression over 2.2.6 breaking COPY ... TO 
> STDOUT.
> In pylib/cqlshlib/copyutil.py, in CopyTask.__init__, self.printmsg is 
> conditionally defined to EITHER a module level function accepting arguments 
> (msg, eol=, encoding=), OR a lambda accepting arguments only (_, eol=).
> Consequently, when the lambda is in use (which requires COPY ... TO STDOUT 
> without --debug), any attempt to call CopyTask.printmsg with an encoding 
> parameter causes an exception.
> This occurs in ExportTask.run, thus rendering all COPY ... TO STDOUT without 
> --debug broken.
> The fix is to update the lambda's arguments to include encoding, or better, 
> replace it with a module-level function defined next to printmsg, so that 
> people realize the two argument lists must be kept in sync.
> The regression was introduced in this commit:
> commit 5de9de1f5832f2a0e92783e2f4412874423e6e15
> Author: Tyler Hobbs 
> Date:   Thu May 5 11:33:35 2016 -0500
> cqlsh: Handle non-ascii chars in error messages
> 
> Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-12497) COPY ... TO STDOUT regression in 2.2.7

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta reassigned CASSANDRA-12497:
---

Assignee: Márton Salomváry  (was: Tyler Hobbs)
Reviewer: Paulo Motta

> COPY ... TO STDOUT regression in 2.2.7
> --
>
> Key: CASSANDRA-12497
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12497
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Max Bowsher
>Assignee: Márton Salomváry
>
> Cassandra 2.2.7 introduces a regression over 2.2.6 breaking COPY ... TO 
> STDOUT.
> In pylib/cqlshlib/copyutil.py, in CopyTask.__init__, self.printmsg is 
> conditionally defined to EITHER a module level function accepting arguments 
> (msg, eol=, encoding=), OR a lambda accepting arguments only (_, eol=).
> Consequently, when the lambda is in use (which requires COPY ... TO STDOUT 
> without --debug), any attempt to call CopyTask.printmsg with an encoding 
> parameter causes an exception.
> This occurs in ExportTask.run, thus rendering all COPY ... TO STDOUT without 
> --debug broken.
> The fix is to update the lambda's arguments to include encoding, or better, 
> replace it with a module-level function defined next to printmsg, so that 
> people realize the two argument lists must be kept in sync.
> The regression was introduced in this commit:
> commit 5de9de1f5832f2a0e92783e2f4412874423e6e15
> Author: Tyler Hobbs 
> Date:   Thu May 5 11:33:35 2016 -0500
> cqlsh: Handle non-ascii chars in error messages
> 
> Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13181) test failure in paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869994#comment-15869994
 ] 

Jason Brown commented on CASSANDRA-13181:
-

open PR on dtest repo: https://github.com/riptano/cassandra-dtest/pull/1443

> test failure in 
> paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys
> 
>
> Key: CASSANDRA-13181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13181
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Jason Brown
>  Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/352/testReport/paging_test/TestPagingData/test_select_in_clause_with_duplicate_keys
> {noformat}
> Error Message
> 'TestPagingData' object has no attribute 'create_ks'
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-I7zCAw
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/paging_test.py", line 1721, in 
> test_select_in_clause_with_duplicate_keys
> self.create_ks(session, 'test_paging_static_cols', 2)
> "'TestPagingData' object has no attribute 'create_ks'\n 
> >> begin captured logging << \ndtest: DEBUG: cluster ccm 
> directory: /tmp/dtest-I7zCAw\ndtest: DEBUG: Done setting configuration 
> options:\n{   'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ncassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster 
> nodes\ncassandra.cluster: INFO: New Cassandra host  datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host  127.0.0.2 datacenter1> discovered\n- >> end captured 
> logging << -"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-10520) Compressed writer and reader should support non-compressed data.

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869945#comment-15869945
 ] 

Branimir Lambov commented on CASSANDRA-10520:
-

Rebased and updated the patch and triggered another round of testing.

bq. The micro benchmark looks different on my Linux machine

That's very similar to what I get with page cache enabled. Is it possible you 
have run the benchmark without turning it off?

bq. When writing compressed chunks, the compressed buffer is sized to the max 
compression length. WDYT about just passing a buffer that's bounded to 
maxCompressedLength and handle the buffer-overflow-exception to write it 
uncompressed?

This is a possibility but as the use of exceptions on non-exceptional code 
paths is a bit of a frowned-upon practice I am worried that it can cause 
optimization headaches -- JIT refusing to optimize or doing the wrong thing, 
resulting in compression always taking longer than it should. At this point I 
don't really want to risk something like that, but it's an option to explore if 
we get some free cycles later on to verify that there are no performance issues 
in all relevant configurations.

bq. Just for clarification - is the following correct?

Yes, that is correct. {{<=}}/compressed is the typical path, hence placed first 
on the read side, and on the write path we have an {{if}} that is only 
triggered on the alternative. The latter could use a {{! <=}} pattern to make 
the subcondition identical, but that feels unnatural and more complex than 
necessary.

bq. Even if CRC checks are disabled...

Suggested patch included, thanks.


> Compressed writer and reader should support non-compressed data.
> 
>
> Key: CASSANDRA-10520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10520
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>  Labels: messaging-service-bump-required
> Fix For: 4.x
>
> Attachments: ReadWriteTestCompression.java
>
>
> Compressing uncompressible data, as done, for instance, to write SSTables 
> during stress-tests, results in chunks larger than 64k which are a problem 
> for the buffer pooling mechanisms employed by the 
> {{CompressedRandomAccessReader}}. This results in non-negligible performance 
> issues due to excessive memory allocation.
> To solve this problem and avoid decompression delays in the cases where it 
> does not provide benefits, I think we should allow compressed files to store 
> uncompressed chunks as alternative to compressed data. Such a chunk could be 
> written after compression returns a buffer larger than, for example, 90% of 
> the input, and would not result in additional delays in writing. On reads it 
> could be recognized by size (using a single global threshold constant in the 
> compression metadata) and data could be directly transferred into the 
> decompressed buffer, skipping the decompression step and ensuring a 64k 
> buffer for compressed data always suffices.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13228) SASI index on partition key part doesn't match

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869926#comment-15869926
 ] 

Alex Petrov commented on CASSANDRA-13228:
-

We can (and probably should) just add a validation before the real solution is 
implemented. Creating an index should fail if it's not supported.

> SASI index on partition key part doesn't match
> --
>
> Key: CASSANDRA-13228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13228
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Hannu Kröger
>  Labels: sasi
>
> I created a SASI index on first part of multi-part partition key. Running 
> query using that index doesn't seem to work.
> I have here a log of queries that should indicate the issue:
> {code}cqlsh:test> CREATE TABLE test1(name text, event_date date, data_type 
> text, bytes int, PRIMARY KEY ((name, event_date), data_type));
> cqlsh:test> CREATE CUSTOM INDEX test_index ON test1(name) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('1234', '2010-01-01', 'sensor', 128);
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('abcd', '2010-01-02', 'sensor', 500);
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows)
> cqlsh:test> CONSISTENCY ALL;
> Consistency level set to ALL.
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows){code}
> Note! Creating a SASI index on single part partition key, SASI index creation 
> fails. Apparently this should not work at all, so is it about missing 
> validation on index creation?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13228) SASI index on partition key part doesn't match

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13228:

Labels: sasi  (was: )

> SASI index on partition key part doesn't match
> --
>
> Key: CASSANDRA-13228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13228
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Hannu Kröger
>  Labels: sasi
>
> I created a SASI index on first part of multi-part partition key. Running 
> query using that index doesn't seem to work.
> I have here a log of queries that should indicate the issue:
> {code}cqlsh:test> CREATE TABLE test1(name text, event_date date, data_type 
> text, bytes int, PRIMARY KEY ((name, event_date), data_type));
> cqlsh:test> CREATE CUSTOM INDEX test_index ON test1(name) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('1234', '2010-01-01', 'sensor', 128);
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('abcd', '2010-01-02', 'sensor', 500);
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows)
> cqlsh:test> CONSISTENCY ALL;
> Consistency level set to ALL.
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows){code}
> Note! Creating a SASI index on single part partition key, SASI index creation 
> fails. Apparently this should not work at all, so is it about missing 
> validation on index creation?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13228) SASI index on partition key part doesn't match

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-13228:

Component/s: sasi

> SASI index on partition key part doesn't match
> --
>
> Key: CASSANDRA-13228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13228
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Hannu Kröger
>
> I created a SASI index on first part of multi-part partition key. Running 
> query using that index doesn't seem to work.
> I have here a log of queries that should indicate the issue:
> {code}cqlsh:test> CREATE TABLE test1(name text, event_date date, data_type 
> text, bytes int, PRIMARY KEY ((name, event_date), data_type));
> cqlsh:test> CREATE CUSTOM INDEX test_index ON test1(name) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('1234', '2010-01-01', 'sensor', 128);
> cqlsh:test> INSERT INTO test1(name, event_date, data_type, bytes) 
> values('abcd', '2010-01-02', 'sensor', 500);
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows)
> cqlsh:test> CONSISTENCY ALL;
> Consistency level set to ALL.
> cqlsh:test> select * from test1 where NAME = '1234';
>  name | event_date | data_type | bytes
> --++---+---
> (0 rows){code}
> Note! Creating a SASI index on single part partition key, SASI index creation 
> fails. Apparently this should not work at all, so is it about missing 
> validation on index creation?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (CASSANDRA-13181) test failure in paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-13181:
---

Assignee: Jason Brown

> test failure in 
> paging_test.TestPagingData.test_select_in_clause_with_duplicate_keys
> 
>
> Key: CASSANDRA-13181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13181
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Jason Brown
>  Labels: dtest, test-failure
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/352/testReport/paging_test/TestPagingData/test_select_in_clause_with_duplicate_keys
> {noformat}
> Error Message
> 'TestPagingData' object has no attribute 'create_ks'
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-I7zCAw
> dtest: DEBUG: Done setting configuration options:
> {   'num_tokens': None,
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> cassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster nodes
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> cassandra.cluster: INFO: New Cassandra host  
> discovered
> - >> end captured logging << -
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/paging_test.py", line 1721, in 
> test_select_in_clause_with_duplicate_keys
> self.create_ks(session, 'test_paging_static_cols', 2)
> "'TestPagingData' object has no attribute 'create_ks'\n 
> >> begin captured logging << \ndtest: DEBUG: cluster ccm 
> directory: /tmp/dtest-I7zCAw\ndtest: DEBUG: Done setting configuration 
> options:\n{   'num_tokens': None,\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms': 
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms': 
> 1}\ncassandra.policies: INFO: Using datacenter 'datacenter1' for 
> DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify 
> a local_dc to the constructor, or limit contact points to local cluster 
> nodes\ncassandra.cluster: INFO: New Cassandra host  datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host  127.0.0.2 datacenter1> discovered\n- >> end captured 
> logging << -"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13222) Paging with reverse queries and static columns may return incorrectly sized pages

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869891#comment-15869891
 ] 

Jason Brown commented on CASSANDRA-13222:
-

3.0 dtest hadn't run yet so I kicked it off. The 2.1/2.2 dtests have failures 
wrt the paging test:

{code}
"'TestPagingData' object has no attribute 'create_ks'
{code}

That appears to be an incorrectly written dtest. I'll submit a PR to fix that 
later.

I'm +1 on the code. The only tiny nit I would have is to add a comment to the 
3.0+ versions of {{QueryPagerTest#pagingReversedQueriesWithStaticColumnsTest}} 
to indicate that we had a problem with reverse paging/statics/et al in pre-3.0, 
but that it shouldn't be problem now (and the test is to make sure everything 
stays legit into the future).

As for which versions to commit to, this doesn't seem critical enough for 2.1, 
so let's do 2.2+.

> Paging with reverse queries and static columns may return incorrectly sized 
> pages
> -
>
> Key: CASSANDRA-13222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13222
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Local Write-Read Paths
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>
> There are 2 specialisations of {{ColumnCounter}} that deal with static 
> columns differently depending on the order of iteration through the column 
> family and which impl is used generally depends on whether or not the 
> {{ColumnFilter}} in use is reversed. However, the base method 
> {{ColumnCounter::countAll}} always uses forward iteration, which can result 
> in overcounting when the query is reversed and there are statics involved. In 
> turn, this leads to incorrectly sized pages being returned to the client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (CASSANDRA-13222) Paging with reverse queries and static columns may return incorrectly sized pages

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13222:

Status: Ready to Commit  (was: Patch Available)

> Paging with reverse queries and static columns may return incorrectly sized 
> pages
> -
>
> Key: CASSANDRA-13222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13222
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Local Write-Read Paths
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>
> There are 2 specialisations of {{ColumnCounter}} that deal with static 
> columns differently depending on the order of iteration through the column 
> family and which impl is used generally depends on whether or not the 
> {{ColumnFilter}} in use is reversed. However, the base method 
> {{ColumnCounter::countAll}} always uses forward iteration, which can result 
> in overcounting when the query is reversed and there are statics involved. In 
> turn, this leads to incorrectly sized pages being returned to the client.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13225) Best Consistency Level

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869855#comment-15869855
 ] 

Sergio Bossa commented on CASSANDRA-13225:
--

bq. If what you are worried about is overloading nodes and want to get all 3 
acks back to slow down the writes, then you might want to take a look at using 
a back_pressure_strategy, this is exactly the case that was implemented for, 
reducing the number of dropped mutations when all nodes are up and there is a 
constant stream of writes coming in.

[~Connor Warrington], I fully second [~jjordan]'s suggestion here.

The main problem with a "variable" CL is that under failure conditions it 
doesn't really give you any guarantees, as it will succeed if both quorum or 
"more than quorum" are met. Hence, it's practically equal to QUORUM in terms of 
guarantees.

Configuring the back-pressure strategy at SLOW will probably be a better way to 
reduce the number of dropped mutations *and* allow hints to readily fix the 
ones you'll still get.

> Best Consistency Level
> --
>
> Key: CASSANDRA-13225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13225
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Connor Warrington
>Priority: Minor
>
> When writing data into a cluster there are a few consistency levels to choose 
> from. When choosing the consistency level to write with you are making a 
> tradeoff between consistency and failover availability. If you choose 
> consistency level ALL then all replicas have to be up and when a write 
> succeeds all replicas received the write. If you choose consistency level 
> QUORUM then a quorum number of replicas have to be up and when a write 
> succeeds at quorum number of replicas received the write. The tradeoff comes 
> in when there are more then quorum nodes available for the write. We would 
> like a write to succeed only when all replicas that are up have received the 
> write. Hence the suggestion of best as a consistency level. This would be 
> available for the existing consistency levels. The main idea behind this 
> feature request is that we are okay with a replica going down (fault 
> tolerance) but when the cluster is in a good state we don't mind waiting for 
> all nodes to get the write. This would enable the writer to operate at speed 
> of the slowest node instead of potentially getting into a state where that 
> slow node gets even further behind. This would also enable back pressure to 
> be better propagated through the system as the slowest node likely has back 
> pressure which is trying to tell the client about but if we don't wait for 
> that node the writer loses that information.
> Example scenarios:
> If we have replication factor of 3: 
> ALL consistency means 3 replicas have to be up and 3 replicas have to 
> successfully get the write. 
> QUORUM consistency means 2 replicas have to be up and 2 replicas have to 
> successfully get the write. 
> BEST_QUORUM consistency means 2 replicas have be up and all up replicas have 
> to successfully get the write.
> If 3 replicas are up with replication factor of 3: 
> ALL would succeed as all 3 replicas are up and would return success when all 
> 3 replicas get the write 
> QUORUM would succeed as all 3 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as all 3 replicas are up and would return success 
> when all 3 replicas get the write
> If 2 replicas are up with replication factor of 3: 
> ALL would fail as only 2 replicas are up 
> QUORUM would succeed as 2 replicas are up and would return success when 2 
> replicas get the write 
> BEST_QUORUM would succeed as 2 replicas are up and would return success when 
> 2 replicas get the write



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13153) Reappeared Data when Mixing Incremental and Full Repairs

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869836#comment-15869836
 ] 

Marcus Eriksson commented on CASSANDRA-13153:
-

bq. Can we really throw them into the same level locally, just because they 
have been at level X on other nodes?
no, we check if it would create overlap before adding it to the manifest:
https://github.com/apache/cassandra/blob/98d74ed998706e9e047dc0f7886a1e9b18df3ce9/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java#L149

> Reappeared Data when Mixing Incremental and Full Repairs
> 
>
> Key: CASSANDRA-13153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13153
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Tools
> Environment: Apache Cassandra 2.2
>Reporter: Amanda Debrot
>  Labels: Cassandra
> Attachments: log-Reappeared-Data.txt, 
> Step-by-Step-Simulate-Reappeared-Data.txt
>
>
> This happens for both LeveledCompactionStrategy and 
> SizeTieredCompactionStrategy.  I've only tested it on Cassandra version 2.2 
> but it most likely also affects all Cassandra versions after 2.2, if they 
> have anticompaction with full repair.
> When mixing incremental and full repairs, there are a few scenarios where the 
> Data SSTable is marked as unrepaired and the Tombstone SSTable is marked as 
> repaired.  Then if it is past gc_grace, and the tombstone and data has been 
> compacted out on other replicas, the next incremental repair will push the 
> Data to other replicas without the tombstone.
> Simplified scenario:
> 3 node cluster with RF=3
> Intial config:
>   Node 1 has data and tombstone in separate SSTables.
>   Node 2 has data and no tombstone.
>   Node 3 has data and tombstone in separate SSTables.
> Incremental repair (nodetool repair -pr) is run every day so now we have 
> tombstone on each node.
> Some minor compactions have happened since so data and tombstone get merged 
> to 1 SSTable on Nodes 1 and 3.
>   Node 1 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
>   Node 2 has data and tombstone in separate SSTables.
>   Node 3 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
> Incremental repairs keep running every day.
> Full repairs run weekly (nodetool repair -full -pr). 
> Now there are 2 scenarios where the Data SSTable will get marked as 
> "Unrepaired" while Tombstone SSTable will get marked as "Repaired".
> Scenario 1:
> Since the Data and Tombstone SSTable have been marked as "Repaired" 
> and anticompacted, they have had minor compactions with other SSTables 
> containing keys from other ranges.  During full repair, if the last node to 
> run it doesn't own this particular key in it's partitioner range, the Data 
> and Tombstone SSTable will get anticompacted and marked as "Unrepaired".  Now 
> in the next incremental repair, if the Data SSTable is involved in a minor 
> compaction during the repair but the Tombstone SSTable is not, the resulting 
> compacted SSTable will be marked "Unrepaired" and Tombstone SSTable is marked 
> "Repaired".
> Scenario 2:
> Only the Data SSTable had minor compaction with other SSTables 
> containing keys from other ranges after being marked as "Repaired".  The 
> Tombstone SSTable was never involved in a minor compaction so therefore all 
> keys in that SSTable belong to 1 particular partitioner range. During full 
> repair, if the last node to run it doesn't own this particular key in it's 
> partitioner range, the Data SSTable will get anticompacted and marked as 
> "Unrepaired".   The Tombstone SSTable stays marked as Repaired.
> Then it’s past gc_grace.  Since Node’s #1 and #3 only have 1 SSTable for that 
> key, the tombstone will get compacted out.
>   Node 1 has nothing.
>   Node 2 has data (in unrepaired SSTable) and tombstone (in repaired 
> SSTable) in separate SSTables.
>   Node 3 has nothing.
> Now when the next incremental repair runs, it will only use the Data SSTable 
> to build the merkle tree since the tombstone SSTable is flagged as repaired 
> and data SSTable is marked as unrepaired.  And the data will get repaired 
> against the other two nodes.
>   Node 1 has data.
>   Node 2 has data and tombstone in separate SSTables.
>   Node 3 has data.
> If a read request hits Node 1 and 3, it will return data.  If it hits 1 and 
> 2, or 2 and 3, however, it would return no data.
> Tested this with single range tokens for simplicity.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-13153) Reappeared Data when Mixing Incremental and Full Repairs

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869826#comment-15869826
 ] 

Stefan Podkowinski commented on CASSANDRA-13153:


bq.  If operators stop using incremental repairs, there's no harm in doing an 
anticompaction after a full repair. 

Even if there's no harm when it comes to consistency, it's still causing 
fragmentation of existing sstables. All repaired ranges will cause all replicas 
to go through all local, intersecting sstables and rewrite them segregated by 
affected and unaffected token ranges. This will cause unnecessary load and is 
probably pretty bad for LCS or STCS, as we constantly break up bigger sstables 
by doing so.

One option to avoid this would be just to never run anti-compaction on repaired 
sstables. See 
[here|https://github.com/spodkowinski/cassandra/commit/684d1c72cda58fecea15b46f928a451df38d87cb]
 for a simple approach. I don't think anti-compaction was ever meant to work on 
already repaired sstables, so that's probably the most non-intrusive fix to 
avoid most of the known issues around incremental repairs discussed here.

Btw, I'm also a bit confused by looking at 
[createWriterForAntiCompaction|https://github.com/apache/cassandra/blob/98d74ed998706e9e047dc0f7886a1e9b18df3ce9/src/java/org/apache/cassandra/db/compaction/CompactionManager.java#L1285].
 Each sstable's level will be streamed as well, doesn't it? Can we really throw 
them into the same level locally, just because they have been at level X on 
other nodes? Won't this potentially break the "non-overlapping sstables" 
guarantee by dropping them blindly to level X?


> Reappeared Data when Mixing Incremental and Full Repairs
> 
>
> Key: CASSANDRA-13153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13153
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction, Tools
> Environment: Apache Cassandra 2.2
>Reporter: Amanda Debrot
>  Labels: Cassandra
> Attachments: log-Reappeared-Data.txt, 
> Step-by-Step-Simulate-Reappeared-Data.txt
>
>
> This happens for both LeveledCompactionStrategy and 
> SizeTieredCompactionStrategy.  I've only tested it on Cassandra version 2.2 
> but it most likely also affects all Cassandra versions after 2.2, if they 
> have anticompaction with full repair.
> When mixing incremental and full repairs, there are a few scenarios where the 
> Data SSTable is marked as unrepaired and the Tombstone SSTable is marked as 
> repaired.  Then if it is past gc_grace, and the tombstone and data has been 
> compacted out on other replicas, the next incremental repair will push the 
> Data to other replicas without the tombstone.
> Simplified scenario:
> 3 node cluster with RF=3
> Intial config:
>   Node 1 has data and tombstone in separate SSTables.
>   Node 2 has data and no tombstone.
>   Node 3 has data and tombstone in separate SSTables.
> Incremental repair (nodetool repair -pr) is run every day so now we have 
> tombstone on each node.
> Some minor compactions have happened since so data and tombstone get merged 
> to 1 SSTable on Nodes 1 and 3.
>   Node 1 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
>   Node 2 has data and tombstone in separate SSTables.
>   Node 3 had a minor compaction that merged data with tombstone. 1 
> SSTable with tombstone.
> Incremental repairs keep running every day.
> Full repairs run weekly (nodetool repair -full -pr). 
> Now there are 2 scenarios where the Data SSTable will get marked as 
> "Unrepaired" while Tombstone SSTable will get marked as "Repaired".
> Scenario 1:
> Since the Data and Tombstone SSTable have been marked as "Repaired" 
> and anticompacted, they have had minor compactions with other SSTables 
> containing keys from other ranges.  During full repair, if the last node to 
> run it doesn't own this particular key in it's partitioner range, the Data 
> and Tombstone SSTable will get anticompacted and marked as "Unrepaired".  Now 
> in the next incremental repair, if the Data SSTable is involved in a minor 
> compaction during the repair but the Tombstone SSTable is not, the resulting 
> compacted SSTable will be marked "Unrepaired" and Tombstone SSTable is marked 
> "Repaired".
> Scenario 2:
> Only the Data SSTable had minor compaction with other SSTables 
> containing keys from other ranges after being marked as "Repaired".  The 
> Tombstone SSTable was never involved in a minor compaction so therefore all 
> keys in that SSTable belong to 1 particular partitioner range. During full 
> repair, if the last node to run it doesn't own this particular key in it's 
> partitioner range, the Data SSTable will get anticompacted and marked as 
> "Unrepaired".   The 

[jira] [Commented] (CASSANDRA-13227) remove CompactionManager#submitAntiCompaction

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869508#comment-15869508
 ] 

Sylvain Lebresne commented on CASSANDRA-13227:
--

+1 (for the record, I personally wouldn't mind if you feel free to ninja-commit 
that kind of changes in the future).

> remove CompactionManager#submitAntiCompaction
> -
>
> Key: CASSANDRA-13227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13227
> Project: Cassandra
>  Issue Type: Task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Trivial
> Fix For: 4.0
>
>
> Method is no longer used



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)