[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165589#comment-16165589
 ] 

ZhaoYang commented on CASSANDRA-13595:
--

[~jjirsa] thanks, I will move to new repo.

[~iamaleksey] I haven't looked at 2.2 yet, just started off with 3.0, will do 
that after you finalized 13794, 12872. 

bq. Does it count as critical?

As a user, yes..

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13866) Clock-dependent integer overflow in tests CellTest and RowsTest

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165494#comment-16165494
 ] 

Jeff Jirsa commented on CASSANDRA-13866:


lgtm.


> Clock-dependent integer overflow in tests CellTest and RowsTest
> ---
>
> Key: CASSANDRA-13866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13866
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>Priority: Trivial
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> These tests create timestamps from Unix time, but this is done as int math 
> with the result stored in a long. This means that if the test is run at 
> certain times, like 1505177731, corresponding to Tuesday, September 12, 2017, 
> 12:55:31, the test can have two timestamps separated by a single second that 
> reverse their ordering when multiplied by 100, such as 1505177731 -> 
> 2147149504 and 1505177732 -> -2146817792. This causes a variety of test 
> failures, since it changes the reconciliation order of these cells.
> Note that I've tagged this as trivial because the problem is in the manual 
> construction of timestamps in the test; I know of nowhere  that we make this 
> mistake with real data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13866) Clock-dependent integer overflow in tests CellTest and RowsTest

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13866:
---
 Reviewer: Jeff Jirsa
Fix Version/s: 4.x
   3.11.x
   3.0.x

> Clock-dependent integer overflow in tests CellTest and RowsTest
> ---
>
> Key: CASSANDRA-13866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13866
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>Priority: Trivial
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> These tests create timestamps from Unix time, but this is done as int math 
> with the result stored in a long. This means that if the test is run at 
> certain times, like 1505177731, corresponding to Tuesday, September 12, 2017, 
> 12:55:31, the test can have two timestamps separated by a single second that 
> reverse their ordering when multiplied by 100, such as 1505177731 -> 
> 2147149504 and 1505177732 -> -2146817792. This causes a variety of test 
> failures, since it changes the reconciliation order of these cells.
> Note that I've tagged this as trivial because the problem is in the manual 
> construction of timestamps in the test; I know of nowhere  that we make this 
> mistake with real data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13866) Clock-dependent integer overflow in tests CellTest and RowsTest

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13866:
---
Status: Ready to Commit  (was: Patch Available)

> Clock-dependent integer overflow in tests CellTest and RowsTest
> ---
>
> Key: CASSANDRA-13866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13866
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>Priority: Trivial
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> These tests create timestamps from Unix time, but this is done as int math 
> with the result stored in a long. This means that if the test is run at 
> certain times, like 1505177731, corresponding to Tuesday, September 12, 2017, 
> 12:55:31, the test can have two timestamps separated by a single second that 
> reverse their ordering when multiplied by 100, such as 1505177731 -> 
> 2147149504 and 1505177732 -> -2146817792. This causes a variety of test 
> failures, since it changes the reconciliation order of these cells.
> Note that I've tagged this as trivial because the problem is in the manual 
> construction of timestamps in the test; I know of nowhere  that we make this 
> mistake with real data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13866) Clock-dependent integer overflow in tests CellTest and RowsTest

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-13866:
--
Status: Patch Available  (was: Open)

Trivial patches for 
[3.0|https://github.com/jkni/cassandra/tree/CASSANDRA-13866-3.0], 
[3.11|https://github.com/jkni/cassandra/tree/CASSANDRA-13866-3.11], and 
[trunk|https://github.com/jkni/cassandra/tree/CASSANDRA-13866-trunk]. The 3.0 
patch merges forward cleanly.

> Clock-dependent integer overflow in tests CellTest and RowsTest
> ---
>
> Key: CASSANDRA-13866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13866
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>Priority: Trivial
>
> These tests create timestamps from Unix time, but this is done as int math 
> with the result stored in a long. This means that if the test is run at 
> certain times, like 1505177731, corresponding to Tuesday, September 12, 2017, 
> 12:55:31, the test can have two timestamps separated by a single second that 
> reverse their ordering when multiplied by 100, such as 1505177731 -> 
> 2147149504 and 1505177732 -> -2146817792. This causes a variety of test 
> failures, since it changes the reconciliation order of these cells.
> Note that I've tagged this as trivial because the problem is in the manual 
> construction of timestamps in the test; I know of nowhere  that we make this 
> mistake with real data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-12872) Paging reads and limit reads are missing some data

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reassigned CASSANDRA-12872:
-

Assignee: Aleksey Yeschenko  (was: Benjamin Lerer)

> Paging reads and limit reads are missing some data
> --
>
> Key: CASSANDRA-12872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Bhaskar Muppana
>Assignee: Aleksey Yeschenko
>Priority: Critical
>  Labels: Correctness
> Fix For: 3.0.x
>
> Attachments: limiterr-reproduce.sh
>
>
> We are seeing an issue with paging reads missing some small number of columns 
> when we do paging/limit reads. We get this on a single DC cluster itself when 
> both reads and writes are happening with QUORUM. Paging/limit reads see this 
> issue. I have attached the ccm based script which reproduces the problem.
> * Keyspace RF - 3
> * Table (id int, course text, marks int, primary key(id, course))
> * replicas for partition key 1 - r1, r2 and r3
> * insert (1, '1', 1) ,  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5) 
> - succeeded on all 3 replicas
> * insert (1, '6', 6) succeeded on r1 and r3, failed on r2
> * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and r2, 
> failed on r3
> * insert (1, '7', 7) succeeded on r1 and r2, failed on r3
> Local data on 3 nodes looks like as below now
> r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7)
> r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7)
> r3: (1, '1', 1),  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5), (1, 
> '6', 6)
> If we do a paging read with page_size 2, and if it gets data from r2 and r3, 
> then it will only get the data (1, '1', 1) and (1, '7', 7) skipping record 6. 
> This problem would happen if the same query is not doing paging but limit set 
> to 2 records.
> Resolution code for reads works same for paging queries and normal queries. 
> Co-ordinator shouldn't respond back to client with records/columns that it 
> didn't have complete visibility on all required replicas (in this case 2 
> replicas). In above case, it is sending back record (1, '7', 7) back to 
> client, but its visibility on r3 is limited up to (1, '2', 2) and it is 
> relying on just r2 data to assume (1, '6', 6) doesn't exist, which is wrong. 
> End of the resolution all it can conclusively say any thing about is (1, '1', 
>  and the other one is that we  and and and and and and the and the and the 
> and d and the other is and 1), which exists and (1, '2', 2), which is deleted.
> Ideally we should have different resolution implementation for paging/limit 
> queries.
> We could reproduce this on 2.0.17, 2.1.16 and 3.0.9.
> Seems like 3.0.9 we have ShortReadProtection transformation on list queries. 
> I assume that is to protect against the cases like above. But, we can 
> reproduce the issue in 3.0.9 as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12872) Paging reads and limit reads are missing some data

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12872:
--
Fix Version/s: 3.0.x

> Paging reads and limit reads are missing some data
> --
>
> Key: CASSANDRA-12872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Bhaskar Muppana
>Assignee: Aleksey Yeschenko
>Priority: Critical
>  Labels: Correctness
> Fix For: 3.0.x
>
> Attachments: limiterr-reproduce.sh
>
>
> We are seeing an issue with paging reads missing some small number of columns 
> when we do paging/limit reads. We get this on a single DC cluster itself when 
> both reads and writes are happening with QUORUM. Paging/limit reads see this 
> issue. I have attached the ccm based script which reproduces the problem.
> * Keyspace RF - 3
> * Table (id int, course text, marks int, primary key(id, course))
> * replicas for partition key 1 - r1, r2 and r3
> * insert (1, '1', 1) ,  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5) 
> - succeeded on all 3 replicas
> * insert (1, '6', 6) succeeded on r1 and r3, failed on r2
> * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and r2, 
> failed on r3
> * insert (1, '7', 7) succeeded on r1 and r2, failed on r3
> Local data on 3 nodes looks like as below now
> r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7)
> r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7)
> r3: (1, '1', 1),  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5), (1, 
> '6', 6)
> If we do a paging read with page_size 2, and if it gets data from r2 and r3, 
> then it will only get the data (1, '1', 1) and (1, '7', 7) skipping record 6. 
> This problem would happen if the same query is not doing paging but limit set 
> to 2 records.
> Resolution code for reads works same for paging queries and normal queries. 
> Co-ordinator shouldn't respond back to client with records/columns that it 
> didn't have complete visibility on all required replicas (in this case 2 
> replicas). In above case, it is sending back record (1, '7', 7) back to 
> client, but its visibility on r3 is limited up to (1, '2', 2) and it is 
> relying on just r2 data to assume (1, '6', 6) doesn't exist, which is wrong. 
> End of the resolution all it can conclusively say any thing about is (1, '1', 
>  and the other one is that we  and and and and and and the and the and the 
> and d and the other is and 1), which exists and (1, '2', 2), which is deleted.
> Ideally we should have different resolution implementation for paging/limit 
> queries.
> We could reproduce this on 2.0.17, 2.1.16 and 3.0.9.
> Seems like 3.0.9 we have ShortReadProtection transformation on list queries. 
> I assume that is to protect against the cases like above. But, we can 
> reproduce the issue in 3.0.9 as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12872) Paging reads and limit reads are missing some data

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165477#comment-16165477
 ] 

Aleksey Yeschenko commented on CASSANDRA-12872:
---

A minimal dtest reproducing the issue 
[here|https://github.com/iamaleksey/cassandra-dtest/commits/12872]. 
CASSANDRA-13794 changes make it pass, but unfortunately the bug itself is still 
here - it's just masked by min(limit, 16) optimisation in CASSANDRA-13794 
branch.

[~blerer] I'm close-ish to completion here. Mind if I take over? It's a bit 
time-sensitive. If you do mind and/or have something in progress, feel free to 
take it back.

> Paging reads and limit reads are missing some data
> --
>
> Key: CASSANDRA-12872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Bhaskar Muppana
>Assignee: Benjamin Lerer
>Priority: Critical
>  Labels: Correctness
> Fix For: 3.0.x
>
> Attachments: limiterr-reproduce.sh
>
>
> We are seeing an issue with paging reads missing some small number of columns 
> when we do paging/limit reads. We get this on a single DC cluster itself when 
> both reads and writes are happening with QUORUM. Paging/limit reads see this 
> issue. I have attached the ccm based script which reproduces the problem.
> * Keyspace RF - 3
> * Table (id int, course text, marks int, primary key(id, course))
> * replicas for partition key 1 - r1, r2 and r3
> * insert (1, '1', 1) ,  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5) 
> - succeeded on all 3 replicas
> * insert (1, '6', 6) succeeded on r1 and r3, failed on r2
> * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and r2, 
> failed on r3
> * insert (1, '7', 7) succeeded on r1 and r2, failed on r3
> Local data on 3 nodes looks like as below now
> r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7)
> r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7)
> r3: (1, '1', 1),  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5), (1, 
> '6', 6)
> If we do a paging read with page_size 2, and if it gets data from r2 and r3, 
> then it will only get the data (1, '1', 1) and (1, '7', 7) skipping record 6. 
> This problem would happen if the same query is not doing paging but limit set 
> to 2 records.
> Resolution code for reads works same for paging queries and normal queries. 
> Co-ordinator shouldn't respond back to client with records/columns that it 
> didn't have complete visibility on all required replicas (in this case 2 
> replicas). In above case, it is sending back record (1, '7', 7) back to 
> client, but its visibility on r3 is limited up to (1, '2', 2) and it is 
> relying on just r2 data to assume (1, '6', 6) doesn't exist, which is wrong. 
> End of the resolution all it can conclusively say any thing about is (1, '1', 
>  and the other one is that we  and and and and and and the and the and the 
> and d and the other is and 1), which exists and (1, '2', 2), which is deleted.
> Ideally we should have different resolution implementation for paging/limit 
> queries.
> We could reproduce this on 2.0.17, 2.1.16 and 3.0.9.
> Seems like 3.0.9 we have ShortReadProtection transformation on list queries. 
> I assume that is to protect against the cases like above. But, we can 
> reproduce the issue in 3.0.9 as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13862) Optimize Paxos prepare and propose stage for local requests

[Jaydeepkumar Chovatia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaydeepkumar Chovatia updated CASSANDRA-13862:
--
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)

> Optimize Paxos prepare and propose stage for local requests 
> 
>
> Key: CASSANDRA-13862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
>
> Currently Paxos prepare and propose messages always go through entire 
> MessagingService stack in Cassandra even if request is to be served locally, 
> we can enhance and make local requests severed w/o involving 
> MessagingService. Similar things are done at may [other places | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1244]
>  in Cassandra which skips MessagingService stage for local requests.
> This is what it looks like currently if we have tracing on and run Cassandra 
> light weight transaction.
> {quote}
> Sending PAXOS_PREPARE message to /A.B.C.D 
> [MessagingService-Outgoing-/A.B.C.D] | 2017-09-11 21:55:18.971000 |  A.B.C.D 
> |  15045
> … 
>  
> REQUEST_RESPONSE message received from /A.B.C.D 
> [MessagingService-Incoming-/A.B.C.D] | 2017-09-11 21:55:18.976000 |  A.B.C.D 
> |  20270
> … 
>   
>  Processing response from /A.B.C.D [SharedPool-Worker-4] 
> | 2017-09-11 21:55:18.976000 |  A.B.C.D |  20372
> {quote}
> Same thing applies for {{Propose stage}} as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13866) Clock-dependent integer overflow in tests CellTest and RowsTest

[Joel Knighton (JIRA)]

Joel Knighton created CASSANDRA-13866:
-

 Summary: Clock-dependent integer overflow in tests CellTest and 
RowsTest
 Key: CASSANDRA-13866
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13866
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Joel Knighton
Assignee: Joel Knighton
Priority: Trivial


These tests create timestamps from Unix time, but this is done as int math with 
the result stored in a long. This means that if the test is run at certain 
times, like 1505177731, corresponding to Tuesday, September 12, 2017, 12:55:31, 
the test can have two timestamps separated by a single second that reverse 
their ordering when multiplied by 100, such as 1505177731 -> 2147149504 and 
1505177732 -> -2146817792. This causes a variety of test failures, since it 
changes the reconciliation order of these cells.

Note that I've tagged this as trivial because the problem is in the manual 
construction of timestamps in the test; I know of nowhere  that we make this 
mistake with real data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13862) Optimize Paxos prepare and propose stage for local requests

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165400#comment-16165400
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13862:
---

Hi [~aweisberg]

Thanks for code review! Please find my response here:

{quote}
Thanks Jaydeepkumar. Because this is a performance improvement it can only go 
in trunk and not 3.0 or 3.11.
{quote}
Sure

{quote}
Playing devil's advocate does this do much for performance? PAXOS is already so 
slow due to other aspects of the implementation and it's generally always going 
to require waiting for remote responses anyways. If we had a better 
implementation of the rest of it then I can definitely see how it's a bigger 
advantage like it is with a regular mutation which is pretty lightweight in a 
lot of cases.
{quote}
It does not make much difference if we test against low transaction rate (few 
hundreds) but as we start hitting millions of transactions then it makes 
significant improvement. I tried test with 40K lwt/second with and without this 
change, and with this patch it definitely helps with latency specifically p99 
latency. I believe if we go higher transaction rate (in millions) then impact 
will be bigger.

{quote}
Should the exception path here invoke an error callback or message of some 
sort? Are there counters that should be incremented? I'm looking at 
StorageProxy.performLocally for clues as to the right way to do this.
{quote}
[PrepareCallback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/paxos/PrepareCallback.java#L42]
 and 
[ProposeCallback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/paxos/ProposeCallback.java#L45]
 have been derived from [AbstractPaxosCallback | 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/paxos/AbstractPaxosCallback.java#L33]
 which is further derived from 
[IAsyncCallback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/IAsyncCallback.java#L32]
 and 
[IAsyncCallback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/IAsyncCallback.java#L32]
 doesn’t have a support for error callback, hence we don’t need to worry about 
it. If it would have been derived from 
[IAsyncCallbackWithFailure|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/IAsyncCallbackWithFailure.java#L24]
 then we have to handle error callback.
No counter increment also required here.

{quote}
Will passing it off to the other thread in this situation correctly preserve 
any necessary thread local tracing state?
{quote}
Yes, It will be similar to the other places where we are doing this type of 
local requests.

{quote}
Is it possible to re-use the existing remote message handling path to some 
degree? Right now those verb handlers are pretty trivial, but I would like to 
avoid the situation where the two diverge and no one 
notices. Factoring it out into a common method makes it obvious this process 
occurs in two places.
{quote}
I’ve modified code to re-use existing Verb handler code and made it a common, 
please review it in the updated code:

||Branch||uTest||
|[trunk|https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13862-trunk]|[circleci
 |https://circleci.com/gh/jaydeepkumar1984/cassandra/18]|

Jaydeep

> Optimize Paxos prepare and propose stage for local requests 
> 
>
> Key: CASSANDRA-13862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Currently Paxos prepare and propose messages always go through entire 
> MessagingService stack in Cassandra even if request is to be served locally, 
> we can enhance and make local requests severed w/o involving 
> MessagingService. Similar things are done at may [other places | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1244]
>  in Cassandra which skips MessagingService stage for local requests.
> This is what it looks like currently if we have tracing on and run Cassandra 
> light weight transaction.
> {quote}
> Sending PAXOS_PREPARE message to /A.B.C.D 
> [MessagingService-Outgoing-/A.B.C.D] | 2017-09-11 21:55:18.971000 |  A.B.C.D 
> |  15045
> … 
>  
> REQUEST_RESPONSE message received from /A.B.C.D 
> [MessagingService-Incoming-/A.B.C.D] | 2017-09-11 21:55:18.976000 | 

[jira] [Updated] (CASSANDRA-13794) Fix short read protection logic for querying more rows

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13794:
---
Labels: Correctness  (was: )

> Fix short read protection logic for querying more rows
> --
>
> Key: CASSANDRA-13794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13794
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Benedict
>Assignee: Aleksey Yeschenko
>  Labels: Correctness
> Fix For: 3.0.x, 3.11.x
>
>
> Discovered by [~benedict] while reviewing CASSANDRA-13747:
> {quote}
> While reviewing I got a little suspicious of the modified line 
> {{DataResolver}} :479, as it seemed that n and x were the wrong way around... 
> and, reading the comment of intent directly above, and reproducing the 
> calculation, they are indeed.
> This is probably a significant enough bug that it warrants its own ticket for 
> record keeping, though I'm fairly agnostic on that decision.
> I'm a little concerned about our current short read behaviour, as right now 
> it seems we should be requesting exactly one row, for any size of under-read, 
> which could mean extremely poor performance in case of large under-reads.
> I would suggest that the outer unconditional {{Math.max}} is a bad idea, has 
> been (poorly) insulating us from this error, and that we should first be 
> asserting that the calculation yields a value >= 0 before setting to 1.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12872) Paging reads and limit reads are missing some data

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12872:
---
Labels: Correctness  (was: )

> Paging reads and limit reads are missing some data
> --
>
> Key: CASSANDRA-12872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12872
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Bhaskar Muppana
>Assignee: Benjamin Lerer
>Priority: Critical
>  Labels: Correctness
> Attachments: limiterr-reproduce.sh
>
>
> We are seeing an issue with paging reads missing some small number of columns 
> when we do paging/limit reads. We get this on a single DC cluster itself when 
> both reads and writes are happening with QUORUM. Paging/limit reads see this 
> issue. I have attached the ccm based script which reproduces the problem.
> * Keyspace RF - 3
> * Table (id int, course text, marks int, primary key(id, course))
> * replicas for partition key 1 - r1, r2 and r3
> * insert (1, '1', 1) ,  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5) 
> - succeeded on all 3 replicas
> * insert (1, '6', 6) succeeded on r1 and r3, failed on r2
> * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and r2, 
> failed on r3
> * insert (1, '7', 7) succeeded on r1 and r2, failed on r3
> Local data on 3 nodes looks like as below now
> r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7)
> r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7)
> r3: (1, '1', 1),  (1, '2', 2),  (1, '3', 3),  (1, '4', 4),  (1, '5', 5), (1, 
> '6', 6)
> If we do a paging read with page_size 2, and if it gets data from r2 and r3, 
> then it will only get the data (1, '1', 1) and (1, '7', 7) skipping record 6. 
> This problem would happen if the same query is not doing paging but limit set 
> to 2 records.
> Resolution code for reads works same for paging queries and normal queries. 
> Co-ordinator shouldn't respond back to client with records/columns that it 
> didn't have complete visibility on all required replicas (in this case 2 
> replicas). In above case, it is sending back record (1, '7', 7) back to 
> client, but its visibility on r3 is limited up to (1, '2', 2) and it is 
> relying on just r2 data to assume (1, '6', 6) doesn't exist, which is wrong. 
> End of the resolution all it can conclusively say any thing about is (1, '1', 
>  and the other one is that we  and and and and and and the and the and the 
> and d and the other is and 1), which exists and (1, '2', 2), which is deleted.
> Ideally we should have different resolution implementation for paging/limit 
> queries.
> We could reproduce this on 2.0.17, 2.1.16 and 3.0.9.
> Seems like 3.0.9 we have ShortReadProtection transformation on list queries. 
> I assume that is to protect against the cases like above. But, we can 
> reproduce the issue in 3.0.9 as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13797) RepairJob blocks on syncTasks

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-13797:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Got clean utest runs on 3.0, 3.11, and trunk. 3.0 and trunk dtest failures were 
either also failing in their respective parent branches, or not reproducible 
locally. Committed to 3.0 as {{e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d}} and 
merged up

> RepairJob blocks on syncTasks
> -
>
> Key: CASSANDRA-13797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> The thread running {{RepairJob}} blocks while it waits for the validations it 
> starts to complete ([see 
> here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
>  However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't 
> waiting for {{RepairJob#run}} to return, they're waiting for a result to be 
> set on RepairJob the future, which happens after the sync tasks have 
> completed. This post repair cleanup stuff also immediately shuts down the 
> executor {{RepairJob#run}} is running in. So in noop repair sessions, where 
> there's nothing to stream, I'm seeing the callbacks sometimes fire before 
> {{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.
> I'm pretty sure this can just be removed, but I'd like a second opinion. This 
> appears to just be a holdover from before repair coordination became async. I 
> thought it might be doing some throttling by blocking, but each repair 
> session gets it's own executor, and validation is  throttled by the fixed 
> size executors doing the actual work of validation, so I don't think we need 
> to keep this around.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[bdeggleston]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/772f0ea9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/772f0ea9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/772f0ea9

Branch: refs/heads/trunk
Commit: 772f0ea9d7760976bda8d735c81ae10fad096119
Parents: e7e9367 28b75ba
Author: Blake Eggleston 
Authored: Wed Sep 13 15:03:14 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 15:03:14 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/772f0ea9/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/772f0ea9/src/java/org/apache/cassandra/repair/RepairJob.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Don't block RepairJob execution on validation futures

[bdeggleston]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 d826c8187 -> e7299c08f
  refs/heads/cassandra-3.11 cbc26d81d -> 28b75ba35
  refs/heads/trunk e7e936735 -> 772f0ea9d


Don't block RepairJob execution on validation futures

Patch by Blake Eggleston; reviewed by Marcus Eriksson for CASSANDRA-13797


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7299c08
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7299c08
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7299c08

Branch: refs/heads/cassandra-3.0
Commit: e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d
Parents: d826c81
Author: Blake Eggleston 
Authored: Thu Aug 24 15:16:37 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 14:58:03 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d2d5e7..7ce298d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
  * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index cba176c..0711a64 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,9 +155,6 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
-
-// Wait for validation to complete
-Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Don't block RepairJob execution on validation futures

[bdeggleston]

Don't block RepairJob execution on validation futures

Patch by Blake Eggleston; reviewed by Marcus Eriksson for CASSANDRA-13797


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7299c08
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7299c08
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7299c08

Branch: refs/heads/trunk
Commit: e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d
Parents: d826c81
Author: Blake Eggleston 
Authored: Thu Aug 24 15:16:37 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 14:58:03 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d2d5e7..7ce298d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
  * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index cba176c..0711a64 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,9 +155,6 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
-
-// Wait for validation to complete
-Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Don't block RepairJob execution on validation futures

[bdeggleston]

Don't block RepairJob execution on validation futures

Patch by Blake Eggleston; reviewed by Marcus Eriksson for CASSANDRA-13797


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7299c08
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7299c08
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7299c08

Branch: refs/heads/cassandra-3.11
Commit: e7299c08f940057e8fd4dfa3f24dcc6e0cb5f78d
Parents: d826c81
Author: Blake Eggleston 
Authored: Thu Aug 24 15:16:37 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 14:58:03 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d2d5e7..7ce298d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
  * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7299c08/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java 
b/src/java/org/apache/cassandra/repair/RepairJob.java
index cba176c..0711a64 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -155,9 +155,6 @@ public class RepairJob extends AbstractFuture 
implements Runnable
 setException(t);
 }
 }, taskExecutor);
-
-// Wait for validation to complete
-Futures.getUnchecked(validations);
 }
 
 /**


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28b75ba3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28b75ba3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28b75ba3

Branch: refs/heads/cassandra-3.11
Commit: 28b75ba355b0d69ce09789a737df2546cd5c0954
Parents: cbc26d8 e7299c0
Author: Blake Eggleston 
Authored: Wed Sep 13 15:01:41 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 15:01:41 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28b75ba3/CHANGES.txt
--
diff --cc CHANGES.txt
index b3fb5a6,7ce298d..bc84477
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.15
 +3.11.1
 + * Add a compaction option to TWCS to ignore sstables overlapping checks 
(CASSANDRA-13418)
 + * BTree.Builder memory leak (CASSANDRA-13754)
 + * Revert CASSANDRA-10368 of supporting non-pk column filtering due to 
correctness (CASSANDRA-13798)
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
   * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
   * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
   * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/28b75ba3/src/java/org/apache/cassandra/repair/RepairJob.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[bdeggleston]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28b75ba3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28b75ba3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28b75ba3

Branch: refs/heads/trunk
Commit: 28b75ba355b0d69ce09789a737df2546cd5c0954
Parents: cbc26d8 e7299c0
Author: Blake Eggleston 
Authored: Wed Sep 13 15:01:41 2017 -0700
Committer: Blake Eggleston 
Committed: Wed Sep 13 15:01:41 2017 -0700

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/repair/RepairJob.java | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28b75ba3/CHANGES.txt
--
diff --cc CHANGES.txt
index b3fb5a6,7ce298d..bc84477
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.15
 +3.11.1
 + * Add a compaction option to TWCS to ignore sstables overlapping checks 
(CASSANDRA-13418)
 + * BTree.Builder memory leak (CASSANDRA-13754)
 + * Revert CASSANDRA-10368 of supporting non-pk column filtering due to 
correctness (CASSANDRA-13798)
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Don't block RepairJob execution on validation futures (CASSANDRA-13797)
   * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
   * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
   * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/28b75ba3/src/java/org/apache/cassandra/repair/RepairJob.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13734) BufferUnderflowException when using uppercase UUID

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165349#comment-16165349
 ] 

Jason Brown commented on CASSANDRA-13734:
-

[~clauds2x] Thanks for all the info you've provided. Similar to [~bdeggleston], 
i haven't been able to reproduce yet. Can you share examples of how you 
insert/update data? Also, do you use TTLs on the table? If you can share the 
data from a failing row that would be great, as well, but it looks like your 
data domain is probably rather sensitive, judging from the column names.

The stack trace shows that we're failing to serialize the {{participants 
frozen>}} correctly, as we've reached the end of the 
{{ByteBuffer}}. Looking at your log dump, the line SliceQueryFilter that 
describes loaded/merged {{participants}} is:

{code}
TRACE [SharedPool-Worker-2] 2017-07-28 20:40:45,316 SliceQueryFilter.java:269 - 
collecting 1 of 100: participants:false:0@1501267173323000
{code}

The "0" after the {{false}} indicated how many bytes are in the buffer for that 
{{participants}} column. When we try to get the size of the set 
([here|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/serializers/CollectionSerializer.java#L85]),
 it tries to read an int from the buffer, but because it's empty the 
{{BufferUnderflowException}} is thrown.

It's not clear to me how we would get an empty buffer there; further, I'm not 
sure how the UUID is related. Providing the insert/update statements will help 
a lot. Also, do you perform a lot of rapid updates to a given partition, 
especially of the {{participants}} field?

If you can create an explicit example of how to repro that would be great. I'll 
keep digging in the meantime...

> BufferUnderflowException when using uppercase UUID
> --
>
> Key: CASSANDRA-13734
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13734
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.2.8 running on OSX 10.12.5
> * org.apache.cassandra:cassandra-all:jar:2.2.8
> * com.datastax.cassandra:cassandra-driver-core:jar:3.0.0
> * org.apache.cassandra:cassandra-thrift:jar:2.2.8
>Reporter: Claudia S
>
> We have a table with a primary key of type uuid which we query for results in 
> JSON format. When I accidentally caused a query passing a UUID which has an 
> uppercase letter I noticed that this causes a BufferUnderflowException on 
> Cassandra.
> I directly attempted the queries using cqlsh, I can retrieve the entry using 
> standard select but whenever I pass JSON I get a BufferUnderflowException.
> {code:title=cql queries}
> cassandra@cqlsh:event_log_system> SELECT * FROM event WHERE id = 
> 559a4d83-9410-4b69-b459-566b8cf57aaa;
> [RESULT REMOVED]
> (1 rows)
> cassandra@cqlsh:event_log_system> SELECT * FROM event WHERE id = 
> 559a4d83-9410-4b69-b459-566b8cf57AAA;
> [RESULT REMOVED]
> (1 rows)
> cassandra@cqlsh:event_log_system> SELECT JSON * FROM event WHERE id = 
> 559a4d83-9410-4b69-b459-566b8cf57AAA;
> ServerError: java.nio.BufferUnderflowException
> cassandra@cqlsh:event_log_system> SELECT JSON * FROM event WHERE id = 
> 559a4d83-9410-4b69-b459-566b8cf57aaa;
> ServerError: java.nio.BufferUnderflowException
> {code}
> {code:title=log}
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,392 Message.java:506 - 
> Received: QUERY SELECT JSON * FROM event WHERE id = 
> 559a4d83-9410-4b69-b459-566b8cf57AAA;, v=4
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,392 QueryProcessor.java:221 - 
> Process org.apache.cassandra.cql3.statements.SelectStatement@67e6c0c @CL.ONE
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,392 ReadCallback.java:76 - 
> Blockfor is 1; setting up requests to localhost/127.0.0.1
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,393 
> AbstractReadExecutor.java:118 - reading data locally
> TRACE [SharedPool-Worker-2] 2017-07-28 20:40:41,393 SliceQueryFilter.java:269 
> - collecting 0 of 2147483647: :false:0@150126701983
> TRACE [SharedPool-Worker-2] 2017-07-28 20:40:41,393 SliceQueryFilter.java:269 
> - collecting 1 of 2147483647: can_login:false:1@150126701983
> TRACE [SharedPool-Worker-2] 2017-07-28 20:40:41,393 SliceQueryFilter.java:269 
> - collecting 1 of 2147483647: is_superuser:false:1@150126701983
> TRACE [SharedPool-Worker-2] 2017-07-28 20:40:41,393 SliceQueryFilter.java:269 
> - collecting 1 of 2147483647: salted_hash:false:60@150126701983
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,393 StorageProxy.java:1449 - 
> Read: 0 ms.
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,394 ReadCallback.java:76 - 
> Blockfor is 1; setting up requests to localhost/127.0.0.1
> TRACE [SharedPool-Worker-1] 2017-07-28 20:40:41,394 
> AbstractReadExecutor.java:118 - reading data locally
> TRACE [SharedPool-Worker-2] 2017-07-28 20:40:41,394 

[jira] [Reopened] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reopened CASSANDRA-13865:
---

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="madeup_ks")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165317#comment-16165317
 ] 

Aleksey Yeschenko commented on CASSANDRA-13865:
---

I see. I'm not sure that this is a problem necessarily, but now at least I 
understand what you mean.

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="madeup_ks")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Jaume M (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaume M updated CASSANDRA-13865:

Description: 
Using the python driver this works and the query is prepared against 
{{my_keyspace}}, {{system}} is ignored.
{code}
prepared_statement = self.session.prepare("SELECT * from my_keyspace.my_table", 
keyspace="system")
{code}

However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
query] message="keyspace madeup_ks does not exist"}}
{code}
prepared_statement = self.session.prepare("SELECT * from my_keyspace.my_table", 
keyspace="madeup_ks")
{code}

  was:
Using the python driver this works and the query is prepared against 
{{my_keyspace}}, {{system}} is ignored.
{code}
prepared_statement = self.session.prepare("SELECT * from my_keyspace.my_table", 
keyspace="system")
{code}

However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
query] message="keyspace madeup_ks does not exist"}}
{code}
prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
keyspace="system")
{code}


> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="madeup_ks")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Jaume M (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165311#comment-16165311
 ] 

Jaume M commented on CASSANDRA-13865:
-

Sorry [~iamaleksey], I've updated the description of the ticket, which was 
wrong, maybe it makes sense now

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="madeup_ks")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165304#comment-16165304
 ] 

Aleksey Yeschenko commented on CASSANDRA-13865:
---

Sorry, but I'm not getting how this is an issue. What behaviour do you expect, 
and why?

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
> keyspace="system")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Jaume M (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165301#comment-16165301
 ] 

Jaume M commented on CASSANDRA-13865:
-

[~iamaleksey] yes, that's what's happening, the fully qualified keyspace is 
being used. The issue is that depending on if the other keyspace exists or not 
the query will succeed or it will return an InvalidRequest

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
> keyspace="system")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165171#comment-16165171
 ] 

Aleksey Yeschenko commented on CASSANDRA-13865:
---

As it should. The keyspace in the fully qualified keyspace.table name always 
take precedence over session's current keyspace in use.

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
> keyspace="system")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-13865.
---
Resolution: Not A Problem

> Non consistent response when setting the keysace in query an explicily
> --
>
> Key: CASSANDRA-13865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jaume M
>
> Using the python driver this works and the query is prepared against 
> {{my_keyspace}}, {{system}} is ignored.
> {code}
> prepared_statement = self.session.prepare("SELECT * from 
> my_keyspace.my_table", keyspace="system")
> {code}
> However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
> query] message="keyspace madeup_ks does not exist"}}
> {code}
> prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
> keyspace="system")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13865) Non consistent response when setting the keysace in query an explicily

[Jaume M (JIRA)]

Jaume M created CASSANDRA-13865:
---

 Summary: Non consistent response when setting the keysace in query 
an explicily
 Key: CASSANDRA-13865
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13865
 Project: Cassandra
  Issue Type: Bug
Reporter: Jaume M


Using the python driver this works and the query is prepared against 
{{my_keyspace}}, {{system}} is ignored.
{code}
prepared_statement = self.session.prepare("SELECT * from my_keyspace.my_table", 
keyspace="system")
{code}

However this throws a {{InvalidRequest: Error from server: code=2200 [Invalid 
query] message="keyspace madeup_ks does not exist"}}
{code}
prepared_statement = self.session.prepare("SELECT * from madeup_ks.my_table", 
keyspace="system")
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13862) Optimize Paxos prepare and propose stage for local requests

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13862:
---
Reviewer: Ariel Weisberg

> Optimize Paxos prepare and propose stage for local requests 
> 
>
> Key: CASSANDRA-13862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Currently Paxos prepare and propose messages always go through entire 
> MessagingService stack in Cassandra even if request is to be served locally, 
> we can enhance and make local requests severed w/o involving 
> MessagingService. Similar things are done at may [other places | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1244]
>  in Cassandra which skips MessagingService stage for local requests.
> This is what it looks like currently if we have tracing on and run Cassandra 
> light weight transaction.
> {quote}
> Sending PAXOS_PREPARE message to /A.B.C.D 
> [MessagingService-Outgoing-/A.B.C.D] | 2017-09-11 21:55:18.971000 |  A.B.C.D 
> |  15045
> … 
>  
> REQUEST_RESPONSE message received from /A.B.C.D 
> [MessagingService-Incoming-/A.B.C.D] | 2017-09-11 21:55:18.976000 |  A.B.C.D 
> |  20270
> … 
>   
>  Processing response from /A.B.C.D [SharedPool-Worker-4] 
> | 2017-09-11 21:55:18.976000 |  A.B.C.D |  20372
> {quote}
> Same thing applies for {{Propose stage}} as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13862) Optimize Paxos prepare and propose stage for local requests

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg reassigned CASSANDRA-13862:
--

Assignee: Jaydeepkumar Chovatia

> Optimize Paxos prepare and propose stage for local requests 
> 
>
> Key: CASSANDRA-13862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Currently Paxos prepare and propose messages always go through entire 
> MessagingService stack in Cassandra even if request is to be served locally, 
> we can enhance and make local requests severed w/o involving 
> MessagingService. Similar things are done at may [other places | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1244]
>  in Cassandra which skips MessagingService stage for local requests.
> This is what it looks like currently if we have tracing on and run Cassandra 
> light weight transaction.
> {quote}
> Sending PAXOS_PREPARE message to /A.B.C.D 
> [MessagingService-Outgoing-/A.B.C.D] | 2017-09-11 21:55:18.971000 |  A.B.C.D 
> |  15045
> … 
>  
> REQUEST_RESPONSE message received from /A.B.C.D 
> [MessagingService-Incoming-/A.B.C.D] | 2017-09-11 21:55:18.976000 |  A.B.C.D 
> |  20270
> … 
>   
>  Processing response from /A.B.C.D [SharedPool-Worker-4] 
> | 2017-09-11 21:55:18.976000 |  A.B.C.D |  20372
> {quote}
> Same thing applies for {{Propose stage}} as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13862) Optimize Paxos prepare and propose stage for local requests

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165010#comment-16165010
 ] 

Ariel Weisberg commented on CASSANDRA-13862:


Thanks Jaydeepkumar. Because this is a performance improvement it can only go 
in trunk and not 3.0 or 3.11.

Playing devil's advocate does this do much for performance? PAXOS is already so 
slow due to other aspects of the implementation and it's generally always going 
to require waiting for remote responses anyways. If we had a better 
implementation of the rest of it then I can definitely see how it's a bigger 
advantage like it is with a regular mutation which is pretty lightweight in a 
lot of cases.

Should the exception path here invoke an error callback or message of some 
sort? Are there counters that should be incremented? I'm looking at 
{{StorageProxy.performLocally}} for clues as to the right way to do this.

Will passing it off to the other thread in this situation correctly preserve 
any necessary thread local tracing state?

Is it possible to re-use the existing remote message handling path to some 
degree? Right now those verb handlers are pretty trivial, but I would like to 
avoid the situation where the two diverge and no one notices. Factoring it out 
into a common method makes it obvious this process occurs in two places.

> Optimize Paxos prepare and propose stage for local requests 
> 
>
> Key: CASSANDRA-13862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Currently Paxos prepare and propose messages always go through entire 
> MessagingService stack in Cassandra even if request is to be served locally, 
> we can enhance and make local requests severed w/o involving 
> MessagingService. Similar things are done at may [other places | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1244]
>  in Cassandra which skips MessagingService stage for local requests.
> This is what it looks like currently if we have tracing on and run Cassandra 
> light weight transaction.
> {quote}
> Sending PAXOS_PREPARE message to /A.B.C.D 
> [MessagingService-Outgoing-/A.B.C.D] | 2017-09-11 21:55:18.971000 |  A.B.C.D 
> |  15045
> … 
>  
> REQUEST_RESPONSE message received from /A.B.C.D 
> [MessagingService-Incoming-/A.B.C.D] | 2017-09-11 21:55:18.976000 |  A.B.C.D 
> |  20270
> … 
>   
>  Processing response from /A.B.C.D [SharedPool-Worker-4] 
> | 2017-09-11 21:55:18.976000 |  A.B.C.D |  20372
> {quote}
> Same thing applies for {{Propose stage}} as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13847) test failure in cqlsh_tests.cqlsh_tests.CqlLoginTest.test_list_roles_after_login

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13847:
--
   Resolution: Fixed
Fix Version/s: 2.2.x
   Status: Resolved  (was: Ready to Commit)

> test failure in 
> cqlsh_tests.cqlsh_tests.CqlLoginTest.test_list_roles_after_login
> 
>
> Key: CASSANDRA-13847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13847
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Joel Knighton
>Assignee: Andrés de la Peña
>  Labels: test-failure
> Fix For: 2.1.x, 2.2.x
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/546/testReport/cqlsh_tests.cqlsh_tests/CqlLoginTest/test_list_roles_after_login
> This test was added for [CASSANDRA-13640]. The comments seem to indicated 
> this is only a problem on 3.0+, but the added test certainly seems to 
> reproduce the problem on 2.1 and 2.2. Even if the issue does affect 2.1/2.2, 
> it seems insufficiently critical for 2.1, so we need to limit the test to run 
> on 2.2+ at the very least, possibly 3.0+ if we don't fix the cause on 2.2.
> Thoughts [~adelapena]?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13754) BTree.Builder memory leak

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp resolved CASSANDRA-13754.
--
Resolution: Fixed

[~urandom], it's not about the {{BTree.Builder}} instances, it's about what's 
kept referenced by those - and that is the bunch of {{HeapByteBuffer}} 
instances, as reported by [~markusdlugi] and fixed by the patch for this 
ticket. The screenshot you attached, matches the fixed issue. Therefore, I 
recommend to try the patch or a build from the recent 3.11/trunk branches and 
test again. Going go resolve this issue. If it's really something else that's 
causing the issue, I'd prefer to open another ticket, because there is already 
something committed for this ticket that addresses a very particular issue.

> BTree.Builder memory leak
> -
>
> Key: CASSANDRA-13754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13754
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 3.11.0, Netty 4.0.44.Final, OpenJDK 8u141-b15
>Reporter: Eric Evans
>Assignee: Robert Stupp
> Fix For: 3.11.1
>
> Attachments: Screenshot from 2017-09-11 16-54-43.png, Screenshot from 
> 2017-09-13 10-39-58.png
>
>
> After a chronic bout of {{OutOfMemoryError}} in our development environment, 
> a heap analysis is showing that more than 10G of our 12G heaps are consumed 
> by the {{threadLocals}} members (instances of {{java.lang.ThreadLocalMap}}) 
> of various {{io.netty.util.concurrent.FastThreadLocalThread}} instances.  
> Reverting 
> [cecbe17|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=cecbe17e3eafc052acc13950494f7dddf026aa54]
>  fixes the issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13847) test failure in cqlsh_tests.cqlsh_tests.CqlLoginTest.test_list_roles_after_login

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164957#comment-16164957
 ] 

Andrés de la Peña commented on CASSANDRA-13847:
---

Thanks for the review.

Committed to 2.2 as 
[43e2a107072bc86c0e26bc2036a61a9ad600f213|https://github.com/apache/cassandra/commit/43e2a107072bc86c0e26bc2036a61a9ad600f213].

Dtest patch committed as 
[3435b0f2121fa4c6099098e562d0bf5f4bd78d5f|https://github.com/apache/cassandra-dtest/commit/3435b0f2121fa4c6099098e562d0bf5f4bd78d5f].

> test failure in 
> cqlsh_tests.cqlsh_tests.CqlLoginTest.test_list_roles_after_login
> 
>
> Key: CASSANDRA-13847
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13847
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing, Tools
>Reporter: Joel Knighton
>Assignee: Andrés de la Peña
>  Labels: test-failure
> Fix For: 2.1.x
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/546/testReport/cqlsh_tests.cqlsh_tests/CqlLoginTest/test_list_roles_after_login
> This test was added for [CASSANDRA-13640]. The comments seem to indicated 
> this is only a problem on 3.0+, but the added test certainly seems to 
> reproduce the problem on 2.1 and 2.2. Even if the issue does affect 2.1/2.2, 
> it seems insufficiently critical for 2.1, so we need to limit the test to run 
> on 2.2+ at the very least, possibly 3.0+ if we don't fix the cause on 2.2.
> Thoughts [~adelapena]?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: Add missed `@since` tag to `CqlLoginTest.test_list_roles_after_login`

[adelapena]

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master c39a85c3e -> 3435b0f21


Add missed `@since` tag to `CqlLoginTest.test_list_roles_after_login`

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13847


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/3435b0f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/3435b0f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/3435b0f2

Branch: refs/heads/master
Commit: 3435b0f2121fa4c6099098e562d0bf5f4bd78d5f
Parents: c39a85c
Author: Andrés de la Peña 
Authored: Wed Sep 13 18:01:48 2017 +0100
Committer: Andrés de la Peña 
Committed: Wed Sep 13 18:01:48 2017 +0100

--
 cqlsh_tests/cqlsh_tests.py | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/3435b0f2/cqlsh_tests/cqlsh_tests.py
--
diff --git a/cqlsh_tests/cqlsh_tests.py b/cqlsh_tests/cqlsh_tests.py
index 8b66a53..8146ace 100644
--- a/cqlsh_tests/cqlsh_tests.py
+++ b/cqlsh_tests/cqlsh_tests.py
@@ -2033,6 +2033,7 @@ class CqlLoginTest(Tester):
 self.assertEqual([x for x in cqlsh_stdout.split() if x], ['ks1table'])
 self.assert_login_not_allowed('user1', cqlsh_stderr)
 
+@since('2.2')
 def test_list_roles_after_login(self):
 """
 @jira_ticket CASSANDRA-13640


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Copy session properties on cqlsh.py do_login

[adelapena]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 717dbe741 -> 43e2a1070


Copy session properties on cqlsh.py do_login

patch by Andres de la Peña; reviewed by Zhao Yang for CASSANDRA-13847


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43e2a107
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43e2a107
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43e2a107

Branch: refs/heads/cassandra-2.2
Commit: 43e2a107072bc86c0e26bc2036a61a9ad600f213
Parents: 717dbe7
Author: Andrés de la Peña 
Authored: Wed Sep 13 17:58:26 2017 +0100
Committer: Andrés de la Peña 
Committed: Wed Sep 13 17:58:26 2017 +0100

--
 CHANGES.txt  | 1 +
 bin/cqlsh.py | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e2a107/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1abd7de..d4d9e4f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.11
+ * Copy session properties on cqlsh.py do_login (CASSANDRA-13847)
  * Fix load over calculated issue in IndexSummaryRedistribution 
(CASSANDRA-13738)
  * Fix compaction and flush exception not captured (CASSANDRA-13833)
  * Make BatchlogManagerMBean.forceBatchlogReplay() blocking (CASSANDRA-13809)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e2a107/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index a9c5ff1..1f63826 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2115,6 +2115,12 @@ class Shell(cmd.Cmd):
 else:
 session = conn.connect()
 
+# Copy session properties
+session.default_timeout = self.session.default_timeout
+session.row_factory = self.session.row_factory
+session.default_consistency_level = 
self.session.default_consistency_level
+session.max_trace_wait = self.session.max_trace_wait
+
 # Update after we've connected in case we fail to authenticate
 self.conn = conn
 self.auth_provider = auth_provider


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164951#comment-16164951
 ] 

Aleksey Yeschenko commented on CASSANDRA-13740:
---

A bit busy currently, sorry. Will have a look as soon as I can.

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164912#comment-16164912
 ] 

Aleksey Yeschenko commented on CASSANDRA-13595:
---

[~jasonstack] Yours is not a duplicate, I'd just recommend holding it off a bit 
until the fixes for CASSANDRA-13794 and CASSANDRA-12872 get in. The code you've 
submitted doesn't seem very far off of what it should be, either. I will 
probably review/collaborate after I'm done with the other two tickets.

But this is 3.0. Have you looked into 2.1? I'm not sure how I feel about 
committing it to 2.1. On one hand, it's a huge correctness issues. Does it 
count as critical?

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164885#comment-16164885
 ] 

Jeff Jirsa commented on CASSANDRA-13595:


Quick note that the dtest repo has moved to 
[apache/cassandra-dtest|https://github.com/apache/cassandra-dtest] - so we 
should move 
[e706952e|https://github.com/riptano/cassandra-dtest/commit/e706952e8d3bb18af1f96cb0cfc283e53260513e]
 over to the new repo

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13754) BTree.Builder memory leak

[Eric Evans (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164860#comment-16164860
 ] 

Eric Evans commented on CASSANDRA-13754:



{quote}
I think it might still be the same issue. The threads you mentioned are all 
created by the SEPWorker as well, as you can also see in your screenshot where 
your FastThreadLocalThread has a reference to an instance of that class. Now 
I'm not sure whether the actual content of your ThreadLocalMap s is the same as 
in my heap dump - in my case, the maps mostly held instances of BTree$Builder , 
which then had references to many byte[] arrays. Maybe you can check if this is 
the case for you as well?
{quote}

There are no instances of {{BTree}} here, (see new screenshot attached).

{quote}
Other than that, you could also try and see if the patches created by Robert 
Stupp alleviate your issue.
{quote}

Do you mean 
[bed7fa5|https://github.com/apache/cassandra/commit/bed7fa5ef8492d1ff3852cf299622a5ad4e0b621]?
  I haven't applied that, no, but it doesn't look like I'm leaking anything 
{{BTree}} so I don't think that would help.

> BTree.Builder memory leak
> -
>
> Key: CASSANDRA-13754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13754
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 3.11.0, Netty 4.0.44.Final, OpenJDK 8u141-b15
>Reporter: Eric Evans
>Assignee: Robert Stupp
> Fix For: 3.11.1
>
> Attachments: Screenshot from 2017-09-11 16-54-43.png, Screenshot from 
> 2017-09-13 10-39-58.png
>
>
> After a chronic bout of {{OutOfMemoryError}} in our development environment, 
> a heap analysis is showing that more than 10G of our 12G heaps are consumed 
> by the {{threadLocals}} members (instances of {{java.lang.ThreadLocalMap}}) 
> of various {{io.netty.util.concurrent.FastThreadLocalThread}} instances.  
> Reverting 
> [cecbe17|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=cecbe17e3eafc052acc13950494f7dddf026aa54]
>  fixes the issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13754) BTree.Builder memory leak

[Eric Evans (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans updated CASSANDRA-13754:
---
Attachment: Screenshot from 2017-09-13 10-39-58.png

> BTree.Builder memory leak
> -
>
> Key: CASSANDRA-13754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13754
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Cassandra 3.11.0, Netty 4.0.44.Final, OpenJDK 8u141-b15
>Reporter: Eric Evans
>Assignee: Robert Stupp
> Fix For: 3.11.1
>
> Attachments: Screenshot from 2017-09-11 16-54-43.png, Screenshot from 
> 2017-09-13 10-39-58.png
>
>
> After a chronic bout of {{OutOfMemoryError}} in our development environment, 
> a heap analysis is showing that more than 10G of our 12G heaps are consumed 
> by the {{threadLocals}} members (instances of {{java.lang.ThreadLocalMap}}) 
> of various {{io.netty.util.concurrent.FastThreadLocalThread}} instances.  
> Reverting 
> [cecbe17|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=cecbe17e3eafc052acc13950494f7dddf026aa54]
>  fixes the issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164833#comment-16164833
 ] 

ZhaoYang commented on CASSANDRA-13595:
--

[~iamaleksey] Thanks for the heads up.

| Source |
| [3.0|https://github.com/jasonstack/cassandra/commits/13595-3.0] |
| [dtest|https://github.com/riptano/cassandra-dtest/commits/13595] |

Here is draft of making {{ShortReadProtection}} extend {{MorePartitions}}.

If you have better solution or integration, feel free to mark it as duplicated. 

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13149) AssertionError prepending to a list

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13149:

Status: Patch Available  (was: Open)

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Jason Brown
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.8.jar:3.0.8]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13123) Draining a node might fail to delete all inactive commitlogs

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-13123:

   Resolution: Fixed
Fix Version/s: (was: 3.8)
   4.0
   3.11.1
   3.0.15
Reproduced In: 3.0.10, 2.2.8  (was: 2.2.8, 3.0.10)
   Status: Resolved  (was: Patch Available)

committed to 3.0 and up as sha {{d826c81874e5d13a30a418f8b982531cb7e5d158}}.

Thanks for the patch, and sorry for the delay!

> Draining a node might fail to delete all inactive commitlogs
> 
>
> Key: CASSANDRA-13123
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13123
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Jan Urbański
>Assignee: Jan Urbański
> Fix For: 3.0.15, 3.11.1, 4.0
>
> Attachments: 13123-2.2.8.txt, 13123-3.0.10.txt, 13123-3.9.txt, 
> 13123-trunk.txt
>
>
> After issuing a drain command, it's possible that not all of the inactive 
> commitlogs are removed.
> The drain command shuts down the CommitLog instance, which in turn shuts down 
> the CommitLogSegmentManager. This has the effect of discarding any pending 
> management tasks it might have, like the removal of inactive commitlogs.
> This in turn leads to an excessive amount of commitlogs being left behind 
> after a drain and a lengthy recovery after a restart. With a fleet of dozens 
> of nodes, each of them leaving several GB of commitlogs after a drain and 
> taking up to two minutes to recover them on restart, the additional time 
> required to restart the entire fleet becomes noticeable.
> This problem is not present in 3.x or trunk because of the CLSM rewrite done 
> in CASSANDRA-8844.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cbc26d81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cbc26d81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cbc26d81

Branch: refs/heads/cassandra-3.11
Commit: cbc26d81d53822d16eb83052a69426e4f61f77ef
Parents: 60a2d97 d826c81
Author: Jason Brown 
Authored: Wed Sep 13 06:29:42 2017 -0700
Committer: Jason Brown 
Committed: Wed Sep 13 06:30:33 2017 -0700

--
 CHANGES.txt |  1 +
 .../commitlog/CommitLogSegmentManagerTest.java  | 30 
 2 files changed, 31 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cbc26d81/CHANGES.txt
--
diff --cc CHANGES.txt
index ec9d126,4d2d5e7..b3fb5a6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.15
 +3.11.1
 + * Add a compaction option to TWCS to ignore sstables overlapping checks 
(CASSANDRA-13418)
 + * BTree.Builder memory leak (CASSANDRA-13754)
 + * Revert CASSANDRA-10368 of supporting non-pk column filtering due to 
correctness (CASSANDRA-13798)
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
   * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
   * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
   * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cbc26d81/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index e22e86f,41f5ed5..dfbf5ad
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@@ -20,9 -20,10 +20,11 @@@
   */
  package org.apache.cassandra.db.commitlog;
  
+ import java.io.File;
  import java.nio.ByteBuffer;
 +import java.util.ArrayList;
  import java.util.Random;
+ import java.util.UUID;
  import java.util.concurrent.Semaphore;
  
  import com.google.common.collect.ImmutableMap;
@@@ -139,4 -136,32 +141,32 @@@ public class CommitLogSegmentManagerTes
  Thread.currentThread().interrupt();
  }
  }
+ 
+ @Test
+ @BMRule(name = "Make removing commitlog segments slow",
+ targetClass = "CommitLogSegment",
+ targetMethod = "discard",
+ action = "Thread.sleep(50)")
+ public void testShutdownWithPendingTasks() throws Throwable {
+ CommitLog.instance.resetUnsafe(true);
+ ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+ 
+ final Mutation m = new RowUpdateBuilder(cfs1.metadata, 0, "k")
+.clustering("bytes")
+.add("val", ByteBuffer.wrap(entropy))
+.build();
+ 
+ // force creating several commitlog files
+ for (int i = 0; i < 10; i++) {
+ CommitLog.instance.add(m);
+ }
+ 
+ // schedule discarding completed segments and immediately issue a 
shutdown
+ UUID cfid = m.getColumnFamilyIds().iterator().next();
 -CommitLog.instance.discardCompletedSegments(cfid, 
ReplayPosition.NONE, CommitLog.instance.getContext());
++CommitLog.instance.discardCompletedSegments(cfid, 
CommitLogPosition.NONE, CommitLog.instance.getCurrentPosition());
+ CommitLog.instance.shutdownBlocking();
+ 
+ // the shutdown should block until all logs except the currently 
active one and perhaps a new, empty one are gone
 -Assert.assertTrue(new 
File(CommitLog.instance.location).listFiles().length <= 2);
++Assert.assertTrue(new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles().length <= 2);
+ }
  }

[1/6] cassandra git commit: Wait for all management tasks to complete before shutting down CLSM

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9882cd825 -> d826c8187
  refs/heads/cassandra-3.11 60a2d97a6 -> cbc26d81d
  refs/heads/trunk 2b57fb67d -> e7e936735


Wait for all management tasks to complete before shutting down CLSM

patch by  Jan Urbański; reviewed by jasobrown for CASSANDRA-13123


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d826c818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d826c818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d826c818

Branch: refs/heads/cassandra-3.0
Commit: d826c81874e5d13a30a418f8b982531cb7e5d158
Parents: 9882cd8
Author: Jan Urbański 
Authored: Sat Jan 14 21:26:52 2017 +0100
Committer: Jason Brown 
Committed: Wed Sep 13 06:28:47 2017 -0700

--
 CHANGES.txt |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  8 --
 .../commitlog/CommitLogSegmentManagerTest.java  | 30 
 3 files changed, 37 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d3903e..4d2d5e7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
  * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
--
diff --git 
a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
index 79dd316..7651d1c 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
@@ -108,14 +108,18 @@ public class CommitLogSegmentManager
 {
 public void runMayThrow() throws Exception
 {
-while (run)
+while (true)
 {
 try
 {
 Runnable task = segmentManagementTasks.poll();
 if (task == null)
 {
-// if we have no more work to do, check if we 
should create a new segment
+// if we have no more work to do, check if we were 
requested to exit before starting background tasks
+if (!run)
+return;
+
+// check if we should create a new segment
 if (!atSegmentLimit() && 
availableSegments.isEmpty() && (activeSegments.isEmpty() || 
createReserveSegments))
 {
 logger.trace("No segments in reserve; creating 
a fresh one");

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index a9b0669..41f5ed5 100644
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@ -20,8 +20,10 @@
  */
 package org.apache.cassandra.db.commitlog;
 
+import java.io.File;
 import java.nio.ByteBuffer;
 import java.util.Random;
+import java.util.UUID;
 import java.util.concurrent.Semaphore;
 
 import com.google.common.collect.ImmutableMap;
@@ -134,4 +136,32 @@ public class CommitLogSegmentManagerTest
 Thread.currentThread().interrupt();
 }
 }
+
+@Test
+@BMRule(name = "Make removing commitlog segments slow",
+targetClass = "CommitLogSegment",
+targetMethod = "discard",
+action = "Thread.sleep(50)")
+public void testShutdownWithPendingTasks() throws Throwable {
+CommitLog.instance.resetUnsafe(true);
+ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+
+final Mutation m = new 

[2/6] cassandra git commit: Wait for all management tasks to complete before shutting down CLSM

[jasobrown]

Wait for all management tasks to complete before shutting down CLSM

patch by  Jan Urbański; reviewed by jasobrown for CASSANDRA-13123


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d826c818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d826c818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d826c818

Branch: refs/heads/cassandra-3.11
Commit: d826c81874e5d13a30a418f8b982531cb7e5d158
Parents: 9882cd8
Author: Jan Urbański 
Authored: Sat Jan 14 21:26:52 2017 +0100
Committer: Jason Brown 
Committed: Wed Sep 13 06:28:47 2017 -0700

--
 CHANGES.txt |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  8 --
 .../commitlog/CommitLogSegmentManagerTest.java  | 30 
 3 files changed, 37 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d3903e..4d2d5e7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
  * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
--
diff --git 
a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
index 79dd316..7651d1c 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
@@ -108,14 +108,18 @@ public class CommitLogSegmentManager
 {
 public void runMayThrow() throws Exception
 {
-while (run)
+while (true)
 {
 try
 {
 Runnable task = segmentManagementTasks.poll();
 if (task == null)
 {
-// if we have no more work to do, check if we 
should create a new segment
+// if we have no more work to do, check if we were 
requested to exit before starting background tasks
+if (!run)
+return;
+
+// check if we should create a new segment
 if (!atSegmentLimit() && 
availableSegments.isEmpty() && (activeSegments.isEmpty() || 
createReserveSegments))
 {
 logger.trace("No segments in reserve; creating 
a fresh one");

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index a9b0669..41f5ed5 100644
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@ -20,8 +20,10 @@
  */
 package org.apache.cassandra.db.commitlog;
 
+import java.io.File;
 import java.nio.ByteBuffer;
 import java.util.Random;
+import java.util.UUID;
 import java.util.concurrent.Semaphore;
 
 import com.google.common.collect.ImmutableMap;
@@ -134,4 +136,32 @@ public class CommitLogSegmentManagerTest
 Thread.currentThread().interrupt();
 }
 }
+
+@Test
+@BMRule(name = "Make removing commitlog segments slow",
+targetClass = "CommitLogSegment",
+targetMethod = "discard",
+action = "Thread.sleep(50)")
+public void testShutdownWithPendingTasks() throws Throwable {
+CommitLog.instance.resetUnsafe(true);
+ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+
+final Mutation m = new RowUpdateBuilder(cfs1.metadata, 0, "k")
+   .clustering("bytes")
+   .add("val", ByteBuffer.wrap(entropy))
+   .build();
+

[3/6] cassandra git commit: Wait for all management tasks to complete before shutting down CLSM

[jasobrown]

Wait for all management tasks to complete before shutting down CLSM

patch by  Jan Urbański; reviewed by jasobrown for CASSANDRA-13123


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d826c818
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d826c818
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d826c818

Branch: refs/heads/trunk
Commit: d826c81874e5d13a30a418f8b982531cb7e5d158
Parents: 9882cd8
Author: Jan Urbański 
Authored: Sat Jan 14 21:26:52 2017 +0100
Committer: Jason Brown 
Committed: Wed Sep 13 06:28:47 2017 -0700

--
 CHANGES.txt |  1 +
 .../db/commitlog/CommitLogSegmentManager.java   |  8 --
 .../commitlog/CommitLogSegmentManagerTest.java  | 30 
 3 files changed, 37 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d3903e..4d2d5e7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.15
+ * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
  * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
  * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
  * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
--
diff --git 
a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
index 79dd316..7651d1c 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManager.java
@@ -108,14 +108,18 @@ public class CommitLogSegmentManager
 {
 public void runMayThrow() throws Exception
 {
-while (run)
+while (true)
 {
 try
 {
 Runnable task = segmentManagementTasks.poll();
 if (task == null)
 {
-// if we have no more work to do, check if we 
should create a new segment
+// if we have no more work to do, check if we were 
requested to exit before starting background tasks
+if (!run)
+return;
+
+// check if we should create a new segment
 if (!atSegmentLimit() && 
availableSegments.isEmpty() && (activeSegments.isEmpty() || 
createReserveSegments))
 {
 logger.trace("No segments in reserve; creating 
a fresh one");

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d826c818/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index a9b0669..41f5ed5 100644
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@ -20,8 +20,10 @@
  */
 package org.apache.cassandra.db.commitlog;
 
+import java.io.File;
 import java.nio.ByteBuffer;
 import java.util.Random;
+import java.util.UUID;
 import java.util.concurrent.Semaphore;
 
 import com.google.common.collect.ImmutableMap;
@@ -134,4 +136,32 @@ public class CommitLogSegmentManagerTest
 Thread.currentThread().interrupt();
 }
 }
+
+@Test
+@BMRule(name = "Make removing commitlog segments slow",
+targetClass = "CommitLogSegment",
+targetMethod = "discard",
+action = "Thread.sleep(50)")
+public void testShutdownWithPendingTasks() throws Throwable {
+CommitLog.instance.resetUnsafe(true);
+ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+
+final Mutation m = new RowUpdateBuilder(cfs1.metadata, 0, "k")
+   .clustering("bytes")
+   .add("val", ByteBuffer.wrap(entropy))
+   .build();
+
+

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cbc26d81
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cbc26d81
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cbc26d81

Branch: refs/heads/trunk
Commit: cbc26d81d53822d16eb83052a69426e4f61f77ef
Parents: 60a2d97 d826c81
Author: Jason Brown 
Authored: Wed Sep 13 06:29:42 2017 -0700
Committer: Jason Brown 
Committed: Wed Sep 13 06:30:33 2017 -0700

--
 CHANGES.txt |  1 +
 .../commitlog/CommitLogSegmentManagerTest.java  | 30 
 2 files changed, 31 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cbc26d81/CHANGES.txt
--
diff --cc CHANGES.txt
index ec9d126,4d2d5e7..b3fb5a6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,14 -1,5 +1,15 @@@
 -3.0.15
 +3.11.1
 + * Add a compaction option to TWCS to ignore sstables overlapping checks 
(CASSANDRA-13418)
 + * BTree.Builder memory leak (CASSANDRA-13754)
 + * Revert CASSANDRA-10368 of supporting non-pk column filtering due to 
correctness (CASSANDRA-13798)
 + * Fix cassandra-stress hang issues when an error during cluster connection 
happens (CASSANDRA-12938)
 + * Better bootstrap failure message when blocked by (potential) range 
movement (CASSANDRA-13744)
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
+  * Wait for all management tasks to complete before shutting down CLSM 
(CASSANDRA-13123)
   * INSERT statement fails when Tuple type is used as clustering column with 
default DESC order (CASSANDRA-13717)
   * Fix pending view mutations handling and cleanup batchlog when there are 
local and remote paired mutations (CASSANDRA-13069)
   * Improve config validation and documentation on overflow and NPE 
(CASSANDRA-13622)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cbc26d81/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index e22e86f,41f5ed5..dfbf5ad
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@@ -20,9 -20,10 +20,11 @@@
   */
  package org.apache.cassandra.db.commitlog;
  
+ import java.io.File;
  import java.nio.ByteBuffer;
 +import java.util.ArrayList;
  import java.util.Random;
+ import java.util.UUID;
  import java.util.concurrent.Semaphore;
  
  import com.google.common.collect.ImmutableMap;
@@@ -139,4 -136,32 +141,32 @@@ public class CommitLogSegmentManagerTes
  Thread.currentThread().interrupt();
  }
  }
+ 
+ @Test
+ @BMRule(name = "Make removing commitlog segments slow",
+ targetClass = "CommitLogSegment",
+ targetMethod = "discard",
+ action = "Thread.sleep(50)")
+ public void testShutdownWithPendingTasks() throws Throwable {
+ CommitLog.instance.resetUnsafe(true);
+ ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+ 
+ final Mutation m = new RowUpdateBuilder(cfs1.metadata, 0, "k")
+.clustering("bytes")
+.add("val", ByteBuffer.wrap(entropy))
+.build();
+ 
+ // force creating several commitlog files
+ for (int i = 0; i < 10; i++) {
+ CommitLog.instance.add(m);
+ }
+ 
+ // schedule discarding completed segments and immediately issue a 
shutdown
+ UUID cfid = m.getColumnFamilyIds().iterator().next();
 -CommitLog.instance.discardCompletedSegments(cfid, 
ReplayPosition.NONE, CommitLog.instance.getContext());
++CommitLog.instance.discardCompletedSegments(cfid, 
CommitLogPosition.NONE, CommitLog.instance.getCurrentPosition());
+ CommitLog.instance.shutdownBlocking();
+ 
+ // the shutdown should block until all logs except the currently 
active one and perhaps a new, empty one are gone
 -Assert.assertTrue(new 
File(CommitLog.instance.location).listFiles().length <= 2);
++Assert.assertTrue(new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles().length <= 2);
+ }
  }


-
To 

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[jasobrown]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7e93673
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7e93673
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7e93673

Branch: refs/heads/trunk
Commit: e7e936735df8e0fdd7e7d0df92d7bd60c4602648
Parents: 2b57fb6 cbc26d8
Author: Jason Brown 
Authored: Wed Sep 13 06:30:50 2017 -0700
Committer: Jason Brown 
Committed: Wed Sep 13 06:31:37 2017 -0700

--
 CHANGES.txt |  1 +
 .../commitlog/CommitLogSegmentManagerTest.java  | 31 
 2 files changed, 32 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7e93673/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7e93673/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
--
diff --cc 
test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
index 06513a5,dfbf5ad..5db7149
--- 
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
+++ 
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerTest.java
@@@ -44,6 -46,6 +46,7 @@@ import org.apache.cassandra.db.compacti
  import org.apache.cassandra.db.marshal.AsciiType;
  import org.apache.cassandra.db.marshal.BytesType;
  import org.apache.cassandra.schema.KeyspaceParams;
++import org.apache.cassandra.schema.TableId;
  import org.jboss.byteman.contrib.bmunit.BMRule;
  import org.jboss.byteman.contrib.bmunit.BMRules;
  import org.jboss.byteman.contrib.bmunit.BMUnitRunner;
@@@ -139,4 -141,32 +142,32 @@@ public class CommitLogSegmentManagerTes
  Thread.currentThread().interrupt();
  }
  }
+ 
+ @Test
+ @BMRule(name = "Make removing commitlog segments slow",
+ targetClass = "CommitLogSegment",
+ targetMethod = "discard",
+ action = "Thread.sleep(50)")
+ public void testShutdownWithPendingTasks() throws Throwable {
+ CommitLog.instance.resetUnsafe(true);
+ ColumnFamilyStore cfs1 = 
Keyspace.open(KEYSPACE1).getColumnFamilyStore(STANDARD1);
+ 
 -final Mutation m = new RowUpdateBuilder(cfs1.metadata, 0, "k")
++final Mutation m = new RowUpdateBuilder(cfs1.metadata.get(), 0, "k")
+.clustering("bytes")
+.add("val", ByteBuffer.wrap(entropy))
+.build();
+ 
+ // force creating several commitlog files
+ for (int i = 0; i < 10; i++) {
+ CommitLog.instance.add(m);
+ }
+ 
+ // schedule discarding completed segments and immediately issue a 
shutdown
 -UUID cfid = m.getColumnFamilyIds().iterator().next();
 -CommitLog.instance.discardCompletedSegments(cfid, 
CommitLogPosition.NONE, CommitLog.instance.getCurrentPosition());
++TableId tableId = m.getTableIds().iterator().next();
++CommitLog.instance.discardCompletedSegments(tableId, 
CommitLogPosition.NONE, CommitLog.instance.getCurrentPosition());
+ CommitLog.instance.shutdownBlocking();
+ 
+ // the shutdown should block until all logs except the currently 
active one and perhaps a new, empty one are gone
+ Assert.assertTrue(new 
File(DatabaseDescriptor.getCommitLogLocation()).listFiles().length <= 2);
+ }
  }


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13123) Draining a node might fail to delete all inactive commitlogs

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163579#comment-16163579
 ] 

Jason Brown edited comment on CASSANDRA-13123 at 9/13/17 1:16 PM:
--

Sorry this fell off my review radar (more than) a few months ago. For the last 
month, however, I've been trying to run this patch, rebased on 3.0/3.11/trunk, 
on circleci and the results have almost always been broken (in ways seemingly 
unrelated to this ticket). I've run it locally and everything seemed legit, and 
I've now run the utests on the apache jenkins server, and things were good (a 
few completely unrelated things failed);

||3.0||3.11||trunk||
|[apache 
utest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/9/]|[apache
 
utest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/10/]|[apache
 
utest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/11/]|

Running the 
[dtests|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/304/]
 now (only for 3.0), and if it looks good I'll commit.


was (Author: jasobrown):
Sorry this fell off my review radar (more than) a few months ago. For the last 
month, however, I've been trying to run this patch, rebased on 3.0/3.11/trunk, 
on circleci and the results have almost always been broken (in ways seemingly 
unrelated to this ticket). I've run it locally and everything seemed legit, and 
I've now run the utests on the apache jenkins server, and things were good (a 
few completely unrelated things failed);

||3.0||3.11||trunk||
|[apache 
dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/9/]|[apache
 
dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/10/]|[apache
 
dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/11/]|

Running the 
[dtests|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/304/]
 now (only for 3.0), and if it looks good I'll commit.

> Draining a node might fail to delete all inactive commitlogs
> 
>
> Key: CASSANDRA-13123
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13123
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Jan Urbański
>Assignee: Jan Urbański
> Fix For: 3.8
>
> Attachments: 13123-2.2.8.txt, 13123-3.0.10.txt, 13123-3.9.txt, 
> 13123-trunk.txt
>
>
> After issuing a drain command, it's possible that not all of the inactive 
> commitlogs are removed.
> The drain command shuts down the CommitLog instance, which in turn shuts down 
> the CommitLogSegmentManager. This has the effect of discarding any pending 
> management tasks it might have, like the removal of inactive commitlogs.
> This in turn leads to an excessive amount of commitlogs being left behind 
> after a drain and a lengthy recovery after a restart. With a fleet of dozens 
> of nodes, each of them leaving several GB of commitlogs after a drain and 
> taking up to two minutes to recover them on restart, the additional time 
> required to restart the entire fleet becomes noticeable.
> This problem is not present in 3.x or trunk because of the CLSM rewrite done 
> in CASSANDRA-8844.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13149) AssertionError prepending to a list

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164613#comment-16164613
 ] 

Jason Brown edited comment on CASSANDRA-13149 at 9/13/17 1:14 PM:
--

Patch here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13149-trunk]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/305/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/309/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/308/]|
|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.0]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.11]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-trunk]|

It mostly follows [~slebresne]'s proposed changes as they are pretty reasonable.


was (Author: jasobrown):
Patch here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13149-trunk]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/305/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/306/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/308/]|
|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.0]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.11]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-trunk]|

It mostly follows [~slebresne]'s proposed changes as they are pretty reasonable.

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Jason Brown
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> 

[jira] [Comment Edited] (CASSANDRA-13149) AssertionError prepending to a list

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164613#comment-16164613
 ] 

Jason Brown edited comment on CASSANDRA-13149 at 9/13/17 1:11 PM:
--

Patch here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13149-trunk]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/305/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/306/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/308/]|
|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.0]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.11]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-trunk]|

It mostly follows [~slebresne]'s proposed changes as they are pretty reasonable.


was (Author: jasobrown):
Patch here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13149-trunk]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/305/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/306/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/307/]|
|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.0]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.11]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-trunk]|

It mostly follows [~slebresne]'s proposed changes as they are pretty reasonable.

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Jason Brown
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> 

[jira] [Assigned] (CASSANDRA-13149) AssertionError prepending to a list

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-13149:
---

Assignee: Jason Brown  (was: Andrés de la Peña)

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Jason Brown
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.8.jar:3.0.8]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13149) AssertionError prepending to a list

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164613#comment-16164613
 ] 

Jason Brown commented on CASSANDRA-13149:
-

Patch here:

||3.0||3.11||trunk||
|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13149-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13149-trunk]|
|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/305/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/306/]|[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/307/]|
|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.0]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-3.11]|[utests|https://circleci.com/gh/jasobrown/cassandra/tree/13149-trunk]|

It mostly follows [~slebresne]'s proposed changes as they are pretty reasonable.

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>Assignee: Andrés de la Peña
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.8.jar:3.0.8]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13797) RepairJob blocks on syncTasks

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-13797:

Fix Version/s: (was: 3.0.15)
   (was: 4.0)
   4.x
   3.11.x
   3.0.x

> RepairJob blocks on syncTasks
> -
>
> Key: CASSANDRA-13797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> The thread running {{RepairJob}} blocks while it waits for the validations it 
> starts to complete ([see 
> here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
>  However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't 
> waiting for {{RepairJob#run}} to return, they're waiting for a result to be 
> set on RepairJob the future, which happens after the sync tasks have 
> completed. This post repair cleanup stuff also immediately shuts down the 
> executor {{RepairJob#run}} is running in. So in noop repair sessions, where 
> there's nothing to stream, I'm seeing the callbacks sometimes fire before 
> {{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.
> I'm pretty sure this can just be removed, but I'd like a second opinion. This 
> appears to just be a holdover from before repair coordination became async. I 
> thought it might be doing some throttling by blocking, but each repair 
> session gets it's own executor, and validation is  throttled by the fixed 
> size executors doing the actual work of validation, so I don't think we need 
> to keep this around.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13797) RepairJob blocks on syncTasks

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164601#comment-16164601
 ] 

Jeremiah Jordan commented on CASSANDRA-13797:
-

If adding 3.0 you also need 3.11

> RepairJob blocks on syncTasks
> -
>
> Key: CASSANDRA-13797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> The thread running {{RepairJob}} blocks while it waits for the validations it 
> starts to complete ([see 
> here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
>  However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't 
> waiting for {{RepairJob#run}} to return, they're waiting for a result to be 
> set on RepairJob the future, which happens after the sync tasks have 
> completed. This post repair cleanup stuff also immediately shuts down the 
> executor {{RepairJob#run}} is running in. So in noop repair sessions, where 
> there's nothing to stream, I'm seeing the callbacks sometimes fire before 
> {{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.
> I'm pretty sure this can just be removed, but I'd like a second opinion. This 
> appears to just be a holdover from before repair coordination became async. I 
> thought it might be doing some throttling by blocking, but each repair 
> session gets it's own executor, and validation is  throttled by the fixed 
> size executors doing the actual work of validation, so I don't think we need 
> to keep this around.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13863) Speculative retry causes read repair even if read_repair_chance is 0.0.

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164538#comment-16164538
 ] 

Aleksey Yeschenko commented on CASSANDRA-13863:
---

bq. read_repair_chance = 0.0 and dclocal_read_repair_chance = 0.0 should cause 
no read repair, but read repair happens with speculative retry. I think 
read_repair_chance = 0.0 and dclocal_read_repair_chance = 0.0 should stop read 
repair completely because the user wants to stop read repair in some cases.

{{(dclocal_)?read_repair_chance}} only determines whether or not Cassandra 
should attempt to fetch responses from additional replicas (for the purposes of 
repair). Whenever we have more than one response (because or non-zero read 
repair chances, speculative retry, or just CL > ONE) and some responses 
mismatch, we perform repair, unconditionally. So you could say the settings are 
confusingly named, and that maybe we could use a way to disable repair of 
mismatched responses, but yeah, this is not a bug.

> Speculative retry causes read repair even if read_repair_chance is 0.0.
> ---
>
> Key: CASSANDRA-13863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13863
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Hiro Wakabayashi
>
> {{read_repair_chance = 0.0}} and {{dclocal_read_repair_chance = 0.0}} should 
> cause no read repair, but read repair happens with speculative retry. I think 
> {{read_repair_chance = 0.0}} and {{dclocal_read_repair_chance = 0.0}} should 
> stop read repair completely because the user wants to stop read repair in 
> some cases.
> {panel:title=Case 1: TWCS users}
> The 
> [documentation|http://cassandra.apache.org/doc/latest/operating/compaction.html?highlight=read_repair_chance]
>  states how to disable read repair.
> {quote}While TWCS tries to minimize the impact of comingled data, users 
> should attempt to avoid this behavior. Specifically, users should avoid 
> queries that explicitly set the timestamp via CQL USING TIMESTAMP. 
> Additionally, users should run frequent repairs (which streams data in such a 
> way that it does not become comingled), and disable background read repair by 
> setting the table’s read_repair_chance and dclocal_read_repair_chance to 0.
> {quote}
> {panel}
> {panel:title=Case 2. Strict SLA for read latency}
> In a peak time, read latency is a key for us but, read repair causes latency 
> higher than no read repair. We can use anti entropy repair in off peak time 
> for consistency.
> {panel}
>  
> Here is my procedure to reproduce the problem.
> h3. 1. Create a cluster and set {{hinted_handoff_enabled}} to false.
> {noformat}
> $ ccm create -v 3.0.14 -n 3 cluster_3.0.14
> $ for h in $(seq 1 3) ; do perl -pi -e 's/hinted_handoff_enabled: 
> true/hinted_handoff_enabled: false/' 
> ~/.ccm/cluster_3.0.14/node$h/conf/cassandra.yaml ; done
> $ for h in $(seq 1 3) ; do grep "hinted_handoff_enabled:" 
> ~/.ccm/cluster_3.0.14/node$h/conf/cassandra.yaml ; done
> hinted_handoff_enabled: false
> hinted_handoff_enabled: false
> hinted_handoff_enabled: false
> $ ccm start{noformat}
> h3. 2. Create a keyspace and a table.
> {noformat}
> $ ccm node1 cqlsh
> DROP KEYSPACE IF EXISTS ks1;
> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': '3'}  AND durable_writes = true;
> CREATE TABLE ks1.t1 (
> key text PRIMARY KEY,
> value blob
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = 'ALWAYS';
> QUIT;
> {noformat}
> h3. 3. Stop node2 and node3. Insert a row.
> {noformat}
> $ ccm node3 stop && ccm node2 stop && ccm status
> Cluster: 'cluster_3.0.14'
> --
> node1: UP
> node3: DOWN
> node2: DOWN
> $ ccm node1 cqlsh -k ks1 -e "consistency; tracing on; insert into ks1.t1 
> (key, value) values ('mmullass', bigintAsBlob(1));"
> Current consistency level is ONE.
> Now Tracing is enabled
> Tracing session: 01d74590-97cb-11e7-8ea7-c1bd4d549501
>  activity 
>| timestamp  | source 

[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164502#comment-16164502
 ] 

Aleksey Yeschenko commented on CASSANDRA-13595:
---

bq. The idea is to extend current ShortReadProtection function with 
across-partition support.

Correct. Short read protection hasn't been implemented properly for range 
reads, which causes correctness issues in particular with paging.

I'm currently addressing the few outstanding issues with single partition short 
reads on 3.0 and above (CASSANDRA-13794, CASSANDRA-12872). This would be an 
extension of that work, I guess - or at least there is strong overlap. Feel 
free to give it a go though - however it may or may not have to be altered 
afterwards to harmonise both implementations.

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13819) Surprising under-documented behavior with DELETE...USING TIMESTAMP

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164463#comment-16164463
 ] 

Kurt Greaves commented on CASSANDRA-13819:
--

This is certainly the intended behaviour, but you're right and if it's not 
explicitly documented already it should be. I've seen people be misled by this 
before as well.
Would probably be worthwhile noting the behaviour and also linking to a more in 
depth run through of timestamps/USING TIMESTAMP in general, probably with some 
examples.
Is this something you would be willing to contribute?

> Surprising under-documented behavior with DELETE...USING TIMESTAMP
> --
>
> Key: CASSANDRA-13819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13819
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Eric Wolak
>Priority: Minor
>
> While investigating differences between various Bigtable derivatives, I‘ve 
> run into an odd behavior of Cassandra. I’m guessing this is intended 
> behavior, but it's surprising enough to me that I think it should be 
> explicitly documented.
> Let‘s say I have a sensor device reporting data with timestamps. It has a 
> great clock, so I use its timestamps in a USING TIMESTAMP clause in my INSERT 
> statements. One day Jeff realizes that we had a hardware bug with the sensor, 
> and data before timestamp T is incorrect. He issues a DELETE...USING 
> TIMESTAMP T to remove the old data. In the meantime, Sam figures out a way to 
> backfill the data, and she writes a job to insert corrected data into the 
> same table. In keeping with the schema, her job issues INSERT...USING 
> TIMESTAMP statememts, with timestamps before T (because that’s the time the 
> data points correspond to). When testing her job, Sam discovers that the 
> backfilled data isn‘t appearing in the database! In fact, there’s no way for 
> her to insert data with a TIMESTAMP <= T, because the tombstone written by 
> Jeff several days ago is masking them. How can Sam backfill the corrected 
> data?
> This behavior seems to match the HBase “Current Limitation” that Deletes Mask 
> Puts, documented at http://hbase.apache.org/book.html#_deletes_mask_puts. 
> Should the Cassandra docs also explicitly call-out this behavior?
> Related:
> http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html
> https://docs.datastax.com/en/cassandra/3.0/cassandra/dml/dmlAboutDeletes.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13595) Short read protection doesn't work at the end of a partition

[ZhaoYang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164449#comment-16164449
 ] 

ZhaoYang commented on CASSANDRA-13595:
--

It seems that current ShortReadProtection only supports the same partition.

The idea is to extend current ShortReadProtection function with 
across-partition support. If {{current response (UnfilteredPartitionIterator)}} 
reached the end and its last key is "behind" (token smaller than) other 
responses' current key, we will do a {{PartitionRange retry}} from the last key 
to make sure no response is falling short of unfetched partitions. 

> Short read protection doesn't work at the end of a partition
> 
>
> Key: CASSANDRA-13595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13595
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Andrés de la Peña
>Assignee: ZhaoYang
>  Labels: Correctness
>
> It seems that short read protection doesn't work when the short read is done 
> at the end of a partition in a range query. The final assertion of this dtest 
> fails:
> {code}
> def short_read_partitions_delete_test(self):
> cluster = self.cluster
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> cluster.populate(2).start(wait_other_notice=True)
> node1, node2 = self.cluster.nodelist()
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 2)
> session.execute("CREATE TABLE t (k int, c int, PRIMARY KEY(k, c)) 
> WITH read_repair_chance = 0.0")
> # we write 1 and 2 in a partition: all nodes get it.
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (1, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> session.execute(SimpleStatement("INSERT INTO t (k, c) VALUES (2, 1)", 
> consistency_level=ConsistencyLevel.ALL))
> # we delete partition 1: only node 1 gets it.
> node2.flush()
> node2.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 1"))
> node2.start(wait_other_notice=True)
> # we delete partition 2: only node 2 gets it.
> node1.flush()
> node1.stop(wait_other_notice=True)
> session = self.patient_cql_connection(node2, 'ks', 
> consistency_level=ConsistencyLevel.ONE)
> session.execute(SimpleStatement("DELETE FROM t WHERE k = 2"))
> node1.start(wait_other_notice=True)
> # read from both nodes
> session = self.patient_cql_connection(node1, 'ks', 
> consistency_level=ConsistencyLevel.ALL)
> assert_none(session, "SELECT * FROM t LIMIT 1")
> {code}
> However, the dtest passes if we remove the {{LIMIT 1}}.
> Short read protection [uses a 
> {{SinglePartitionReadCommand}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DataResolver.java#L484],
>  maybe it should use a {{PartitionRangeReadCommand}} instead?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12014:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> IndexSummary > 2G causes an assertion error
> ---
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444  
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should 
> have a better method of coping with this than throwing the AE.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164338#comment-16164338
 ] 

Stefania commented on CASSANDRA-12014:
--

Thanks. Committed fix relying on env. variable to 3.0 as 
9882cd825ff95c88264c78dfcdc481aad6fc3522 and merged upwards.

> IndexSummary > 2G causes an assertion error
> ---
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444  
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should 
> have a better method of coping with this than throwing the AE.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[stefania]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2b57fb67
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2b57fb67
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2b57fb67

Branch: refs/heads/trunk
Commit: 2b57fb67dbf74b9fa87ce2ce5ac407f065ac1c3f
Parents: 3cec208 60a2d97
Author: Stefania Alborghetti 
Authored: Wed Sep 13 16:48:42 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:48:42 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b57fb67/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Ignore large IndexSummaryTest tests on Circle CI

[stefania]

Ignore large IndexSummaryTest tests on Circle CI

patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-12014 (follow 
up)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9882cd82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9882cd82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9882cd82

Branch: refs/heads/trunk
Commit: 9882cd825ff95c88264c78dfcdc481aad6fc3522
Parents: a08a816
Author: Stefania Alborghetti 
Authored: Wed Sep 13 10:23:48 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:44:26 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9882cd82/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java 
b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
index ad08ba0..6f37d8f 100644
--- a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
@@ -26,6 +26,7 @@ import java.util.*;
 import com.google.common.collect.Lists;
 import org.junit.BeforeClass;
 import org.junit.Test;
+import org.junit.Assume;
 
 import org.apache.cassandra.Util;
 import org.apache.cassandra.db.*;
@@ -58,6 +59,9 @@ public class IndexSummaryTest
 @Test
 public void testIndexSummaryKeySizes() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 testIndexSummaryProperties(32, 100);
 testIndexSummaryProperties(64, 100);
 testIndexSummaryProperties(100, 100);
@@ -100,8 +104,11 @@ public class IndexSummaryTest
  * create an index summary, albeit one that does not cover the entire 
sstable.
  */
 @Test
-public void tesLargeIndexSummary() throws IOException
+public void testLargeIndexSummary() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;
@@ -131,8 +138,11 @@ public class IndexSummaryTest
  * the index summary should be downsampled automatically.
  */
 @Test
-public void tesLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
+public void testLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Ignore large IndexSummaryTest tests on Circle CI

[stefania]

Ignore large IndexSummaryTest tests on Circle CI

patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-12014 (follow 
up)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9882cd82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9882cd82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9882cd82

Branch: refs/heads/cassandra-3.11
Commit: 9882cd825ff95c88264c78dfcdc481aad6fc3522
Parents: a08a816
Author: Stefania Alborghetti 
Authored: Wed Sep 13 10:23:48 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:44:26 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9882cd82/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java 
b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
index ad08ba0..6f37d8f 100644
--- a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
@@ -26,6 +26,7 @@ import java.util.*;
 import com.google.common.collect.Lists;
 import org.junit.BeforeClass;
 import org.junit.Test;
+import org.junit.Assume;
 
 import org.apache.cassandra.Util;
 import org.apache.cassandra.db.*;
@@ -58,6 +59,9 @@ public class IndexSummaryTest
 @Test
 public void testIndexSummaryKeySizes() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 testIndexSummaryProperties(32, 100);
 testIndexSummaryProperties(64, 100);
 testIndexSummaryProperties(100, 100);
@@ -100,8 +104,11 @@ public class IndexSummaryTest
  * create an index summary, albeit one that does not cover the entire 
sstable.
  */
 @Test
-public void tesLargeIndexSummary() throws IOException
+public void testLargeIndexSummary() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;
@@ -131,8 +138,11 @@ public class IndexSummaryTest
  * the index summary should be downsampled automatically.
  */
 @Test
-public void tesLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
+public void testLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60a2d97a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60a2d97a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60a2d97a

Branch: refs/heads/cassandra-3.11
Commit: 60a2d97a63c151c3f76791775d6ab09d97c74b54
Parents: cb2a1c8 9882cd8
Author: Stefania Alborghetti 
Authored: Wed Sep 13 16:46:49 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:46:49 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60a2d97a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--
diff --cc test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
index ab0486d,6f37d8f..d83b96c
--- a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
@@@ -26,9 -26,9 +26,10 @@@ import java.util.*
  import com.google.common.collect.Lists;
  import org.junit.BeforeClass;
  import org.junit.Test;
+ import org.junit.Assume;
  
  import org.apache.cassandra.Util;
 +import org.apache.cassandra.config.DatabaseDescriptor;
  import org.apache.cassandra.db.*;
  import org.apache.cassandra.dht.IPartitioner;
  import org.apache.cassandra.dht.RandomPartitioner;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[stefania]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60a2d97a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60a2d97a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60a2d97a

Branch: refs/heads/trunk
Commit: 60a2d97a63c151c3f76791775d6ab09d97c74b54
Parents: cb2a1c8 9882cd8
Author: Stefania Alborghetti 
Authored: Wed Sep 13 16:46:49 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:46:49 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60a2d97a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--
diff --cc test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
index ab0486d,6f37d8f..d83b96c
--- a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
@@@ -26,9 -26,9 +26,10 @@@ import java.util.*
  import com.google.common.collect.Lists;
  import org.junit.BeforeClass;
  import org.junit.Test;
+ import org.junit.Assume;
  
  import org.apache.cassandra.Util;
 +import org.apache.cassandra.config.DatabaseDescriptor;
  import org.apache.cassandra.db.*;
  import org.apache.cassandra.dht.IPartitioner;
  import org.apache.cassandra.dht.RandomPartitioner;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Ignore large IndexSummaryTest tests on Circle CI

[stefania]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 a08a816a6 -> 9882cd825
  refs/heads/cassandra-3.11 cb2a1c8f4 -> 60a2d97a6
  refs/heads/trunk 3cec208c4 -> 2b57fb67d


Ignore large IndexSummaryTest tests on Circle CI

patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-12014 (follow 
up)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9882cd82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9882cd82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9882cd82

Branch: refs/heads/cassandra-3.0
Commit: 9882cd825ff95c88264c78dfcdc481aad6fc3522
Parents: a08a816
Author: Stefania Alborghetti 
Authored: Wed Sep 13 10:23:48 2017 +0800
Committer: Stefania Alborghetti 
Committed: Wed Sep 13 16:44:26 2017 +0800

--
 .../apache/cassandra/io/sstable/IndexSummaryTest.java | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9882cd82/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java 
b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
index ad08ba0..6f37d8f 100644
--- a/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/IndexSummaryTest.java
@@ -26,6 +26,7 @@ import java.util.*;
 import com.google.common.collect.Lists;
 import org.junit.BeforeClass;
 import org.junit.Test;
+import org.junit.Assume;
 
 import org.apache.cassandra.Util;
 import org.apache.cassandra.db.*;
@@ -58,6 +59,9 @@ public class IndexSummaryTest
 @Test
 public void testIndexSummaryKeySizes() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 testIndexSummaryProperties(32, 100);
 testIndexSummaryProperties(64, 100);
 testIndexSummaryProperties(100, 100);
@@ -100,8 +104,11 @@ public class IndexSummaryTest
  * create an index summary, albeit one that does not cover the entire 
sstable.
  */
 @Test
-public void tesLargeIndexSummary() throws IOException
+public void testLargeIndexSummary() throws IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;
@@ -131,8 +138,11 @@ public class IndexSummaryTest
  * the index summary should be downsampled automatically.
  */
 @Test
-public void tesLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
+public void testLargeIndexSummaryWithExpectedSizeMatching() throws 
IOException
 {
+// On Circle CI we normally don't have enough off-heap memory for this 
test so ignore it
+Assume.assumeTrue(System.getenv("CIRCLECI") == null);
+
 final int numKeys = 100;
 final int keySize = 3000;
 final int minIndexInterval = 1;


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13864) Failure to execute cql script using cqlsh with nested SOURCE on cassandra 3.11.0

[Haim Raman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haim Raman updated CASSANDRA-13864:
---
Description: 
I have a script I used to execute queries and DDL on cassandra 2.1.15 (based on 
DSE 4.8.10).

The script include nested use of the SOURCE command.

{code:title=1.sql|borderStyle=solid}
USE test;
SOURCE '2.sql'
exit;
{code}

{code:title=2.sql|borderStyle=solid}
SELECT count(1) FROM user;
SOURCE '3.sql';
{code}

{code:title=3.sql|borderStyle=solid}
SELECT count(1) FROM user;
{code}

When executing this script with DSE 4.8.10 it runs correctly and output

{code:title=cassandra  2.1.15|borderStyle=none}
cqlsh –f  1.sql
count

 0

(1 rows)

count

 0

(1 rows)
{code}
Running the same script with cassandra 3.11.0 (based on DSE 5.1.2).

{code:title=cassandra  3.11.0|borderStyle=none}
cqlsh –f  1.sql
 count
---
 0

(1 rows)

Warnings :
Aggregation query used without partition key

2.sql:3:DSEShell instance has no attribute 'execution_profiles'
{code}

*The actual issue is that the script in 3.sql is not executed.*

I did some additional investigations
# With DSE-5.1.2 I switch the off to authenticator: AllowAllAuthenticator 
authorizer: AllowAllAuthorizer, but I am still experiencing the issue
# With DSE-5.0.9 (Cassandra 3.0.13.1735) it works

h4.Steps to reproduce: 
Use the attached test.zip
execute
cqlsh -f  create_keyspace.sql
cqlsh -f 1.sql

  was:
I have a script I used to execute queries and DDL on cassandra 2.1.15 (based on 
DSE 4.8.10).

The script include nested use of the SOURCE command. E.g.
1.sql

USE test;
SOURCE '2.sql'
exit;
2.sql

SELECT count(1) FROM user;
SOURCE '3.sql';
3.sql

SELECT count(1) FROM user;
When executing this script with DSE 4.8.10 it runs correctly and output

cqlsh –f  1.sql
count

 0

(1 rows)

count

 0

(1 rows)
Running the same script with cassandra 3.11.0 (based on DSE 5.1.2).

cqlsh –f  1.sql
 count
---
 0

(1 rows)

Warnings :
Aggregation query used without partition key

2.sql:3:DSEShell instance has no attribute 'execution_profiles'
The actual issue is that the script in 3.sql is not executed.
I failed to find any us full information on the error "DSEShell instance has no 
attribute 'execution_profiles'"

I failed to figure out what are execution_profiles although they are mentioned 
int the python docs here

I did some additional investigations
# With DSE-5.1.2 I switch the off to authenticator: AllowAllAuthenticator 
authorizer: AllowAllAuthorizer, but I am still experiencing the issue
# With DSE-5.0.9 (Cassandra 3.0.13.1735) it works

Steps to reproduce: 
Use the attached test.zip
execute
cqlsh -f  create_keyspace.sql
cqlsh -f 1.sql


> Failure to execute cql script  using cqlsh with nested SOURCE on cassandra 
> 3.11.0
> -
>
> Key: CASSANDRA-13864
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13864
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: SUSE Linux Enterprise Server 12
> Python 2.7.7
>Reporter: Haim Raman
> Attachments: test.zip
>
>
> I have a script I used to execute queries and DDL on cassandra 2.1.15 (based 
> on DSE 4.8.10).
> The script include nested use of the SOURCE command.
> {code:title=1.sql|borderStyle=solid}
> USE test;
> SOURCE '2.sql'
> exit;
> {code}
> {code:title=2.sql|borderStyle=solid}
> SELECT count(1) FROM user;
> SOURCE '3.sql';
> {code}
> {code:title=3.sql|borderStyle=solid}
> SELECT count(1) FROM user;
> {code}
> When executing this script with DSE 4.8.10 it runs correctly and output
> {code:title=cassandra  2.1.15|borderStyle=none}
> cqlsh –f  1.sql
> count
> 
>  0
> (1 rows)
> count
> 
>  0
> (1 rows)
> {code}
> Running the same script with cassandra 3.11.0 (based on DSE 5.1.2).
> {code:title=cassandra  3.11.0|borderStyle=none}
> cqlsh –f  1.sql
>  count
> ---
>  0
> (1 rows)
> Warnings :
> Aggregation query used without partition key
> 2.sql:3:DSEShell instance has no attribute 'execution_profiles'
> {code}
> *The actual issue is that the script in 3.sql is not executed.*
> I did some additional investigations
> # With DSE-5.1.2 I switch the off to authenticator: AllowAllAuthenticator 
> authorizer: AllowAllAuthorizer, but I am still experiencing the issue
> # With DSE-5.0.9 (Cassandra 3.0.13.1735) it works
> h4.Steps to reproduce: 
> Use the attached test.zip
> execute
> cqlsh -f  create_keyspace.sql
> cqlsh -f 1.sql



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13864) Failure to execute cql script using cqlsh with nested SOURCE on cassandra 3.11.0

[Haim Raman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164230#comment-16164230
 ] 

Haim Raman commented on CASSANDRA-13864:


See my question on stackoverflow 
https://stackoverflow.com/questions/46174238/failure-to-execute-cql-script-from-using-cqlsh-with-nested-soure-on-cassandra-3

> Failure to execute cql script  using cqlsh with nested SOURCE on cassandra 
> 3.11.0
> -
>
> Key: CASSANDRA-13864
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13864
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: SUSE Linux Enterprise Server 12
> Python 2.7.7
>Reporter: Haim Raman
> Attachments: test.zip
>
>
> I have a script I used to execute queries and DDL on cassandra 2.1.15 (based 
> on DSE 4.8.10).
> The script include nested use of the SOURCE command. E.g.
> 1.sql
> USE test;
> SOURCE '2.sql'
> exit;
> 2.sql
> SELECT count(1) FROM user;
> SOURCE '3.sql';
> 3.sql
> SELECT count(1) FROM user;
> When executing this script with DSE 4.8.10 it runs correctly and output
> cqlsh –f  1.sql
> count
> 
>  0
> (1 rows)
> count
> 
>  0
> (1 rows)
> Running the same script with cassandra 3.11.0 (based on DSE 5.1.2).
> cqlsh –f  1.sql
>  count
> ---
>  0
> (1 rows)
> Warnings :
> Aggregation query used without partition key
> 2.sql:3:DSEShell instance has no attribute 'execution_profiles'
> The actual issue is that the script in 3.sql is not executed.
> I failed to find any us full information on the error "DSEShell instance has 
> no attribute 'execution_profiles'"
> I failed to figure out what are execution_profiles although they are 
> mentioned int the python docs here
> I did some additional investigations
> # With DSE-5.1.2 I switch the off to authenticator: AllowAllAuthenticator 
> authorizer: AllowAllAuthorizer, but I am still experiencing the issue
> # With DSE-5.0.9 (Cassandra 3.0.13.1735) it works
> Steps to reproduce: 
> Use the attached test.zip
> execute
> cqlsh -f  create_keyspace.sql
> cqlsh -f 1.sql



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13864) Failure to execute cql script using cqlsh with nested SOURCE on cassandra 3.11.0

[Haim Raman (JIRA)]

Haim Raman created CASSANDRA-13864:
--

 Summary: Failure to execute cql script  using cqlsh with nested 
SOURCE on cassandra 3.11.0
 Key: CASSANDRA-13864
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13864
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: SUSE Linux Enterprise Server 12
Python 2.7.7
Reporter: Haim Raman
 Attachments: test.zip

I have a script I used to execute queries and DDL on cassandra 2.1.15 (based on 
DSE 4.8.10).

The script include nested use of the SOURCE command. E.g.
1.sql

USE test;
SOURCE '2.sql'
exit;
2.sql

SELECT count(1) FROM user;
SOURCE '3.sql';
3.sql

SELECT count(1) FROM user;
When executing this script with DSE 4.8.10 it runs correctly and output

cqlsh –f  1.sql
count

 0

(1 rows)

count

 0

(1 rows)
Running the same script with cassandra 3.11.0 (based on DSE 5.1.2).

cqlsh –f  1.sql
 count
---
 0

(1 rows)

Warnings :
Aggregation query used without partition key

2.sql:3:DSEShell instance has no attribute 'execution_profiles'
The actual issue is that the script in 3.sql is not executed.
I failed to find any us full information on the error "DSEShell instance has no 
attribute 'execution_profiles'"

I failed to figure out what are execution_profiles although they are mentioned 
int the python docs here

I did some additional investigations
# With DSE-5.1.2 I switch the off to authenticator: AllowAllAuthenticator 
authorizer: AllowAllAuthorizer, but I am still experiencing the issue
# With DSE-5.0.9 (Cassandra 3.0.13.1735) it works

Steps to reproduce: 
Use the attached test.zip
execute
cqlsh -f  create_keyspace.sql
cqlsh -f 1.sql



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164225#comment-16164225
 ] 

Marcus Eriksson commented on CASSANDRA-12014:
-

Sure, lets rely on the env variable for now, if we get bigger containers in the 
future, we should remember to remove the check

> IndexSummary > 2G causes an assertion error
> ---
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> {noformat}
> ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444  
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:1546280,1,main]
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>  ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {noformat}
> I believe this can be fixed by raising the min_index_interval, but we should 
> have a better method of coping with this than throwing the AE.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org