date:20180319

[jira] [Commented] (CASSANDRA-14260) Refactor pair to avoid boxing longs/ints

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405824#comment-16405824
 ] 

Dinesh Joshi commented on CASSANDRA-14260:
--

+1

The changes look good. You did not swap left & right AFAICT.

> Refactor pair to avoid boxing longs/ints
> 
>
> Key: CASSANDRA-14260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14260
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
>
> We uses Pair all over the place, and in many cases either/both of X and 
> Y are primitives (ints, longs), and we end up boxing them into Integers and 
> Longs. We should have specialized versions that take primitives. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14260) Refactor pair to avoid boxing longs/ints

2018-03-19 Thread Dinesh Joshi (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-14260:
-
Status: Ready to Commit  (was: Patch Available)

> Refactor pair to avoid boxing longs/ints
> 
>
> Key: CASSANDRA-14260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14260
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
>
> We uses Pair all over the place, and in many cases either/both of X and 
> Y are primitives (ints, longs), and we end up boxing them into Integers and 
> Longs. We should have specialized versions that take primitives. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to

2018-03-19 Thread Alex Lourie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405779#comment-16405779
 ] 

Alex Lourie edited comment on CASSANDRA-13010 at 3/20/18 4:12 AM:
--

[~rustyrazorblade] Thinking about your suggestion, and with some discussion 
with others, I think that adding an additional option flag to *compactionstats* 
would be more beneficial. I really like that I can see _all_ the information 
about all running compactions at the same time, including the directories. 
Hence, we could keep the current presentation intact and just add a flag to 
show any additional info. Also, if we wanted to, we could then expand 
*compactionstats* to accept an ID and only show info for that specific 
compaction.

What do you think?


was (Author: alourie):
[~rustyrazorblade] Thinking about your suggestion, and with some discussion 
with others, I think that adding an additional option flag to compactionstats
would be more beneficial. I really like that I can see _all_ the information 
about all running compactions at the same time, including the directories. 
Hence, we could keep the current presentation intact and just add a flag to 
show any additional info. Also, if we wanted to, we could then expand 
compactionstats to accept an ID and only show info for that specific compaction.

What do you think?

> nodetool compactionstats should say which disk a compaction is writing to
> -
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Tools
>Reporter: Jon Haddad
>Assignee: Alex Lourie
>Priority: Major
>  Labels: lhf
> Attachments: 13010.patch, cleanup.png, multiple operations.png
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to

2018-03-19 Thread Alex Lourie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405779#comment-16405779
 ] 

Alex Lourie edited comment on CASSANDRA-13010 at 3/20/18 4:10 AM:
--

[~rustyrazorblade] Thinking about your suggestion, and with some discussion 
with others, I think that adding an additional option flag to compactionstats
would be more beneficial. I really like that I can see _all_ the information 
about all running compactions at the same time, including the directories. 
Hence, we could keep the current presentation intact and just add a flag to 
show any additional info. Also, if we wanted to, we could then expand 
compactionstats to accept an ID and only show info for that specific compaction.

What do you think?


was (Author: alourie):
[~rustyrazorblade] Thinking about your suggestion, and with some discussion 
with others, I think that adding an additional option flag to `compactionstats` 
would be more beneficial. I really like that I can see _all_ the information 
about all running compactions at the same time, including the directories. 
Hence, we could keep the current presentation intact and just add a flag to 
show any additional info. Also, if we wanted to, we could then expand 
`compactionstats` to accept an ID and only show info for that specific 
compaction.

What do you think?

> nodetool compactionstats should say which disk a compaction is writing to
> -
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Tools
>Reporter: Jon Haddad
>Assignee: Alex Lourie
>Priority: Major
>  Labels: lhf
> Attachments: 13010.patch, cleanup.png, multiple operations.png
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to

2018-03-19 Thread Alex Lourie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405779#comment-16405779
 ] 

Alex Lourie commented on CASSANDRA-13010:
-

[~rustyrazorblade] Thinking about your suggestion, and with some discussion 
with others, I think that adding an additional option flag to `compactionstats` 
would be more beneficial. I really like that I can see _all_ the information 
about all running compactions at the same time, including the directories. 
Hence, we could keep the current presentation intact and just add a flag to 
show any additional info. Also, if we wanted to, we could then expand 
`compactionstats` to accept an ID and only show info for that specific 
compaction.

What do you think?

> nodetool compactionstats should say which disk a compaction is writing to
> -
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Tools
>Reporter: Jon Haddad
>Assignee: Alex Lourie
>Priority: Major
>  Labels: lhf
> Attachments: 13010.patch, cleanup.png, multiple operations.png
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14104) Index target doesn't correctly recognise non-UTF column names after COMPACT STORAGE drop

2018-03-19 Thread Kurt Greaves (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14104:
-
Fix Version/s: 3.0.16
   3.11.2
   4.0

> Index target doesn't correctly recognise non-UTF column names after COMPACT 
> STORAGE drop
> 
>
> Key: CASSANDRA-14104
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14104
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Major
> Fix For: 3.0.16, 3.11.2, 4.0
>
> Attachments: ifesdjeen-14104-3.0-dtest.png, 
> ifesdjeen-14104-3.0-testall.png, ifesdjeen-14104-3.11-dtest.png, 
> ifesdjeen-14104-3.11-testall.png, ifesdjeen-14104-trunk-dtest.png, 
> ifesdjeen-14104-trunk-testall.png
>
>
> Creating a compact storage table with dynamic composite type, then running 
> {{ALTER TALBE ... DROP COMPACT STORAGE}} and then restarting the node will 
> crash Cassandra node, since the Index Target is fetched using hashmap / 
> strict equality. We need to fall back to linear search when index target 
> can't be found (which should not be happening often).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14315) ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only partition level info

2018-03-19 Thread ZhaoYang (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405732#comment-16405732
 ] 

ZhaoYang commented on CASSANDRA-14315:
--

Thanks for the review!

> ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only 
> partition level info
> --
>
> Key: CASSANDRA-14315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14315
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Major
> Fix For: 4.0
>
> Attachments: dtest.png, unit test.png
>
>
> When repairing base table with MV, in order to avoid OOM, Cassandra-13299 
> added ThrottledUnfilteredIterator to split large partition into small chunks, 
> but it didn't handle partition without unfiltered properly.
> {code:title=repro}
> // create cell tombstone, range tombstone, partition deletion
> createTable("CREATE TABLE %s (pk int, ck1 int, ck2 int, v1 int, v2 int, 
> PRIMARY KEY (pk, ck1, ck2))");
> // partition deletion
> execute("DELETE FROM %s USING TIMESTAMP 160 WHERE pk=1");
> // flush and generate 1 sstable
> ColumnFamilyStore cfs = 
> Keyspace.open(keyspace()).getColumnFamilyStore(currentTable());
> cfs.forceBlockingFlush();
> cfs.disableAutoCompaction();
> cfs.forceMajorCompaction();
> assertEquals(1, cfs.getLiveSSTables().size());
> SSTableReader reader = cfs.getLiveSSTables().iterator().next();
> try (ISSTableScanner scanner = reader.getScanner();
> CloseableIterator throttled = 
> ThrottledUnfilteredIterator.throttle(scanner, 100))
> {
> assertTrue(throttled.hasNext());
> UnfilteredRowIterator iterator = throttled.next();
> assertFalse(throttled.hasNext());
> assertFalse(iterator.hasNext());
> assertEquals(iterator.partitionLevelDeletion().markedForDeleteAt(), 160);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14315) ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only partition level info

2018-03-19 Thread Paulo Motta (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405722#comment-16405722
 ] 

Paulo Motta commented on CASSANDRA-14315:
-

Committed dtest as {{2c1b986bc82ad29a4db06158043aceaaf473e17c}}.

> ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only 
> partition level info
> --
>
> Key: CASSANDRA-14315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14315
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Major
> Fix For: 4.0
>
> Attachments: dtest.png, unit test.png
>
>
> When repairing base table with MV, in order to avoid OOM, Cassandra-13299 
> added ThrottledUnfilteredIterator to split large partition into small chunks, 
> but it didn't handle partition without unfiltered properly.
> {code:title=repro}
> // create cell tombstone, range tombstone, partition deletion
> createTable("CREATE TABLE %s (pk int, ck1 int, ck2 int, v1 int, v2 int, 
> PRIMARY KEY (pk, ck1, ck2))");
> // partition deletion
> execute("DELETE FROM %s USING TIMESTAMP 160 WHERE pk=1");
> // flush and generate 1 sstable
> ColumnFamilyStore cfs = 
> Keyspace.open(keyspace()).getColumnFamilyStore(currentTable());
> cfs.forceBlockingFlush();
> cfs.disableAutoCompaction();
> cfs.forceMajorCompaction();
> assertEquals(1, cfs.getLiveSSTables().size());
> SSTableReader reader = cfs.getLiveSSTables().iterator().next();
> try (ISSTableScanner scanner = reader.getScanner();
> CloseableIterator throttled = 
> ThrottledUnfilteredIterator.throttle(scanner, 100))
> {
> assertTrue(throttled.hasNext());
> UnfilteredRowIterator iterator = throttled.next();
> assertFalse(throttled.hasNext());
> assertFalse(iterator.hasNext());
> assertEquals(iterator.partitionLevelDeletion().markedForDeleteAt(), 160);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: Add test for CASSANDRA-14315

2018-03-19 Thread paulo

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 1888c4048 -> 2c1b986bc


Add test for CASSANDRA-14315

Patch by Zhao Yang; Reviewed by Paulo motta for CASSANDRA-14315


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/2c1b986b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/2c1b986b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/2c1b986b

Branch: refs/heads/master
Commit: 2c1b986bc82ad29a4db06158043aceaaf473e17c
Parents: 1888c40
Author: Zhao Yang 
Authored: Thu Mar 15 15:47:02 2018 +0800
Committer: Paulo Motta 
Committed: Mon Mar 19 23:40:58 2018 -0300

--
 materialized_views_test.py | 8 
 1 file changed, 8 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/2c1b986b/materialized_views_test.py
--
diff --git a/materialized_views_test.py b/materialized_views_test.py
index a723c4f..7771f9d 100644
--- a/materialized_views_test.py
+++ b/materialized_views_test.py
@@ -2084,6 +2084,8 @@ class TestMaterializedViews(Tester):
 
 # partition deletion for ck1 <= partition_deletion_ts
 session.execute("DELETE FROM ks.t USING TIMESTAMP {} WHERE 
pk=1".format(partition_deletion_ts))
+# only partition deletion for the pk=2000
+session.execute("DELETE FROM ks.t USING TIMESTAMP {} WHERE 
pk=2000".format(partition_deletion_ts))
 self._replay_batchlogs()
 
 # start nodes with different batch size
@@ -2096,6 +2098,9 @@ class TestMaterializedViews(Tester):
 
 logger.debug('repairing base table')
 node1.nodetool("repair ks t")
+# insert data to the deleted partition with pk=2000, they should be 
considered dead
+session.execute("INSERT INTO ks.t (pk, ck1, ck2, v1, v2)"
+" VALUES (2000, 0, 0, 0, 0) USING TIMESTAMP 
{}".format(partition_deletion_ts - 1))
 self._replay_batchlogs()
 
 logger.debug('stop cluster')
@@ -2127,6 +2132,9 @@ class TestMaterializedViews(Tester):
 "ck1={} AND ck2={}".format(ck1, 
ck2), [1, ck1, ck2, ck1, ck2])
 assert_one(session, "SELECT pk,ck1,ck2,v1,v2 FROM ks.t 
WHERE pk=1 AND "
 "ck1={} AND ck2={}".format(ck1, 
ck2), [1, ck1, ck2, ck1, ck2])
+# Verify partition deletion with pk=2000 has no live data
+assert_none(session, "SELECT pk,ck1,ck2,v1,v2 FROM ks.t WHERE 
pk=2000")
+assert_none(session, "SELECT pk,ck1,ck2,v1,v2 FROM ks.t_by_v WHERE 
pk=2000")
 logger.debug('stopping {}'.format(node.name))
 node.stop(wait_other_notice=True, wait_for_binary_proto=True)
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405711#comment-16405711
 ] 

Sumanth Pasupuleti commented on CASSANDRA-14232:


Thanks for your inputs [~chovatia.jayd...@gmail.com] . Fixed and updated the 
patch.

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-14232:
---
Attachment: (was: 14232-trunk.txt)

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-14232:
---
Attachment: 14232-trunk.txt

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14315) ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only partition level info

2018-03-19 Thread Paulo Motta (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta resolved CASSANDRA-14315.
-
Resolution: Fixed

Good catch! Patch and tests LGTM. Committed as 
{{5b9e985474e696a83d23e7cf4bedaf360cdb1eaf}} to trunk. Thanks!

> ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only 
> partition level info
> --
>
> Key: CASSANDRA-14315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14315
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Major
> Fix For: 4.0
>
> Attachments: dtest.png, unit test.png
>
>
> When repairing base table with MV, in order to avoid OOM, Cassandra-13299 
> added ThrottledUnfilteredIterator to split large partition into small chunks, 
> but it didn't handle partition without unfiltered properly.
> {code:title=repro}
> // create cell tombstone, range tombstone, partition deletion
> createTable("CREATE TABLE %s (pk int, ck1 int, ck2 int, v1 int, v2 int, 
> PRIMARY KEY (pk, ck1, ck2))");
> // partition deletion
> execute("DELETE FROM %s USING TIMESTAMP 160 WHERE pk=1");
> // flush and generate 1 sstable
> ColumnFamilyStore cfs = 
> Keyspace.open(keyspace()).getColumnFamilyStore(currentTable());
> cfs.forceBlockingFlush();
> cfs.disableAutoCompaction();
> cfs.forceMajorCompaction();
> assertEquals(1, cfs.getLiveSSTables().size());
> SSTableReader reader = cfs.getLiveSSTables().iterator().next();
> try (ISSTableScanner scanner = reader.getScanner();
> CloseableIterator throttled = 
> ThrottledUnfilteredIterator.throttle(scanner, 100))
> {
> assertTrue(throttled.hasNext());
> UnfilteredRowIterator iterator = throttled.next();
> assertFalse(throttled.hasNext());
> assertFalse(iterator.hasNext());
> assertEquals(iterator.partitionLevelDeletion().markedForDeleteAt(), 160);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[03/16] cassandra git commit: ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option

2018-03-19 Thread paulo

ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d26bcb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d26bcb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d26bcb

Branch: refs/heads/trunk
Commit: 19d26bcb80219bce0089fbe8942a34e3a331fd17
Parents: 4bbd28a
Author: Paulo Motta 
Authored: Mon Mar 19 22:02:26 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:02:26 2018 -0300

--
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d26bcb/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 54d7fb7..9963dc3 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1283,7 +1283,7 @@ public class NodeTool
 private int jobs = 2;
 
 @Option(title = "reinsert_overflowed_ttl",
-name = {"r", "--reinsert-overflowed-ttl"},
+name = {"-r", "--reinsert-overflowed-ttl"},
 description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 private boolean reinsertOverflowedTTL = false;
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[15/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-03-19 Thread paulo

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/656cca77
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/656cca77
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/656cca77

Branch: refs/heads/trunk
Commit: 656cca7783f1dec596dad8a2bab9a7b687b22e62
Parents: f57d12e cccaf7c
Author: Paulo Motta 
Authored: Mon Mar 19 22:22:33 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:22:33 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[11/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-03-19 Thread paulo

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cccaf7ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cccaf7ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cccaf7ca

Branch: refs/heads/trunk
Commit: cccaf7ca20ca521272835863aec91fff85b2ea06
Parents: 1d05bda 53b6116
Author: Paulo Motta 
Authored: Mon Mar 19 22:22:22 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:22:22 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cccaf7ca/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index ead2fd4,3c726b9..263291d
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -49,8 -49,13 +49,8 @@@ public class Scrub extends NodeToolCm
 description = "Do not validate columns using column 
validator")
  private boolean noValidation = false;
  
 -@Option(title = "jobs",
 -name = {"-j", "--jobs"},
 -description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 -private int jobs = 2;
 -
  @Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
+ name = {"-r", "--reinsert-overflowed-ttl"},
  description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
  private boolean reinsertOverflowedTTL = false;
  


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[09/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2018-03-19 Thread paulo

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/53b6116d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/53b6116d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/53b6116d

Branch: refs/heads/cassandra-3.0
Commit: 53b6116d56a9353ad46fb90f181cd33fcabb9e0e
Parents: 9715fc0 19d26bc
Author: Paulo Motta 
Authored: Mon Mar 19 22:03:46 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:15:55 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/53b6116d/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index 50224a0,000..3c726b9
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -1,82 -1,0 +1,82 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.tools.nodetool;
 +
 +import io.airlift.command.Arguments;
 +import io.airlift.command.Command;
 +import io.airlift.command.Option;
 +
 +import java.util.ArrayList;
 +import java.util.List;
 +
 +import org.apache.cassandra.tools.NodeProbe;
 +import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 +import org.apache.cassandra.tools.StandaloneScrubber;
 +
 +@Command(name = "scrub", description = "Scrub (rebuild sstables for) one or 
more tables")
 +public class Scrub extends NodeToolCmd
 +{
 +@Arguments(usage = "[ ...]", description = "The 
keyspace followed by one or many tables")
 +private List args = new ArrayList<>();
 +
 +@Option(title = "disable_snapshot",
 +name = {"-ns", "--no-snapshot"},
 +description = "Scrubbed CFs will be snapshotted first, if 
disableSnapshot is false. (default false)")
 +private boolean disableSnapshot = false;
 +
 +@Option(title = "skip_corrupted",
 +name = {"-s", "--skip-corrupted"},
 +description = "Skip corrupted partitions even when scrubbing 
counter tables. (default false)")
 +private boolean skipCorrupted = false;
 +
 +@Option(title = "no_validate",
 +   name = {"-n", "--no-validate"},
 +   description = "Do not validate columns using column 
validator")
 +private boolean noValidation = false;
 +
 +@Option(title = "jobs",
 +name = {"-j", "--jobs"},
 +description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 +private int jobs = 2;
 +
 +@Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
++name = {"-r", "--reinsert-overflowed-ttl"},
 +description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 +private boolean reinsertOverflowedTTL = false;
 +
 +@Override
 +public void execute(NodeProbe probe)
 +{
 +List keyspaces = parseOptionalKeyspace(args, probe);
 +String[] cfnames = parseOptionalColumnFamilies(args);
 +
 +for (String keyspace : keyspaces)
 +{
 +try
 +{
 +probe.scrub(System.out, disableSnapshot, skipCorrupted, 
!noValidation, reinsertOverflowedTTL, jobs, keyspace, cfnames);
 +} catch (IllegalArgumentException e)
 +{
 +throw e;
 +} catch (Exception e)
 +{
 +throw new RuntimeException("Error occurred during scrubbing", 
e);
 +}
 +}
 +}
 +}


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[06/16] cassandra git commit: ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option

2018-03-19 Thread paulo

ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d26bcb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d26bcb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d26bcb

Branch: refs/heads/cassandra-3.11
Commit: 19d26bcb80219bce0089fbe8942a34e3a331fd17
Parents: 4bbd28a
Author: Paulo Motta 
Authored: Mon Mar 19 22:02:26 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:02:26 2018 -0300

--
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d26bcb/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 54d7fb7..9963dc3 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1283,7 +1283,7 @@ public class NodeTool
 private int jobs = 2;
 
 @Option(title = "reinsert_overflowed_ttl",
-name = {"r", "--reinsert-overflowed-ttl"},
+name = {"-r", "--reinsert-overflowed-ttl"},
 description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 private boolean reinsertOverflowedTTL = false;
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[04/16] cassandra git commit: ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option

2018-03-19 Thread paulo

ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d26bcb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d26bcb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d26bcb

Branch: refs/heads/cassandra-3.0
Commit: 19d26bcb80219bce0089fbe8942a34e3a331fd17
Parents: 4bbd28a
Author: Paulo Motta 
Authored: Mon Mar 19 22:02:26 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:02:26 2018 -0300

--
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d26bcb/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 54d7fb7..9963dc3 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1283,7 +1283,7 @@ public class NodeTool
 private int jobs = 2;
 
 @Option(title = "reinsert_overflowed_ttl",
-name = {"r", "--reinsert-overflowed-ttl"},
+name = {"-r", "--reinsert-overflowed-ttl"},
 description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 private boolean reinsertOverflowedTTL = false;
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[07/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2018-03-19 Thread paulo

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/53b6116d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/53b6116d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/53b6116d

Branch: refs/heads/trunk
Commit: 53b6116d56a9353ad46fb90f181cd33fcabb9e0e
Parents: 9715fc0 19d26bc
Author: Paulo Motta 
Authored: Mon Mar 19 22:03:46 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:15:55 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/53b6116d/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index 50224a0,000..3c726b9
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -1,82 -1,0 +1,82 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.tools.nodetool;
 +
 +import io.airlift.command.Arguments;
 +import io.airlift.command.Command;
 +import io.airlift.command.Option;
 +
 +import java.util.ArrayList;
 +import java.util.List;
 +
 +import org.apache.cassandra.tools.NodeProbe;
 +import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 +import org.apache.cassandra.tools.StandaloneScrubber;
 +
 +@Command(name = "scrub", description = "Scrub (rebuild sstables for) one or 
more tables")
 +public class Scrub extends NodeToolCmd
 +{
 +@Arguments(usage = "[ ...]", description = "The 
keyspace followed by one or many tables")
 +private List args = new ArrayList<>();
 +
 +@Option(title = "disable_snapshot",
 +name = {"-ns", "--no-snapshot"},
 +description = "Scrubbed CFs will be snapshotted first, if 
disableSnapshot is false. (default false)")
 +private boolean disableSnapshot = false;
 +
 +@Option(title = "skip_corrupted",
 +name = {"-s", "--skip-corrupted"},
 +description = "Skip corrupted partitions even when scrubbing 
counter tables. (default false)")
 +private boolean skipCorrupted = false;
 +
 +@Option(title = "no_validate",
 +   name = {"-n", "--no-validate"},
 +   description = "Do not validate columns using column 
validator")
 +private boolean noValidation = false;
 +
 +@Option(title = "jobs",
 +name = {"-j", "--jobs"},
 +description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 +private int jobs = 2;
 +
 +@Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
++name = {"-r", "--reinsert-overflowed-ttl"},
 +description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 +private boolean reinsertOverflowedTTL = false;
 +
 +@Override
 +public void execute(NodeProbe probe)
 +{
 +List keyspaces = parseOptionalKeyspace(args, probe);
 +String[] cfnames = parseOptionalColumnFamilies(args);
 +
 +for (String keyspace : keyspaces)
 +{
 +try
 +{
 +probe.scrub(System.out, disableSnapshot, skipCorrupted, 
!noValidation, reinsertOverflowedTTL, jobs, keyspace, cfnames);
 +} catch (IllegalArgumentException e)
 +{
 +throw e;
 +} catch (Exception e)
 +{
 +throw new RuntimeException("Error occurred during scrubbing", 
e);
 +}
 +}
 +}
 +}


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[02/16] cassandra git commit: ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option

2018-03-19 Thread paulo

ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d26bcb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d26bcb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d26bcb

Branch: refs/heads/cassandra-2.2
Commit: 19d26bcb80219bce0089fbe8942a34e3a331fd17
Parents: 4bbd28a
Author: Paulo Motta 
Authored: Mon Mar 19 22:02:26 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:02:26 2018 -0300

--
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d26bcb/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 54d7fb7..9963dc3 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1283,7 +1283,7 @@ public class NodeTool
 private int jobs = 2;
 
 @Option(title = "reinsert_overflowed_ttl",
-name = {"r", "--reinsert-overflowed-ttl"},
+name = {"-r", "--reinsert-overflowed-ttl"},
 description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 private boolean reinsertOverflowedTTL = false;
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[16/16] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

2018-03-19 Thread paulo

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c09e298a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c09e298a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c09e298a

Branch: refs/heads/trunk
Commit: c09e298a4e78bf0dc0e1b6e9a78a21801caef999
Parents: 5b9e985 656cca7
Author: Paulo Motta 
Authored: Mon Mar 19 22:25:07 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:25:07 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c09e298a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[05/16] cassandra git commit: ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option

2018-03-19 Thread paulo

ninja: fix nodetool scrub -r (--reinsert-overflowed-ttl) option


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d26bcb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d26bcb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d26bcb

Branch: refs/heads/cassandra-2.1
Commit: 19d26bcb80219bce0089fbe8942a34e3a331fd17
Parents: 4bbd28a
Author: Paulo Motta 
Authored: Mon Mar 19 22:02:26 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:02:26 2018 -0300

--
 src/java/org/apache/cassandra/tools/NodeTool.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d26bcb/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java 
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 54d7fb7..9963dc3 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1283,7 +1283,7 @@ public class NodeTool
 private int jobs = 2;
 
 @Option(title = "reinsert_overflowed_ttl",
-name = {"r", "--reinsert-overflowed-ttl"},
+name = {"-r", "--reinsert-overflowed-ttl"},
 description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 private boolean reinsertOverflowedTTL = false;
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[13/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-03-19 Thread paulo

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cccaf7ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cccaf7ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cccaf7ca

Branch: refs/heads/cassandra-3.0
Commit: cccaf7ca20ca521272835863aec91fff85b2ea06
Parents: 1d05bda 53b6116
Author: Paulo Motta 
Authored: Mon Mar 19 22:22:22 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:22:22 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cccaf7ca/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index ead2fd4,3c726b9..263291d
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -49,8 -49,13 +49,8 @@@ public class Scrub extends NodeToolCm
 description = "Do not validate columns using column 
validator")
  private boolean noValidation = false;
  
 -@Option(title = "jobs",
 -name = {"-j", "--jobs"},
 -description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 -private int jobs = 2;
 -
  @Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
+ name = {"-r", "--reinsert-overflowed-ttl"},
  description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
  private boolean reinsertOverflowedTTL = false;
  


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[14/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

2018-03-19 Thread paulo

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/656cca77
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/656cca77
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/656cca77

Branch: refs/heads/cassandra-3.11
Commit: 656cca7783f1dec596dad8a2bab9a7b687b22e62
Parents: f57d12e cccaf7c
Author: Paulo Motta 
Authored: Mon Mar 19 22:22:33 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:22:33 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[12/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2018-03-19 Thread paulo

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cccaf7ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cccaf7ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cccaf7ca

Branch: refs/heads/cassandra-3.11
Commit: cccaf7ca20ca521272835863aec91fff85b2ea06
Parents: 1d05bda 53b6116
Author: Paulo Motta 
Authored: Mon Mar 19 22:22:22 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:22:22 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cccaf7ca/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index ead2fd4,3c726b9..263291d
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -49,8 -49,13 +49,8 @@@ public class Scrub extends NodeToolCm
 description = "Do not validate columns using column 
validator")
  private boolean noValidation = false;
  
 -@Option(title = "jobs",
 -name = {"-j", "--jobs"},
 -description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 -private int jobs = 2;
 -
  @Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
+ name = {"-r", "--reinsert-overflowed-ttl"},
  description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
  private boolean reinsertOverflowedTTL = false;
  


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[01/16] cassandra git commit: Handle static and partition deletion properly on ThrottledUnfilteredIterator

2018-03-19 Thread paulo

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 4bbd28a04 -> 19d26bcb8
  refs/heads/cassandra-2.2 9715fc09b -> 53b6116d5
  refs/heads/cassandra-3.0 1d05bdab2 -> cccaf7ca2
  refs/heads/cassandra-3.11 f57d12ee7 -> 656cca778
  refs/heads/trunk 59814db54 -> c09e298a4


Handle static and partition deletion properly on ThrottledUnfilteredIterator

Patch by Zhao Yang; Reviewed by Paulo Motta for CASSANDRA-14315


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5b9e9854
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5b9e9854
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5b9e9854

Branch: refs/heads/trunk
Commit: 5b9e985474e696a83d23e7cf4bedaf360cdb1eaf
Parents: 59814db
Author: Zhao Yang 
Authored: Thu Mar 15 11:47:54 2018 +0800
Committer: Paulo Motta 
Committed: Mon Mar 19 21:53:44 2018 -0300

--
 CHANGES.txt |  1 +
 .../db/rows/ThrottledUnfilteredIterator.java| 20 -
 .../rows/ThrottledUnfilteredIteratorTest.java   | 88 +++-
 3 files changed, 102 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5b9e9854/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fed0cd1..fbcc1bb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Handle static and partition deletion properly on 
ThrottledUnfilteredIterator (CASSANDRA-14315)
  * NodeTool clientstats should show SSL Cipher (CASSANDRA-14322)
  * Add ability to specify driver name and version (CASSANDRA-14275)
  * Abstract streaming for pluggable storage (CASSANDRA-14115)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5b9e9854/src/java/org/apache/cassandra/db/rows/ThrottledUnfilteredIterator.java
--
diff --git 
a/src/java/org/apache/cassandra/db/rows/ThrottledUnfilteredIterator.java 
b/src/java/org/apache/cassandra/db/rows/ThrottledUnfilteredIterator.java
index dd33b1e..a2e8425 100644
--- a/src/java/org/apache/cassandra/db/rows/ThrottledUnfilteredIterator.java
+++ b/src/java/org/apache/cassandra/db/rows/ThrottledUnfilteredIterator.java
@@ -75,8 +75,14 @@ public class ThrottledUnfilteredIterator extends 
AbstractIteratormaxBatchSize
+ * Splits a {@link UnfilteredPartitionIterator} in {@link 
UnfilteredRowIterator} batches with size no higher than
+ * maxBatchSize
+ *
+ * @param partitionIterator
+ * @param maxBatchSize max number of unfiltereds in the 
UnfilteredRowIterator. if 0 is given, it means no throttle.
+ * @return
  */
 public static CloseableIterator 
throttle(UnfilteredPartitionIterator partitionIterator, int maxBatchSize)
 {
+if (maxBatchSize == 0) // opt out
+return partitionIterator;
+
 return new AbstractIterator()
 {
 ThrottledUnfilteredIterator current = null;
@@ -232,7 +245,6 @@ public class ThrottledUnfilteredIterator extends 
AbstractIteratorhttp://git-wip-us.apache.org/repos/asf/cassandra/blob/5b9e9854/test/unit/org/apache/cassandra/db/rows/ThrottledUnfilteredIteratorTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/rows/ThrottledUnfilteredIteratorTest.java 
b/test/unit/org/apache/cassandra/db/rows/ThrottledUnfilteredIteratorTest.java
index 2d2cce0..a530521 100644
--- 
a/test/unit/org/apache/cassandra/db/rows/ThrottledUnfilteredIteratorTest.java
+++ 
b/test/unit/org/apache/cassandra/db/rows/ThrottledUnfilteredIteratorTest.java
@@ -84,6 +84,7 @@ public class ThrottledUnfilteredIteratorTest extends CQLTester
 static final TableMetadata metadata;
 static final ColumnMetadata v1Metadata;
 static final ColumnMetadata v2Metadata;
+static final ColumnMetadata staticMetadata;
 
 static
 {
@@ -93,9 +94,81 @@ public class ThrottledUnfilteredIteratorTest extends 
CQLTester
 .addClusteringColumn("ck2", Int32Type.instance)
 .addRegularColumn("v1", Int32Type.instance)
 .addRegularColumn("v2", Int32Type.instance)
+.addStaticColumn("s1", Int32Type.instance)
 .build();
 v1Metadata = 
metadata.regularAndStaticColumns().columns(false).getSimple(0);
 v2Metadata = 
metadata.regularAndStaticColumns().columns(false).getSimple(1);
+staticMetadata = 
metadata.regularAndStaticColumns().columns(true).getSimple(0);
+}
+
+@Test
+public void emptyPartitionDeletionTest() throws Throwable
+{
+// create cell tombstone, range tombstone,

[10/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2018-03-19 Thread paulo

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/53b6116d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/53b6116d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/53b6116d

Branch: refs/heads/cassandra-3.11
Commit: 53b6116d56a9353ad46fb90f181cd33fcabb9e0e
Parents: 9715fc0 19d26bc
Author: Paulo Motta 
Authored: Mon Mar 19 22:03:46 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:15:55 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/53b6116d/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index 50224a0,000..3c726b9
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -1,82 -1,0 +1,82 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.tools.nodetool;
 +
 +import io.airlift.command.Arguments;
 +import io.airlift.command.Command;
 +import io.airlift.command.Option;
 +
 +import java.util.ArrayList;
 +import java.util.List;
 +
 +import org.apache.cassandra.tools.NodeProbe;
 +import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 +import org.apache.cassandra.tools.StandaloneScrubber;
 +
 +@Command(name = "scrub", description = "Scrub (rebuild sstables for) one or 
more tables")
 +public class Scrub extends NodeToolCmd
 +{
 +@Arguments(usage = "[ ...]", description = "The 
keyspace followed by one or many tables")
 +private List args = new ArrayList<>();
 +
 +@Option(title = "disable_snapshot",
 +name = {"-ns", "--no-snapshot"},
 +description = "Scrubbed CFs will be snapshotted first, if 
disableSnapshot is false. (default false)")
 +private boolean disableSnapshot = false;
 +
 +@Option(title = "skip_corrupted",
 +name = {"-s", "--skip-corrupted"},
 +description = "Skip corrupted partitions even when scrubbing 
counter tables. (default false)")
 +private boolean skipCorrupted = false;
 +
 +@Option(title = "no_validate",
 +   name = {"-n", "--no-validate"},
 +   description = "Do not validate columns using column 
validator")
 +private boolean noValidation = false;
 +
 +@Option(title = "jobs",
 +name = {"-j", "--jobs"},
 +description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 +private int jobs = 2;
 +
 +@Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
++name = {"-r", "--reinsert-overflowed-ttl"},
 +description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 +private boolean reinsertOverflowedTTL = false;
 +
 +@Override
 +public void execute(NodeProbe probe)
 +{
 +List keyspaces = parseOptionalKeyspace(args, probe);
 +String[] cfnames = parseOptionalColumnFamilies(args);
 +
 +for (String keyspace : keyspaces)
 +{
 +try
 +{
 +probe.scrub(System.out, disableSnapshot, skipCorrupted, 
!noValidation, reinsertOverflowedTTL, jobs, keyspace, cfnames);
 +} catch (IllegalArgumentException e)
 +{
 +throw e;
 +} catch (Exception e)
 +{
 +throw new RuntimeException("Error occurred during scrubbing", 
e);
 +}
 +}
 +}
 +}


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[08/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2018-03-19 Thread paulo

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/53b6116d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/53b6116d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/53b6116d

Branch: refs/heads/cassandra-2.2
Commit: 53b6116d56a9353ad46fb90f181cd33fcabb9e0e
Parents: 9715fc0 19d26bc
Author: Paulo Motta 
Authored: Mon Mar 19 22:03:46 2018 -0300
Committer: Paulo Motta 
Committed: Mon Mar 19 22:15:55 2018 -0300

--
 src/java/org/apache/cassandra/tools/nodetool/Scrub.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/53b6116d/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
--
diff --cc src/java/org/apache/cassandra/tools/nodetool/Scrub.java
index 50224a0,000..3c726b9
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/Scrub.java
@@@ -1,82 -1,0 +1,82 @@@
 +/*
 + * Licensed to the Apache Software Foundation (ASF) under one
 + * or more contributor license agreements.  See the NOTICE file
 + * distributed with this work for additional information
 + * regarding copyright ownership.  The ASF licenses this file
 + * to you under the Apache License, Version 2.0 (the
 + * "License"); you may not use this file except in compliance
 + * with the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an "AS IS" BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +package org.apache.cassandra.tools.nodetool;
 +
 +import io.airlift.command.Arguments;
 +import io.airlift.command.Command;
 +import io.airlift.command.Option;
 +
 +import java.util.ArrayList;
 +import java.util.List;
 +
 +import org.apache.cassandra.tools.NodeProbe;
 +import org.apache.cassandra.tools.NodeTool.NodeToolCmd;
 +import org.apache.cassandra.tools.StandaloneScrubber;
 +
 +@Command(name = "scrub", description = "Scrub (rebuild sstables for) one or 
more tables")
 +public class Scrub extends NodeToolCmd
 +{
 +@Arguments(usage = "[ ...]", description = "The 
keyspace followed by one or many tables")
 +private List args = new ArrayList<>();
 +
 +@Option(title = "disable_snapshot",
 +name = {"-ns", "--no-snapshot"},
 +description = "Scrubbed CFs will be snapshotted first, if 
disableSnapshot is false. (default false)")
 +private boolean disableSnapshot = false;
 +
 +@Option(title = "skip_corrupted",
 +name = {"-s", "--skip-corrupted"},
 +description = "Skip corrupted partitions even when scrubbing 
counter tables. (default false)")
 +private boolean skipCorrupted = false;
 +
 +@Option(title = "no_validate",
 +   name = {"-n", "--no-validate"},
 +   description = "Do not validate columns using column 
validator")
 +private boolean noValidation = false;
 +
 +@Option(title = "jobs",
 +name = {"-j", "--jobs"},
 +description = "Number of sstables to scrub simultanously, set to 
0 to use all available compaction threads")
 +private int jobs = 2;
 +
 +@Option(title = "reinsert_overflowed_ttl",
- name = {"r", "--reinsert-overflowed-ttl"},
++name = {"-r", "--reinsert-overflowed-ttl"},
 +description = 
StandaloneScrubber.REINSERT_OVERFLOWED_TTL_OPTION_DESCRIPTION)
 +private boolean reinsertOverflowedTTL = false;
 +
 +@Override
 +public void execute(NodeProbe probe)
 +{
 +List keyspaces = parseOptionalKeyspace(args, probe);
 +String[] cfnames = parseOptionalColumnFamilies(args);
 +
 +for (String keyspace : keyspaces)
 +{
 +try
 +{
 +probe.scrub(System.out, disableSnapshot, skipCorrupted, 
!noValidation, reinsertOverflowedTTL, jobs, keyspace, cfnames);
 +} catch (IllegalArgumentException e)
 +{
 +throw e;
 +} catch (Exception e)
 +{
 +throw new RuntimeException("Error occurred during scrubbing", 
e);
 +}
 +}
 +}
 +}


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Jaydeepkumar Chovatia (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405595#comment-16405595
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-14232:
---

Hi [~sumanth.pasupuleti]

Can you please use {{Keyspace.open(metadata.keyspace)}} instead of 
{{Schema.instance.getKeyspaceInstance(keyspaceName)}} due to following two 
reasons:
1. {{Keyspace.open}} checks for {{initialization}}
2. To keep code consistent with {{coordinatorReadLatency}}

Jaydeep


> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14318) Debug logging can create massive performance issues

2018-03-19 Thread Jeff Jirsa (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405580#comment-16405580
 ] 

Jeff Jirsa commented on CASSANDRA-14318:


If you havent yet started the second run of benchmarks, you may also want to 
move the debug log statements from ReadCallback.java (one deals with digest 
mismatch exceptions, forgot what the other is) to trace as well. 

> Debug logging can create massive performance issues
> ---
>
> Key: CASSANDRA-14318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14318
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Major
>  Labels: lhf, performance
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: debuglogging.png, flame22 nodebug sjk svg.png, 
> flame22-nodebug-sjk.svg, flame22-sjk.svg, flame_graph_snapshot.png
>
>
> Debug logging can involve in many cases (especially very low latency ones) a 
> very important overhead on the read path in 2.2 as we've seen when upgrading 
> clusters from 2.0 to 2.2.
> The performance impact was especially noticeable on the client side metrics, 
> where p99 could go up to 10 times higher, while ClientRequest metrics 
> recorded by Cassandra didn't show any overhead.
> Below shows latencies recorded on the client side with debug logging on 
> first, and then without it :
> !debuglogging.png!  
> We generated a flame graph before turning off debug logging that shows the 
> read call stack is dominated by debug logging : 
> !flame_graph_snapshot.png!
> I've attached the original flame graph for exploration.
> Once disabled, the new flame graph shows that the read call stack gets 
> extremely thin, which is further confirmed by client recorded metrics : 
> !flame22 nodebug sjk svg.png!
> The query pager code has been reworked since 3.0 and it looks like 
> log.debug() calls are gone there, but for 2.2 users and to prevent such 
> issues to appear with default settings, I really think debug logging should 
> be disabled by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

2018-03-19 Thread Jaydeepkumar Chovatia (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405567#comment-16405567
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13740:
---

Hi [~iamaleksey]

Please find updated patch here:
||trunk||3.0||
|[patch|https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13740-trunk?expand=1]|[patch|https://github.com/apache/cassandra/compare/cassandra-3.0...jaydeepkumar1984:CASSANDRA-13740_1?expand=1]|
|[utest|https://circleci.com/gh/jaydeepkumar1984/cassandra/46]|[utest|https://circleci.com/gh/jaydeepkumar1984/cassandra/44]|

Jaydeep

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Hints
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Dikang Gu (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405543#comment-16405543
 ] 

Dikang Gu commented on CASSANDRA-14252:
---

Could be. We had serious problems with speculative retry before, which 
overloaded hot replicas, so we turned it off in our production clusters. 

Still I feel it's bad for dynamic endpoint snitch can not fallback due to lack 
of a default score. 

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Jay Zhuang (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405519#comment-16405519
 ] 

Jay Zhuang commented on CASSANDRA-14252:


Should "Speculative Retry" help in this situation?

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405483#comment-16405483
 ] 

Sumanth Pasupuleti commented on CASSANDRA-14232:


[~vinaykumarcse] Thanks! Added updates to NEWS.txt and metrics.rst in the 
documentation.

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-14232:
---
Attachment: (was: 14232-trunk.txt)

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-14232:
---
Fix Version/s: 4.0

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14232) Add metric for coordinator writes per column family

2018-03-19 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-14232:
---
Attachment: 14232-trunk.txt

> Add metric for coordinator writes per column family
> ---
>
> Key: CASSANDRA-14232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14232
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Major
> Fix For: 4.0
>
> Attachments: 14232-trunk.txt
>
>
> Includes write ops and latencies at coordinator per column family.
> Relevant discussion in dev mailing list - 
> [https://lists.apache.org/thread.html/f68f694b13b670a1fa28fa75620304603fc89e94ec515933199f4c37@%3Cdev.cassandra.apache.org%3E]
> Below are a few advantages of having such metric
>  * Ability to identify specific column family that coordinator writes are 
> slow to
>  * Also useful in a multi-tenant cluster, where different column families are 
> owned by different teams



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Dikang Gu (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405231#comment-16405231
 ] 

Dikang Gu commented on CASSANDRA-14252:
---

[~jay.zhuang], the scenario is:
 # Coordinator and node A is in DC1, node B is another replica of node A and in 
DC2, we use DynamicEndpointSnitch and NetworkTopologyStrategy.
 # In normal situation, coordinator only talks to node A, it has correct score 
of node A. Coordinator never talks to node B, and do not have score for node B.
 # Then node A is degraded, it is slow but still alive. Coordinator set the 
score of node A to be very high, like 1.
 # But still, Coordinator do not have score for node B, which makes it never 
has the chance to talk to node B, which is a healthy of the replica in a 
different region.

 

My patch is provide a default score for node B, so coordinator will have chance 
to talk to node B at least once, to get the correct latency number between 
coordinator and node B, and can use it to decide whether to switch from node A 
to node B, if necessary.

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14118) Refactor write path

2018-03-19 Thread Dikang Gu (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405221#comment-16405221
 ] 

Dikang Gu commented on CASSANDRA-14118:
---

update the PR, handle commit log inside the WriteHandler, and handle the cache 
inside the CacheHandler.

> Refactor write path
> ---
>
> Key: CASSANDRA-14118
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14118
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
>
> As part of the pluggable storage engine effort, we'd like to modularize the 
> write path related code, make it to be independent from existing storage 
> engine implementation details.
> For now, refer to 
> https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc
>  for high level designs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Jay Zhuang (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405172#comment-16405172
 ] 

Jay Zhuang commented on CASSANDRA-14252:


Hi [~dikanggu], would you please help me to understand the scenario?
Assume there're 3 nodes: coordinator node, a degraded node, and a healthy node:

!IMG_3180.jpg!

When the issue happens, the coordinator node doesn't have the score for either 
degraded node nor healthy node, so it follows subsnitch ordering and always 
talk to the degraded node, is that right? Or are the coordinator node and the 
degraded node the same node?

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Jay Zhuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14252:
---
Attachment: IMG_3180.jpg

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Jay Zhuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14252:
---
Attachment: (was: IMG_3180.jpg)

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch

2018-03-19 Thread Jay Zhuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14252:
---
Attachment: IMG_3180.jpg

> Use zero as default score in DynamicEndpointSnitch
> --
>
> Key: CASSANDRA-14252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14252
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: IMG_3180.jpg
>
>
> The problem I want to solve is that I found in our deployment, one slow but 
> alive data node can slow down the whole cluster, even caused timeout of our 
> requests. 
> We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the 
> DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node 
> latency is too high.
> I added some debug log, and figured out that in a lot of cases, the score 
> from remote data node was not populated, so the fallback to 
> sortByProximityWithScore never happened. That's why a single slow data node, 
> can cause huge problems to the whole cluster.
> In this jira, I'd like to use zero as default score, so that we will get a 
> chance to try remote data node, if local one is slow. 
> I tested it in our test cluster, it improved the client latency in single 
> slow data node case significantly.  
> I flag this as a Bug, because it caused problems to our use cases multiple 
> times.
>   logs ===
> _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [0.0]_
>  _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: 
> sortByProximityWithBadness: after sorting by proximity, addresses order 
> change to [ip1, ip2], with scores [1.0]_
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405135#comment-16405135
 ] 

Dinesh Joshi commented on CASSANDRA-14322:
--

[~jasobrown] thank you!

> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0
>
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Jason Brown (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14322:

   Resolution: Fixed
Fix Version/s: 4.0
   Status: Resolved  (was: Patch Available)

+1. committed as sha {{59814db54375d4002eb11403c72861765d9eb356}}. Thanks!

> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0
>
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405112#comment-16405112
 ] 

Dinesh Joshi commented on CASSANDRA-14322:
--

I have rebased my patch on trunk since there were some overlapping changes that 
would've caused conflicts.

> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: NodeTool clientstats should show SSL Cipher

2018-03-19 Thread jasobrown

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5dc55e715 -> 59814db54


NodeTool clientstats should show SSL Cipher

patch by Dinesh Joshi; reviewed by jasobrown for CASSANDRA-14322


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59814db5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59814db5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59814db5

Branch: refs/heads/trunk
Commit: 59814db54375d4002eb11403c72861765d9eb356
Parents: 5dc55e7
Author: Dinesh Joshi 
Authored: Mon Mar 19 09:54:11 2018 -0700
Committer: Jason Brown 
Committed: Mon Mar 19 09:54:11 2018 -0700

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/tools/nodetool/ClientStats.java | 4 ++--
 src/java/org/apache/cassandra/transport/Server.java   | 6 +-
 3 files changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/59814db5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9436539..fed0cd1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * NodeTool clientstats should show SSL Cipher (CASSANDRA-14322)
  * Add ability to specify driver name and version (CASSANDRA-14275)
  * Abstract streaming for pluggable storage (CASSANDRA-14115)
  * Forced incremental repairs should promote sstables if they can 
(CASSANDRA-14294)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/59814db5/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
index 4529c86..0469074 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
@@ -43,10 +43,10 @@ public class ClientStats extends NodeToolCmd
 if (!clients.isEmpty())
 {
 TableBuilder table = new TableBuilder();
-table.add("Address", "SSL", "Version", "User", "Keyspace", 
"Requests", "Driver-Name", "Driver-Version");
+table.add("Address", "SSL", "Cipher", "Protocol", "Version", 
"User", "Keyspace", "Requests", "Driver-Name", "Driver-Version");
 for (Map conn : clients)
 {
-table.add(conn.get("address"), conn.get("ssl"), 
conn.get("version"), 
+table.add(conn.get("address"), conn.get("ssl"), 
conn.get("cipher"), conn.get("protocol"), conn.get("version"),
   conn.get("user"), conn.get("keyspace"), 
conn.get("requests"), conn.get("driverName"), conn.get("driverVersion"));
 }
 table.printTo(System.out);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/59814db5/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index 0504b2f..0f666d8 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -189,13 +189,17 @@ public class Server implements CassandraDaemon.Server
 if (connection instanceof ServerConnection)
 {
 ServerConnection conn = (ServerConnection) connection;
+SslHandler sslHandler = 
conn.channel().pipeline().get(SslHandler.class);
+
 result.add(new ImmutableMap.Builder()
 .put("user", conn.getClientState().getUser().getName())
 .put("keyspace", 
conn.getClientState().getRawKeyspace() == null ? "" : 
conn.getClientState().getRawKeyspace())
 .put("address", 
conn.getClientState().getRemoteAddress().toString())
 .put("version", 
String.valueOf(conn.getVersion().asInt()))
 .put("requests", 
String.valueOf(conn.requests.getCount()))
-.put("ssl", 
conn.channel().pipeline().get(SslHandler.class) == null ? "false" : "true")
+.put("ssl", Boolean.toString(sslHandler == null))
+.put("cipher", sslHandler != null ? 
sslHandler.engine().getSession().getCipherSuite() : "undefined")
+.put("protocol", sslHandler != null ? 
sslHandler.engine().getSession().getProtocol() : "undefined")
 .put("driverName",

[jira] [Commented] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405088#comment-16405088
 ] 

Dinesh Joshi commented on CASSANDRA-14322:
--

[~iamaleksey] gave me some feedback over IRC that I have incorporated in the 
PR. Looks a lot better now.

> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14325) Java executable check succeeds despite no java on PATH

2018-03-19 Thread Angelo Polo (JIRA)

Angelo Polo created CASSANDRA-14325:
---

 Summary: Java executable check succeeds despite no java on PATH
 Key: CASSANDRA-14325
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14325
 Project: Cassandra
  Issue Type: Bug
  Components: Lifecycle
Reporter: Angelo Polo
 Attachments: bin_cassandra.patch

The check -z $JAVA on line 102 of bin/cassandra currently always succeeds if 
JAVA_HOME is not set since in this case JAVA gets set directly to 'java'. The 
error message "_Unable to find java executable. Check JAVA_HOME and PATH 
environment variables._" will never be echoed on a PATH misconfiguration. If 
java isn't on the PATH the failure will instead occur on line 95 of 
cassandra-env.sh at the java version check.

It would be better to check consistently for the java executable in one place 
in bin/cassandra. Also we don't want users to mistakenly think they have a java 
version problem when they in fact have a PATH problem.

See proposed patch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14256) Renaming Reporting Bugs and Contributions to just Reporting Bugs

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405063#comment-16405063
 ] 

Dinesh Joshi commented on CASSANDRA-14256:
--

Thanks [~kenbrotman]. It would be nice to put your commits / PR references in 
the ticket for easy lookup.

> Renaming Reporting Bugs and Contributions to just Reporting Bugs
> 
>
> Key: CASSANDRA-14256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Kenneth Brotman
>Priority: Major
>
> The top level menu title and web page title of 
> [http://cassandra.apache.org/doc/latest/bugs.html] should be rename from 
> “Reporting Bugs and Contributions” to just Reporting Bugs. There is a 
> separate section for “Contributing to Cassandra” already and it has the 
> information on contributing over there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14318) Debug logging can create massive performance issues

2018-03-19 Thread Alexander Dejanovski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405051#comment-16405051
 ] 

Alexander Dejanovski commented on CASSANDRA-14318:
--

Fair point. I'll write the patch and run the benchmarks.

> Debug logging can create massive performance issues
> ---
>
> Key: CASSANDRA-14318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14318
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Major
>  Labels: lhf, performance
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: debuglogging.png, flame22 nodebug sjk svg.png, 
> flame22-nodebug-sjk.svg, flame22-sjk.svg, flame_graph_snapshot.png
>
>
> Debug logging can involve in many cases (especially very low latency ones) a 
> very important overhead on the read path in 2.2 as we've seen when upgrading 
> clusters from 2.0 to 2.2.
> The performance impact was especially noticeable on the client side metrics, 
> where p99 could go up to 10 times higher, while ClientRequest metrics 
> recorded by Cassandra didn't show any overhead.
> Below shows latencies recorded on the client side with debug logging on 
> first, and then without it :
> !debuglogging.png!  
> We generated a flame graph before turning off debug logging that shows the 
> read call stack is dominated by debug logging : 
> !flame_graph_snapshot.png!
> I've attached the original flame graph for exploration.
> Once disabled, the new flame graph shows that the read call stack gets 
> extremely thin, which is further confirmed by client recorded metrics : 
> !flame22 nodebug sjk svg.png!
> The query pager code has been reworked since 3.0 and it looks like 
> log.debug() calls are gone there, but for 2.2 users and to prevent such 
> issues to appear with default settings, I really think debug logging should 
> be disabled by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405045#comment-16405045
 ] 

Dinesh Joshi commented on CASSANDRA-14322:
--

Hi [~jasobrown], I have addressed #2 and updated my commit. [~iamaleksey] and I 
settled on using {{undefined}} for missing driver information. So keep things 
consistent, I used {{undefined}} here.

> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14275) Cassandra Driver should send identification information to Server

2018-03-19 Thread Dinesh Joshi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405009#comment-16405009
 ] 

Dinesh Joshi commented on CASSANDRA-14275:
--

Thanks, [~iamaleksey]!

> Cassandra Driver should send identification information to Server
> -
>
> Key: CASSANDRA-14275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14275
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Fix For: 4.0
>
>
> Currently there doesn't seem to be any way to readily identify the driver 
> that clients are using to connect to Cassandra. Add the capability of 
> identifying the driver through metadata information much like how HTTP 
> Clients identify themselves through User-Agent HTTP header. This is useful 
> for debugging in large deployments where clients tend to use different 
> drivers, wrappers and language bindings to connect to Cassandra. This can 
> help surface issues as well as help detect clients which are using older or 
> unsupported drivers.
> The identification information should be a string that unambiguously 
> identifies the driver. It should include information such as the name of the 
> driver, it's version, CQL version, Platform (Linux, macOS, Windows, etc.) and 
> architecture (x86, x86_64).
> We should surface this information in `nodetool clientstats` command.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14256) Renaming Reporting Bugs and Contributions to just Reporting Bugs

2018-03-19 Thread Kenneth Brotman (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404939#comment-16404939
 ] 

Kenneth Brotman commented on CASSANDRA-14256:
-

[https://github.com/apache/cassandra/commit/a7a36e703c8b4a0a126b2f750e2dbe56440658ea]

 

> Renaming Reporting Bugs and Contributions to just Reporting Bugs
> 
>
> Key: CASSANDRA-14256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14256
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Kenneth Brotman
>Priority: Major
>
> The top level menu title and web page title of 
> [http://cassandra.apache.org/doc/latest/bugs.html] should be rename from 
> “Reporting Bugs and Contributions” to just Reporting Bugs. There is a 
> separate section for “Contributing to Cassandra” already and it has the 
> information on contributing over there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14318) Debug logging can create massive performance issues

2018-03-19 Thread Jeremiah Jordan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404814#comment-16404814
 ] 

Jeremiah Jordan commented on CASSANDRA-14318:
-

No matter what is decided we should move those log messages to TRACE.  So I 
think we can proceed here no matter what.

> Debug logging can create massive performance issues
> ---
>
> Key: CASSANDRA-14318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14318
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Major
>  Labels: lhf, performance
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: debuglogging.png, flame22 nodebug sjk svg.png, 
> flame22-nodebug-sjk.svg, flame22-sjk.svg, flame_graph_snapshot.png
>
>
> Debug logging can involve in many cases (especially very low latency ones) a 
> very important overhead on the read path in 2.2 as we've seen when upgrading 
> clusters from 2.0 to 2.2.
> The performance impact was especially noticeable on the client side metrics, 
> where p99 could go up to 10 times higher, while ClientRequest metrics 
> recorded by Cassandra didn't show any overhead.
> Below shows latencies recorded on the client side with debug logging on 
> first, and then without it :
> !debuglogging.png!  
> We generated a flame graph before turning off debug logging that shows the 
> read call stack is dominated by debug logging : 
> !flame_graph_snapshot.png!
> I've attached the original flame graph for exploration.
> Once disabled, the new flame graph shows that the read call stack gets 
> extremely thin, which is further confirmed by client recorded metrics : 
> !flame22 nodebug sjk svg.png!
> The query pager code has been reworked since 3.0 and it looks like 
> log.debug() calls are gone there, but for 2.2 users and to prevent such 
> issues to appear with default settings, I really think debug logging should 
> be disabled by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14275) Cassandra Driver should send identification information to Server

2018-03-19 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14275:
--
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Patch Available)

Committed to 4.0 as [5dc55e715eba6667c388da9f8f1eb7a46489b35c 
|https://github.com/apache/cassandra/commit/5dc55e715eba6667c388da9f8f1eb7a46489b35c].

> Cassandra Driver should send identification information to Server
> -
>
> Key: CASSANDRA-14275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14275
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Fix For: 4.0
>
>
> Currently there doesn't seem to be any way to readily identify the driver 
> that clients are using to connect to Cassandra. Add the capability of 
> identifying the driver through metadata information much like how HTTP 
> Clients identify themselves through User-Agent HTTP header. This is useful 
> for debugging in large deployments where clients tend to use different 
> drivers, wrappers and language bindings to connect to Cassandra. This can 
> help surface issues as well as help detect clients which are using older or 
> unsupported drivers.
> The identification information should be a string that unambiguously 
> identifies the driver. It should include information such as the name of the 
> driver, it's version, CQL version, Platform (Linux, macOS, Windows, etc.) and 
> architecture (x86, x86_64).
> We should surface this information in `nodetool clientstats` command.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14318) Debug logging can create massive performance issues

2018-03-19 Thread Alexander Dejanovski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404797#comment-16404797
 ] 

Alexander Dejanovski commented on CASSANDRA-14318:
--

[~pauloricardomg], thanks for assigning the ticket to me.

Whatever the implementation, I'll run new performance tests and generate flame 
graphs to verify the impact of the changes, no worries.

I'll wait a bit for a consensus to come out of the discussion on the dev ML 
before moving on, as there seem to be some conflicting views on what should be 
done.

> Debug logging can create massive performance issues
> ---
>
> Key: CASSANDRA-14318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14318
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Major
>  Labels: lhf, performance
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: debuglogging.png, flame22 nodebug sjk svg.png, 
> flame22-nodebug-sjk.svg, flame22-sjk.svg, flame_graph_snapshot.png
>
>
> Debug logging can involve in many cases (especially very low latency ones) a 
> very important overhead on the read path in 2.2 as we've seen when upgrading 
> clusters from 2.0 to 2.2.
> The performance impact was especially noticeable on the client side metrics, 
> where p99 could go up to 10 times higher, while ClientRequest metrics 
> recorded by Cassandra didn't show any overhead.
> Below shows latencies recorded on the client side with debug logging on 
> first, and then without it :
> !debuglogging.png!  
> We generated a flame graph before turning off debug logging that shows the 
> read call stack is dominated by debug logging : 
> !flame_graph_snapshot.png!
> I've attached the original flame graph for exploration.
> Once disabled, the new flame graph shows that the read call stack gets 
> extremely thin, which is further confirmed by client recorded metrics : 
> !flame22 nodebug sjk svg.png!
> The query pager code has been reworked since 3.0 and it looks like 
> log.debug() calls are gone there, but for 2.2 users and to prevent such 
> issues to appear with default settings, I really think debug logging should 
> be disabled by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14118) Refactor write path

2018-03-19 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404752#comment-16404752
 ] 

Joshua McKenzie edited comment on CASSANDRA-14118 at 3/19/18 12:45 PM:
---

bq. And yes, not using the C* commit log would probably preclude supporting cdc.
Correct. Its current implementation is constrained by the 1:1 requirement (if 
it persists to disk, it's available for consumption, if it doesn't, it's not) + 
desire to keep heap pressure from the feature to a minimum (i.e. don't cache 
AllTheThings, then mark them in memory as persisted on CL flush, then fragment 
that memory out if offheap or pressure GC if on on reclaim, etc etc etc).

i.e. it's using the CL as the mechanism of persistence + "communication" with 
an external process reading the files, so you don't write it to the CL, you 
don't get "CDC".


was (Author: joshuamckenzie):
bq. And yes, not using the C* commit log would probably preclude supporting cdc.
Correct. It's current implementation is constrained by the 1:1 requirement (if 
it persists to disk, it's available for consumption, if it doesn't, it's not) + 
desire to keep heap pressure from the feature to a minimum (i.e. don't cache 
AllTheThings, then mark them in memory as persisted on CL flush, then fragment 
that memory out if offheap or pressure GC if on on reclaim, etc etc etc).

i.e. it's using the CL as the mechanism of persistence + "communication" with 
an external process reading the files, so you don't write it to the CL, you 
don't get "CDC".

> Refactor write path
> ---
>
> Key: CASSANDRA-14118
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14118
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
>
> As part of the pluggable storage engine effort, we'd like to modularize the 
> write path related code, make it to be independent from existing storage 
> engine implementation details.
> For now, refer to 
> https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc
>  for high level designs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14118) Refactor write path

2018-03-19 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404752#comment-16404752
 ] 

Joshua McKenzie commented on CASSANDRA-14118:
-

bq. And yes, not using the C* commit log would probably preclude supporting cdc.
Correct. It's current implementation is constrained by the 1:1 requirement (if 
it persists to disk, it's available for consumption, if it doesn't, it's not) + 
desire to keep heap pressure from the feature to a minimum (i.e. don't cache 
AllTheThings, then mark them in memory as persisted on CL flush, then fragment 
that memory out if offheap or pressure GC if on on reclaim, etc etc etc).

i.e. it's using the CL as the mechanism of persistence + "communication" with 
an external process reading the files, so you don't write it to the CL, you 
don't get "CDC".

> Refactor write path
> ---
>
> Key: CASSANDRA-14118
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14118
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
>
> As part of the pluggable storage engine effort, we'd like to modularize the 
> write path related code, make it to be independent from existing storage 
> engine implementation details.
> For now, refer to 
> https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc
>  for high level designs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] Git Push Summary

2018-03-19 Thread aleksey

Repository: cassandra
Updated Branches:
  refs/heads/14275-4.0 [deleted] 5dc55e715

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Add ability to specify driver name and version

2018-03-19 Thread aleksey

Repository: cassandra
Updated Branches:
  refs/heads/trunk 9714a7c81 -> 5dc55e715


Add ability to specify driver name and version

patch by Dinesh Joshi; reviewed by Aleksey Yeschenko for CASSANDRA-14275


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5dc55e71
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5dc55e71
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5dc55e71

Branch: refs/heads/trunk
Commit: 5dc55e715eba6667c388da9f8f1eb7a46489b35c
Parents: 9714a7c
Author: Dinesh Joshi 
Authored: Mon Feb 26 16:59:23 2018 -0800
Committer: Aleksey Yeshchenko 
Committed: Mon Mar 19 12:25:41 2018 +

--
 CHANGES.txt |  1 +
 .../apache/cassandra/service/ClientState.java   | 27 
 .../cassandra/tools/nodetool/ClientStats.java   |  4 +--
 .../org/apache/cassandra/transport/Server.java  |  2 ++
 .../transport/messages/StartupMessage.java  | 11 
 5 files changed, 43 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9458f19..9436539 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Add ability to specify driver name and version (CASSANDRA-14275)
  * Abstract streaming for pluggable storage (CASSANDRA-14115)
  * Forced incremental repairs should promote sstables if they can 
(CASSANDRA-14294)
  * Use Murmur3 for validation compactions (CASSANDRA-14002)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/src/java/org/apache/cassandra/service/ClientState.java
--
diff --git a/src/java/org/apache/cassandra/service/ClientState.java 
b/src/java/org/apache/cassandra/service/ClientState.java
index 97b6172..3e2be80 100644
--- a/src/java/org/apache/cassandra/service/ClientState.java
+++ b/src/java/org/apache/cassandra/service/ClientState.java
@@ -21,6 +21,7 @@ import java.net.InetSocketAddress;
 import java.net.SocketAddress;
 import java.util.Arrays;
 import java.util.HashSet;
+import java.util.Optional;
 import java.util.Set;
 import java.util.concurrent.atomic.AtomicLong;
 
@@ -106,6 +107,10 @@ public class ClientState
 // The remote address of the client - null for internal clients.
 private final InetSocketAddress remoteAddress;
 
+// Driver String for the client
+private volatile String driverName;
+private volatile String driverVersion;
+
 // The biggest timestamp that was returned by getTimestamp/assigned to a 
query. This is global to ensure that the
 // timestamp assigned are strictly monotonic on a node, which is likely 
what user expect intuitively (more likely,
 // most new user will intuitively expect timestamp to be strictly 
monotonic cluster-wise, but while that last part
@@ -135,6 +140,8 @@ public class ClientState
 this.remoteAddress = source.remoteAddress;
 this.user = source.user;
 this.keyspace = source.keyspace;
+this.driverName = source.driverName;
+this.driverVersion = source.driverVersion;
 }
 
 /**
@@ -243,6 +250,26 @@ public class ClientState
 }
 }
 
+public Optional getDriverName()
+{
+return Optional.ofNullable(driverName);
+}
+
+public Optional getDriverVersion()
+{
+return Optional.ofNullable(driverVersion);
+}
+
+public void setDriverName(String driverName)
+{
+this.driverName = driverName;
+}
+
+public void setDriverVersion(String driverVersion)
+{
+this.driverVersion = driverVersion;
+}
+
 public static QueryHandler getCQLQueryHandler()
 {
 return cqlQueryHandler;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
index 21915cb..4529c86 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
@@ -43,11 +43,11 @@ public class ClientStats extends NodeToolCmd
 if (!clients.isEmpty())
 {
 TableBuilder table = new TableBuilder();
-table.add("Address", "SSL", "Version", "User", "Keyspace", 
"Requests");
+table.add("Address", "SSL", "Version", "User", "Keyspace", 
"Requests", "Driver-Name", "Driver-Version");
 for (Map conn :

cassandra git commit: Add ability to specify driver name and version

2018-03-19 Thread aleksey

Repository: cassandra
Updated Branches:
  refs/heads/14275-4.0 [created] 5dc55e715


Add ability to specify driver name and version

patch by Dinesh Joshi; reviewed by Aleksey Yeschenko for CASSANDRA-14275


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5dc55e71
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5dc55e71
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5dc55e71

Branch: refs/heads/14275-4.0
Commit: 5dc55e715eba6667c388da9f8f1eb7a46489b35c
Parents: 9714a7c
Author: Dinesh Joshi 
Authored: Mon Feb 26 16:59:23 2018 -0800
Committer: Aleksey Yeshchenko 
Committed: Mon Mar 19 12:25:41 2018 +

--
 CHANGES.txt |  1 +
 .../apache/cassandra/service/ClientState.java   | 27 
 .../cassandra/tools/nodetool/ClientStats.java   |  4 +--
 .../org/apache/cassandra/transport/Server.java  |  2 ++
 .../transport/messages/StartupMessage.java  | 11 
 5 files changed, 43 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9458f19..9436539 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Add ability to specify driver name and version (CASSANDRA-14275)
  * Abstract streaming for pluggable storage (CASSANDRA-14115)
  * Forced incremental repairs should promote sstables if they can 
(CASSANDRA-14294)
  * Use Murmur3 for validation compactions (CASSANDRA-14002)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/src/java/org/apache/cassandra/service/ClientState.java
--
diff --git a/src/java/org/apache/cassandra/service/ClientState.java 
b/src/java/org/apache/cassandra/service/ClientState.java
index 97b6172..3e2be80 100644
--- a/src/java/org/apache/cassandra/service/ClientState.java
+++ b/src/java/org/apache/cassandra/service/ClientState.java
@@ -21,6 +21,7 @@ import java.net.InetSocketAddress;
 import java.net.SocketAddress;
 import java.util.Arrays;
 import java.util.HashSet;
+import java.util.Optional;
 import java.util.Set;
 import java.util.concurrent.atomic.AtomicLong;
 
@@ -106,6 +107,10 @@ public class ClientState
 // The remote address of the client - null for internal clients.
 private final InetSocketAddress remoteAddress;
 
+// Driver String for the client
+private volatile String driverName;
+private volatile String driverVersion;
+
 // The biggest timestamp that was returned by getTimestamp/assigned to a 
query. This is global to ensure that the
 // timestamp assigned are strictly monotonic on a node, which is likely 
what user expect intuitively (more likely,
 // most new user will intuitively expect timestamp to be strictly 
monotonic cluster-wise, but while that last part
@@ -135,6 +140,8 @@ public class ClientState
 this.remoteAddress = source.remoteAddress;
 this.user = source.user;
 this.keyspace = source.keyspace;
+this.driverName = source.driverName;
+this.driverVersion = source.driverVersion;
 }
 
 /**
@@ -243,6 +250,26 @@ public class ClientState
 }
 }
 
+public Optional getDriverName()
+{
+return Optional.ofNullable(driverName);
+}
+
+public Optional getDriverVersion()
+{
+return Optional.ofNullable(driverVersion);
+}
+
+public void setDriverName(String driverName)
+{
+this.driverName = driverName;
+}
+
+public void setDriverVersion(String driverVersion)
+{
+this.driverVersion = driverVersion;
+}
+
 public static QueryHandler getCQLQueryHandler()
 {
 return cqlQueryHandler;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5dc55e71/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java 
b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
index 21915cb..4529c86 100644
--- a/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
+++ b/src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
@@ -43,11 +43,11 @@ public class ClientStats extends NodeToolCmd
 if (!clients.isEmpty())
 {
 TableBuilder table = new TableBuilder();
-table.add("Address", "SSL", "Version", "User", "Keyspace", 
"Requests");
+table.add("Address", "SSL", "Version", "User", "Keyspace", 
"Requests", "Driver-Name", "Driver-Version");
 for (Map conn :

[jira] [Commented] (CASSANDRA-14322) Cassandra NodeTool clientstats should show SSL Cipher

2018-03-19 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404713#comment-16404713
 ] 

Jason Brown commented on CASSANDRA-14322:
-

two minor points:

- I'm not a fan of printing {{"undefined"}} when the connection is not using 
ssl. Maybe {{"--"}} is better? 
- As long as we're adding the cipher suite, should we print out the protocol, 
as well?


> Cassandra NodeTool clientstats should show SSL Cipher
> -
>
> Key: CASSANDRA-14322
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14322
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
>
> Currently nodetool prints out some information that does not include the SSL 
> Cipher being used by the client. It would be helpful to add this in for 
> better visibility.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14275) Cassandra Driver should send identification information to Server

2018-03-19 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404686#comment-16404686
 ] 

Aleksey Yeschenko commented on CASSANDRA-14275:
---

That's almost it, thanks.

You shouldn't have {{Optional}} as a field, though - the proper way is to 
create them in getters, with {{Optional.ofNullable()}}. And I'd move that 
conditional logic to {{StartupMessage}} itself, to keep {{ClientState}} dumb.

Made these small adjustments myself 
[here|https://github.com/iamaleksey/cassandra/commit/0df1039f0c5f5260534974891c33711bf956c34a]
 so that we can save one trivial round-trip. Will commit as soon as CI is green.

> Cassandra Driver should send identification information to Server
> -
>
> Key: CASSANDRA-14275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14275
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Fix For: 4.x
>
>
> Currently there doesn't seem to be any way to readily identify the driver 
> that clients are using to connect to Cassandra. Add the capability of 
> identifying the driver through metadata information much like how HTTP 
> Clients identify themselves through User-Agent HTTP header. This is useful 
> for debugging in large deployments where clients tend to use different 
> drivers, wrappers and language bindings to connect to Cassandra. This can 
> help surface issues as well as help detect clients which are using older or 
> unsupported drivers.
> The identification information should be a string that unambiguously 
> identifies the driver. It should include information such as the name of the 
> driver, it's version, CQL version, Platform (Linux, macOS, Windows, etc.) and 
> architecture (x86, x86_64).
> We should surface this information in `nodetool clientstats` command.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14315) ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only partition level info

2018-03-19 Thread ZhaoYang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-14315:
-
Attachment: unit test.png
dtest.png

> ThrottledUnfilteredIterator failed on UnfilteredRowIterator with only 
> partition level info
> --
>
> Key: CASSANDRA-14315
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14315
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Major
> Fix For: 4.0
>
> Attachments: dtest.png, unit test.png
>
>
> When repairing base table with MV, in order to avoid OOM, Cassandra-13299 
> added ThrottledUnfilteredIterator to split large partition into small chunks, 
> but it didn't handle partition without unfiltered properly.
> {code:title=repro}
> // create cell tombstone, range tombstone, partition deletion
> createTable("CREATE TABLE %s (pk int, ck1 int, ck2 int, v1 int, v2 int, 
> PRIMARY KEY (pk, ck1, ck2))");
> // partition deletion
> execute("DELETE FROM %s USING TIMESTAMP 160 WHERE pk=1");
> // flush and generate 1 sstable
> ColumnFamilyStore cfs = 
> Keyspace.open(keyspace()).getColumnFamilyStore(currentTable());
> cfs.forceBlockingFlush();
> cfs.disableAutoCompaction();
> cfs.forceMajorCompaction();
> assertEquals(1, cfs.getLiveSSTables().size());
> SSTableReader reader = cfs.getLiveSSTables().iterator().next();
> try (ISSTableScanner scanner = reader.getScanner();
> CloseableIterator throttled = 
> ThrottledUnfilteredIterator.throttle(scanner, 100))
> {
> assertTrue(throttled.hasNext());
> UnfilteredRowIterator iterator = throttled.next();
> assertFalse(throttled.hasNext());
> assertFalse(iterator.hasNext());
> assertEquals(iterator.partitionLevelDeletion().markedForDeleteAt(), 160);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14286:
---
Since Version: 2.2.0

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14245) SELECT JSON prints null on empty strings

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14245:
---
Since Version: 3.11.0  (was: 2.2.0)

> SELECT JSON prints null on empty strings
> 
>
> Key: CASSANDRA-14245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14245
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS
>  
>Reporter: Norbert Schultz
>Assignee: Francisco Fernandez
>Priority: Major
>
> SELECT JSON reports an empty string as null.
>  
> Example:
> {code:java}
> cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id));
> cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo');
> cqlsh:unittest> insert into test (id, name) VALUES (2, '');
> cqlsh:unittest> insert into test (id, name) VALUES (3, null);
> cqlsh:unittest> select * from test;
> id | name
> +--
>   1 |  Foo
>   2 |     
>   3 | null
> (3 rows)
> cqlsh:unittest> select JSON * from test;
> [json]
> --
> {"id": 1, "name": "Foo"}
> {"id": 2, "name": null}
> {"id": 3, "name": null}
> (3 rows){code}
>  
> This even happens, if the string is part of the Primary Key, which makes the 
> generated string not insertable.
>  
> {code:java}
> cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY 
> KEY(id, name));
> cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42);
> cqlsh:unittest> select JSON * from test2;
> [json]
> 
> {"id": 1, "name": null, "age": 42}
> (1 rows)
> cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
> null value in condition for column name"{code}
>  
> On an older version of Cassandra (3.0.8) does not have this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14286:
---
Fix Version/s: (was: 3.11.x)

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14245) SELECT JSON prints null on empty strings

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14245:
---
Since Version: 2.2.0

> SELECT JSON prints null on empty strings
> 
>
> Key: CASSANDRA-14245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14245
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS
>  
>Reporter: Norbert Schultz
>Assignee: Francisco Fernandez
>Priority: Major
>
> SELECT JSON reports an empty string as null.
>  
> Example:
> {code:java}
> cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id));
> cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo');
> cqlsh:unittest> insert into test (id, name) VALUES (2, '');
> cqlsh:unittest> insert into test (id, name) VALUES (3, null);
> cqlsh:unittest> select * from test;
> id | name
> +--
>   1 |  Foo
>   2 |     
>   3 | null
> (3 rows)
> cqlsh:unittest> select JSON * from test;
> [json]
> --
> {"id": 1, "name": "Foo"}
> {"id": 2, "name": null}
> {"id": 3, "name": null}
> (3 rows){code}
>  
> This even happens, if the string is part of the Primary Key, which makes the 
> generated string not insertable.
>  
> {code:java}
> cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY 
> KEY(id, name));
> cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42);
> cqlsh:unittest> select JSON * from test2;
> [json]
> 
> {"id": 1, "name": null, "age": 42}
> (1 rows)
> cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
> null value in condition for column name"{code}
>  
> On an older version of Cassandra (3.0.8) does not have this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14245) SELECT JSON prints null on empty strings

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14245:
---
Reviewer: Benjamin Lerer

> SELECT JSON prints null on empty strings
> 
>
> Key: CASSANDRA-14245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14245
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS
>  
>Reporter: Norbert Schultz
>Assignee: Francisco Fernandez
>Priority: Major
>
> SELECT JSON reports an empty string as null.
>  
> Example:
> {code:java}
> cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id));
> cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo');
> cqlsh:unittest> insert into test (id, name) VALUES (2, '');
> cqlsh:unittest> insert into test (id, name) VALUES (3, null);
> cqlsh:unittest> select * from test;
> id | name
> +--
>   1 |  Foo
>   2 |     
>   3 | null
> (3 rows)
> cqlsh:unittest> select JSON * from test;
> [json]
> --
> {"id": 1, "name": "Foo"}
> {"id": 2, "name": null}
> {"id": 3, "name": null}
> (3 rows){code}
>  
> This even happens, if the string is part of the Primary Key, which makes the 
> generated string not insertable.
>  
> {code:java}
> cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY 
> KEY(id, name));
> cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42);
> cqlsh:unittest> select JSON * from test2;
> [json]
> 
> {"id": 1, "name": null, "age": 42}
> (1 rows)
> cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
> null value in condition for column name"{code}
>  
> On an older version of Cassandra (3.0.8) does not have this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14245) SELECT JSON prints null on empty strings

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-14245:
--

Assignee: Francisco Fernandez  (was: Benjamin Lerer)

> SELECT JSON prints null on empty strings
> 
>
> Key: CASSANDRA-14245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14245
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS
>  
>Reporter: Norbert Schultz
>Assignee: Francisco Fernandez
>Priority: Major
>
> SELECT JSON reports an empty string as null.
>  
> Example:
> {code:java}
> cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id));
> cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo');
> cqlsh:unittest> insert into test (id, name) VALUES (2, '');
> cqlsh:unittest> insert into test (id, name) VALUES (3, null);
> cqlsh:unittest> select * from test;
> id | name
> +--
>   1 |  Foo
>   2 |     
>   3 | null
> (3 rows)
> cqlsh:unittest> select JSON * from test;
> [json]
> --
> {"id": 1, "name": "Foo"}
> {"id": 2, "name": null}
> {"id": 3, "name": null}
> (3 rows){code}
>  
> This even happens, if the string is part of the Primary Key, which makes the 
> generated string not insertable.
>  
> {code:java}
> cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY 
> KEY(id, name));
> cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42);
> cqlsh:unittest> select JSON * from test2;
> [json]
> 
> {"id": 1, "name": null, "age": 42}
> (1 rows)
> cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
> null value in condition for column name"{code}
>  
> On an older version of Cassandra (3.0.8) does not have this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14286:
---
Reviewer: Benjamin Lerer

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Fix For: 3.11.x
>
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

2018-03-19 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-14286:
--

Assignee: Francisco Fernandez  (was: Benjamin Lerer)

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Fix For: 3.11.x
>
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-14324) Cassandra front page links to planetcassandra.org

2018-03-19 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hannu Kröger resolved CASSANDRA-14324.
--
Resolution: Duplicate

> Cassandra front page links to planetcassandra.org
> -
>
> Key: CASSANDRA-14324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14324
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation and Website
>Reporter: Hannu Kröger
>Priority: Minor
>
> Under header "Proven" following links point to non-existing 
> planetcassandra.org:
>  * CERN
>  * Github
>  * GoDaddy
>  * Hulu
>  * Instagram
>  * Reddit
>  * The Weather Channel
>  * over 1500 more companies



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14324) Cassandra front page links to planetcassandra.org

2018-03-19 Thread JIRA

Hannu Kröger created CASSANDRA-14324:


 Summary: Cassandra front page links to planetcassandra.org
 Key: CASSANDRA-14324
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14324
 Project: Cassandra
  Issue Type: Task
  Components: Documentation and Website
Reporter: Hannu Kröger


Under header "Proven" following links point to non-existing planetcassandra.org:
 * CERN
 * Github
 * GoDaddy
 * Hulu
 * Instagram
 * Reddit
 * The Weather Channel
 * over 1500 more companies



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14323) Same timestamp insert conflict resolution breaks row-level data consistency

2018-03-19 Thread Rishi Kathera (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rishi Kathera updated CASSANDRA-14323:
--
Description: 
When inserting multiple rows with the same primary key and timestamp, memtable 
update logic does not maintain row-level consistency for the key inserted. For 
example,
{code:java}
create table test.consistency(pk int PRIMARY KEY , nk1 text, nk2 text);
BEGIN UNLOGGED BATCH USING TIMESTAMP 1521080773000 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk1','nk2'); 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk2','nk1'); 
APPLY BATCH; 
select * from test.consistency;
{code}
In this case, I would expect either one row overwrites the other so the result 
of the read would be either
{code:java}
2, nk1, nk2{code}
or
{code:java}
2, nk2, nk1{code}
but the row retrieved is
{code:java}
2, nk2, nk2{code}
 which breaks consistency of the writes. This behavior comes from this logic, 

[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/Conflicts.java#L45]

where it appears that the value of the cell itself is used to resolve overwrite 
conflict which I don't think is the correct way of handling the situation. 
Shouldn't it either be overwrite or not overwrite for all cases?

  was:
When inserting multiple rows with the same primary key and timestamp, memtable 
update logic does not maintain row-level consistency for the key inserted. For 
example,
{code:java}
create table test.consistency(pk int PRIMARY KEY , nk1 text, nk2 text);
BEGIN UNLOGGED BATCH USING TIMESTAMP 1521080773000 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk1','nk2'); 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk2','nk1'); 
APPLY BATCH; 
select * from test.consistency;
{code}
In this case, I would expect either one row overwrites the other so the result 
of the read would be either
{code:java}
2, nk1, nk2{code}
or
{code:java}
2, nk2, nk1{code}
but the row retrieved is
{code:java}
2, nk2, nk2{code}
 which breaks consistency of the writes. This behavior comes from this logic, 

[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/Conflicts.java#L45]

where it appears that the value of the cell itself if used to resolve overwrite 
conflict which I don't think is the correct way of handling the situation. 
Shouldn't it either be overwrite or not overwrite for all cases?


> Same timestamp insert conflict resolution breaks row-level data consistency
> ---
>
> Key: CASSANDRA-14323
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14323
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Rishi Kathera
>Priority: Minor
>
> When inserting multiple rows with the same primary key and timestamp, 
> memtable update logic does not maintain row-level consistency for the key 
> inserted. For example,
> {code:java}
> create table test.consistency(pk int PRIMARY KEY , nk1 text, nk2 text);
> BEGIN UNLOGGED BATCH USING TIMESTAMP 1521080773000 
> insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk1','nk2'); 
> insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk2','nk1'); 
> APPLY BATCH; 
> select * from test.consistency;
> {code}
> In this case, I would expect either one row overwrites the other so the 
> result of the read would be either
> {code:java}
> 2, nk1, nk2{code}
> or
> {code:java}
> 2, nk2, nk1{code}
> but the row retrieved is
> {code:java}
> 2, nk2, nk2{code}
>  which breaks consistency of the writes. This behavior comes from this logic, 
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/Conflicts.java#L45]
> where it appears that the value of the cell itself is used to resolve 
> overwrite conflict which I don't think is the correct way of handling the 
> situation. Shouldn't it either be overwrite or not overwrite for all cases?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14323) Same timestamp insert conflict resolution breaks row-level data consistency

2018-03-19 Thread Rishi Kathera (JIRA)

Rishi Kathera created CASSANDRA-14323:
-

 Summary: Same timestamp insert conflict resolution breaks 
row-level data consistency
 Key: CASSANDRA-14323
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14323
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Rishi Kathera


When inserting multiple rows with the same primary key and timestamp, memtable 
update logic does not maintain row-level consistency for the key inserted. For 
example,
{code:java}
create table test.consistency(pk int PRIMARY KEY , nk1 text, nk2 text);
BEGIN UNLOGGED BATCH USING TIMESTAMP 1521080773000 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk1','nk2'); 
insert into test.consistency (pk,nk1,nk2) VALUES (2,'nk2','nk1'); 
APPLY BATCH; 
select * from test.consistency;
{code}
In this case, I would expect either one row overwrites the other so the result 
of the read would be either
{code:java}
2, nk1, nk2{code}
or
{code:java}
2, nk2, nk1{code}
but the row retrieved is
{code:java}
2, nk2, nk2{code}
 which breaks consistency of the writes. This behavior comes from this logic, 

[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/Conflicts.java#L45]

where it appears that the value of the cell itself if used to resolve overwrite 
conflict which I don't think is the correct way of handling the situation. 
Shouldn't it either be overwrite or not overwrite for all cases?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

78 matches

Mail list logo