[jira] [Commented] (CASSANDRA-14336) sstableloader fails if sstables contains removed columns

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410837#comment-16410837
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-14336:
---

Hi [~hkroger]

To me this looks like an expected behavior. If you try to load old SSTables 
with column {{d}} to destination table {{bug3_dest}} which doesn't have column 
{{d}} in its schema then SSTable loader is supposed to fail complaining schema 
mismatch and it does exactly the same thing above.

If you change your script as following:

{quote}
cqlsh -e "ALTER TABLE $KS.$SRCTABLE DROP d;"
cqlsh -e "INSERT INTO $KS.$SRCTABLE(a,b,c) values(5,6,7);"
nodetool flush $KS $SRCTABLE <--- new SSTables
{quote}

and after this if you try {{sstableloader}} with new SSTables then it works.

Can you please clarify what you think should be the right outcome?

Jaydeep


> sstableloader fails if sstables contains removed columns
> 
>
> Key: CASSANDRA-14336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14336
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Jaydeepkumar Chovatia
>Priority: Major
>
> If I copy the schema and try to load in sstables with sstableloader, loading 
> sometimes fails with
> {code:java}
> Exception in thread "main" org.apache.cassandra.tools.BulkLoadException: 
> java.lang.RuntimeException: Failed to list files in /tmp/test/bug3_dest-acdc
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:93)
>     at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:48)
> Caused by: java.lang.RuntimeException: Failed to list files in 
> /tmp/test/bug3_dest-acdc
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
>     at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:561)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:80)
>     ... 1 more
> Caused by: java.lang.RuntimeException: Unknown column d during deserialization
>     at 
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:321)
>     at 
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:440)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$0(SSTableLoader.java:121)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$2(LogAwareFileLister.java:99)
>     at 
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
>     at java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2969)
>     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>     at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>     at 
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:101)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:73)
>     ... 5 more{code}
> This requires that we have dropped columns in the source table and sstables 
> exist from the "old schema" time.
> This can be very easily reproduced. I used following script:
> {code:java}
> KS=test
> SRCTABLE=bug3_source
> DESTTABLE=bug3_dest
> DATADIR=/usr/local/var/lib/cassandra/data
> TMPDIR=/tmp
> cqlsh -e "CREATE TABLE $KS.$SRCTABLE(a int primary key, b int, c int, d int);"
> cqlsh -e "CREATE TABLE $KS.$DESTTABLE(a int primary key, b int, c int);"
> cqlsh -e "INSERT INTO $KS.$SRCTABLE(a,b,c,d) values(1,2,3,4);"
> nodetool flush $KS $SRCTABLE
> cqlsh -e "ALTER TABLE $KS.$SRCTABLE DROP d;"
> nodetool flush $KS $SRCTABLE
> mkdir -p $TMPDIR/$KS/$DESTTABLE-acdc
> cp $DATADIR/$KS/$SRCTABLE-*/* $TMPDIR/$KS/$DESTTABLE-acdc
> sstableloader -d 127.0.0.1 $TMPDIR/$KS/$DESTTABLE-acdc{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14336) sstableloader fails if sstables contains removed columns

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410826#comment-16410826
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-14336:
---

Hi [~hkroger]

Let me take a look at this and then I will get back to you.

Jaydeep

> sstableloader fails if sstables contains removed columns
> 
>
> Key: CASSANDRA-14336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14336
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Jaydeepkumar Chovatia
>Priority: Major
>
> If I copy the schema and try to load in sstables with sstableloader, loading 
> sometimes fails with
> {code:java}
> Exception in thread "main" org.apache.cassandra.tools.BulkLoadException: 
> java.lang.RuntimeException: Failed to list files in /tmp/test/bug3_dest-acdc
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:93)
>     at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:48)
> Caused by: java.lang.RuntimeException: Failed to list files in 
> /tmp/test/bug3_dest-acdc
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
>     at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:561)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:80)
>     ... 1 more
> Caused by: java.lang.RuntimeException: Unknown column d during deserialization
>     at 
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:321)
>     at 
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:440)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$0(SSTableLoader.java:121)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$2(LogAwareFileLister.java:99)
>     at 
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
>     at java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2969)
>     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>     at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>     at 
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:101)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:73)
>     ... 5 more{code}
> This requires that we have dropped columns in the source table and sstables 
> exist from the "old schema" time.
> This can be very easily reproduced. I used following script:
> {code:java}
> KS=test
> SRCTABLE=bug3_source
> DESTTABLE=bug3_dest
> DATADIR=/usr/local/var/lib/cassandra/data
> TMPDIR=/tmp
> cqlsh -e "CREATE TABLE $KS.$SRCTABLE(a int primary key, b int, c int, d int);"
> cqlsh -e "CREATE TABLE $KS.$DESTTABLE(a int primary key, b int, c int);"
> cqlsh -e "INSERT INTO $KS.$SRCTABLE(a,b,c,d) values(1,2,3,4);"
> nodetool flush $KS $SRCTABLE
> cqlsh -e "ALTER TABLE $KS.$SRCTABLE DROP d;"
> nodetool flush $KS $SRCTABLE
> mkdir -p $TMPDIR/$KS/$DESTTABLE-acdc
> cp $DATADIR/$KS/$SRCTABLE-*/* $TMPDIR/$KS/$DESTTABLE-acdc
> sstableloader -d 127.0.0.1 $TMPDIR/$KS/$DESTTABLE-acdc{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14336) sstableloader fails if sstables contains removed columns

[Jaydeepkumar Chovatia (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaydeepkumar Chovatia reassigned CASSANDRA-14336:
-

Assignee: Jaydeepkumar Chovatia

> sstableloader fails if sstables contains removed columns
> 
>
> Key: CASSANDRA-14336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14336
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Hannu Kröger
>Assignee: Jaydeepkumar Chovatia
>Priority: Major
>
> If I copy the schema and try to load in sstables with sstableloader, loading 
> sometimes fails with
> {code:java}
> Exception in thread "main" org.apache.cassandra.tools.BulkLoadException: 
> java.lang.RuntimeException: Failed to list files in /tmp/test/bug3_dest-acdc
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:93)
>     at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:48)
> Caused by: java.lang.RuntimeException: Failed to list files in 
> /tmp/test/bug3_dest-acdc
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
>     at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:561)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
>     at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:80)
>     ... 1 more
> Caused by: java.lang.RuntimeException: Unknown column d during deserialization
>     at 
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:321)
>     at 
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:440)
>     at 
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$0(SSTableLoader.java:121)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$2(LogAwareFileLister.java:99)
>     at 
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
>     at java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2969)
>     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>     at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>     at 
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:101)
>     at 
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:73)
>     ... 5 more{code}
> This requires that we have dropped columns in the source table and sstables 
> exist from the "old schema" time.
> This can be very easily reproduced. I used following script:
> {code:java}
> KS=test
> SRCTABLE=bug3_source
> DESTTABLE=bug3_dest
> DATADIR=/usr/local/var/lib/cassandra/data
> TMPDIR=/tmp
> cqlsh -e "CREATE TABLE $KS.$SRCTABLE(a int primary key, b int, c int, d int);"
> cqlsh -e "CREATE TABLE $KS.$DESTTABLE(a int primary key, b int, c int);"
> cqlsh -e "INSERT INTO $KS.$SRCTABLE(a,b,c,d) values(1,2,3,4);"
> nodetool flush $KS $SRCTABLE
> cqlsh -e "ALTER TABLE $KS.$SRCTABLE DROP d;"
> nodetool flush $KS $SRCTABLE
> mkdir -p $TMPDIR/$KS/$DESTTABLE-acdc
> cp $DATADIR/$KS/$SRCTABLE-*/* $TMPDIR/$KS/$DESTTABLE-acdc
> sstableloader -d 127.0.0.1 $TMPDIR/$KS/$DESTTABLE-acdc{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13889) cfstats should take sorting and limit parameters

[Jon Haddad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410701#comment-16410701
 ] 

Jon Haddad commented on CASSANDRA-13889:


Hey [~ptbannister], yes, I'm thinking we keep the existing output and use those 
options to limit to top K with {{--sort}}.  

I agree with you, defaulting to descending is the most reasonable approach.  
{{--ascending}} is useful to find tables that aren't used often or at all, but 
we can punt that to a follow up ticket..  Let's also wait on doing the more 
complex sort, as it wouldn't change the command - we'd maybe want to see 
{{--sort reads,writes}}, but I'm having a hard time imagining very many use 
cases for that either.  

Regarding keyspace metrics, I'm OK with omitting them as they don't make any 
sense anymore.  It changes cfstats from a keyspace centric report to a table 
one. 

> cfstats should take sorting and limit parameters
> 
>
> Key: CASSANDRA-13889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13889
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jon Haddad
>Priority: Major
> Fix For: 4.0
>
>
> When looking at a problematic node I'm not familiar with, one of the first 
> things I do is check cfstats to identify the tables with the most reads, 
> writes, and data.  This is fine as long as there aren't a lot of tables but 
> once it goes above a dozen it's quite difficult.  cfstats should allow me to 
> sort the results and limit to top K tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Joseph Lynch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410693#comment-16410693
 ] 

Joseph Lynch commented on CASSANDRA-12151:
--

I took a look at the patch and it looks good to me. Comments left on the PR, 
but I like that the interface is pluggable and future implementations can do 
what they like (percentage based, remote/network logs, different output 
formats, etc).

Should we do some dtests and ccm multi node testing locally?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13889) cfstats should take sorting and limit parameters

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410632#comment-16410632
 ] 

Patrick Bannister commented on CASSANDRA-13889:
---

Jon, are you thinking that the output would remain in one of its current output 
forms - text, json, or yaml - but sorted and truncated according to the options 
provided by the user?

To sort, the user would have to provide a command line argument indicating 
which metric is the sorting key - something like --sort "read count".

I think sorting would default to descending, with an optional argument to 
indicate an ascending sort (probably --ascending). Actually it's not clear to 
me that any of these metrics would be interesting sorted ascending, except 
maybe compression ratio, but I'd defer on that question to somebody with more 
experience troubleshooting production clusters, and in any case the 
implementation of an optional ascending sort would not be difficult

Is there a compelling use case for a complex sort, for example, sort first by 
write latency descending, then by write count descending, then by memtable off 
heap memory used descending?

If a user wants the top K tables, that implies they must also specify a sort 
key.

Currently we output some overall metrics for keyspaces before outputting 
table-specific metrics for that keyspace. If a user requests sorting or top K 
tables, with more than one keyspace or without specifying anything, what do we 
do with the keyspace metrics? Would we just omit the keyspace metrics from the 
output?

> cfstats should take sorting and limit parameters
> 
>
> Key: CASSANDRA-13889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13889
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jon Haddad
>Priority: Major
> Fix For: 4.0
>
>
> When looking at a problematic node I'm not familiar with, one of the first 
> things I do is check cfstats to identify the tables with the most reads, 
> writes, and data.  This is fine as long as there aren't a lot of tables but 
> once it goes above a dozen it's quite difficult.  cfstats should allow me to 
> sort the results and limit to top K tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-7622) Implement virtual tables

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-7622:
--
Component/s: CQL

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-7622) Implement virtual tables

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-7622:
--
Reviewer: Dinesh Joshi

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410535#comment-16410535
 ] 

sankalp kohli commented on CASSANDRA-7622:
--

[~djoshi3] Can do it [~jjirsa]

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-7622) Implement virtual tables

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410528#comment-16410528
 ] 

Jeff Jirsa commented on CASSANDRA-7622:
---

[~spo...@gmail.com] any chance you're interested in reviewing?

 

(Failing that, [~jasobrown] , I know your queue is huge, but interested in 
throwing this onto the tail of it ?)

> Implement virtual tables
> 
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any 
> table that would be backed by an API, rather than data explicitly managed and 
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a 
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml 
> configuration information. So it would be an alternate approach to 
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not 
> presupposing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14293:
--
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Patch Available)

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.0
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410212#comment-16410212
 ] 

Aleksey Yeschenko commented on CASSANDRA-14293:
---

Committed to trunk as {{d6a67598f196214bc30d22e1b6b4e3586fe6f057}}, cheers.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Add support for hybrid MIN(), MAX() speculative retry policies

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 11496039f -> d6a67598f


Add support for hybrid MIN(), MAX() speculative retry policies

patch by Michael Kjellman; reviewed by Aleksey Yeschenko for
CASSANDRA-14293


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d6a67598
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d6a67598
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d6a67598

Branch: refs/heads/trunk
Commit: d6a67598f196214bc30d22e1b6b4e3586fe6f057
Parents: 1149603
Author: Michael Kjellman 
Authored: Tue Mar 6 22:05:26 2018 -0800
Committer: Aleksey Yeshchenko 
Committed: Thu Mar 22 20:07:05 2018 +

--
 CHANGES.txt |   1 +
 doc/source/cql/ddl.rst  |  39 +++--
 .../cql3/statements/TableAttributes.java|   3 +-
 .../apache/cassandra/db/ColumnFamilyStore.java  |  41 ++---
 .../apache/cassandra/schema/SchemaKeyspace.java |   4 +-
 .../cassandra/schema/SpeculativeRetryParam.java | 164 ---
 .../apache/cassandra/schema/TableMetadata.java  |   3 +-
 .../apache/cassandra/schema/TableParams.java|   7 +-
 .../service/reads/AbstractReadExecutor.java |   7 +-
 .../reads/AlwaysSpeculativeRetryPolicy.java |  68 
 .../reads/FixedSpeculativeRetryPolicy.java  |  74 +
 .../reads/HybridSpeculativeRetryPolicy.java |  90 ++
 .../reads/NeverSpeculativeRetryPolicy.java  |  68 
 .../reads/PercentileSpeculativeRetryPolicy.java |  72 
 .../service/reads/SpeculativeRetryPolicy.java   | 150 +
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   3 +-
 .../schema/SpeculativeRetryParamParseTest.java  | 104 
 .../reads/SpeculativeRetryParseTest.java| 143 
 18 files changed, 728 insertions(+), 313 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d6a67598/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f86a380..ac5269c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Add support for hybrid MIN(), MAX() speculative retry policies 
(CASSANDRA-14293)
  * Correct and clarify SSLFactory.getSslContext method and call sites 
(CASSANDRA-14314)
  * Use zero as default score in DynamicEndpointSnitch (CASSANDRA-14252)
  * Handle static and partition deletion properly on 
ThrottledUnfilteredIterator (CASSANDRA-14315)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d6a67598/doc/source/cql/ddl.rst
--
diff --git a/doc/source/cql/ddl.rst b/doc/source/cql/ddl.rst
index dfed687..be5e2c6 100644
--- a/doc/source/cql/ddl.rst
+++ b/doc/source/cql/ddl.rst
@@ -495,19 +495,32 @@ consistency levels: one for consistency level ``ONE``, a 
quorum for ``QUORUM``,
 ``speculative_retry`` determines when coordinators may query additional 
replicas, which is useful
 when replicas are slow or unresponsive.  The following are legal values 
(case-insensitive):
 
-=  
=
- FormatExample  Description
-=  
=
- ``XPERCENTILE``   90.5PERCENTILE   Coordinators record average 
per-table response times for all replicas.
-If a replica takes longer than 
``X`` percent of this table's average
-response time, the coordinator 
queries an additional replica.
-``X`` must be between 0 and 100.
- ``XP``90.5PSynonym for ``XPERCENTILE``
- ``Yms``   25ms If a replica takes more than ``Y`` 
milliseconds to respond,
-the coordinator queries an 
additional replica.
- ``ALWAYS`` Coordinators always query all 
replicas.
- ``NONE``   Coordinators never query 
additional replicas.
-=  
=
+  
=
+ Format   Example  Description
+  
=
+ ``XPERCENTILE`` 

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410167#comment-16410167
 ] 

Michael Kjellman commented on CASSANDRA-14293:
--

changes look fine to me. let's get this committed on to trunk to start with so 
we can start working on a 3.0 version...

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14333) nodetool repair causes out of memory issues

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410115#comment-16410115
 ] 

Vinay Chella commented on CASSANDRA-14333:
--

Thanks for sharing JVM settings. Can you provide the following

{quote}C* logs from the repair coordinator node and its peers, schema of the 
tables that you are trying to repair. 
 {quote}


> nodetool repair causes out of  memory issues 
> -
>
> Key: CASSANDRA-14333
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14333
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: chinta kiran
>Priority: Major
> Fix For: 3.9
>
> Attachments: cassandra-env.sh, cassandra.yaml
>
>
> When ever we try to run nodetool repair we are getting OOM ] . We have tried 
> all types of nodetool repair, full,incremental range repair. all the times it 
> is failing . how to overcome this issue. We have 32GB Ram.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13529) cassandra-stress light-weight transaction support

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409904#comment-16409904
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13529:
---

 
Hi [~djoshi3]

I've incorporated your review comments, only thing I've done bit different is 
moved factory inside {{StressProfile::getQuery}} class instead of 
{{SettingsCommandUser::getFactory}} to avoid code duplication as {{CASQuery}} 
and {{SchemaQuery}} has quite a few stuff in common, please find updated patch 
details here:
||trunk||
|[utest |https://circleci.com/gh/jaydeepkumar1984/cassandra/55]|
|[patch 
|https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:CASSANDRA-13529?expand=1]|

Jaydeep

> cassandra-stress light-weight transaction support
> -
>
> Key: CASSANDRA-13529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13529
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Stress
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 13529.txt, lwttest.yaml
>
>
> It would be nice to have a light-weight transaction support in 
> cassandra-stress.
> Although currently in cassandra-stress we can achieve light-weight 
> transaction partially by using static conditions like "IF col1 != null" or 
> "IF not EXIST". 
> If would be ideal to have full fledged light-weight transaction support like 
> "IF col1 = ? and col2 = ?". One way to implement is to read values from 
> Cassandra and use that in the condition so it will execute all the paxos 
> phases in Cassandra.
> Please find git link for the patch: 
> https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13529-trunk?expand=1
> ||trunk|
> |[branch|https://github.com/jaydeepkumar1984/cassandra/tree/13529-trunk]|
> |[utests|https://circleci.com/gh/jaydeepkumar1984/cassandra/8]|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409892#comment-16409892
 ] 

Aleksey Yeschenko commented on CASSANDRA-14293:
---

And then another one to change the default toString() of percentile policy from 
the incredibly loud and obnoxious {{PERCENTILE}} to more common and reasonable 
{{p}}, to make it consistent with {{ms}} and just not loud.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13823) The Getting Started page should have instructions on setting up a cluster

[Jon Haddad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad reassigned CASSANDRA-13823:
--

Assignee: Sumanth Pasupuleti

> The Getting Started page should have instructions on setting up a cluster
> -
>
> Key: CASSANDRA-13823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13823
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sumanth Pasupuleti
>Priority: Minor
>  Labels: lhf
>
> Currently the docs don't have an easy to follow guide on setting up a 
> cluster.  I think it would benefit from a nice easy to follow walkthrough.
> https://cassandra.apache.org/doc/latest/getting_started/index.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13749) add documentation about upgrade process to docs

[Jon Haddad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad reassigned CASSANDRA-13749:
--

Assignee: Sumanth Pasupuleti

> add documentation about upgrade process to docs
> ---
>
> Key: CASSANDRA-13749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13749
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sumanth Pasupuleti
>Priority: Major
>  Labels: documentation
>
> The docs don't have any information on how to upgrade.  This question gets 
> asked constantly on the mailing list.
> Seems like it belongs under the "Operating Cassandra" section.
> https://cassandra.apache.org/doc/latest/operating/index.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409879#comment-16409879
 ] 

Aleksey Yeschenko commented on CASSANDRA-14293:
---

Pushed some minor changes (addressing my review feedback) 
[here|https://github.com/iamaleksey/cassandra/tree/14293-4.0]. Lmk if you are 
cool with these and if so I'll commit.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14293:
--
Status: Patch Available  (was: Open)

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13823) The Getting Started page should have instructions on setting up a cluster

[Sumanth Pasupuleti (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409874#comment-16409874
 ] 

Sumanth Pasupuleti commented on CASSANDRA-13823:


I would like to contribute to this JIRA. Could you please assign this to me?

> The Getting Started page should have instructions on setting up a cluster
> -
>
> Key: CASSANDRA-13823
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13823
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Priority: Minor
>  Labels: lhf
>
> Currently the docs don't have an easy to follow guide on setting up a 
> cluster.  I think it would benefit from a nice easy to follow walkthrough.
> https://cassandra.apache.org/doc/latest/getting_started/index.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13749) add documentation about upgrade process to docs

[Sumanth Pasupuleti (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409870#comment-16409870
 ] 

Sumanth Pasupuleti commented on CASSANDRA-13749:


[~rustyrazorblade] If this Jira is still up for grabs, I can take it up and 
work on it. Could you please assign this Jira to me?

> add documentation about upgrade process to docs
> ---
>
> Key: CASSANDRA-13749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13749
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Priority: Major
>  Labels: documentation
>
> The docs don't have any information on how to upgrade.  This question gets 
> asked constantly on the mailing list.
> Seems like it belongs under the "Operating Cassandra" section.
> https://cassandra.apache.org/doc/latest/operating/index.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative

[Preetika Tyagi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409851#comment-16409851
 ] 

Preetika Tyagi commented on CASSANDRA-13853:


No, I haven't used DTests before. I will start looking into it and put together 
some tests.

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.x
>
> Attachments: cassandra-13853-v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409813#comment-16409813
 ] 

Aleksey Yeschenko commented on CASSANDRA-14293:
---

The roundtrip tests should be failing but aren't, because the loss of precision 
occurs in the constructor. So we pass bad values to {{assertEquals}} in both 
arguments, and don't notice the loss.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14293:
--
Status: Open  (was: Patch Available)

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409723#comment-16409723
 ] 

Aleksey Yeschenko commented on CASSANDRA-14293:
---

I like this overall. Some issues:

1. {{PercentileSpeculativeRetryPolicy.isDynamic()}} should return {{true}}
2. {{FixedSpeculativeRetryPolicy.calculateThreshold()}} casts the double to 
long before converting to nanos, losing precision - should do the opposite 
(pre-existing problem). But really we've been treating it as an int all this 
time.. might as well recognise it now and ditch the double.
3. {{PercentileSpeculativeRetryPolicy.calculateThreshold()}} method invokes 
{{readLatency.getSnapshot().getValue()}} with threshold divided by another 100, 
making us off by 100. Also the value returned is in nanos already, so we don't 
have to mul by 1k (though this one is on me).
4. Moving to {{BigDecimal}} actually made things lossy quite a bit.. 
{{toString()}} with the patch results in the following now.. 99.5 -> "99.0P", 
99.95 -> "100.0P", 99.99 -> "100.0P"

And a small nit: in {{FixedSpeculativeRetryPolicy.toString()}}, you changed 
what used to be {{ms}}, lower case, to {{MS}}, upper case.

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14293) Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE and FIXED Policies

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14293:
--
Fix Version/s: 4.x

> Speculative Retry Policy Should Support Specifying MIN/MAX of 2 PERCENTILE 
> and FIXED Policies
> -
>
> Key: CASSANDRA-14293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14293
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Major
> Fix For: 4.x
>
>
> Currently the Speculative Retry Policy takes a single string as a parameter, 
> this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
> The problem we have is when a single host goes into a bad state this drags up 
> the percentiles. This means if we are set to use p99 alone, we might not 
> speculate when we intended to to because the value at the specified 
> percentile has gone so high.
> As a fix we need to have support for something like min(99percentile,50ms)
> this means if the normal p99 for the table is <50ms, we will still speculate 
> at this value and not drag the happy path tail latencies up... but if the 
> p99th goes above what we know we should never exceed we use that instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14314) Fix argument passing for SSLContext in trunk

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14314:

   Resolution: Fixed
Fix Version/s: 4.0
   Status: Resolved  (was: Patch Available)

committed, with minor nits addressed, as sha 
{{11496039fb18bb45407246602e31740c56d28157}}. Thanks!

> Fix argument passing for SSLContext in trunk
> 
>
> Key: CASSANDRA-14314
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14314
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
>  Labels: security
> Fix For: 4.0
>
>
> Argument passing has a minor bug while creating the SSLContext. Audit and 
> make sure that the client & server SSL contexts are created at appropriate 
> locations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Correct and clarify SSLFactory.getSslContext method and call sites

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk f109f200a -> 11496039f


Correct and clarify SSLFactory.getSslContext method and call sites

patch by Dinesh Joshi; reviewed by jasobrown for CASSANDRA-14314


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/11496039
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/11496039
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/11496039

Branch: refs/heads/trunk
Commit: 11496039fb18bb45407246602e31740c56d28157
Parents: f109f20
Author: Dinesh A. Joshi 
Authored: Sat Mar 17 17:17:42 2018 -0700
Committer: Jason Brown 
Committed: Thu Mar 22 06:38:56 2018 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/config/DatabaseDescriptor.java|   6 +-
 .../cassandra/config/EncryptionOptions.java |  35 +
 .../apache/cassandra/net/MessagingService.java  |   6 +-
 .../cassandra/net/async/NettyFactory.java   |   6 +-
 .../cassandra/net/async/OptionalSslHandler.java |   2 +-
 .../apache/cassandra/security/SSLFactory.java   | 154 +++
 .../service/NativeTransportService.java |   3 +-
 .../streaming/DefaultConnectionFactory.java |   2 +-
 .../org/apache/cassandra/transport/Server.java  |   7 +-
 .../cassandra/transport/SimpleClient.java   |   5 +-
 .../cassandra/net/MessagingServiceTest.java |   8 +-
 .../async/OutboundMessagingConnectionTest.java  |   7 +-
 .../cassandra/security/SSLFactoryTest.java  |  46 +-
 .../service/NativeTransportServiceTest.java |  13 +-
 15 files changed, 194 insertions(+), 107 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/11496039/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c092a9f..f86a380 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Correct and clarify SSLFactory.getSslContext method and call sites 
(CASSANDRA-14314)
  * Use zero as default score in DynamicEndpointSnitch (CASSANDRA-14252)
  * Handle static and partition deletion properly on 
ThrottledUnfilteredIterator (CASSANDRA-14315)
  * NodeTool clientstats should show SSL Cipher (CASSANDRA-14322)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/11496039/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 2e772c5..bf00d40 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -2005,17 +2005,17 @@ public class DatabaseDescriptor
 conf.dynamic_snitch_badness_threshold = dynamicBadnessThreshold;
 }
 
-public static EncryptionOptions.ServerEncryptionOptions 
getServerEncryptionOptions()
+public static EncryptionOptions.ServerEncryptionOptions 
getInternodeMessagingEncyptionOptions()
 {
 return conf.server_encryption_options;
 }
 
-public static void 
setServerEncryptionOptions(EncryptionOptions.ServerEncryptionOptions 
encryptionOptions)
+public static void 
setInternodeMessagingEncyptionOptions(EncryptionOptions.ServerEncryptionOptions 
encryptionOptions)
 {
 conf.server_encryption_options = encryptionOptions;
 }
 
-public static EncryptionOptions getClientEncryptionOptions()
+public static EncryptionOptions getNativeProtocolEncryptionOptions()
 {
 return conf.client_encryption_options;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/11496039/src/java/org/apache/cassandra/config/EncryptionOptions.java
--
diff --git a/src/java/org/apache/cassandra/config/EncryptionOptions.java 
b/src/java/org/apache/cassandra/config/EncryptionOptions.java
index 5260dff..45579fb 100644
--- a/src/java/org/apache/cassandra/config/EncryptionOptions.java
+++ b/src/java/org/apache/cassandra/config/EncryptionOptions.java
@@ -17,6 +17,9 @@
  */
 package org.apache.cassandra.config;
 
+import java.util.Arrays;
+import java.util.Objects;
+
 public class EncryptionOptions
 {
 public String keystore = "conf/.keystore";
@@ -54,6 +57,38 @@ public class EncryptionOptions
 optional = options.optional;
 }
 
+@Override
+public boolean equals(Object o)
+{
+if (o == this)
+return true;
+if (o == null || getClass() != o.getClass())
+return false;
+
+EncryptionOptions opt = (EncryptionOptions)o;
+return Objects.equals(keystore, opt.keystore) &&
+   

[jira] [Created] (CASSANDRA-14337) cast function is not working in INSERT statement

[Rakesh Roshan (JIRA)]

Rakesh Roshan created CASSANDRA-14337:
-

 Summary: cast function is not working in INSERT statement
 Key: CASSANDRA-14337
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14337
 Project: Cassandra
  Issue Type: Bug
 Environment: cqlsh
Reporter: Rakesh Roshan
 Fix For: 3.11.x


INSERT INTO test.table(last_modified) values(cast(

('2018-03-20 13:05:56.063000+' as timestamp));

 

fails with error 

*no viable alternative at input '('*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14245) SELECT JSON prints null on empty strings

[Francisco Fernandez (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409364#comment-16409364
 ] 

Francisco Fernandez commented on CASSANDRA-14245:
-

The problem is that the json serializer writes empty buffers as null instead of 
"".

[Patch|https://github.com/apache/cassandra/compare/cassandra-3.11...fcofdez:CASSANDRA-14245-3.11?expand=1]

> SELECT JSON prints null on empty strings
> 
>
> Key: CASSANDRA-14245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14245
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS
>  
>Reporter: Norbert Schultz
>Assignee: Francisco Fernandez
>Priority: Major
>
> SELECT JSON reports an empty string as null.
>  
> Example:
> {code:java}
> cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id));
> cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo');
> cqlsh:unittest> insert into test (id, name) VALUES (2, '');
> cqlsh:unittest> insert into test (id, name) VALUES (3, null);
> cqlsh:unittest> select * from test;
> id | name
> +--
>   1 |  Foo
>   2 |     
>   3 | null
> (3 rows)
> cqlsh:unittest> select JSON * from test;
> [json]
> --
> {"id": 1, "name": "Foo"}
> {"id": 2, "name": null}
> {"id": 3, "name": null}
> (3 rows){code}
>  
> This even happens, if the string is part of the Primary Key, which makes the 
> generated string not insertable.
>  
> {code:java}
> cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY 
> KEY(id, name));
> cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42);
> cqlsh:unittest> select JSON * from test2;
> [json]
> 
> {"id": 1, "name": null, "age": 42}
> (1 rows)
> cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid 
> null value in condition for column name"{code}
>  
> On an older version of Cassandra (3.0.8) does not have this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14336) sstableloader fails if sstables contains removed columns

[JIRA]

Hannu Kröger created CASSANDRA-14336:


 Summary: sstableloader fails if sstables contains removed columns
 Key: CASSANDRA-14336
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14336
 Project: Cassandra
  Issue Type: Bug
Reporter: Hannu Kröger


If I copy the schema and try to load in sstables with sstableloader, loading 
sometimes fails with
{code:java}
Exception in thread "main" org.apache.cassandra.tools.BulkLoadException: 
java.lang.RuntimeException: Failed to list files in /tmp/test/bug3_dest-acdc
    at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:93)
    at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:48)
Caused by: java.lang.RuntimeException: Failed to list files in 
/tmp/test/bug3_dest-acdc
    at 
org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
    at 
org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:561)
    at 
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
    at 
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
    at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:80)
    ... 1 more
Caused by: java.lang.RuntimeException: Unknown column d during deserialization
    at 
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:321)
    at 
org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:440)
    at 
org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$0(SSTableLoader.java:121)
    at 
org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$2(LogAwareFileLister.java:99)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
    at java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2969)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at 
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at 
org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:101)
    at 
org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:73)
    ... 5 more{code}
This requires that we have dropped columns in the source table and sstables 
exist from the "old schema" time.

This can be very easily reproduced. I used following script:
{code:java}
KS=test
SRCTABLE=bug3_source
DESTTABLE=bug3_dest
DATADIR=/usr/local/var/lib/cassandra/data
TMPDIR=/tmp

cqlsh -e "CREATE TABLE $KS.$SRCTABLE(a int primary key, b int, c int, d int);"
cqlsh -e "CREATE TABLE $KS.$DESTTABLE(a int primary key, b int, c int);"
cqlsh -e "INSERT INTO $KS.$SRCTABLE(a,b,c,d) values(1,2,3,4);"
nodetool flush $KS $SRCTABLE
cqlsh -e "ALTER TABLE $KS.$SRCTABLE DROP d;"
nodetool flush $KS $SRCTABLE

mkdir -p $TMPDIR/$KS/$DESTTABLE-acdc
cp $DATADIR/$KS/$SRCTABLE-*/* $TMPDIR/$KS/$DESTTABLE-acdc
sstableloader -d 127.0.0.1 $TMPDIR/$KS/$DESTTABLE-acdc{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14286) IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY

[Francisco Fernandez (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409339#comment-16409339
 ] 

Francisco Fernandez commented on CASSANDRA-14286:
-

The problem was that columns were serialized as json and then discarded, while 
{{OrderingComparator}} expected those columns to be present. With this patch, 
all columns are present and only the json column is sent back to clients. It 
uses the same mechanism as the regular ordering of non-selected columns.

[Patch|https://github.com/apache/cassandra/compare/cassandra-2.2...fcofdez:CASSANDRA-14286?expand=1]
 

> IndexOutOfBoundsException with SELECT JSON using IN and ORDER BY
> 
>
> Key: CASSANDRA-14286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14286
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kubernetes cluster using cassandra:3.11.1 Docker image.
>Reporter: Szymon Acedański
>Assignee: Francisco Fernandez
>Priority: Major
> Attachments: orderbug-traceback.txt
>
>
> When running the following code:
> {code}
> public class CassandraJsonOrderingBug {
> public static void main(String[] args) {
> Session session = CassandraFactory.getSession();
> session.execute("CREATE TABLE thebug ( PRIMARY KEY (a, b), a INT, b 
> INT)");
> try {
> session.execute("INSERT INTO thebug (a, b) VALUES (20, 30)");
> session.execute("INSERT INTO thebug (a, b) VALUES (100, 200)");
> Statement statement = new SimpleStatement("SELECT JSON a, b FROM 
> thebug WHERE a IN (20, 100) ORDER BY b");
> statement.setFetchSize(Integer.MAX_VALUE);
> for (Row w: session.execute(statement)) {
> System.out.println(w.toString());
> }
> } finally {
> session.execute("DROP TABLE thebug");
> }
> }
> }
> {code}
> The following exception is thrown server-side:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.Collections$SingletonList.get(Collections.java:4815) 
> ~[na:1.8.0_151]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1297)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$SingleColumnComparator.compare(SelectStatement.java:1284)
>  ~[apache-cassandra-3.11.1.jar:3.11.1]
>   at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) 
> ~[na:1.8.0_151]
>   at java.util.TimSort.sort(TimSort.java:220) ~[na:1.8.0_151]
>   at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_151]
>   at java.util.ArrayList.sort(ArrayList.java:1460) ~[na:1.8.0_151]
>   at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_151]
> {noformat}
> (full traceback attached)
> The accessed index is the index of the sorted column in the SELECT JSON 
> fields list.
> Similarly, if the select clause is changed to
> SELECT JSON b, a FROM thebug WHERE a IN (20, 100) ORDER BY b
> then the query finishes, but the output is sorted incorrectly (by textual 
> JSON representation):
> {noformat}
> Row[{"b": 200, "a": 100}]
> Row[{"b": 30, "a": 20}]
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14334) dtest - lingering Python 3 bytes/string decoding problems

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14334:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

committed as {{0b4f34a9af3e800d7cdbb432de3aac1e0c524c2d}}, thanks!

> dtest - lingering Python 3 bytes/string decoding problems
> -
>
> Key: CASSANDRA-14334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14334
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
> Environment: Platform: x64_64 (AWS EC2 t2_large)
> OS: Linux Ubuntu 16.04 LTS
> cassandra-3.0 at commit f57d12ee7af1bf3b93d36b486749defd2204d066
> cassandra-dtest at commit 1888c4048b65c447733a2759f32472a7004eb46f
> Python 3.5.2, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
> Attachments: 14334-master.txt
>
>
> We have a few dtests failing because we're trying to decode a string object 
> returned from ccm Node functions.
> repair_tests/repair_test.py::TestRepair::test_no_anticompaction_of_already_repaired
>  fails at line 283 because we try to decode data from ccm node.nodetool().
> sstablesplit_test.py::TestSSTableSplit::test_single_file_split fails at line 
> 106 because because we try to decode data from ccm node.run_sstablesplit().
> In both cases, ccm is returning strings instead of bytes now, so we shouldn't 
> decode them anymore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14320) dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409293#comment-16409293
 ] 

ASF GitHub Bot commented on CASSANDRA-14320:


Github user asfgit closed the pull request at:

https://github.com/apache/cassandra-dtest/pull/21


> dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on 
> bytes
> ---
>
> Key: CASSANDRA-14320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
>  Labels: Python3, dtest, python3, repair
>
> JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its 
> _query function tries to use json.loads (which only accepts string input) on 
> a bytes object.
> {code:java}
>     def _query(self, body, verbose=True):
>     request_data = json.dumps(body).encode("utf-8")
>     url = 'http://%s:8778/jolokia/' % 
> (self.node.network_interfaces['binary'][0],)
>     req = urllib.request.Request(url)
>     response = urllib.request.urlopen(req, data=request_data, 
> timeout=10.0)
>     if response.code != 200:
>     raise Exception("Failed to query Jolokia agent; HTTP response 
> code: %d; response: %s" % (response.code, response.readlines()))
>     raw_response = response.readline() # response is 
> http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes 
> when read
>     response = json.loads(raw_response) # this raises a TypeError now
>     if response['status'] != 200:
>     stacktrace = response.get('stacktrace')
>     if stacktrace and verbose:
>     print("Stacktrace from Jolokia error follows:")
>     for line in stacktrace.splitlines():
>     print(line)
>     raise Exception("Jolokia agent returned non-200 status: %s" % 
> (response,))
>     return response{code}
> This can be seen clearly by running the deprecated repair tests 
> (repair_tests/deprecated_repair_test.py). They all fail right now because of 
> this TypeError.
> This is a side effect of the migration to Python 3, which makes bytes objects 
> fundamentally different from strings. This will also happen anytime we try to 
> json.loads data returned from stdout or stderr piped from subprocess. I need 
> to take a closer look at offline_tools_test.py and 
> cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
> We can fix this issue by decoding bytes objects to strings before calling 
> json.loads(). For example, in the above:
> {code:java}
>     response = json.loads(raw_response.decode(encoding='utf-8')){code}
> I have a fix for the JolokiaAgent problem - I'll submit a pull request to 
> cassandra-dtest once I have this issue number to reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14320) dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14320:

   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   Status: Resolved  (was: Patch Available)

committed as {{5057b8273cddfeff1788fd2861de4ac1c80369cf}}, thanks!

> dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on 
> bytes
> ---
>
> Key: CASSANDRA-14320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
>  Labels: Python3, dtest, python3, repair
>
> JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its 
> _query function tries to use json.loads (which only accepts string input) on 
> a bytes object.
> {code:java}
>     def _query(self, body, verbose=True):
>     request_data = json.dumps(body).encode("utf-8")
>     url = 'http://%s:8778/jolokia/' % 
> (self.node.network_interfaces['binary'][0],)
>     req = urllib.request.Request(url)
>     response = urllib.request.urlopen(req, data=request_data, 
> timeout=10.0)
>     if response.code != 200:
>     raise Exception("Failed to query Jolokia agent; HTTP response 
> code: %d; response: %s" % (response.code, response.readlines()))
>     raw_response = response.readline() # response is 
> http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes 
> when read
>     response = json.loads(raw_response) # this raises a TypeError now
>     if response['status'] != 200:
>     stacktrace = response.get('stacktrace')
>     if stacktrace and verbose:
>     print("Stacktrace from Jolokia error follows:")
>     for line in stacktrace.splitlines():
>     print(line)
>     raise Exception("Jolokia agent returned non-200 status: %s" % 
> (response,))
>     return response{code}
> This can be seen clearly by running the deprecated repair tests 
> (repair_tests/deprecated_repair_test.py). They all fail right now because of 
> this TypeError.
> This is a side effect of the migration to Python 3, which makes bytes objects 
> fundamentally different from strings. This will also happen anytime we try to 
> json.loads data returned from stdout or stderr piped from subprocess. I need 
> to take a closer look at offline_tools_test.py and 
> cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
> We can fix this issue by decoding bytes objects to strings before calling 
> json.loads(). For example, in the above:
> {code:java}
>     response = json.loads(raw_response.decode(encoding='utf-8')){code}
> I have a fix for the JolokiaAgent problem - I'll submit a pull request to 
> cassandra-dtest once I have this issue number to reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/2] cassandra-dtest git commit: more fixes to stop decoding strings

[marcuse]

more fixes to stop decoding strings

Patch by Patrick Bannister; reviewed by marcuse for CASSANDRA-14334


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/0b4f34a9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/0b4f34a9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/0b4f34a9

Branch: refs/heads/master
Commit: 0b4f34a9af3e800d7cdbb432de3aac1e0c524c2d
Parents: 5057b82
Author: Patrick Bannister 
Authored: Wed Mar 21 20:15:45 2018 +
Committer: Marcus Eriksson 
Committed: Thu Mar 22 10:40:24 2018 +0100

--
 repair_tests/repair_test.py | 8 +---
 sstablesplit_test.py| 4 ++--
 2 files changed, 7 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/0b4f34a9/repair_tests/repair_test.py
--
diff --git a/repair_tests/repair_test.py b/repair_tests/repair_test.py
index 53ab2af..66189cf 100644
--- a/repair_tests/repair_test.py
+++ b/repair_tests/repair_test.py
@@ -278,7 +278,7 @@ class TestRepair(BaseRepairTest):
 out = node.run_sstablemetadata(keyspace=keyspace).stdout
 
 def matches(pattern):
-return [_f for _f in [pattern.match(l) for l in 
out.decode("utf-8").split('\n')] if _f]
+return [_f for _f in [pattern.match(l) for l in out.split('\n')] 
if _f]
 
 names = [m.group(1) for m in matches(_sstable_name)]
 repaired_times = [int(m.group(1)) for m in matches(_repaired_at)]
@@ -650,7 +650,7 @@ class TestRepair(BaseRepairTest):
 
 for line, m in out_of_sync_logs:
 num_out_of_sync_ranges, out_of_sync_nodes = m.group(3), 
{m.group(1), m.group(2)}
-assert int(num_out_of_sync_ranges) == 1, "Expecting 1 range out of 
sync for {}, but saw {}".format(out_of_sync_nodes , num_out_of_sync_ranges)
+assert int(num_out_of_sync_ranges) == 1, "Expecting 1 range out of 
sync for {}, but saw {}".format(out_of_sync_nodes, num_out_of_sync_ranges)
 assert out_of_sync_nodes, valid_out_of_sync_pairs in 
str(out_of_sync_nodes)
 
 # Check node2 now has the key
@@ -1090,6 +1090,7 @@ class TestRepair(BaseRepairTest):
 cluster.populate(3).start(wait_for_binary_proto=True)
 node1, node2, node3 = cluster.nodelist()
 node1.stress(['write', 'n=100k', '-schema', 'replication(factor=3)', 
'-rate', 'threads=30'])
+
 def run_repair():
 try:
 if cluster.version() >= "2.2":
@@ -1097,7 +1098,8 @@ class TestRepair(BaseRepairTest):
 else:
 node1.nodetool('repair keyspace1 standard1 -inc -par')
 except ToolError:
-debug("got expected exception during repair, ignoring")
+logger.debug("got expected exception during repair, ignoring")
+
 t1 = threading.Thread(target=run_repair)
 t1.start()
 if cluster.version() > "2.2":

http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/0b4f34a9/sstablesplit_test.py
--
diff --git a/sstablesplit_test.py b/sstablesplit_test.py
index 382f618..4d86195 100644
--- a/sstablesplit_test.py
+++ b/sstablesplit_test.py
@@ -103,8 +103,8 @@ class TestSSTableSplit(Tester):
 result = node.run_sstablesplit(keyspace='keyspace1', size=1, 
no_snapshot=True)
 
 for (stdout, stderr, rc) in result:
-logger.debug(stderr.decode("utf-8"))
-failure = stderr.decode("utf-8").find("java.lang.AssertionError: 
Data component is missing")
+logger.debug(stderr)
+failure = stderr.find("java.lang.AssertionError: Data component is 
missing")
 assert failure, -1 == "Error during sstablesplit"
 
 sstables = node.get_sstables('keyspace1', '')


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/2] cassandra-dtest git commit: tools/jmxutils.py decode bytes to string before passing to json.loads

[marcuse]

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 2c1b986bc -> 0b4f34a9a


tools/jmxutils.py decode bytes to string before passing to json.loads

Patch by Patrick Bannister; reviewed by marcuse for CASSANDRA-14320

Closes #21


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/5057b827
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/5057b827
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/5057b827

Branch: refs/heads/master
Commit: 5057b8273cddfeff1788fd2861de4ac1c80369cf
Parents: 2c1b986
Author: Patrick Bannister 
Authored: Sat Mar 17 03:32:50 2018 +
Committer: Marcus Eriksson 
Committed: Thu Mar 22 10:39:57 2018 +0100

--
 tools/jmxutils.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/5057b827/tools/jmxutils.py
--
diff --git a/tools/jmxutils.py b/tools/jmxutils.py
index 83459e0..8c78022 100644
--- a/tools/jmxutils.py
+++ b/tools/jmxutils.py
@@ -22,7 +22,7 @@ def jolokia_classpath():
 return CLASSPATH_SEP.join((tools_jar, JOLOKIA_JAR))
 else:
 logger.warning("Environment variable $JAVA_HOME not present: jmx-based 
" +
-"tests may fail because of missing $JAVA_HOME/lib/tools.jar.")
+   "tests may fail because of missing 
$JAVA_HOME/lib/tools.jar.")
 return JOLOKIA_JAR
 
 
@@ -237,7 +237,7 @@ class JolokiaAgent(object):
 raise Exception("Failed to query Jolokia agent; HTTP response 
code: %d; response: %s" % (response.code, response.readlines()))
 
 raw_response = response.readline()
-response = json.loads(raw_response)
+response = json.loads(raw_response.decode(encoding='utf-8'))
 if response['status'] != 200:
 stacktrace = response.get('stacktrace')
 if stacktrace and verbose:


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14334) dtest - lingering Python 3 bytes/string decoding problems

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14334:

Reviewer: Marcus Eriksson

> dtest - lingering Python 3 bytes/string decoding problems
> -
>
> Key: CASSANDRA-14334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14334
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
> Environment: Platform: x64_64 (AWS EC2 t2_large)
> OS: Linux Ubuntu 16.04 LTS
> cassandra-3.0 at commit f57d12ee7af1bf3b93d36b486749defd2204d066
> cassandra-dtest at commit 1888c4048b65c447733a2759f32472a7004eb46f
> Python 3.5.2, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
> Attachments: 14334-master.txt
>
>
> We have a few dtests failing because we're trying to decode a string object 
> returned from ccm Node functions.
> repair_tests/repair_test.py::TestRepair::test_no_anticompaction_of_already_repaired
>  fails at line 283 because we try to decode data from ccm node.nodetool().
> sstablesplit_test.py::TestSSTableSplit::test_single_file_split fails at line 
> 106 because because we try to decode data from ccm node.run_sstablesplit().
> In both cases, ccm is returning strings instead of bytes now, so we shouldn't 
> decode them anymore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14334) dtest - lingering Python 3 bytes/string decoding problems

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-14334:
---

Assignee: Patrick Bannister

> dtest - lingering Python 3 bytes/string decoding problems
> -
>
> Key: CASSANDRA-14334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14334
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
> Environment: Platform: x64_64 (AWS EC2 t2_large)
> OS: Linux Ubuntu 16.04 LTS
> cassandra-3.0 at commit f57d12ee7af1bf3b93d36b486749defd2204d066
> cassandra-dtest at commit 1888c4048b65c447733a2759f32472a7004eb46f
> Python 3.5.2, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
> Attachments: 14334-master.txt
>
>
> We have a few dtests failing because we're trying to decode a string object 
> returned from ccm Node functions.
> repair_tests/repair_test.py::TestRepair::test_no_anticompaction_of_already_repaired
>  fails at line 283 because we try to decode data from ccm node.nodetool().
> sstablesplit_test.py::TestSSTableSplit::test_single_file_split fails at line 
> 106 because because we try to decode data from ccm node.run_sstablesplit().
> In both cases, ccm is returning strings instead of bytes now, so we shouldn't 
> decode them anymore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14320) dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14320:

Reviewer: Marcus Eriksson

> dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on 
> bytes
> ---
>
> Key: CASSANDRA-14320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
>  Labels: Python3, dtest, python3, repair
> Fix For: 3.0.x, 3.11.x
>
>
> JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its 
> _query function tries to use json.loads (which only accepts string input) on 
> a bytes object.
> {code:java}
>     def _query(self, body, verbose=True):
>     request_data = json.dumps(body).encode("utf-8")
>     url = 'http://%s:8778/jolokia/' % 
> (self.node.network_interfaces['binary'][0],)
>     req = urllib.request.Request(url)
>     response = urllib.request.urlopen(req, data=request_data, 
> timeout=10.0)
>     if response.code != 200:
>     raise Exception("Failed to query Jolokia agent; HTTP response 
> code: %d; response: %s" % (response.code, response.readlines()))
>     raw_response = response.readline() # response is 
> http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes 
> when read
>     response = json.loads(raw_response) # this raises a TypeError now
>     if response['status'] != 200:
>     stacktrace = response.get('stacktrace')
>     if stacktrace and verbose:
>     print("Stacktrace from Jolokia error follows:")
>     for line in stacktrace.splitlines():
>     print(line)
>     raise Exception("Jolokia agent returned non-200 status: %s" % 
> (response,))
>     return response{code}
> This can be seen clearly by running the deprecated repair tests 
> (repair_tests/deprecated_repair_test.py). They all fail right now because of 
> this TypeError.
> This is a side effect of the migration to Python 3, which makes bytes objects 
> fundamentally different from strings. This will also happen anytime we try to 
> json.loads data returned from stdout or stderr piped from subprocess. I need 
> to take a closer look at offline_tools_test.py and 
> cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
> We can fix this issue by decoding bytes objects to strings before calling 
> json.loads(). For example, in the above:
> {code:java}
>     response = json.loads(raw_response.decode(encoding='utf-8')){code}
> I have a fix for the JolokiaAgent problem - I'll submit a pull request to 
> cassandra-dtest once I have this issue number to reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14320) dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on bytes

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-14320:
---

Assignee: Patrick Bannister

> dtest tools/jmxutils.py JolokiaAgent raises TypeError using json.loads on 
> bytes
> ---
>
> Key: CASSANDRA-14320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14320
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Patrick Bannister
>Assignee: Patrick Bannister
>Priority: Minor
>  Labels: Python3, dtest, python3, repair
> Fix For: 3.0.x, 3.11.x
>
>
> JolokiaAgent in tools/jmxutils.py raises a TypeError when used, because its 
> _query function tries to use json.loads (which only accepts string input) on 
> a bytes object.
> {code:java}
>     def _query(self, body, verbose=True):
>     request_data = json.dumps(body).encode("utf-8")
>     url = 'http://%s:8778/jolokia/' % 
> (self.node.network_interfaces['binary'][0],)
>     req = urllib.request.Request(url)
>     response = urllib.request.urlopen(req, data=request_data, 
> timeout=10.0)
>     if response.code != 200:
>     raise Exception("Failed to query Jolokia agent; HTTP response 
> code: %d; response: %s" % (response.code, response.readlines()))
>     raw_response = response.readline() # response is 
> http.client.HTTPResponse, which subclasses RawIOBase, which returns bytes 
> when read
>     response = json.loads(raw_response) # this raises a TypeError now
>     if response['status'] != 200:
>     stacktrace = response.get('stacktrace')
>     if stacktrace and verbose:
>     print("Stacktrace from Jolokia error follows:")
>     for line in stacktrace.splitlines():
>     print(line)
>     raise Exception("Jolokia agent returned non-200 status: %s" % 
> (response,))
>     return response{code}
> This can be seen clearly by running the deprecated repair tests 
> (repair_tests/deprecated_repair_test.py). They all fail right now because of 
> this TypeError.
> This is a side effect of the migration to Python 3, which makes bytes objects 
> fundamentally different from strings. This will also happen anytime we try to 
> json.loads data returned from stdout or stderr piped from subprocess. I need 
> to take a closer look at offline_tools_test.py and 
> cqlsh_tests/cqlsh_copy_tests.py, because I suspect they're impacted as well.
> We can fix this issue by decoding bytes objects to strings before calling 
> json.loads(). For example, in the above:
> {code:java}
>     response = json.loads(raw_response.decode(encoding='utf-8')){code}
> I have a fix for the JolokiaAgent problem - I'll submit a pull request to 
> cassandra-dtest once I have this issue number to reference.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14333) nodetool repair causes out of memory issues

[chinta kiran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chinta kiran updated CASSANDRA-14333:
-
Attachment: cassandra-env.sh
cassandra.yaml

The JVM file and Cassandra.yaml file is attached.

Regards,
Chinta kiran 



> nodetool repair causes out of  memory issues 
> -
>
> Key: CASSANDRA-14333
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14333
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: chinta kiran
>Priority: Major
> Fix For: 3.9
>
> Attachments: cassandra-env.sh, cassandra.yaml
>
>
> When ever we try to run nodetool repair we are getting OOM ] . We have tried 
> all types of nodetool repair, full,incremental range repair. all the times it 
> is failing . how to overcome this issue. We have 32GB Ram.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14319) nodetool rebuild from DC lets you pass invalid datacenters

[Vinay Chella (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407169#comment-16407169
 ] 

Vinay Chella edited comment on CASSANDRA-14319 at 3/22/18 7:04 AM:
---

Hi [~rustyrazorblade],

Are you looking for something like below when we provide invalid datacenter as 
an option to rebuild?
{code:java}
$ bin/nodetool rebuild no_dc
nodetool: Provided datacenter: no_dc is not a valid datacenter, available 
datacenters are: datacenter1, datacenter2
See 'nodetool help' or 'nodetool help '.
$ 
{code}
And for
{quote}3. Ideally, we indicate which keyspaces are set to replicate to this DC 
and which aren't
{quote}
Are you referring {{this}} as the datacenter from where the {{rebuild}} command 
is being executed from or the one which is provided as an option (e.g., 
{{no_dc}} in above example)? if it is the later, invalid dc would not have any 
keyspaces, so what is expected for #3 in this scenario?

 

I am interested in working on this ticket. Can you assign this ticket to me? 


was (Author: vinaykumarcse):
Hi Jon,

Are you looking for something like below when we provide invalid datacenter as 
an option to rebuild?
{code:java}
$ bin/nodetool rebuild no_dc
nodetool: Provided datacenter: no_dc is not a valid datacenter, available 
datacenters are: datacenter1, datacenter2
See 'nodetool help' or 'nodetool help '.
$ 
{code}
And for
{quote}3. Ideally, we indicate which keyspaces are set to replicate to this DC 
and which aren't
{quote}
Are you referring {{this}} as the datacenter from where the {{rebuild}} command 
is being executed from or the one which is provided as an option (e.g., 
{{no_dc}} in above example)? if it is the later, invalid dc would not have any 
keyspaces, so what is expected for #3 in this scenario?

 

I am interested in working on this ticket. Can you assign this ticket to me? 

> nodetool rebuild from DC lets you pass invalid datacenters 
> ---
>
> Key: CASSANDRA-14319
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14319
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> If you pass an invalid datacenter to nodetool rebuild, you'll get an error 
> like this:
> {code}
> Unable to find sufficient sources for streaming range 
> (3074457345618258602,-9223372036854775808] in keyspace system_distributed
> {code}
> Unfortunately, this is a rabbit hole of frustration if you are using caps for 
> your DC names and you pass in a lowercase DC name, or you just typo the DC.  
> Let's do the following:
> # Check the DC name that's passed in against the list of DCs we know about
> # If we don't find it, let's output a reasonable error, and list all the DCs 
> someone could put in.
> # Ideally we indicate which keyspaces are set to replicate to this DC and 
> which aren't



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13529) cassandra-stress light-weight transaction support

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409128#comment-16409128
 ] 

Dinesh Joshi commented on CASSANDRA-13529:
--

Hi [~chovatia.jayd...@gmail.com], the patch looks like it's in the right 
direction. There are a few things we can do to make it better -

 # In {{SchemaQuery}}'s constructor, you're passing it's partially constructed 
reference to {{CASQueryDynamicConditions}}. Don't do this (See: 
https://www.ibm.com/developerworks/library/j-jtp0618/).
 # {{CASQueryDynamicConditions}} should ideally inherit from 
{{SchemaStatement}} class and perhaps call it {{CASQuery}} (Making it a sibling 
of {{SchemaQuery}} class).
 # You can modify {{SettingsCommandUser::getFactory}} so {{CASQuery}} type 
would be a new query type to choose.

> cassandra-stress light-weight transaction support
> -
>
> Key: CASSANDRA-13529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13529
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Stress
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 13529.txt, lwttest.yaml
>
>
> It would be nice to have a light-weight transaction support in 
> cassandra-stress.
> Although currently in cassandra-stress we can achieve light-weight 
> transaction partially by using static conditions like "IF col1 != null" or 
> "IF not EXIST". 
> If would be ideal to have full fledged light-weight transaction support like 
> "IF col1 = ? and col2 = ?". One way to implement is to read values from 
> Cassandra and use that in the condition so it will execute all the paxos 
> phases in Cassandra.
> Please find git link for the patch: 
> https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13529-trunk?expand=1
> ||trunk|
> |[branch|https://github.com/jaydeepkumar1984/cassandra/tree/13529-trunk]|
> |[utests|https://circleci.com/gh/jaydeepkumar1984/cassandra/8]|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13529) cassandra-stress light-weight transaction support

[Dinesh Joshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-13529:
-
Reviewer: Dinesh Joshi  (was: Jason Brown)

> cassandra-stress light-weight transaction support
> -
>
> Key: CASSANDRA-13529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13529
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Stress
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 13529.txt, lwttest.yaml
>
>
> It would be nice to have a light-weight transaction support in 
> cassandra-stress.
> Although currently in cassandra-stress we can achieve light-weight 
> transaction partially by using static conditions like "IF col1 != null" or 
> "IF not EXIST". 
> If would be ideal to have full fledged light-weight transaction support like 
> "IF col1 = ? and col2 = ?". One way to implement is to read values from 
> Cassandra and use that in the condition so it will execute all the paxos 
> phases in Cassandra.
> Please find git link for the patch: 
> https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13529-trunk?expand=1
> ||trunk|
> |[branch|https://github.com/jaydeepkumar1984/cassandra/tree/13529-trunk]|
> |[utests|https://circleci.com/gh/jaydeepkumar1984/cassandra/8]|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org