[jira] [Updated] (CASSANDRA-8857) Batching SELECTs

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8857:

Component/s: CQL

> Batching SELECTs
> 
>
> Key: CASSANDRA-8857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8857
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: CQL
>Reporter: Jay Patel
>Priority: Major
>
> Support for batching Selects across multiple table.
> Also check out CASSANDRA-8855



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8634) Multi-thread commit log compression

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8634:

Component/s: Compression

> Multi-thread commit log compression
> ---
>
> Key: CASSANDRA-8634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8634
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compression
>Reporter: Ariel Weisberg
>Priority: Major
>  Labels: performance
>
> The initial commit log compression implementation can do about 240 
> megabytes/second (uncompressed input) on a fast (500 megabytes/sec)  SSD.
> This is basically what I measured as the throughput of LZ4 against the same 
> data.
> Assuming a 1.5x compression ratio that is 160 megabytes/second of output. 
> Enough to saturate a spinning disk, but not an SSD.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8721) expose "shadowed" columns in tracing output

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8721:

Component/s: Observability

> expose "shadowed" columns in tracing output
> ---
>
> Key: CASSANDRA-8721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8721
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Robert Coli
>Priority: Trivial
>
> Current tracing messages expose how many tombstones are read in order to read 
> live columns, but they do not expose shadowed columns. Shadowed columns are 
> columns where the timestamp for a given column is lower than the highest 
> timestamp for that column.
> It would be useful for users who are tracing queries to understand how many 
> shadowed columns are being read-than-ignored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8583) Check for Thread.start()

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8583:

Component/s: Core

> Check for Thread.start()
> 
>
> Key: CASSANDRA-8583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8583
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Robert Stupp
>Priority: Minor
>
> Old classes sometimes still use 
> {noformat}
>   new Thread(...).start()
> {noformat}
> which might be costly.
> This ticket's about to find and possibly fix such code.
> Locations in code worth to investigate (IMO). This list is not prioritized - 
> it's just the order I've found "Thread.start()"
> # 
> {{org.apache.cassandra.streaming.compress.CompressedInputStream#CompressedInputStream}}
>  creates one thread per input stream to decompress in a separate thread. If 
> necessary, should be easily replaceable with a thread-pool
> # 
> {{org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter#SSTableSimpleUnsortedWriter(java.io.File,
>  org.apache.cassandra.config.CFMetaData, 
> org.apache.cassandra.dht.IPartitioner, long)}} creates one thread per write. 
> If necessary, should be easily replaceable with a thread-pool
> # {{org.apache.cassandra.streaming.ConnectionHandler.MessageHandler#start}} 
> creates one thread. If necessary, should be easily replaceable with a 
> thread-pool.
> # {{org.apache.cassandra.net.OutboundTcpConnection#handshakeVersion}} creates 
> one thread just to implement a timeout. Not sure why not just using 
> {{Socket.setSoTimeout}}
> # 
> {{org.apache.cassandra.service.StorageService#forceRepairAsync(java.lang.String,
>  org.apache.cassandra.repair.messages.RepairOption)}} creates one thread per 
> repair. Not sure whether it's worth to investigate this one, since repairs 
> are "long running" operations
> # {{org.apache.cassandra.db.index.SecondaryIndex#buildIndexAsync}} creates a 
> thread. Not sure whether it's worth to investigate this one.
> Beside these, there are threads used in {{MessagingService}} and for 
> streaming (blocking I/O model). These could be changed by using non-blocking 
> I/O - but that's a much bigger task with much higher risks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8732:

Component/s: Streaming and Messaging

> Make inter-node timeouts tolerate clock skew and drift
> --
>
> Key: CASSANDRA-8732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8732
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Priority: Major
> Attachments: maximalskew.png
>
>
> Right now internode timeouts rely on currentTimeMillis() (and NTP) to make 
> sure that tasks don't expire before they arrive.
> Every receiver needs to deduce the offset between its nanoTime and the remote 
> nanoTime. I don't think currentTimeMillis is a good choice because it is 
> designed to be manipulated by operators and NTP. I would probably be 
> comfortable assuming that nanoTime isn't going to move in significant ways 
> without something that could be classified as operator error happening.
> I suspect the one timing method you can rely on being accurate is nanoTime 
> within a node (on average) and that a node can report on its own scheduling 
> jitter (on average).
> Finding the offset requires knowing what the network latency is in one 
> direction.
> One way to do this would be to periodically send a ping request which 
> generates a series of ping responses at fixed intervals (maybe by UDP?). The 
> responses should corrected for scheduling jitter since the fixed intervals 
> may not be exactly achieved by the sender. By measuring the time deviation 
> between ping responses and their expected arrival time (based on the 
> interval) and correcting for the remotely reported scheduling jitter, you 
> should be able to measure latency in one direction.
> A weighted moving average (only correct for drift, not readjustment) of these 
> measurements would eventually converge on a close answer and would not be 
> impacted by outlier measurements. It may also make sense to drop the largest 
> N samples to improve accuracy.
> One you know network latency you can add that to the timestamp of each ping 
> and compare to the local clock and know what the offset is.
> These measurements won't calculate the offset to be too small (timeouts fire 
> early), but could calculate the offset to be too large (timeouts fire late). 
> The conditions where you the offset won't be accurate are the conditions 
> where you also want them firing reliably. This and bootstrapping in bad 
> conditions is what I am most uncertain of.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8590) Test repairing large dataset after upgrade

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8590:

Component/s: Testing

> Test repairing large dataset after upgrade
> --
>
> Key: CASSANDRA-8590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8590
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Ryan McGuire
>Assignee: Ryan McGuire
>Priority: Major
>
> * Write large dataset in multiple tables
> * upgrade
> * replace a few nodes
> * repair in round-robin fashion
> * ensure exit codes of cmd line tools are expected
> * verify data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8629) Exceptions in cassandra-stress

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8629:

Component/s: Stress

> Exceptions in cassandra-stress
> --
>
> Key: CASSANDRA-8629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8629
> Project: Cassandra
>  Issue Type: Bug
>  Components: Stress
>Reporter: Aleksey Yeschenko
>Priority: Trivial
>  Labels: stress
>
> cassandra-stress when run with tiny n, throws 
> org.apache.commons.math3.exception.NotStrictlyPositiveException.
> Now, n=1 doesn't really make any sense, w/ 50k writes used just for warmup, 
> but an exception is still an exception. Labeled w/ priority: Trivial.
> Profile used: http://pastebin.com/raw.php?i=9U5EMdVq
> {noformat}
> tools/bin/cassandra-stress user profile=partition.yaml ops\(insert=1\) n=1 
> -rate threads=50
> INFO  18:21:59 Using data-center name 'datacenter1' for 
> DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct 
> datacenter name with DCAwareRoundRobinPolicy constructor)
> Connected to cluster: Test Cluster
> Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
> INFO  18:21:59 New Cassandra host localhost/127.0.0.1:9042 added
> Created schema. Sleeping 1s for propagation.
> Exception in thread "main" 
> org.apache.commons.math3.exception.NotStrictlyPositiveException: standard 
> deviation (0)
>   at 
> org.apache.commons.math3.distribution.NormalDistribution.(NormalDistribution.java:108)
>   at 
> org.apache.cassandra.stress.settings.OptionDistribution$GaussianFactory.get(OptionDistribution.java:418)
>   at 
> org.apache.cassandra.stress.generate.SeedManager.(SeedManager.java:59)
>   at 
> org.apache.cassandra.stress.settings.SettingsCommandUser.getFactory(SettingsCommandUser.java:78)
>   at org.apache.cassandra.stress.StressAction.run(StressAction.java:61)
>   at org.apache.cassandra.stress.Stress.main(Stress.java:109)
> {noformat}
> On cassandra-2.1 HEAD, I cannot reproduce it, but get a different exception, 
> with n=10:
> {noformat}
> Exception in thread "Thread-13" java.lang.AssertionError
>   at 
> org.apache.cassandra.stress.util.DynamicList.remove(DynamicList.java:156)
>   at org.apache.cassandra.stress.generate.Seed.remove(Seed.java:83)
>   at 
> org.apache.cassandra.stress.generate.SeedManager.markLastWrite(SeedManager.java:115)
>   at 
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.setHasNext(PartitionIterator.java:561)
>   at 
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.seek(PartitionIterator.java:333)
>   at 
> org.apache.cassandra.stress.generate.PartitionIterator$MultiRowIterator.reset(PartitionIterator.java:242)
>   at 
> org.apache.cassandra.stress.generate.PartitionIterator.reset(PartitionIterator.java:99)
>   at org.apache.cassandra.stress.Operation.ready(Operation.java:110)
>   at 
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:288)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8686) Introduce Latency Target for Stress

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8686:

Component/s: Stress

> Introduce Latency Target for Stress
> ---
>
> Key: CASSANDRA-8686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8686
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: jonathan lacefield
>Priority: Minor
>  Labels: stress
>
> This item is a request to add a latency target to the rate option for the new 
> stress tool.  The goal of the latency target would be to provide a guideline 
> for SLAs to the stress tool so the stress tool can determine threads and 
> throughputs that can be sustained while meeting the SLA targets.
> For example:
> cassandra-stress [options/commands] -rate latency p90=5 p95=10 p99=100
> The outcome of this command would be a stress execution that would gradually 
> increase threads, and hence throughput (trans/sec), until the latency profile 
> can no longer be satisfied with the current workload (yaml file definition) 
> and/or cluster.  This would provide a ceiling for throughput and connections 
> for the given cluster, workload, and SLA profile.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Resolved] (CASSANDRA-8542) Make get_range_slices and related succeed most of the time on tombstone heavy column families

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas resolved CASSANDRA-8542.
-
Resolution: Won't Fix

> Make get_range_slices and related succeed most of the time on tombstone heavy 
> column families
> -
>
> Key: CASSANDRA-8542
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8542
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Shawn Walker
>Priority: Minor
> Attachments: trunk-8542-InvertibleBloomReconciler.txt
>
>
> I've got a ColumnFamily in which some rows end up being used in a queue-like 
> fashion, and where many tombstones tend to pile up at the left end of the 
> row.  As a result, I run into {{TombstoneOverwhelming}} (e.g. CASSANDRA-6117) 
> when trying to list the columns of said rows.
> Please don't yell at me too loudly. I know the issue I'm describing will 
> generally be considered as being due to poor use of Cassandra.  I understand 
> the rationale of the current behavior, and am hesitant to play with fire by 
> increasing {{tombstone_fail_threshold}} to a high value.  Instead, what I 
> propose is an alternate method of resolving such queries on the read path.
> 
> This is based on the following observation: on the coordinator node, when 
> {{RangeSliceResponseResolver}} is resolving a range slice query, it needn't 
> be aware of all tombstones that all responding nodes have within the 
> specified range.  Instead, it would suffice if it could determine only those 
> tombstones which aren't shared by all responding nodes. E.g. if node #1 
> responds with tombstones (A, B, D), node #2 responds with tombstones (A, B, 
> C), and node #3 responds with tombstones (A, B, C, D), 
> {{RangeSliceResponseResolver}} need only actually know about the tombstones 
> (C, D): All of the responders will already have removed any relevant data for 
> the tombstones (A, B) from their individual responses.
> Arranging for {{RangeSliceResponseResolver}} to discover only the non-common 
> tombstones can be accomplished by using a variant of the "invertible bloom 
> filter" data structure described in e.g. 
> http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p218.pdf.  Using 
> an invertible Bloom filter, each responding node can build a (roughly) fixed 
> size data structure holding a representation of all the tombstones that node 
> encounters.  The coordinator node can then combine the invertible Bloom 
> filters.  If there aren't too many non-common tombstones, the coordinator 
> node will be able to enumerate all of them, and so resolve the range slice 
> query.
> I see accomplishing this in a few discrete steps:
> 1. Implement an appropriate variant of the "invertible bloom filter".  I've 
> started this already, and will shortly upload a patch including my 
> {{InvertibleBloomReconciler}} implementation.  From a black box perspective, 
> {{InvertibleBloomReconcilerTest.verifyFiveWayReconciliation()}} demonstrates 
> how this data structure and algorithm could be used.
> 2. ("db layer") Wrap {{InvertibleBloomReconciler}} into 
> {{TombstoneReconciler}}, a structure for spilling tombstones into.  Refactor 
> large swaths of {{org.apache.cassandra.db}}'s read path to accomodate the 
> possibility of placing tombstones discovered during a read into a 
> {{TombstoneReconciler}} instead of returning them within a {{Row}}, 
> {{List}}, {{ColumnFamily}}, etc.  I've attempted a start on this, though 
> this means carrying the {{TombstoneReconciler}} around through most of 
> {{ColumnFamilyStore}}, practically all of {{org.apache.db.filter}}, and other 
> places I haven't yet discovered.  I'm wondering if I wouldn't be better off 
> waiting for CASSANDRA-8099 before starting this step -- a fully iterator flow 
> through {{org.apache.cassandra.db}} could make this easier, cleaner, and have 
> significantly lower code impact.
> 3. ("dynamo layer") Arrange for {{RangeSliceCommand}} to include parameters 
> for the IBR (table size, hash count, seed), possibly by making these part of 
> {{CFMetaData}}.  Allow a {{RangeSliceResponse}} to optionally return a 
> {{TombstoneReconciler}} in addition to its {{List}}.  Refactor 
> {{RangeSliceResponseResolver}} to be capable of handling 
> {{TombstoneReconciler}} s.  Possibly refactor 
> {{StorageProxy.getRangeSlices(...)}} to disable read repair if any responses 
> contained a {{TombstoneReconciler}}.
> Since the invertible bloom filter is a probabilistic data structure, it is 
> possible that resolving a query in this manner could fail.  As such, I'm 
> proposing that the current read path not make use of the 
> {{TombstoneReconciler}} idea unless it would otherwise encounter a 
> {{TombstoneOverwhelming}} situation.
> Also, there 

[jira] [Updated] (CASSANDRA-8503) Collect important stress profiles for regression analysis done by jenkins

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8503:

Component/s: Testing
 Stress

> Collect important stress profiles for regression analysis done by jenkins
> -
>
> Key: CASSANDRA-8503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8503
> Project: Cassandra
>  Issue Type: Task
>  Components: Stress, Testing
>Reporter: Ryan McGuire
>Assignee: Ryan McGuire
>Priority: Major
> Attachments: inmemory.yaml, ycsb.yaml
>
>
> We have a weekly job setup on CassCI to run a performance benchmark against 
> the dev branches as well as the last stable releases.
> Here's an example:
> http://cstar.datastax.com/tests/id/8223fe2e-8585-11e4-b0bf-42010af0688f
> This test is currently pretty basic, it's running on three nodes, with a the 
> default stress profile. We should crowdsource a collection of stress profiles 
> to run, and then once we have many of these tests running we can collect them 
> all into a weekly email.
> Ideas:
>  * Timeseries (Can this be done with stress? not sure)
>  * compact storage
>  * compression off
>  * ...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8519) Mechanism to determine which nodes own which token ranges without Thrift

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8519:

Component/s: Distributed Metadata

> Mechanism to determine which nodes own which token ranges without Thrift
> 
>
> Key: CASSANDRA-8519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8519
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter:  Brian Hess
>Priority: Minor
>
> Right now the only way to determine which nodes own which token ranges is via 
> the Thrift interface.  There is not a Java/CQL driver mechanism to determine 
> this.  Applications that make multiple connections to Cassandra to extract 
> data in parallel need this ability so they can split the data into pieces, 
> and it is reasonable to want those splits to be on token range boundaries.  
> Of course, once you split this way, you would want to route those queries to 
> nodes that own that token range / split, for efficiency.
> This applies for both Hadoop and Spark, but other applications, too.  Hadoop 
> and Spark currently use Thrift to determine this topology.
> Additionally, different replication strategies and replication factors result 
> in different token range ownership, so there will have to be a different 
> answer based on which keyspace is used. 
> It would be useful if this data was stored in a CQL table and could be simply 
> queried.  A suggestion would be to add a column to the 
> SYSTEM.SCHEMA_KEYSPACES table (maybe a complex Map of Host to a UDT that has 
> a List of (beginRange, endRange) pairs - as an example).  This table would 
> need to be updated on an ALTER KEYSPACE command or on a topology change 
> event.  This would allow the server(s) to hold this information and the 
> drivers could simply query it (as opposed to having each driver manage this 
> separately).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8468) Stress support for multiple asynchronous operations per client

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8468:

Component/s: Stress

> Stress support for multiple asynchronous operations per client
> --
>
> Key: CASSANDRA-8468
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8468
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Stress
>Reporter: Benedict
>Priority: Major
>
> In conjunction with CASSANDRA-8466, this would permit more tunable variation 
> in network load generation characteristics.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8324) Cleanup Directories + BlacklistedDirectories classes

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8324:

Component/s: Core

> Cleanup Directories + BlacklistedDirectories classes
> 
>
> Key: CASSANDRA-8324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8324
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Robert Stupp
>Priority: Major
>
> {{Directories.getLocationForDisk(DataDirectory)}} uses 
> {{File.getAbsolutePath().startsWith(...)}} to return the sstable directory 
> for a data directory. This may return wrong result if two data directory 
> names are similar (e.g. {{/dataDir1}} and {{/dataDir1a}}).
> {{BlacklistedDirectories}} uses two sets that contain blacklisted 
> directories. These could be replaced with two {{AtomicBoolean}} fields in 
> {{Directories.DataDirectory}}.
> Goal of this ticket is to reduce the number of string operations, fix the 
> possible wrong result mentioned above and to refactor the blacklisted 
> directories.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8498) Replaying commit log records that are older than gc_grace is dangerous

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8498:

Component/s: Core

> Replaying commit log records that are older than gc_grace is dangerous
> --
>
> Key: CASSANDRA-8498
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8498
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Priority: Major
>
> If we replay commit log records that are older than gc_grace we could 
> introduce data corruption to the cluster. We should either (1) fail and 
> suggest a repair, or (2) log an exception. I prefer (1).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8335) Support CAS with UPDATE...IF col=val OR NOT EXISTS

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8335:

Component/s: CQL

> Support CAS with UPDATE...IF col=val OR NOT EXISTS
> --
>
> Key: CASSANDRA-8335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8335
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Robert Stupp
>Priority: Minor
>  Labels: LWT, cas, cql
>
> On the -dev mailing list the RFE to extend UPDATE LWT using something like
> {{UPDATE ... IF col=val ... OR NOT EXISTS}}
> RFE is to add the {{OR NOT EXISTS}} which means either the the {{IF}} 
> conditions are met *xor* the row does not exist.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8362) Reduce memory usage of RefCountedMemory

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8362:

Component/s: Core

> Reduce memory usage of RefCountedMemory
> ---
>
> Key: CASSANDRA-8362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8362
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
>
> We can store the references as the first 4 bytes of the Unsafe memory and use 
> case CAS[1] for reference counting of the memory.
> This change will reduce the object over head + additional 4 bytes from java 
> heap. Calling methods can reference as long's.
> [1] 
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/atomic/AtomicInteger.java#AtomicInteger.incrementAndGet%28%29



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8341) Expose time spent in each thread pool

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8341:

Component/s: Observability
 Metrics

> Expose time spent in each thread pool
> -
>
> Key: CASSANDRA-8341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8341
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Metrics, Observability
>Reporter: Chris Lohfink
>Priority: Minor
>  Labels: metrics
> Attachments: 8341.patch, 8341v2.txt
>
>
> Can increment a counter with time spent in each queue.  This can provide 
> context on how much time is spent percentage wise in each stage.  
> Additionally can be used with littles law in future if ever want to try to 
> tune the size of the pools.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8273) Allow filtering queries can return stale data

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8273:

Component/s: CQL

> Allow filtering queries can return stale data
> -
>
> Key: CASSANDRA-8273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8273
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
>Priority: Major
>
> Data filtering is done replica side. That means that a single replica with 
> stale data may make the whole query return that stale data.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v1 text, v2 int);
> CREATE INDEX ON test(v1);
> INSERT INTO test(k, v1, v2) VALUES (0, 'foo', 1);
> {noformat}
> with every replica up to date. Now, suppose that the following queries are 
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v2 = 2 WHERE k = 0;
> SELECT * FROM test WHERE v1 = 'foo' AND v2 = 1;
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before 
> having applied the insert, then the now stale result will be returned. Let's 
> note that this is a problem related to filtering, not 2ndary indexes.
> This issue share similarity with CASSANDRA-8272 but contrarily to that former 
> issue, I'm not sure how to fix it. Obviously, moving the filtering to the 
> coordinator would remove that problem, but doing so would, on top of not 
> being trivial to implmenent, have serious performance impact since we can't 
> know in advance how much data will be filtered and we may have to redo query 
> to replica multiple times.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8130) upgrade tests should run upgradesstables less eagerly

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8130:

Component/s: Testing

> upgrade tests should run upgradesstables less eagerly
> -
>
> Key: CASSANDRA-8130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8130
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Russ Hatch
>Priority: Major
>
> Currently the cassandra upgrade tests in cassandra-dtest are doing an upgrade 
> then running upgradesstables soon after. We are missing some potential 
> coverage with the current approach.
> Instead of upgrading sstables "early", we should be waiting until absolutely 
> necessary to run upgradesstables to test the guarantee that a version can 
> read sstables from the previous version. This will give tests more time to 
> interact with reading previous version sstables and hopefully increase 
> chances of catching a bug.
> Each version of cassandra should be able to read sstables from the previous 
> version (so 2.1.x can read 2.0.x, but is not guaranteed to read 1.2.x), and 
> therefore can work just fine reading and writing data. Writes of course will 
> happen in the current sstable version, so old sstables may be supplanted by 
> current ones over time (subject to write volume and compaction), potentially 
> obviating the need for an sstable upgrade.
> upgradesstables must be run before upgrading when the system could contain 
> sstables from two versions back since read compatibility is not guaranteed 
> (so we must run upgradesstables before upgrading to 2.1.x if there is a 
> chance that sstables still exist from 1.2.x; because this is two versions 
> back).
> This is all a long-winded way of saying that we should keep track in the 
> dtests if we are about to be 2 versions behind for an impending upgrade, and 
> only run upgradesstables at that point.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9100) Gossip is inadequately tested

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9100:

Component/s: Testing
 Distributed Metadata

> Gossip is inadequately tested
> -
>
> Key: CASSANDRA-9100
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9100
> Project: Cassandra
>  Issue Type: Test
>  Components: Distributed Metadata, Testing
>Reporter: Ariel Weisberg
>Priority: Major
>
> We found a few unit tests, but nothing that exercises Gossip under 
> challenging conditions. Maybe consider a long test that hooks up some 
> gossipers over a fake network and then do fault injection on that fake 
> network. Uni-directional and bi-directional partitions, delayed delivery, out 
> of order delivery if that is something that they can see in practice. 
> Connects/disconnects.
> Also play with bad clocks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8218) Internode Authentication and Gossip

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8218:

Component/s: Distributed Metadata

> Internode Authentication and Gossip
> ---
>
> Key: CASSANDRA-8218
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8218
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Minor
>
> We have seen that a bad node with memory corruption introduce random IPs into 
> Gossip which gets propagated across the cluster. We should not allow IPs to 
> be added to Gossip if it is not allowed by inter node Auth.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8197) Decide whether to allow case-insensitive system keyspace names

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8197:

Component/s: Distributed Metadata

> Decide whether to allow case-insensitive system keyspace names
> --
>
> Key: CASSANDRA-8197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8197
> Project: Cassandra
>  Issue Type: Task
>  Components: Distributed Metadata
>Reporter: Robert Stupp
>Priority: Minor
>
> The current implementation does not allow to create a keyspace like 
> {{sYsTeM}} or other case-variations of "system".
> But it allows that for the two other system keyspaces {{system_trace}} and 
> {{system_auth}}.
> IMO we should make that consistent at least for the system keyspaces.
> {code}
> cqlsh> create KEYSPACE "SyStEM" WITH replication = {'class': 
> 'SimpleStrategy', 'replication_factor': 1};
> code=2200 [Invalid query] message="system keyspace is not user-modifiable"
> cqlsh> create KEYSPACE "SYSTEM_tRaCeS" WITH replication = {'class': 
> 'SimpleStrategy', 'replication_factor': 1};
> cqlsh> create KEYSPACE "SYSTEM_aUtH" WITH replication = {'class': 
> 'SimpleStrategy', 'replication_factor': 1};
> cqlsh> DESC KEYSPACES ;
> "SYSTEM_aUtH"  system  "SYSTEM_tRaCeS"  system_traces
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9377) Untested commit log code found via code coverage

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9377:

Component/s: Testing

> Untested commit log code found via code coverage
> 
>
> Key: CASSANDRA-9377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9377
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Ariel Weisberg
>Priority: Major
> Attachments: jacoco.tgz
>
>
> It took some doing but I was finally able to extract coverage for just the 
> unit tests that test the commit log. Attached is the jacoco output as well as 
> the build.xml I used to get test-compression and test to run just the commit 
> log tests.
> This includes
> {noformat}
> CommitLogTest
> CommitLogFailurePolicyTest
> RecoveryManagerTest
> RecoveryManager2Test
> RecoveryManager3Test
> CommitLogStressTest
> {noformat}
> All tests were run with and without test-compression.
> Coverage is pretty good for some things with the missing coverage being 
> exceptional paths for things like files that aren't doing anything 
> exceptional in the tests.
> ReplayPosition implements equals and hashCode but has no coverage.
> CommitLogSegment.waitForFinalSync has no coverage.
> CommitLogDescriptor.fromFileName and fromHeader. CommitLogDescriptor 
> implements several equals methods that are not fully tested and also doesn't 
> implement hashCode to match the equality changes.
> CommitLog does not cover handleCommitError, nor forceRecyle*
> CommitLogReplayer is not well off. Not worth enumerating the issues just a 
> lot of error handling that is untested.
> CommitLogArchiver is in poor shape with no coverage for maybeRestoreArchive().
> CommitLogSegmentManager has a few important looking functions with 0 coverage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8219) Support for overflow disk

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8219:

Component/s: Core

> Support for overflow disk
> -
>
> Key: CASSANDRA-8219
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8219
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: sankalp kohli
>Priority: Minor
>
> During repairs and other operation like force compaction, the requirement of 
> disk space is more than normal.  We should add an option of overflow disk 
> which will only be used when the primary drive is x% full. Once the primary 
> drive has more space, all new data will again start going to primary drive. 
> Optionally, we can expose a way to compact/move all sstable in overflow 
> directory back into primary to free up overflow disk.  One of the use case of 
> this will be to use a spinning disk as overflow disk or share a disk between 
> multiple instances.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8202) Live Size Tests

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8202:

Component/s: Testing

> Live Size Tests
> ---
>
> Key: CASSANDRA-8202
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8202
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Philip Thompson
>Priority: Major
>
> 2.1.X has seen more than a handful of tickets related to incorrectly 
> computing live size. We need to expand our coverage of tests on this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9284) support joins if partition key is the same in both tables

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9284:

Component/s: Core

> support joins if partition key is the same in both tables
> -
>
> Key: CASSANDRA-9284
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9284
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jon Haddad
>Priority: Minor
>
> Based off this conversation: 
> https://mail-archives.apache.org/mod_mbox/cassandra-dev/201505.mbox/%3CCACUnPaAfJqU%2B86fFd4S6MS7Wv0KhpT_vavdkvDS%2Bm4Madi8_cg%40mail.gmail.com%3E
> It would be nice to have the flexibility of joins if we knew the query could 
> be satisfied on a single node.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8270:

Component/s: CQL

> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server 
> side slicing to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix 
> but if we extended slices to include sub-slices we could effectively request 
> for:
> {noformat}
> b(2,) c(3,)
> b(,) c(3,)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9228) Add documentation for Type Cast, Type Conversion and Function Overloading

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9228:

Component/s: Documentation and Website

> Add documentation for Type Cast, Type Conversion and Function Overloading
> -
>
> Key: CASSANDRA-9228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9228
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation and Website
>Reporter: Michaël Figuière
>Priority: Minor
>
> While digging around UDF overloading and type conversion it appeared that 
> several topics aren't currently documented in the CQL spec:
> * Explicit type casting has been added with CASSANDRA-5226 in Cassandra 1.2.2 
> and is currently not explained in the spec.
> * UDFs support overloading, this should be explained in the CQL spec, along 
> with a clear definition of what is considered as an ambiguous function call.
> * In this context it also becomes important to document the type conversion 
> rules, that is, which type A can be converted to which type B without raising 
> an error.
> * Not directly related but it would also be useful to document the column 
> type changes that are allowed in {{ALTER TABLE}} statements.
> All these fairly important topics are currently not obvious or hard to figure 
> out for the user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8048) Add WARN for large resultsets

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8048:

Component/s: Observability

> Add WARN for large resultsets
> -
>
> Key: CASSANDRA-8048
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8048
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Observability
>Reporter: Adam Hattrell
>Priority: Major
>
> Similarly to how we warn on excessive tombstones or large batches - it would 
> be useful to warn if we come across a large row over a specific threshold.
> If not too intrusive it could provide us with a smoking gun for gc thrashing 
> type issues.
> To put some meat on the bones - I've recently seen someone store a 1Gb blob 
> and then try to work round it by splitting it into 3 columns of 330 Mb each.  
> Neither gave particularly good performance.
> On the opposite side of the coin - a large number of columns grabbed at once 
> can also lead to problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8114) Better handle critical thread exits

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8114:

Component/s: Core

> Better handle critical thread exits
> ---
>
> Key: CASSANDRA-8114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8114
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: sankalp kohli
>Priority: Minor
>
> We have seen on 2 occasion where a critical thread exited due to some 
> exception and C* was still running. I see two options to detect such thread 
> deaths
> 1) Write a wrapper around such Runnable which takes some action if the thread 
> is about to exit.
> 2) Write something in uncaught exception handler which identifies a thread by 
> name and alerts if it is a critical thread. 
> Once we can better detect such things, we can configure action on it like 
> alerting or killing C*. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8060) Geography-aware, distributed replication

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8060:

Component/s: Streaming and Messaging
 Coordination

> Geography-aware, distributed replication
> 
>
> Key: CASSANDRA-8060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8060
> Project: Cassandra
>  Issue Type: Wish
>  Components: Coordination, Streaming and Messaging
>Reporter: Donald Smith
>Priority: Major
>
> We have three data centers in the US (CA in California, TX in Texas, and NJ 
> in NJ), two in Europe (UK  and DE), and two in Asia (JP and CH1).  We do all 
> our writing to CA.  That represents a bottleneck, since the coordinator nodes 
> in CA are responsible for all the replication to every data center.
> Far better if we had the option of setting things up so that CA replicated to 
> TX , which replicated to NJ. NJ is closer to UK, so NJ should be responsible 
> for replicating to UK, which should replicate to DE.  Etc, etc.
> This could be controlled by the topology file.
> The replication could be organized in a tree-like structure instead of a 
> daisy-chain.
> It would require architectural changes and would have major ramifications for 
> latency but might be appropriate for some scenarios.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8050) Benchmark huge page sizes

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8050:

Component/s: Testing

> Benchmark huge page sizes
> -
>
> Key: CASSANDRA-8050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8050
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Ryan McGuire
>Priority: Minor
>
> We should run a cstar_perf comparison using [huge page 
> sizes|http://dino.ciuffetti.info/2011/07/howto-java-huge-pages-linux/]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9330) CAS timeout errors should use a different exception than WriteTimeoutException as WTE can happen when nodes fail to respond.

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9330:

Component/s: Coordination

> CAS timeout errors should use a different exception than 
> WriteTimeoutException as WTE can happen when nodes fail to respond.
> 
>
> Key: CASSANDRA-9330
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9330
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
>Reporter: Aaron Whiteside
>Priority: Major
>  Labels: LWT
>
> Perhaps a CASContentionTimeoutException?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9178) Test exposed JMX methods

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-9178:

Component/s: Testing

> Test exposed JMX methods
> 
>
> Key: CASSANDRA-9178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9178
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Carl Yeksigian
>Assignee: Ryan McGuire
>Priority: Major
>
> [~thobbs] added support for JMX testing in dtests, and we have seen issues 
> related to nodetool testing in various different stages of execution. Tests 
> which exercise the different methods which nodetool calls should be added to 
> catch those issues early.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8121) Audit acquire/release SSTable references

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8121:

Component/s: Core

> Audit acquire/release SSTable references
> 
>
> Key: CASSANDRA-8121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8121
> Project: Cassandra
>  Issue Type: Task
>  Components: Core
>Reporter: Richard Low
>Priority: Major
>
> There are instances where SSTable references are not guaranteed to be 
> released (e.g. CompactionTask.runWith) because there is no try/finally around 
> the reference acquire/release. We should audit all places where SSTable 
> references are acquired and wrap them appropriately. Leaked references cause 
> junk files to build up on disk and on a restart can lead to data resurrection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7955) Range movements can violate consistency if RING_DELAY <= write_request_timeout_in_ms

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7955:

Component/s: Coordination

> Range movements can violate consistency if RING_DELAY <= 
> write_request_timeout_in_ms
> 
>
> Key: CASSANDRA-7955
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7955
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
>Reporter: Benedict
>Priority: Minor
>
> Cassandra should probably refuse to start if RING_DELAY is too close to (or 
> below) write_request_timeout_ms, because we depend on this for 
> correctness/consistency during range movements
> We should probably also consider throwing a WriteTimeoutException _even if we 
> don't get interrupted by the timeout, since there are reasons due to 
> scheduling or system overload this could not happen (though it is unlikely to 
> be significant enough to have an impact, it's better safe than sorry)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7900) Snapshot tests are not platform neutral

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7900:

Component/s: Testing

> Snapshot tests are not platform neutral
> ---
>
> Key: CASSANDRA-7900
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7900
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
> Environment: Windows
>Reporter: Philip Thompson
>Priority: Major
>  Labels: Windows
>
> The various snapshot and commit log archiving tests in snapshot_test.py are 
> failing on Windows platforms. This appears to be the fault of test behavior 
> due to extensive operations in the file system that fail to consider the test 
> may be running on Windows. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7958) Create jenkins targets to run CQLTester unit tests with prepared statements and without

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7958:

Component/s: Testing
 Build

> Create jenkins targets to run CQLTester unit tests with prepared statements 
> and without
> ---
>
> Key: CASSANDRA-7958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7958
> Project: Cassandra
>  Issue Type: Test
>  Components: Build, Testing
>Reporter: Ryan McGuire
>Assignee: Michael Shuler
>Priority: Major
>
> The CQL tests within the unit test code has the ability to run with prepared 
> statements, or without, using the cassandra.test.use_prepared flag. We should 
> create two jenkins targets on CassCI to run it both ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7907) Determine how many network threads we need for native transport

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7907:

Component/s: CQL

> Determine how many network threads we need for native transport
> ---
>
> Key: CASSANDRA-7907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7907
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benedict
>Priority: Minor
>
> With the introduction of CASSANDRA-4718, it is highly likely we can cope with 
> just _one_ network IO thread. We could even try pinning it to a single 
> (optionally configurable) core, and (also optionally) pin all other threads 
> to a different core, so that we can guarantee extremely prompt execution (and 
> if pinned to the correct core the OS uses for managing the network, improve 
> throughput further).
> Testing this out will be challenging, as we need to simulate clients from 
> lots of IPs. However, it is quite likely this would reduce the percentage of 
> time spent in kernel networking calls, and the amount of context switching.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7999) Inconsistent logging in CassandraDaemon

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7999:

Component/s: Testing

> Inconsistent logging in CassandraDaemon
> ---
>
> Key: CASSANDRA-7999
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7999
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Shaun Kalley
>Priority: Minor
>
> Both fatal and non-fatal errors are being logged at error level which creates 
> a problem for monitoring.  CassandraDaemon.setup() logs an error-level 
> message if a directory does not exist but is still able to continue if the 
> directory can be created, and CassandraDaemon.activate() logs an error-level 
> message if the MBean cannot be registered but still allows the server to 
> start.  I'd like to see the logging levels for these issues downgraded to 
> warning, or the levels for the fatal errors upgraded to fatal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7746) fix upgrade_through_versions local mode for rolling upgrades

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7746:

Component/s: Testing

> fix upgrade_through_versions local mode for rolling upgrades
> 
>
> Key: CASSANDRA-7746
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7746
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Russ Hatch
>Priority: Minor
>
> Upgrade tests now have a local mode for easier debugging with local code 
> modifications (instead of fetching remote versions). This works by locally 
> changing branches and building cassandra as needed.
> There's a potential problem for rolling upgrades because the code will be 
> updated while some of the nodes are still using the last version. I'm not 
> certain if this really affects much in practice but should be relatively 
> straightforward to fix.
> Instead of switching branches, the tests will need to check out the target 
> branch/tag to another location on the filesystem. When upgrading, another 
> version will be checked out to another location on the filesystem, etc. To 
> keep this (hopefully) platform agnostic we should be able to use the python 
> tempfile module, and git can probably be used to check out a single branch as 
> described here: 
> http://stackoverflow.com/questions/1778088/how-to-clone-a-single-branch-in-git



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7880) Create a new system table "schema_change_history"

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7880:

Component/s: Distributed Metadata

> Create a new system table "schema_change_history"
> -
>
> Key: CASSANDRA-7880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7880
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Distributed Metadata
>Reporter: Michaël Figuière
>Priority: Minor
>
> The current way Cassandra handle schema modification can lead to some schema 
> disagreements as DDL statements execution doesn't come with any absolute 
> guarantee. I understand that entirely seamless schema updates in such a 
> distributed system will be challenging to reach and probably not a high 
> priority for now.
> That being said these disagreements can sometime lead to challenging 
> situation for scripts or tools that need things to be in order to move on. To 
> clarify the situation, help the user to figure out what's going on, as well 
> as to properly log these sensitive operations, it would be interesting to add 
> a {{schema_change_history}} table in the {{system}} keyspace.
> I would expect it to be local to a node and to contain the following 
> information:
> * DDL statement that has been executed
> * User login used for the operation
> * IP of the client that originated the request
> * Date/Time of the change
> * Schema version before the change
> * Schema version after the change
> Under normal conditions, Cassandra shouldn't handle a massive amount of DDL 
> statements so this table should grow at a descent pace. Nevertheless to bound 
> its growth we can consider adding a TTL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7723) sstable2json (and possibly other command-line tools) hang if no write permission to the commitlogs

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7723:

Component/s: Tools

> sstable2json (and possibly other command-line tools) hang if no write 
> permission to the commitlogs
> --
>
> Key: CASSANDRA-7723
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7723
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: J.B. Langston
>Priority: Minor
>
> sstable2json (and potentially other command-line tools that call 
> DatabaseDescriptor.loadSchemas) will hang if the user running them doesn't 
> have write permission on the commit logs.  loadSchemas calls 
> Schema.updateVersion, which causes a mutation to the system tables, then it 
> just spins forever trying to acquire a commit log segment.  See this thread 
> dump: https://gist.github.com/markcurtis1970/837e770d1cad5200943c. The tools 
> should recognize this and present an understandable error message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7720) Add a more consistent snapshot mechanism

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7720:

Component/s: Tools

> Add a more consistent snapshot mechanism
> 
>
> Key: CASSANDRA-7720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Mike Schrag
>Priority: Major
>
> We’ve hit an interesting issue with snapshotting, which makes sense in 
> hindsight, but presents an interesting challenge for consistent restores:
> * initiate snapshot
> * snapshotting flushes table A and takes the snapshot
> * insert into table A
> * insert into table B
> * snapshotting flushes table B and takes the snapshot
> * snapshot finishes
> So what happens here is that we end up having a B, but NOT having an A, even 
> though B was chronologically inserted after A.
> It makes sense when I think about what snapshot is doing, but I wonder if 
> snapshots actually should get a little fancier to behave a little more like 
> what I think most people would expect. What I think should happen is 
> something along the lines of the following:
> For each node:
> * pass a client timestamp in the snapshot call corresponding to "now"
> * snapshot the tables using the existing procedure
> * walk backwards through the linked snapshot sstables in that snapshot
>   * if the earliest update in that sstable is after the client's timestamp, 
> delete the sstable in the snapshot
>   * if the earliest update in the sstable is before the client's timestamp, 
> then look at the last update. Walk backwards through that sstable.
> * if any updates fall after the timestamp, make a copy of that sstable in 
> the snapshot folder only up to the point of the timestamp and then delete the 
> original sstable in the snapshot (we need to copy because we're likely 
> holding a shared hard linked sstable)
> I think this would guarantee that you have a chronologically consistent view 
> of your snapshot across all machines and columnfamilies within a given 
> snapshot.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7548) Disk I/O priority control for Compaction and Flusher

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7548:

Component/s: Core

> Disk I/O priority control for Compaction and Flusher
> 
>
> Key: CASSANDRA-7548
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7548
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Hanson
>Priority: Major
>
> Disk I/O priority: Memtables Flusher shall have higher priority than 
> Compaction.
> This is to avoid DB Insert/Update hung (spikes) during SSTables Compaction. 
> The Compaction shall be able to detect the in-progress of Memtables flushing, 
> and slow down itself for disk I/O.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7589) Option to disable min timestamp check for TTL compaction enhancements

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7589:

Component/s: Compaction

> Option to disable min timestamp check for TTL compaction enhancements
> -
>
> Key: CASSANDRA-7589
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7589
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Matt Stump
>Priority: Minor
>
> As part of CASSANDRA-5228 we added an enhancement to unlink SSTables for TTL 
> use cases if certain checks are met. One of those checks is that for an 
> SSTable to be deleted the maximum timestamp must not be less than the minimum 
> timestamp for all other SSTables. This makes sense for use cases where GC 
> grace is >= the TTL, or use case where deletes are performed by the 
> application.
> For use cases where GC grace is less than the TTL, and where deletes are only 
> performed via TTL then these checks result in SSTables that could be safely 
> deleted sticking around for some time. In practice the TTL related 
> enhancements kick in very infrequently and most SSTables go through the 
> normal compaction process.
> What I propose is a CF level setting that disables the check, so that an 
> SSTable can be unlinked once time() >= max TTL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7530) Repair should be allowed in mixed version clusters

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7530:

Component/s: Repair

> Repair should be allowed in mixed version clusters
> --
>
> Key: CASSANDRA-7530
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7530
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Repair
>Reporter: Jeremiah Jordan
>Priority: Major
>  Labels: ponies
>
> As we try to improve the ops experience for large clusters (or cautious 
> people), we need to make it so repair works across major version upgrades.
> Currently you have to be able to upgrade all of your nodes and then run a 
> full repair, within the gc_grace window.
> The recent ability to stream old sstables makes this a little better, as you 
> don't have to also run upgradesstables first.
> It would be really good if we could let people pause their upgrade of 1000 
> nodes to run some repairs.  Or to upgrade one DC and test the new major 
> version for a while (while still running repairs).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7597) Remove static initializer in DatabaseDescriptor

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7597:

Component/s: Lifecycle

> Remove static initializer in DatabaseDescriptor
> ---
>
> Key: CASSANDRA-7597
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7597
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
> Environment: Cassandra 2.0.9 (earlier version should be affected as 
> well)
>Reporter: Pavel Sakun
>Priority: Major
> Attachments: 7597.txt
>
>
> As discussed below, it's difficult to properly react on invalid configuration 
> values in a client tool that uses cassandra code (here: an sstable loader).
> Reason is that the static initializer in DatabaseDescriptor calls System.exit 
> in case of configuration failures.
> Recommend to implement some "loadAndApplyConfig" method on DatabaseDescriptor 
> and remove the static initializer and let the calling code react accordingly 
> (print error, exit VM).
> All direct and indirect uses of DatabaseDescriptor must be catched to solve 
> this ticket - so this is not a 2.1 ticket.
> --
> Old Description:
> We're using SSTableSimpleUnsortedWriter API to generate SSTable to be loaded 
> into cassandra. In case of any issue with config DatabaseDescriptor calls 
> System.exit() which is apparently not the thing you expect while using API.
> Test case is simple:
> System.setProperty( "cassandra.config", "" );
> new YamlConfigurationLoader().loadConfig();
> Thread.sleep( 5000 );
> System.out.println("We're still alive"); // this will never be called



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7439) There are a number of places where Gossiper.instance.getEndpointStateForEndpoint is not checked for null

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7439:

Component/s: Distributed Metadata

> There are a number of places where 
> Gossiper.instance.getEndpointStateForEndpoint is not checked for null
> 
>
> Key: CASSANDRA-7439
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7439
> Project: Cassandra
>  Issue Type: Task
>  Components: Distributed Metadata
>Reporter: Mike Adamson
>Priority: Minor
>
> In most places the getEndpointStateForEndpoint method is null checked but in 
> a number of places it isn't.
> Without testing each individual call point it is difficult to tell from the 
> context whether the state will be null or not.
> In this case wouldn't it make sense to null every call to this method to 
> avoid race conditions and NPEs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7540) When indicating corruption, specify the sstable

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7540:

Component/s: Observability

> When indicating corruption, specify the sstable
> ---
>
> Key: CASSANDRA-7540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7540
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Jeremy Hanna
>Priority: Major
>
> In the 2.0 line, there was #6285 where hsha introduced some corruption.  If 
> you're hit by this, you get the DecoratedKey message that indicates that one 
> of the sstables getting compacted had the error in it.  It would be much 
> nicer to know which sstable had that problem.  Otherwise, you're left having 
> to ss2j each of the sstables involved in the compaction until you've found 
> it/them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7184) improvement of SizeTieredCompaction

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7184:

Component/s: Compaction

> improvement  of  SizeTieredCompaction
> -
>
> Key: CASSANDRA-7184
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7184
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jianwei Zhang
>Assignee: Jianwei Zhang
>Priority: Minor
>  Labels: compaction
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> 1,  In our usage scenario, there is no duplicated insert and no delete . The 
> data increased all the time, and some big sstables are generated (100GB for 
> example).  we don't want these sstables to participate in the 
> SizeTieredCompaction any more. so we add a max threshold which is set to 
> 100GB . Sstables larger than the threshold will not be compacted. Should this 
> strategy be added to the trunk ?
> 2,  In our usage scenario, maybe hundreds of sstable need to be compacted in 
> a Major Compaction. The total size would be larger to 5TB. So during the 
> compaction, when the size writed reach to a configed threshhold(200GB for 
> example), it switch to write a new sstable. In this way, we avoid to generate 
> too huge sstables. Too huge sstable have some bad infuence: 
>  (1) It will be larger than the capacity of a disk;
>  (2) If the sstable is corrupt, lots of objects will be influenced .
> Should this strategy be added to the trunk ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7426) ALTER SYSTEM / nodetool enhancements

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7426:

Component/s: Distributed Metadata
 Configuration

> ALTER SYSTEM / nodetool enhancements
> 
>
> Key: CASSANDRA-7426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7426
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Distributed Metadata
>Reporter: Robert Stupp
>Priority: Major
>  Labels: cql
>
> CASSANDRA-7370 mentions the idea to enhance nodetool to do something like 
> {{nodetool> set config foo='bar'}}. This could also be done via a "special" 
> CQL {{ALTER SYSTEM SET foo='bar';}}.
> In general this ticket is just meant not keep in mind that there should be 
> some option to change configuration using a generic approach. Either 
> implementation (nodetool via JMX or CQL statement) should internally end at 
> the same code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7213) look into failure testing using libfiu

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7213:

Component/s: Testing

> look into failure testing using libfiu
> --
>
> Key: CASSANDRA-7213
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7213
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>Priority: Major
>
> libfiu might be a useful library for simulating system failures. We should 
> look into using this within cassandra-dtest or independently. Ideally we'll 
> want some control over how specific test nodes should behave ("node A should 
> fail as if problem X has occurred"), so we'll want to look into somehow 
> conditionally failing calls for specific nodes and perhaps intermittent 
> probabilistic failures too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7296) Add CL.COORDINATOR_ONLY

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7296:

Component/s: Coordination

> Add CL.COORDINATOR_ONLY
> ---
>
> Key: CASSANDRA-7296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7296
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
>Reporter: Tupshin Harper
>Priority: Major
>
> For reasons such as CASSANDRA-6340 and similar, it would be nice to have a 
> read that never gets distributed, and only works if the coordinator you are 
> talking to is an owner of the row.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7180) Investigate mutation testing for unit tests

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7180:

Component/s: Testing

> Investigate mutation testing for unit tests
> ---
>
> Key: CASSANDRA-7180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7180
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Russ Hatch
>Assignee: Russ Hatch
>Priority: Major
>
> We might get some useful information from mutation testing tools, possibly 
> PIT (http://pitest.org/). The basic idea is to seed faults in the code and 
> check to see if the unit tests are able to catch them. If we can get this up 
> and running, perhaps it could be run periodically on cassci.datastax.com.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7433) Improve information for Unknown table/cf error message

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7433:

Component/s: Observability

> Improve information for Unknown table/cf error message
> --
>
> Key: CASSANDRA-7433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7433
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Adam Hattrell
>Priority: Minor
>
> Could we add some further details to the following types of error:
> {code:none}
> ERROR [Thread-11235] 2014-06-20 11:08:20,278 StorageService.java (line 2436) 
> Repair session failed: 
> java.lang.IllegalArgumentException: Unknown table/cf pair 
> (counterpartyrisk.deals20140602HPCE_OFFICIAL) 
> at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165) 
> at 
> org.apache.cassandra.service.StorageService.getValidColumnFamilies(StorageService.java:2293)
>  
> at 
> org.apache.cassandra.service.StorageService.forceTableRepair(StorageService.java:2485)
>  
> at 
> org.apache.cassandra.service.StorageService$5.runMayThrow(StorageService.java:2432)
>  
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Specifically that this is expected if you have recently dropped the 
> associated column family.  I don't think this should be dropped to a WARN as 
> if you haven't changed your schema recently then this probably indicates a 
> significant problem.  In the majority of cases though - it can probably be 
> safely ignored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7065) Add some extra metadata in leveled manifest to be able to reduce the amount of sstables searched on read path

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7065:

Component/s: Local Write-Read Paths

> Add some extra metadata in leveled manifest to be able to reduce the amount 
> of sstables searched on read path
> -
>
> Key: CASSANDRA-7065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7065
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Marcus Eriksson
>Priority: Major
>  Labels: lcs
>
> Based on this;
> http://rocksdb.org/blog/431/indexing-sst-files-for-better-lookup-performance/
> By keeping pointers from the sstables in lower to higher levels we could 
> reduce the number of candidates in higher levels, ie, instead of searching 
> all 1000 L3 sstables, we use the information from the L2 search to include 
> less L3 sstables.
> First we need to figure out if this can beat our IntervalTree approach (and 
> if the win is worth it).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8342) Remove historical guidance for concurrent reader and writer tunings.

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8342:

Component/s: Documentation and Website

> Remove historical guidance for concurrent reader and writer tunings.
> 
>
> Key: CASSANDRA-8342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8342
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Matt Stump
>Priority: Major
>
> The cassandra.yaml and documentation provide guidance on tuning concurrent 
> readers or concurrent writers to system resources (cores, spindles). Testing 
> performed by both myself and customers demonstrates no benefit for thread 
> pool sizes above 64 in size, and for thread pools greater than 128 in size a 
> decrease in throughput. This is due to thread scheduling and synchronization 
> bottlenecks within Cassandra. 
> Additionally, for lower end systems reducing these thread pools provides very 
> little benefit because the bottleneck is typically moved to either IO or CPU.
> I propose that we set the default value to 64 (current default is 32), and 
> remove all guidance/recommendations regarding tuning.
> This recommendation may change in 3.0, but that would require further 
> experimentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8433) Add jmx and nodetool controls to reset lifetime metrics to zero

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8433:

Component/s: Metrics

> Add jmx and nodetool controls to reset lifetime metrics to zero
> ---
>
> Key: CASSANDRA-8433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8433
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Metrics
>Reporter: Donald Smith
>Priority: Minor
>
> Often I change some parameter in cassandra, in the OS, or in an external 
> component and want to see the effect on cassandra performance.  Because some 
> the jmx metrics are for the lifetime of the process, it's hard to see the 
> effect of changes.  It's inconvenient to restart all the nodes. And if you 
> restart only some nodes (as I often do) then only those metrics reset to zero.
> The jmx interface should provide a way to reset all lifetime metrics to zero. 
>  And *nodetool* should invoke that to allow resetting metrics from the 
> command line.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8470) Commit Log / Memtable Flush Correctness Stress Test

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8470:

Component/s: Testing

> Commit Log / Memtable Flush Correctness Stress Test
> ---
>
> Key: CASSANDRA-8470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8470
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Testing
>Reporter: Benedict
>Priority: Major
>
> CASSANDRA-8383 should have been detected with automated testing. We should 
> introduce a stress test designed to expose any bugs in the core data paths.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8612) Read metrics should be updated on all types of reads

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8612:

Component/s: Metrics

> Read metrics should be updated on all types of reads
> 
>
> Key: CASSANDRA-8612
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8612
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Metrics
>Reporter: Chris Lohfink
>Priority: Minor
>  Labels: lhf, metrics
>
> Metrics like "sstables per read" are not updated on a range slice.  Although 
> separating things out for each type of read could make sense like we do for 
> latencies, only exposing the metrics for one type can be a little confusing 
> when people do a query and see nothing increases.  I think its sufficient to 
> use the same metrics for all reads.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6990) Log ClosedChannelException at DEBUG at shutdown in CompressedRandomAccessReader

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6990:

Component/s: Observability

> Log ClosedChannelException at DEBUG at shutdown in 
> CompressedRandomAccessReader
> ---
>
> Key: CASSANDRA-6990
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6990
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Jeremy Hanna
>Priority: Minor
>
> Related to CASSANDRA-5368, there is another spot where we are not catching 
> ClosedChannelException messages on shutdown.
> The trace is:
> {quote}
> ERROR [Deserialize 
> SSTableReader(path='/data10/cassandra/ks1/cf1/ks1-cf1-ic-246-Data.db')] 
> 2014-04-06 04:01:00,753 CassandraDaemon.java (line 191) Exception in thread 
> Thread[Deserialize 
> SSTableReader(path='/data10/cassandra/ks1/cf1/ks1-cf1-ic-246-Data.db'),1,main]
> FSReadError in /data10/cassandra/ks1/cf1/ks1-cf1-ic-246-Data.db
>   at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:93)
>   at 
> org.apache.cassandra.io.compress.CompressedThrottledReader.reBuffer(CompressedThrottledReader.java:45)
>   at 
> org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:327)
>   at java.io.RandomAccessFile.readInt(RandomAccessFile.java:756)
>   at java.io.RandomAccessFile.readLong(RandomAccessFile.java:792)
>   at 
> org.apache.cassandra.utils.BytesReadTracker.readLong(BytesReadTracker.java:114)
>   at 
> org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:107)
>   at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:92)
>   at 
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumnsFromSSTable(ColumnFamilySerializer.java:149)
>   at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234)
>   at 
> org.apache.cassandra.db.compaction.ParallelCompactionIterable$Deserializer$1.runMayThrow(ParallelCompactionIterable.java:291)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at java.lang.Thread.run(Thread.java:662)
> Caused by: java.nio.channels.ClosedChannelException
>   at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
>   at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:243)
>   at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.checksum(CompressedRandomAccessReader.java:140)
>   at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:127)
>   at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:85)
>   ... 12 more
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6794) Optimise slab allocator to enable higher number of column families

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6794:

Component/s: Core

> Optimise slab allocator to enable higher number of column families
> --
>
> Key: CASSANDRA-6794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6794
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Jeremy Hanna
>Priority: Minor
>
> Currently the slab allocator allocates 1MB per column family.  This has been 
> very beneficial for gc efficiency.  However, it makes it more difficult to 
> have large numbers of column families.
> It would be preferable to have a more intelligent way to allocate slabs so 
> that there is more flexibility between slab allocator and non-slab allocator 
> behaviour.
> A simple first step is to ramp up size of slabs from small (say  8KB) when 
> empty, to 1MB after a few slabs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7061) High accuracy, low overhead local read/write tracing

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7061:

Component/s: Observability

> High accuracy, low overhead local read/write tracing
> 
>
> Key: CASSANDRA-7061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7061
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Benedict
>Priority: Major
>
> External profilers are pretty inadequate for getting accurate information at 
> the granularity we're working at: tracing is too high overhead, so measures 
> something completely different, and sampling suffers from bias of attribution 
> due to the way the stack traces are retrieved. Hyperthreading can make this 
> even worse.
> I propose to introduce an extremely low overhead tracing feature that must be 
> enabled with a system property that will trace operations within the node 
> only, so that we can perform various accurate low level analyses of 
> performance. This information will include threading info, so that we can 
> trace hand off delays and actual active time spent processing an operation. 
> With the property disabled there will be no increased burden of tracing, 
> however I hope to keep the total trace burden to less than one microsecond, 
> and any single trace command to a few tens of nanos.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6917) enum data type

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6917:

Issue Type: New Feature  (was: Improvement)

> enum data type
> --
>
> Key: CASSANDRA-6917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6917
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Benedict
>Priority: Minor
>  Labels: performance
>
> It seems like it would be useful to support an enum data type, that 
> automatically converts string data from the user into a fixed-width data type 
> with guaranteed uniqueness across the cluster. This data would be replicated 
> to all nodes for lookup, but ideally would use only the keyspace RF to 
> determine nodes for coordinating quorum writes/consistency.
> This would not only permit improved local disk and inter-node network IO for 
> symbology information (e.g. stock tickers, ISINs, etc), but also potentially 
> for column identifiers also, which are currently stored as their full string 
> representation.
> It should be possible then with later updates to propagate the enum map 
> (lazily) to clients through the native protocol, reducing network IO further.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6903) Allow a token scan to filter on partition key columns

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6903:

Component/s: Core

> Allow a token scan to filter on partition key columns
> -
>
> Key: CASSANDRA-6903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6903
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Andy Neilson
>Priority: Major
>
> When extracting data for analysis (e.g., in Hadoop) using a token scan, 
> allowing filtering on column that is part of the partition key allow for more 
> efficient processing. For example, assume that we have the following schema 
> (from the example defined 
> [here|http://planetcassandra.org/blog/post/getting-started-with-time-series-data-modeling/]):
> {noformat}
> CREATE TABLE temperature_by_day (
>weatherstation_id text,
>date text,
>event_time timestamp,
>temperature text,
>PRIMARY KEY ((weatherstation_id,date),event_time)
> );
> {noformat}
> Assume that I am primarily interested in doing analysis of more recent data, 
> so I can use a SELECT like the following to extract the data I am interested 
> in:
> {noformat}
> SELECT *
> FROM temperature_by_day
> WHERE token(weatherstation_id,date) > ? AND token(weatherstation_id,date) <= ?
> AND event_time >= ?
> LIMIT 5000
> ALLOW FILTERING;
> {noformat}
> The filtering is potentially expensive since it touches a lot of columns. 
> Since the {{date}} column that is used to fragment wide rows is related to 
> the {{event_time}}, I could apply a (redundant) filter to {{date}}, as in:
> {noformat}
> SELECT *
> FROM temperature_by_day
> WHERE token(weatherstation_id,date) > ? AND token(weatherstation_id,date) <= ?
> AND event_time >= ?
> AND date >= ?
> LIMIT 5000
> ALLOW FILTERING;
> {noformat}
> ...but currently I can't add the filter on the {{date}} column because it is 
> part of the partition key. However, because this query is doing a token scan, 
> there really is no problem in filtering on the partition key. The predicate 
> on {{date}} can be evaluated directly on the row index without looking at the 
> values in columns at all. The effect is to efficiently filter out a large 
> swath of rows, and not forcing the scan to filter on rows that couldn't 
> possibly contain those dates.
> There are probably lots of ways to optimize predicates on partition key 
> columns. For example, if the {{date}} column was made the first column in the 
> partition key, evaluating a range could be done without scanning the entire 
> row index.
> In this case, if we have a year of data, but are only interested in 
> extracting the last day, so the overhead of filtering is reduced by a factor 
> of 365. 
> What I am looking for is:
> * If the SELECT is a token scan, allow filtering on partition key columns.
> * Predicates on partition key columns are evaluated on for the row as a 
> whole, before filtering on clustering columns.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6897) Add checksum to the Summary File and Bloom Filter file of SSTables

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6897:

Component/s: Local Write-Read Paths

> Add checksum to the Summary File and Bloom Filter file of SSTables
> --
>
> Key: CASSANDRA-6897
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6897
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Adam Hattrell
>Priority: Major
>
> Could we add a checksum to the Summary file and filter file of the SSTable. 
> Since reads the whole bloom filter before actually reading data, it seems 
> like it would make sense to checksum the bloom filter to make sure there is 
> no corruption there. Same is true with the summary file. The core of our 
> question is, can you add checksumming to all elements of the SSTable so if we 
> read anything corrupt we immediately see a failure?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7022) Consider an option to reduce tracing cost by writing only to sessions table (not events)

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7022:

Component/s: Observability

> Consider an option to reduce tracing cost by writing only to sessions table 
> (not events)
> 
>
> Key: CASSANDRA-7022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7022
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
> Environment: C* 2.0.6 running on Ubuntu 12.04
>Reporter: Bill Joyce
>Priority: Minor
>
> With MySQL and SQL Server, I can profile all queries in high traffic 
> production environments. I'm assuming the bulk of the C* tracing cost comes 
> in writing to the system_traces.events table, so it would be great to have an 
> option to write just the system_traces.session info if that allows me to run 
> 'nodetool settraceprobability' with a higher probability (ideally a 
> probability of 1). This along with CASSANDRA-7021 would go a long way in 
> giving us performance analysis closer to what can be done with more mature 
> back ends.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6768) Refresh permissions cache in ClientState periodically to avoid cache miss stampede effect

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6768:

Component/s: Auth

> Refresh permissions cache in ClientState periodically to avoid cache miss 
> stampede effect
> -
>
> Key: CASSANDRA-6768
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6768
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Auth
>Reporter: Michał Michalski
>Assignee: Michał Michalski
>Priority: Major
>  Labels: authentication
>
> h2. Background
> We want to password-protect Cassandra by using the built-in 
> PasswordAuthenticator and PasswordAuthorizer. In general we are happy with 
> this solution, but after reviewing the code we're a bit afraid of default  
> permissionsCache behaviour in org.apache.cassandra.service.ClientState.
> h2. Problem
> From what I understand, at the moment cache expires every N seconds (2 by 
> default) and it gets repopulated when permissionsCache.get() is being  
> called. However, as we're talking about at least a few hundreds requests to 
> Cassandra per second, we're afraid of the "stampede" effect once the cache 
> expires and a number of queries will "trigger" its reload simultaneously 
> during the short period of time when it will be empty.
> h2. Proposed Solutions
> Therefore, instead of the current solution, we'd prefer this cache to be 
> reloaded "in background" every N seconds, so it's only a single request every 
> N seconds, rather than tens (hundreds?) of them just after the cache expires 
> during the period when it's empty.
> In other words, we're thinking about two solutions (updated after comments 
> from [~jjordan] and [~iamaleksey]):
> h3. Make the ClientState's permissionsCache configurable. 
> Let's define additional configurable variable called refreshPeriod and make 
> it 0 by default (0 means no refresh - nothing changes in current code). If 
> it's > 0, add the refreshAfterWrite to the existing code.
> This way we keep the defaults "safe", but add possibility to "tune" the cache 
> when someone needs it. Any cons?
> h3. Make Authorizer responsible for its own cache
> As I said, I believe that Authorizer should be responsible for defining its 
> cache, so I'd prefer to add a getPermissionsCache method to IAuthorizer and 
> get rid of the cache creating code in ClientState. Of course it's a bigger 
> change, it breaks the existing interface, but from my point of view it's the 
> better choice. 
> Of course there's no problem to combine these two options and make the 
> per-Authorizer cache configurable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6760) Nodetool command to disable reads

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6760:

Component/s: Configuration

> Nodetool command to disable reads
> -
>
> Key: CASSANDRA-6760
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6760
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: sankalp kohli
>Priority: Minor
>
> There is no way to stop a node from accepting reads and still be part of the 
> ring. 
> This will be helpful in case node does not bootstrap properly and we need to 
> run node tool rebuild to fetch the data. 
> The node can use gossip to propagate this fact to other nodes so that they 
> don't forward reads to it. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6588) Add a 'NO EMPTY RESULTS' filter to SELECT

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6588:

Component/s: CQL

> Add a 'NO EMPTY RESULTS' filter to SELECT
> -
>
> Key: CASSANDRA-6588
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6588
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Sylvain Lebresne
>Priority: Minor
>
> It is the semantic of CQL that a (CQL) row exists as long as it has one 
> non-null column (including the PK columns, which, given that no PK columns 
> can be null, means that it's enough to have the PK set for a row to exist). 
> This does means that the result to
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v1 int, v2 int);
> INSERT INTO test(k, v1) VALUES (0, 4);
> SELECT v2 FROM test;
> {noformat}
> must be (and is)
> {noformat}
>  v2
> --
>  null
> {noformat}
> That fact does mean however that when we only select a few columns of a row, 
> we still need to find out rows that exist but have no values for the selected 
> columns. Long story short, given how the storage engine works, this means we 
> need to query full (CQL) rows even when only some of the columns are selected 
> because that's the only way to distinguish between "the row exists but have 
> no value for the selected columns" and "the row doesn't exist". I'll note in 
> particular that, due to CASSANDRA-5762, we can't unfortunately rely on the 
> row marker to optimize that out.
> Now, when you selects only a subsets of the columns of a row, there is many 
> cases where you don't care about rows that exists but have no value for the 
> columns you requested and are happy to filter those out. So, for those cases, 
> we could provided a new SELECT filter. Outside the potential convenience (not 
> having to filter empty results client side), one interesting part is that 
> when this filter is provided, we could optimize a bit by only querying the 
> columns selected, since we wouldn't need to return rows that exists but have 
> no values for the selected columns.
> For the exact syntax, there is probably a bunch of options. For instance:
> * {{SELECT NON EMPTY(v2, v3) FROM test}}: the vague rational for putting it 
> in the SELECT part is that such filter is kind of in the spirit to DISTINCT.  
> Possibly a bit ugly outside of that.
> * {{SELECT v2, v3 FROM test NO EMPTY RESULTS}} or {{SELECT v2, v3 FROM test 
> NO EMPTY ROWS}} or {{SELECT v2, v3 FROM test NO EMPTY}}: the last one is 
> shorter but maybe a bit less explicit. As for {{RESULTS}} versus {{ROWS}}, 
> the only small object to {{NO EMPTY ROWS}} could be that it might suggest it 
> is filtering non existing rows (I mean, the fact we never ever return non 
> existing rows should hint that it's not what it does but well...) while we're 
> just filtering empty "resultSet rows".
> Of course, if there is a pre-existing SQL syntax for that, it's even better, 
> though a very quick search didn't turn anything. Other suggestions welcome 
> too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6672) Support for Microsecond Resolution Time UUIDs in CQL3

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6672:

Component/s: CQL

> Support for Microsecond Resolution Time UUIDs in CQL3
> -
>
> Key: CASSANDRA-6672
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6672
> Project: Cassandra
>  Issue Type: Wish
>  Components: CQL
>Reporter: Drew Kutcharian
>Priority: Major
>
> Currently CQL3 supports time uuid based functions (now, unixtimestampof, 
> dateof, ...) that deal with millisecond resolution time uuids. I think it 
> will be a good idea to have the microsecond resolution version of those 
> functions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6727) Specify particular endpoints to be used during bootstrap

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6727:

Component/s: Streaming and Messaging

> Specify particular endpoints to be used during bootstrap 
> -
>
> Key: CASSANDRA-6727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6727
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: sankalp kohli
>Priority: Minor
>
> Sometimes there is a need to bootstrap a node from a set of endpoints which 
> you want to specify. We should allow specifying set of endpoints while 
> starting to be used and override the endpoints which get picked up due to 
> snitch. 
> Here is a example in which it will be useful. 
> A cluster is being upgraded and before upgradestable could be run on all 
> replicas, we want to replace a node. 
> With this feature, we can specify a node on which we have already run 
> upgradestable and bootstrap a node. 
> This feature is less useful with v-nodes.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6444) Have a nodetool command which emits true data size

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6444:

Component/s: Tools

> Have a nodetool command which emits true data size
> --
>
> Key: CASSANDRA-6444
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6444
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: sankalp kohli
>Priority: Minor
>
> Sometimes we have an unbalanced clusters and it is difficult to know whether 
> some nodes are taking more space because updated have not yet been compacted 
> away or it is due to distribution of data.
> So we need to know the true fully compacted data size. 
> We can do this with validation compaction and summing up the size of all 
> rows. 
> We should also emit such a sum during repair when the Merkle tree is being 
> generated. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6752) Streaming should pause when receiving host is overwhelmed

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6752:

Component/s: Streaming and Messaging

> Streaming should pause when receiving host is overwhelmed
> -
>
> Key: CASSANDRA-6752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6752
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: sankalp kohli
>Priority: Minor
>  Labels: ponies, streaming
>
> During repair, sometimes repair bring in lot of data and the disk gets full 
> on the node.  A cassandra node should better protect itself by monitoring its 
> disk space and pausing incoming streams. 
> Outgoing throttle can help but this automatic check will make operations 
> easier. 
> Lot of improvements have been done on repair specially incremental repairs 
> which makes this less of a problem but I think this could still be useful and 
> will avoid operator shooting itself in the foot. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6340) Provide a mechanism for retrieving all replicas

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6340:

Component/s: Coordination

> Provide a mechanism for retrieving all replicas
> ---
>
> Key: CASSANDRA-6340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6340
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination
> Environment: Production 
>Reporter: Ahmed Bashir
>Priority: Minor
>  Labels: ponies
>
> In order to facilitate problem diagnosis, there should exist some mechanism 
> to retrieve all copies of specific columns



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6394) Accessing and setting expiration time of a column (instead of TTL)

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6394:

Component/s: CQL

> Accessing and setting expiration time of a column (instead of TTL)
> --
>
> Key: CASSANDRA-6394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6394
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Patrick Varilly
>Priority: Minor
>  Labels: cql3, features, timestamp, ttl
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When selecting and inserting/updating columns, one can get/set the TIMESTAMP 
> / WRITETIME and TTL.  However, this is not enough information to recreate the 
> column's expiration time (clock sync and network latency between the client 
> and server get in the way).  This makes updating columns with a set TTL 
> fragile: there is no way to make the new value of a column *expire* at the 
> same time as the old value.
> Ideally, you'd be able to say something like:
> SELECT x, y EXPIRATIONTIME( y ) FROM my_cf
> and
> UPDATE my_cf USING EXPIRATIONTIME sameAsFromSelect SET y = newy WHERE x = 
> oldx.
> The use case I'm facing is that I write an entire row with a given TTL, and 
> might later want to update a few of its columns.  Currently, that makes the 
> updated columns live longer than the non-updated columns.  Of course, you can 
> come up with a good approximation for the appropriate TTL in the update to 
> make the updated columns expire at *around* the same time, but not at 
> *exactly* the same time.  Since Cassandra stores an expiration time 
> internally, making the expiration *exactly* simultaneous should be possible, 
> but CQL3 does not expose this ability.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6422) Add ability to control HintedHandoff threads per datacenter

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6422:

Component/s: Hints

> Add ability to control HintedHandoff threads per datacenter
> ---
>
> Key: CASSANDRA-6422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6422
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Hints
>Reporter: Adam Hattrell
>Priority: Minor
>
> I have a user that would like to be able to define how many threads and the 
> throughput limits for each DC.  
> To quote:
> Having different thread counts for local DC vs remote as well as separate 
> bandwidth limits certainly makes sense.  Expanding on this, I could foresee 
> that if there are more than two DCs that being able to configure each one 
> independently would be hugely beneficial, as latency/available bandwidth to 
> each DC may differ, as well as the number of nodes per DC.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6177) remove all sleeps in the dtests

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6177:

Component/s: Testing

> remove all sleeps in the dtests
> ---
>
> Key: CASSANDRA-6177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6177
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Brandon Williams
>Priority: Major
>
> The dtests use a ton of sleep calls for various things, most of which is 
> guessing if Cassandra has finished doing something or not.  Guessing is 
> problematic and shouldn't be necessary -- a prime example of this is creating 
> a ks or cf.  When done over cql, we sleep and hope it's done propagating, but 
> when done over thrift we actually check for schema agreement.  We should be 
> able to eliminate the sleeps and reliably detect when it's time for the next 
> step programmatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6193) Move throughput-heavy activities (repair/compaction) into separate process

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6193:

Component/s: Core

> Move throughput-heavy activities (repair/compaction) into separate process
> --
>
> Key: CASSANDRA-6193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6193
> Project: Cassandra
>  Issue Type: Wish
>  Components: Core
>Reporter: André Cruz
>Priority: Minor
>  Labels: ponies
>
> Repairs and compactions are activities that I've seen cause Full GCs. It is 
> difficult to optimize the GC for pauseless behaviour when the jvm is 
> performing such different functions as serving client requests and processing 
> large data files.
> Wouldn't it be possible to separate repairs/compactions into another process 
> where Full GCs would not be a problem?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6288) Make compaction a priority queue

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6288:

Component/s: Compaction

> Make compaction a priority queue
> 
>
> Key: CASSANDRA-6288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6288
> Project: Cassandra
>  Issue Type: Wish
>  Components: Compaction
>Reporter: Jonathan Ellis
>Assignee: Jeff Jirsa
>Priority: Minor
>  Labels: compaction
>
> We should prioritize compacting CFs by how many reads/s its preferred 
> candidate would save, divided by the number of bytes in the sstables.
> (Note that STCS currently divides by number of keys; ISTM that bytes will 
> work better since that does not penalize narrow rows.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6412) Custom creation and merge functions for user-defined column types

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6412:

Component/s: Core

> Custom creation and merge functions for user-defined column types
> -
>
> Key: CASSANDRA-6412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6412
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Nicolas Favre-Felix
>Priority: Major
>
> This is a proposal for a new feature, mapping custom types to Cassandra 
> columns.
> These types would provide a creation function and a merge function, to be 
> implemented in Java by the user.
> This feature relates to the concept of CRDTs; the proposal is to replicate 
> "operations" on these types during write, to apply these operations 
> internally during merge (Column.reconcile), and to also merge their values on 
> read.
> The following operations are made possible without reading back any data:
> * MIN or MAX(value) for a column
> * First value for a column
> * Count Distinct
> * HyperLogLog
> * Count-Min
> And any composition of these too, e.g. a Candlestick type includes first, 
> last, min, and max.
> The merge operations exposed by these types need to be commutative; this is 
> the case for many functions used in analytics.
> This feature is incomplete without some integration with CASSANDRA-4775 
> (Counters 2.0) which provides a Read-Modify-Write implementation for 
> distributed counters. Integrating custom creation and merge functions with 
> new counters would let users implement complex CRDTs in Cassandra, including:
> * Averages & related (sum of squares, standard deviation)
> * Graphs
> * Sets
> * Custom registers (even with vector clocks)
> I have a working prototype with implementations for min, max, and Candlestick 
> at https://github.com/acunu/cassandra/tree/crdts - I'd appreciate any 
> feedback on the design and interfaces.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7592) Ownership changes can violate consistency

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7592:

Component/s: Core

> Ownership changes can violate consistency
> -
>
> Key: CASSANDRA-7592
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7592
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Richard Low
>Priority: Major
>
> CASSANDRA-2434 goes a long way to avoiding consistency violations when 
> growing a cluster. However, there is still a window when consistency can be 
> violated when switching ownership of a range.
> Suppose you have replication factor 3 and all reads and writes at quorum. The 
> first part of the ring looks like this:
> Z: 0
> A: 100
> B: 200
> C: 300
> Choose two random coordinators, C1 and C2. Then you bootstrap node X at token 
> 50.
> Consider the token range 0-50. Before bootstrap, this is stored on A, B, C. 
> During bootstrap, writes go to X, A, B, C (and must succeed on 3) and reads 
> choose two from A, B, C. After bootstrap, the range is on X, A, B.
> When the bootstrap completes, suppose C1 processes the ownership change at t1 
> and C2 at t4. Then the following can give an inconsistency:
> t1: C1 switches ownership.
> t2: C1 performs write, so sends write to X, A, B. A is busy and drops the 
> write, but it succeeds because X and B return.
> t3: C2 performs a read. It hasn’t done the switch and chooses A and C. 
> Neither got the write at t2 so null is returned.
> t4: C2 switches ownership.
> This could be solved by continuing writes to the old replica for some time 
> (maybe ring delay) after the ownership changes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7822) Confusing timeout on CAS contention

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7822:

Component/s: Coordination

> Confusing timeout on CAS contention
> ---
>
> Key: CASSANDRA-7822
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7822
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Coordination
>Reporter: sankalp kohli
>Priority: Minor
>  Labels: LWT
>
> If we have contention in CAS and we hit the cas contention timeout, we throw 
> an exception. In this timeout exception, we pass that 0 replicas responded. 
> This is very confusing to someone looking at the client logs. I think we 
> might need to throw a separate exception for contention or may be add a flag 
> in the timeout exception. 
> We have seen many people confused by this so I think we should fix it. 
> This is how we throw it on contention. 
> throw new WriteTimeoutException(WriteType.CAS, consistencyForPaxos, 0, 
> consistencyForPaxos.blockFor(Keyspace.open(keyspaceName)));



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8224) Checksum Gossip state

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8224:

Component/s: Distributed Metadata

> Checksum Gossip state
> -
>
> Key: CASSANDRA-8224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8224
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Minor
> Attachments: incomplete_trunk_8224.diff
>
>
>  We have seen that a single machine with bad memory can corrupt the gossip of 
> other nodes and cause entire cluster to be affected. If we store and pass the 
> checksum of the entire state, we can detect corruption. If a bad machine 
> tries to bump the generation number or other things, it will be detected and 
> ignored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6408) Efficient multi-partition mutations

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6408:

Component/s: Core

> Efficient multi-partition mutations
> ---
>
> Key: CASSANDRA-6408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6408
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Rick Branson
>Priority: Major
>
> At the SF Summit this year, Sylvain suggested that C* drops a very large 
> amount of write throughput on the floor for multi-partition mutations because 
> they are broken into RowMutations and executed individually. Stress tests 
> that I've run show 10X the throughput for 1-row x 1000-col writes versus 
> 1000-row x 1-col writes. We have a core high-write-skew use case which 
> involves fan-out-on-write against hundreds or up to thousands of keys at a 
> time currently implemented in Redis as it doesn't seem to suffer from the 
> issue. Would love to be able to move this to C* at some point.
> This is likely a pretty large undertaking as it would require touching a 
> large portion of the write path, but I figure I'd put it here for comment 
> and/or debate at this point.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7764) RFC: Range movements will "wake up" previously invisible data

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7764:

Component/s: Core

> RFC: Range movements will "wake up" previously invisible data
> -
>
> Key: CASSANDRA-7764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7764
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Rick Branson
>Priority: Major
>  Labels: triaged
>
> Presumably this has been going on as long as Cassandra has existed, but 
> wanted to capture it here since it came up in an IRC discussion. This issue 
> will probably show up on any cluster eventually.
> Scenario:
> 1) Start with a 3-node cluster, RF=1
> 2) A 4th node is added to the cluster
> 3) Data is deleted on ranges belonging to 4th node
> 4) Wait for GC to clean up some tombstones on 4th node
> 4) 4th node removed from cluster
> 5) Deleted data will reappear since it was dormant on the original 3 nodes
> This could definitely happen in many other situations where dormant data 
> could exist such as inconsistencies that aren't resolved before range 
> movement, but the case above seemed the most reasonable to propose as a 
> real-world problem.
> The cleanup operation can be used to get rid of the dormant data, but from my 
> experience people don't run cleanup unless they're low on disk. It's 
> definitely not a best practice for data integrity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6061) Rewrite TokenMetadata

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6061:

Component/s: Core

> Rewrite TokenMetadata
> -
>
> Key: CASSANDRA-6061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6061
> Project: Cassandra
>  Issue Type: Task
>  Components: Core
>Reporter: Jonathan Ellis
>Priority: Minor
>
> Feels like this "mostly works" but is generally fragile (see: shuffle).
> Would be good to get a fresh perspective on it and see if we can do better.
> Bonus would be, ability to bootstrap multiple nodes w/o Two Minute Rule.  
> Probably would involve using LWT on pending ranges state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6091) Better Vnode support in hadoop/pig

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6091:

Component/s: Core

> Better Vnode support in hadoop/pig
> --
>
> Key: CASSANDRA-6091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6091
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
>Reporter: Alex Liu
>Assignee: mck
>Priority: Minor
> Attachments: cassandra-2.0-6091.txt, cassandra-2.1-6091.txt, 
> trunk-6091.txt
>
>
> CASSANDRA-6084 shows there are some issues during running hadoop/pig job if 
> vnodes are enable. Also the hadoop performance of vnode enabled nodes  are 
> bad for there are so many splits.
> The idea is to combine vnode splits into a big sudo splits so it work like 
> vnode is disable for hadoop/pig job



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8036) Add dtest for ipv6 functionality

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8036:

Component/s: Testing

> Add dtest for ipv6 functionality
> 
>
> Key: CASSANDRA-8036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8036
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Philip Thompson
>Priority: Major
>
> Cassandra can run with ipv6 addresses, and cqlsh should be able to connect 
> via ipv6. We need a dtest to verify this. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-8117) Checksum all communication between Cassandra nodes

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-8117:

Component/s: Streaming and Messaging

> Checksum all communication between Cassandra nodes
> --
>
> Key: CASSANDRA-8117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8117
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: sankalp kohli
>Priority: Major
>
> We have seen corruption in Gossip due to corruption happening in the n/w card 
> between main memory which is ECC protected and TCP checksum. Same corruption 
> can happen to other data flowing b/w machines. We should add checksum to all 
> communication b/w machines in the application. 
> A different JIRA could be to add checksums from the client driver for writes 
> and back to the driver for reads. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5912) Optimize listing partition keys

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5912:

Component/s: CQL

> Optimize listing partition keys
> ---
>
> Key: CASSANDRA-5912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5912
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Sylvain Lebresne
>Assignee: Suresh
>Priority: Minor
>
> That's a small followup to optimize the DISTINCT support added by 
> CASSANDRA-4536 if we feel like it's worth it.
> Quoting the initial ticket: it should be possible to optimize that further if 
> we consider it worth it by adding a 1 bit per key info in the sstable index 
> saying 'is there at least one live column for that key in that sstable' (we 
> could even add that bit-per-key without augmenting the on-disk index size if 
> we want to by using the first bit of the key position (since we use it as a 
> signed long and thus the first bit is unused)).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5888) Need a way to start or bootstrap a node regardless of its current state

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5888:

Component/s: Lifecycle

> Need a way to start or bootstrap a node regardless of its current state
> ---
>
> Key: CASSANDRA-5888
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5888
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Oleg Kibirev
>Priority: Major
>
> Currently, there is no straightforward way to start a cassandra node on a 
> given host without knowing:
> 1. Weather the node's initial_token has been successfully registered through 
> gossip 
> 2. If the node has successfully finished bootstrapping
> 3. Weather data directory has been purged by, say, replacing a dead disk
> Specifically the following cases are problematic:
> 1. If a data directory is purged and the node is restarted with the same host 
> IP and initial token, the node will successfully start but will not 
> bootstrap, creating a silent consistency loss.
> 2. If -Dcassandra.replace_token is given, the node will bootstrap again, even 
> if its successfully bootstrapped already with the same token.
> The information on the correct thing to do can only come from gossip and 
> system keyspace. Its very difficult to infer correct start arguments from a 
> process external to cassandra and have it work 100% of the time on large 
> scale. What if a node already gossiped its token but has not successfully 
> finished bootstrapping - how do I know to drop replace_token and will it 
> still re-bootstrap?
> Cassandra daemon should always bootstrap on start if it hasn't yet finished 
> bootstrapping successfully. -Dcassandra.replace_token (or host replacement 
> equivalent with nodes) should just ALLOW replacing a token of a down node, 
> but not force an unnecessary bootstrap or fail if the token is not present.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-7693) Reminder: clean up Javadoc errors/warnings

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-7693:

Component/s: Documentation and Website

> Reminder: clean up Javadoc errors/warnings
> --
>
> Key: CASSANDRA-7693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7693
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Robert Stupp
>Priority: Trivial
>
> Current C* build (trunk) reports > 100 errors + > 100 warnings (when building 
> with Java8).
> Trivial but long ("keyboard monkey") task: Clean up the errors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5861) Test decomission LEFT state serialization

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5861:

Component/s: Testing

> Test decomission LEFT state serialization
> -
>
> Key: CASSANDRA-5861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5861
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Ryan McGuire
>Assignee: Ryan McGuire
>Priority: Major
>
> Make sure things like CASSANDRA-5857 don't crop up again.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5313) provide a cardinality function for collection types ( CQL3 )

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5313:

Component/s: CQL

> provide a cardinality function for collection types ( CQL3 )
> 
>
> Key: CASSANDRA-5313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5313
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Ahmet AKYOL
>Priority: Major
>  Labels: cql
>
> Currently , cql3 doesn't provide a cardinality function for collection types. 
> It'll be great to have one:
> {code}
> select content, cardinality(like_set),cardinality(dislike_set) from comments 
> where id=?;
> {code}
> or size as keyword
> {code}
> select content, size(like_set),size(dislike_set) from comments where id=?;
> {code}
> Something similar in SQL is [cardinality of nested 
> tables|http://www.techonthenet.com/oracle/functions/cardinality.php] .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5972) Reduce the amount of data to be transferred during repair

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5972:

Component/s: Repair

> Reduce the amount of data to be transferred during repair
> -
>
> Key: CASSANDRA-5972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5972
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Repair
>Reporter: Jacek Lewandowski
>Priority: Minor
>
> Currently, when a validator finds a token range different in n replicas, data 
> streams are initiated simultaneously between each possible pair of these n 
> nodes, in both directions. It yields n*(n-1) data stream in total. 
> It can be done in a sequence - Replica[1] -> R[2], R[2] -> R[3], ... , R[n-1] 
> -> R[n]. After this process, the data in R[n] are up to date. Then, we 
> continue: R[n] -> R[1], R[1] -> R[2], ... , R[n-2] -> R[n-1]. The active 
> repair is done after 2*(n-1) data transfers performed sequentially in 2*(n-1) 
> steps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5962) Support trigger parametrization

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5962:

Component/s: CQL

> Support trigger parametrization
> ---
>
> Key: CASSANDRA-5962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5962
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Aleksey Yeschenko
>Priority: Minor
>  Labels: cql3, triggers
>
> We don't have a convenient way to parametrize triggers, which limits their 
> reusability and usability in general. For any configuration you have to rely 
> on external config files.
> We already have [trigger_options map] column in 
> system.schema_triggers, all we need is to add the right syntax to CQL3 
> (CREATE TRIGGER foo ON bar USING class WITH options = {..}) and modify 
> ITrigger to support it.
> Setting fixver to 2.1, but might move to 2.0.x later.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-6091) Better Vnode support in hadoop/pig

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-6091:

Component/s: (was: Core)
 Libraries

> Better Vnode support in hadoop/pig
> --
>
> Key: CASSANDRA-6091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6091
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
>Reporter: Alex Liu
>Assignee: mck
>Priority: Minor
> Attachments: cassandra-2.0-6091.txt, cassandra-2.1-6091.txt, 
> trunk-6091.txt
>
>
> CASSANDRA-6084 shows there are some issues during running hadoop/pig job if 
> vnodes are enable. Also the hadoop performance of vnode enabled nodes  are 
> bad for there are so many splits.
> The idea is to combine vnode splits into a big sudo splits so it work like 
> vnode is disable for hadoop/pig job



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-5091) Provide a way to sort read results on the coordinator node

2018-11-18 Thread C. Scott Andreas (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

C. Scott Andreas updated CASSANDRA-5091:

Component/s: Coordination

> Provide a way to sort read results on the coordinator node
> --
>
> Key: CASSANDRA-5091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5091
> Project: Cassandra
>  Issue Type: Wish
>  Components: Coordination
>Reporter: Sergio Bossa
>Priority: Major
>  Labels: cql
>
> While Cassandra provides a way to scatter/gather queries with secondary 
> indexes, as well as custom secondary indexes implementations, there is 
> currently no way to sort gathered results on the coordinator node, which 
> would be very useful (actually, essential) if the index implementation 
> provides results in sorted order.
> So, what I'm proposing here is a way to provide (custom) sort 
> implementations, which may either be as easy as just adding one more 
> interface and a bunch of method calls, or get a bit more complex by providing 
> an ORDER BY clause in CQL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



<    2   3   4   5   6   7   8   9   >