[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199976#comment-14199976 ] Robert Stupp commented on CASSANDRA-7438: - Debugging C code via JNI and debugging Unsafe code on large data structures is a nightmare. And a simple, stupid bug in both kinds can quickly let the JVM core dump. Advantage for the Unsafe approach is that all OS are directly supported. Advantage for the JNI approach is that the code that handles the data structures is much easier to read. Proposal: * Extract the changes that support pluggable ICacheProvide from this ticket to a separate ticket and commit that stuff * Let Vijay continue his work on this one * Provide an alternative implementation using Unsafe * Let both implementations compete in some long running tests This is much effort to do - but I don't know how to validate either solution theoretically. Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Vijay Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8220) JDK bug prevents clean shutdown on OSX with Java 1.8.0_20
[
https://issues.apache.org/jira/browse/CASSANDRA-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200055#comment-14200055
]
Wim Deblauwe commented on CASSANDRA-8220:
-
I am running into the same issue. Due to this my unit tests (using
Cassandra-Unit) are no longer running properly.
JDK bug prevents clean shutdown on OSX with Java 1.8.0_20
-
Key: CASSANDRA-8220
URL: https://issues.apache.org/jira/browse/CASSANDRA-8220
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 2.1.2
Attachments: jdk8050499.txt
Clean shutdown on OSX with Java 1.8.0_20 does not work due to [JDK bug
8050499 fixed for Java 9|https://bugs.openjdk.java.net/browse/JDK-8050499].
Supplied patch provides a (very dirty) workaround to (hopefully) allow
shutdown.
Exception during shutdown looks like this:
{code}
ERROR 22:23:15 Fatal exception in thread
Thread[StorageServiceShutdownHook,5,main]
java.io.IOError: java.io.IOException: Unknown error: 316
at
org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:721)
~[trunk/na]
at
org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:553)
~[trunk/na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[trunk/na]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_20]
Caused by: java.io.IOException: Unknown error: 316
at sun.nio.ch.NativeThread.signal(Native Method) ~[na:1.8.0_20]
at
sun.nio.ch.ServerSocketChannelImpl.implCloseSelectableChannel(ServerSocketChannelImpl.java:283)
~[na:1.8.0_20]
at
java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:234)
~[na:1.8.0_20]
at
java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
~[na:1.8.0_20]
at sun.nio.ch.ServerSocketAdaptor.close(ServerSocketAdaptor.java:137)
~[na:1.8.0_20]
at
org.apache.cassandra.net.MessagingService$SocketThread.close(MessagingService.java:958)
~[trunk/na]
at
org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:717)
~[trunk/na]
... 3 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
Branimir Lambov created CASSANDRA-8268: -- Summary: Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Priority: Minor This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
Jose Martinez Poblete created CASSANDRA-8269:
Summary: Large number of system hints other CF's cause heap to
fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line 199)
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 | 2,898,864 | 0.03%
| | | '- java.util.ArrayList @ 0x7b7caaae0
| 24 | 2,898,832 | 0.03%
| | | '- java.lang.Object[12283] @ 0x7b7caaaf8
| 49,152 | 2,898,808 | 0.03%
| | ||- org.apache.cassandra.io.sstable.IndexHelper$IndexInfo @
0x7b7cb6af8 | 40 | 232 | 0.00%
| | ||- org.apache.cassandra.io.sstable.IndexHelper$IndexInfo @
0x7b7cb6be0 |
[jira] [Commented] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200251#comment-14200251 ] Jonathan Ellis commented on CASSANDRA-8268: --- How would serialized tokens be in user data? Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Priority: Minor This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200303#comment-14200303 ] Branimir Lambov commented on CASSANDRA-8268: I think I used the wrong term here. I meant some data that is part of an user's existing database, be it the messages that old version nodes send or any system data stored. I actually don't know if there's anything that does not use the storage service partitioner that is ever serialized. Most probably there isn't or we'd be seeing errors, but this needs to be determined for sure before anything to do with the serialization is changed. Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Priority: Minor This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8266) Upgrade Cassandra node to 2.1.1 with hsha configuration. give OOM error
[ https://issues.apache.org/jira/browse/CASSANDRA-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler resolved CASSANDRA-8266. --- Resolution: Duplicate Set rpc_max_threads to something reasonable for your cluster. Upgrade Cassandra node to 2.1.1 with hsha configuration. give OOM error --- Key: CASSANDRA-8266 URL: https://issues.apache.org/jira/browse/CASSANDRA-8266 Project: Cassandra Issue Type: Bug Components: Core Environment: Ubuntu 14.0 Cluster with 3 nodes. Reporter: neha Upgrade the node with 2.1.1 (change cassandra.yaml to have hsha) Start Cassandra (it gives OOM ) ERROR java.lang.OutOfMemoryError: Requested array size exceeds VM limit Cassandra does not start. If I change configuration back to sync in cassandra.yaml, it works -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200311#comment-14200311 ] Jonathan Ellis commented on CASSANDRA-8268: --- It's definitely safe to assume that anything that's not node-local (i.e. not system table or indexes) is using the cluster partitioner. Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Priority: Minor This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200315#comment-14200315
]
Aleksey Yeschenko commented on CASSANDRA-8269:
--
I think CASSANDRA-6988 (since 2.0.11) would make this a non-issue.
Large number of system hints other CF's cause heap to fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line
199) Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 | 2,898,864 | 0.03%
| | | '- java.util.ArrayList @ 0x7b7caaae0
| 24 |
[jira] [Resolved] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-8269.
---
Resolution: Won't Fix
If you don't have enough heap to deserialize the partition summary you need to
increase column_index_size_in_kb, stop generating huge partitions, or both.
(We're addressing this for hints with CASSANDRA-6230.)
Large number of system hints other CF's cause heap to fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line
199) Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 | 2,898,864 | 0.03%
| |
[jira] [Commented] (CASSANDRA-7409) Allow multiple overlapping sstables in L1
[ https://issues.apache.org/jira/browse/CASSANDRA-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200322#comment-14200322 ] Carl Yeksigian commented on CASSANDRA-7409: --- It's the default settings -- no overlapping in L1, but the compaction candidates are selected slightly differently compared to before (starting at L0 and going up instead of the other way). It's possible that the way this selection is happening is prolonging the compaction process tremendously; I hadn't seen the same behavior locally, so I'll need to figure out a different way to debug. Allow multiple overlapping sstables in L1 - Key: CASSANDRA-7409 URL: https://issues.apache.org/jira/browse/CASSANDRA-7409 Project: Cassandra Issue Type: Improvement Reporter: Carl Yeksigian Assignee: Carl Yeksigian Labels: compaction Fix For: 3.0 Currently, when a normal L0 compaction takes place (not STCS), we take up to MAX_COMPACTING_L0 L0 sstables and all of the overlapping L1 sstables and compact them together. If we didn't have to deal with the overlapping L1 tables, we could compact a higher number of L0 sstables together into a set of non-overlapping L1 sstables. This could be done by delaying the invariant that L1 has no overlapping sstables. Going from L1 to L2, we would be compacting fewer sstables together which overlap. When reading, we will not have the same one sstable per level (except L0) guarantee, but this can be bounded (once we have too many sets of sstables, either compact them back into the same level, or compact them up to the next level). This could be generalized to allow any level to be the maximum for this overlapping strategy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8269:
--
Comment: was deleted
(was: I believe CASSANDRA-6998 is the correct ticket number from the first
comment.)
Large number of system hints other CF's cause heap to fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line
199) Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 | 2,898,864 | 0.03%
| | | '- java.util.ArrayList @ 0x7b7caaae0
| 24 | 2,898,832 | 0.03%
|
[jira] [Commented] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200323#comment-14200323
]
Michael Shuler commented on CASSANDRA-8269:
---
I believe CASSANDRA-6998 is the correct ticket number from the first comment.
Large number of system hints other CF's cause heap to fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line
199) Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 | 2,898,864 | 0.03%
| | | '- java.util.ArrayList @ 0x7b7caaae0
| 24 |
[jira] [Comment Edited] (CASSANDRA-8269) Large number of system hints other CF's cause heap to fill and run OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200315#comment-14200315
]
Aleksey Yeschenko edited comment on CASSANDRA-8269 at 11/6/14 3:30 PM:
---
I think CASSANDRA-6998 (since 2.0.11) would make this a non-issue.
was (Author: iamaleksey):
I think CASSANDRA-6988 (since 2.0.11) would make this a non-issue.
Large number of system hints other CF's cause heap to fill and run OOM
Key: CASSANDRA-8269
URL: https://issues.apache.org/jira/browse/CASSANDRA-8269
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.0 with Apache Cassandra 2.0.5
Reporter: Jose Martinez Poblete
Attachments: alln01-ats-cas2-java_1414110068_Leak_Suspects.zip,
system.log
A 3 node cluster with large amount of sstables for system.hints and other 3
user tables was coming down regularly with OOM on system log showing up the
following:
{noformat}
ERROR [OptionalTasks:1] 2014-10-23 18:51:29,052 CassandraDaemon.java (line
199) Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187)
at
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:229)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:203)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:183)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:144)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:87)
at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:46)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:74)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1586)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1709)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1643)
at
org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:513)
at
org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:91)
at
org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:173)
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
A heapdump would show the following:
{noformat}
Class Name
| Shallow Heap | Retained Heap | Percentage
--
java.lang.Thread @ 0x67b292138 OptionalTasks:1 Thread
| 104 | 4,901,485,768 | 58.60%
|- org.apache.cassandra.utils.MergeIterator$ManyToOne @ 0x7b9dc4ad8
| 40 | 4,900,817,312 | 58.59%
| |- java.util.ArrayList @ 0x6f05f15f0
| 24 | 403,635,848 | 4.83%
| |- org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator @
0x7b5fe7078| 40 |29,669,312 | 0.35%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7caaa28
| 32 |26,770,264 | 0.32%
| | |- org.apache.cassandra.db.RowIndexEntry$IndexedEntry @ 0x7b7f6e670
| 32 |
[jira] [Commented] (CASSANDRA-8265) Disable SSLv3 for POODLE
[ https://issues.apache.org/jira/browse/CASSANDRA-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200335#comment-14200335 ] Jeremiah Jordan commented on CASSANDRA-8265: We already let people specify the protocol in cassandra.yaml to force TLS, but we default it to be commented out. Should we default to having protocol uncommented? Disable SSLv3 for POODLE Key: CASSANDRA-8265 URL: https://issues.apache.org/jira/browse/CASSANDRA-8265 Project: Cassandra Issue Type: Bug Reporter: Jeremiah Jordan We should probably disable SSLv3. http://www.oracle.com/technetwork/java/javase/documentation/cve-2014-3566-2342133.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
git commit: Make paxos reuse the timestamp generation of normal operation
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 cdf80d9d1 - 85ea37356
Make paxos reuse the timestamp generation of normal operation
patch by slebresne; reviewed by iamaleksey for CASSANDRA-7801
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85ea3735
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85ea3735
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85ea3735
Branch: refs/heads/cassandra-2.1
Commit: 85ea37356e666c2780294bbd29daa89a32ebf333
Parents: cdf80d9
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Nov 6 16:38:27 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Nov 6 16:38:27 2014 +0100
--
CHANGES.txt | 3 +
.../cql3/statements/BatchStatement.java | 7 +-
.../cql3/statements/ModificationStatement.java | 3 +-
.../cql3/statements/SelectStatement.java| 8 +-
.../apache/cassandra/service/ClientState.java | 36 +
.../apache/cassandra/service/QueryState.java| 5 +-
.../apache/cassandra/service/StorageProxy.java | 139 ++-
.../service/pager/MultiPartitionPager.java | 13 +-
.../service/pager/NamesQueryPager.java | 7 +-
.../cassandra/service/pager/QueryPagers.java| 30 ++--
.../service/pager/SliceQueryPager.java | 11 +-
.../cassandra/thrift/CassandraServer.java | 35 +++--
12 files changed, 206 insertions(+), 91 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2e60f3a..5348f2f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.3
+ * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
+
2.1.2
* (cqlsh) parse_for_table_meta errors out on queries with undefined
grammars (CASSANDRA-8262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
index 17d1771..d54e4fd 100644
--- a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
@@ -275,7 +275,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
throw new InvalidRequestException(Invalid empty serial
consistency level);
if (hasConditions)
-return executeWithConditions(options, now);
+return executeWithConditions(options, queryState);
executeWithoutConditions(getMutations(options, local, now),
options.getConsistency());
return new ResultMessage.Void();
@@ -297,9 +297,10 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
StorageProxy.mutateWithTriggers(mutations, cl, mutateAtomic);
}
-private ResultMessage executeWithConditions(BatchQueryOptions options,
long now)
+private ResultMessage executeWithConditions(BatchQueryOptions options,
QueryState state)
throws RequestExecutionException, RequestValidationException
{
+long now = state.getTimestamp();
ByteBuffer key = null;
String ksName = null;
String cfName = null;
@@ -339,7 +340,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
casRequest.addRowUpdate(clusteringPrefix, statement,
statementOptions, timestamp);
}
-ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency());
+ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency(),
state.getClientState());
return new
ResultMessage.Rows(ModificationStatement.buildCasResultSet(ksName, key, cfName,
result, columnsWithConditions, true, options.forStatement(0)));
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 974ccc8..846ad3e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -524,7 +524,8
[1/2] git commit: Make paxos reuse the timestamp generation of normal operation
Repository: cassandra
Updated Branches:
refs/heads/trunk aa6233aa1 - d5b2fa206
Make paxos reuse the timestamp generation of normal operation
patch by slebresne; reviewed by iamaleksey for CASSANDRA-7801
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85ea3735
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85ea3735
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85ea3735
Branch: refs/heads/trunk
Commit: 85ea37356e666c2780294bbd29daa89a32ebf333
Parents: cdf80d9
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Nov 6 16:38:27 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Nov 6 16:38:27 2014 +0100
--
CHANGES.txt | 3 +
.../cql3/statements/BatchStatement.java | 7 +-
.../cql3/statements/ModificationStatement.java | 3 +-
.../cql3/statements/SelectStatement.java| 8 +-
.../apache/cassandra/service/ClientState.java | 36 +
.../apache/cassandra/service/QueryState.java| 5 +-
.../apache/cassandra/service/StorageProxy.java | 139 ++-
.../service/pager/MultiPartitionPager.java | 13 +-
.../service/pager/NamesQueryPager.java | 7 +-
.../cassandra/service/pager/QueryPagers.java| 30 ++--
.../service/pager/SliceQueryPager.java | 11 +-
.../cassandra/thrift/CassandraServer.java | 35 +++--
12 files changed, 206 insertions(+), 91 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2e60f3a..5348f2f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.3
+ * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
+
2.1.2
* (cqlsh) parse_for_table_meta errors out on queries with undefined
grammars (CASSANDRA-8262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
index 17d1771..d54e4fd 100644
--- a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
@@ -275,7 +275,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
throw new InvalidRequestException(Invalid empty serial
consistency level);
if (hasConditions)
-return executeWithConditions(options, now);
+return executeWithConditions(options, queryState);
executeWithoutConditions(getMutations(options, local, now),
options.getConsistency());
return new ResultMessage.Void();
@@ -297,9 +297,10 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
StorageProxy.mutateWithTriggers(mutations, cl, mutateAtomic);
}
-private ResultMessage executeWithConditions(BatchQueryOptions options,
long now)
+private ResultMessage executeWithConditions(BatchQueryOptions options,
QueryState state)
throws RequestExecutionException, RequestValidationException
{
+long now = state.getTimestamp();
ByteBuffer key = null;
String ksName = null;
String cfName = null;
@@ -339,7 +340,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
casRequest.addRowUpdate(clusteringPrefix, statement,
statementOptions, timestamp);
}
-ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency());
+ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency(),
state.getClientState());
return new
ResultMessage.Rows(ModificationStatement.buildCasResultSet(ksName, key, cfName,
result, columnsWithConditions, true, options.forStatement(0)));
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 974ccc8..846ad3e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -524,7 +524,8 @@ public
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
src/java/org/apache/cassandra/thrift/CassandraServer.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d5b2fa20
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d5b2fa20
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d5b2fa20
Branch: refs/heads/trunk
Commit: d5b2fa206fbbf308f65e153e865e2a36bccfda89
Parents: aa6233a 85ea373
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Nov 6 16:41:20 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Nov 6 16:41:20 2014 +0100
--
CHANGES.txt | 2 +
.../cql3/statements/BatchStatement.java | 7 +-
.../cql3/statements/ModificationStatement.java | 3 +-
.../cql3/statements/SelectStatement.java| 8 +-
.../apache/cassandra/service/ClientState.java | 36 +
.../apache/cassandra/service/QueryState.java| 5 +-
.../apache/cassandra/service/StorageProxy.java | 139 ++-
.../service/pager/MultiPartitionPager.java | 13 +-
.../service/pager/NamesQueryPager.java | 7 +-
.../cassandra/service/pager/QueryPagers.java| 30 ++--
.../service/pager/SliceQueryPager.java | 11 +-
.../cassandra/thrift/CassandraServer.java | 37 ++---
12 files changed, 206 insertions(+), 92 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5b2fa20/CHANGES.txt
--
diff --cc CHANGES.txt
index b23f756,5348f2f..79318fc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,35 -1,5 +1,37 @@@
+3.0
+ * Mark sstables as repaired after full repair (CASSANDRA-7586)
+ * Extend Descriptor to include a format value and refactor reader/writer
apis (CASSANDRA-7443)
+ * Integrate JMH for microbenchmarks (CASSANDRA-8151)
+ * Keep sstable levels when bootstrapping (CASSANDRA-7460)
+ * Add Sigar library and perform basic OS settings check on startup
(CASSANDRA-7838)
+ * Support for aggregation functions (CASSANDRA-4914)
+ * Remove cassandra-cli (CASSANDRA-7920)
+ * Accept dollar quoted strings in CQL (CASSANDRA-7769)
+ * Make assassinate a first class command (CASSANDRA-7935)
+ * Support IN clause on any clustering column (CASSANDRA-4762)
+ * Improve compaction logging (CASSANDRA-7818)
+ * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
+ * Do anticompaction in groups (CASSANDRA-6851)
+ * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740,
7781, 7929,
+ 7924, 7812, 8063)
+ * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
+ * Move sstable RandomAccessReader to nio2, which allows using the
+ FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
+ * Remove CQL2 (CASSANDRA-5918)
+ * Add Thrift get_multi_slice call (CASSANDRA-6757)
+ * Optimize fetching multiple cells by name (CASSANDRA-6933)
+ * Allow compilation in java 8 (CASSANDRA-7028)
+ * Make incremental repair default (CASSANDRA-7250)
+ * Enable code coverage thru JaCoCo (CASSANDRA-7226)
+ * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369)
+ * Shorten SSTable path (CASSANDRA-6962)
+ * Use unsafe mutations for most unit tests (CASSANDRA-6969)
+ * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
+ * Fail on very large batch sizes (CASSANDRA-8011)
+ * improve concurrency of repair (CASSANDRA-6455)
+
+ 2.1.3
+ * Have paxos reuse the timestamp generation of normal queries
(CASSANDRA-7801)
2.1.2
* (cqlsh) parse_for_table_meta errors out on queries with undefined
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5b2fa20/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5b2fa20/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5b2fa20/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 686f50f,84cbdc0..b6cbd30
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@@ -214,25 -212,27 +214,25 @@@ public class SelectStatement implement
if (pageSize = 0 || command == null ||
!QueryPagers.mayNeedPaging(command, pageSize))
{
- return
[jira] [Commented] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200374#comment-14200374
]
Branimir Lambov commented on CASSANDRA-8244:
Proposed patch is uploaded for review on
https://github.com/blambov/cassandra/compare/8244-partitioner-in-token. Applies
on top of CASSANDRA-8230 on trunk.
Changes tokens and {{RingPosition}} to provide a reference to the partitioner
that created them, which is used to make sure Range/Bounds always use the
correct partitioner for the bounds they contain. Previously this was quite easy
to get wrong without noticing, see e.g. {{DataRange.allData}}. To avoid
increasing the space used by a token, this is done by moving the token type
definitions inside the partitioner, which allows them to have a static
reference to it, except {{LocalToken}} which now replaces the reference to its
comparator with a reference to the issuing partitioner.
Removes custom key construction in {{SecondaryIndex.getIndexKeyFor}} and
delegates the task to the {{indexCfs}} partitioner, the presence of which is
already tested whenever {{getIndexKeyFor}} is used.
Adds static instances of the partitioners and a mechanism to choose them
instead of instantiating a new copy in {{FBUtilities.newPartitioner}}.
Fixes {{KeyCollisionTest}} which wasn't changing the partitioner as expected,
and was also passing with the wrong partitioner.
Token, DecoratedKey, RowPosition and all bound types should not make any
hidden references to the database partitioner
--
Key: CASSANDRA-8244
URL: https://issues.apache.org/jira/browse/CASSANDRA-8244
Project: Cassandra
Issue Type: Bug
Reporter: Branimir Lambov
Assignee: Branimir Lambov
Priority: Minor
Currently some of the functionality of Token refers to
StorageService.getPartitioner() to avoid needing an extra argument. This is
in turn implicitly used by RowPosition and then Range, causing possible
problems, for example when ranges on secondary indices are used in a
murmur-partitioned database.
These references should be removed to force explicit choice of partitioner by
callers; alternatively, the Token interface could be changed to provide a
reference to the partitioner that created it.
(Note: the hidden reference to partitioner in serialization is a separate
issue.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8270) Allow sub slices for composites
T Jake Luciani created CASSANDRA-8270:
-
Summary: Allow sub slices for composites
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
b(2,*) c (3,*)
b (*,*) c (3,*)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
git commit: Fix overflow on histogram computation
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 85ea37356 - 9685622b3
Fix overflow on histogram computation
Patch by cyeksigian; reviewed by jmckenzie for CASSANDRA-8028
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9685622b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9685622b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9685622b
Branch: refs/heads/cassandra-2.1
Commit: 9685622b32d19ffc9315e764a224f41cbe6b6d4a
Parents: 85ea373
Author: Carl Yeksigian c...@apache.org
Authored: Thu Nov 6 10:18:09 2014 -0600
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Thu Nov 6 10:18:09 2014 -0600
--
CHANGES.txt | 1 +
.../cassandra/metrics/ColumnFamilyMetrics.java | 67
.../org/apache/cassandra/tools/NodeTool.java| 32 --
3 files changed, 80 insertions(+), 20 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5348f2f..89db48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix overflow on histogram computation (CASSANDRA-8028)
* Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
2.1.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
index d9d3ed9..2bdaf27 100644
--- a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.metrics;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;
@@ -146,6 +147,46 @@ public class ColumnFamilyMetrics
* Stores all metric names created that can be used when unregistering
*/
public final static SetString all = Sets.newHashSet();
+
+private interface GetHistogram
+{
+public EstimatedHistogram getHistogram(SSTableReader reader);
+}
+
+private static long[] combineHistograms(IterableSSTableReader sstables,
GetHistogram getHistogram)
+{
+IteratorSSTableReader iterator = sstables.iterator();
+if (!iterator.hasNext())
+{
+return new long[0];
+}
+long[] firstBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+long[] values = new long[firstBucket.length];
+System.arraycopy(firstBucket, 0, values, 0, values.length);
+
+while (iterator.hasNext())
+{
+long[] nextBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+if (nextBucket.length values.length)
+{
+long[] newValues = new long[nextBucket.length];
+System.arraycopy(firstBucket, 0, newValues, 0,
firstBucket.length);
+for (int i = 0; i newValues.length; i++)
+{
+newValues[i] += nextBucket[i];
+}
+values = newValues;
+}
+else
+{
+for (int i = 0; i values.length; i++)
+{
+values[i] += nextBucket[i];
+}
+}
+}
+return values;
+}
/**
* Creates metrics for given {@link ColumnFamilyStore}.
@@ -219,28 +260,26 @@ public class ColumnFamilyMetrics
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReader sstable : cfs.getSSTables())
+return combineHistograms(cfs.getSSTables(), new GetHistogram()
{
-long[] rowSize =
sstable.getEstimatedRowSize().getBuckets(false);
-for (int i = 0; i histogram.length; i++)
-histogram[i] += rowSize[i];
-}
-return histogram;
+public EstimatedHistogram getHistogram(SSTableReader
reader)
+{
+return reader.getEstimatedRowSize();
+}
+});
}
});
estimatedColumnCountHistogram =
Metrics.newGauge(factory.createMetricName(EstimatedColumnCountHistogram), new
Gaugelong[]()
{
public long[] value()
{
-
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4cbb28f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4cbb28f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4cbb28f Branch: refs/heads/trunk Commit: b4cbb28f3759fc92ab2a43432fcbce33bac60d7a Parents: d5b2fa2 9685622 Author: Joshua McKenzie jmcken...@apache.org Authored: Thu Nov 6 10:19:03 2014 -0600 Committer: Joshua McKenzie jmcken...@apache.org Committed: Thu Nov 6 10:19:03 2014 -0600 -- CHANGES.txt | 1 + .../cassandra/metrics/ColumnFamilyMetrics.java | 67 .../org/apache/cassandra/tools/NodeTool.java| 32 -- 3 files changed, 80 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/CHANGES.txt -- diff --cc CHANGES.txt index 79318fc,89db48e..92b2b2c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,36 -1,5 +1,37 @@@ +3.0 + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer apis (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * improve concurrency of repair (CASSANDRA-6455) + 2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) 2.1.2 http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/src/java/org/apache/cassandra/tools/NodeTool.java --
[1/2] git commit: Fix overflow on histogram computation
Repository: cassandra
Updated Branches:
refs/heads/trunk d5b2fa206 - b4cbb28f3
Fix overflow on histogram computation
Patch by cyeksigian; reviewed by jmckenzie for CASSANDRA-8028
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9685622b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9685622b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9685622b
Branch: refs/heads/trunk
Commit: 9685622b32d19ffc9315e764a224f41cbe6b6d4a
Parents: 85ea373
Author: Carl Yeksigian c...@apache.org
Authored: Thu Nov 6 10:18:09 2014 -0600
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Thu Nov 6 10:18:09 2014 -0600
--
CHANGES.txt | 1 +
.../cassandra/metrics/ColumnFamilyMetrics.java | 67
.../org/apache/cassandra/tools/NodeTool.java| 32 --
3 files changed, 80 insertions(+), 20 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5348f2f..89db48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix overflow on histogram computation (CASSANDRA-8028)
* Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
2.1.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
index d9d3ed9..2bdaf27 100644
--- a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.metrics;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;
@@ -146,6 +147,46 @@ public class ColumnFamilyMetrics
* Stores all metric names created that can be used when unregistering
*/
public final static SetString all = Sets.newHashSet();
+
+private interface GetHistogram
+{
+public EstimatedHistogram getHistogram(SSTableReader reader);
+}
+
+private static long[] combineHistograms(IterableSSTableReader sstables,
GetHistogram getHistogram)
+{
+IteratorSSTableReader iterator = sstables.iterator();
+if (!iterator.hasNext())
+{
+return new long[0];
+}
+long[] firstBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+long[] values = new long[firstBucket.length];
+System.arraycopy(firstBucket, 0, values, 0, values.length);
+
+while (iterator.hasNext())
+{
+long[] nextBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+if (nextBucket.length values.length)
+{
+long[] newValues = new long[nextBucket.length];
+System.arraycopy(firstBucket, 0, newValues, 0,
firstBucket.length);
+for (int i = 0; i newValues.length; i++)
+{
+newValues[i] += nextBucket[i];
+}
+values = newValues;
+}
+else
+{
+for (int i = 0; i values.length; i++)
+{
+values[i] += nextBucket[i];
+}
+}
+}
+return values;
+}
/**
* Creates metrics for given {@link ColumnFamilyStore}.
@@ -219,28 +260,26 @@ public class ColumnFamilyMetrics
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReader sstable : cfs.getSSTables())
+return combineHistograms(cfs.getSSTables(), new GetHistogram()
{
-long[] rowSize =
sstable.getEstimatedRowSize().getBuckets(false);
-for (int i = 0; i histogram.length; i++)
-histogram[i] += rowSize[i];
-}
-return histogram;
+public EstimatedHistogram getHistogram(SSTableReader
reader)
+{
+return reader.getEstimatedRowSize();
+}
+});
}
});
estimatedColumnCountHistogram =
Metrics.newGauge(factory.createMetricName(EstimatedColumnCountHistogram), new
Gaugelong[]()
{
public long[] value()
{
-long[] histogram
[jira] [Updated] (CASSANDRA-8028) Unable to compute when histogram overflowed
[ https://issues.apache.org/jira/browse/CASSANDRA-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8028: --- Fix Version/s: (was: 2.1.2) 2.1.3 Unable to compute when histogram overflowed --- Key: CASSANDRA-8028 URL: https://issues.apache.org/jira/browse/CASSANDRA-8028 Project: Cassandra Issue Type: Bug Components: Tools Environment: Linux Reporter: Gianluca Borello Assignee: Carl Yeksigian Fix For: 2.1.3 Attachments: 8028-2.1-clean.txt, 8028-2.1-v2.txt, 8028-2.1.txt, 8028-trunk.txt, sstable-histogrambuster.tar.bz2 It seems like with 2.1.0 histograms can't be computed most of the times: $ nodetool cfhistograms draios top_files_by_agent1 nodetool: Unable to compute when histogram overflowed See 'nodetool help' or 'nodetool help command'. I can probably find a way to attach a .cql script to reproduce it, but I suspect it must be obvious to replicate it as it happens on more than 50% of my column families. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c (3,*)
b (*,*) c (3,*)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
b(2,*) c (3,*)
b (*,*) c (3,*)
Allow sub slices for composites
---
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server
side to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c (3,*)
b (*,*) c (3,*)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c(3,*)
b(*,*) c(3,*)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c (3,*)
b (*,*) c (3,*)
{noformat}
Allow sub slices for composites
---
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server
side to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c(3,*)
b(*,*) c(3,*)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c(3,*)
b(*,*) c(3,*)
{noformat}
Allow sub slices for composites
---
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server
side to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-8270:
---
Description:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
Allow sub slices for composites
---
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server
side slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7859) Extend freezing to collections
[
https://issues.apache.org/jira/browse/CASSANDRA-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200421#comment-14200421
]
Sylvain Lebresne commented on CASSANDRA-7859:
-
bq. When frozen lists and maps are used as cell values, the collection value
ordering doesn't matter.
I'm confused because my earlier comment was exactly to say that I'm
uncomfortable making that assumption and you seemed to agree initially. To
reiter, collection values are sorted (even when used as cell values), and so it
felt fishy to me to consider that changing to a type that change the internal
sorting of those values is compatible. Typically sending a value to clients
that is not properly sorted feels wrong. And given we lost little in practice
by requiring the sorting not to change, I'd prefer sticking to that for now.
bq. I think it could potentially introduce assertion errors when the behavior
is reasonable.
Fair enough, it's just that the comments in there pretty much says that it
should only be call for frozen collections. Maybe we should just remove the
comment in that case. Not a big deal though.
Extend freezing to collections
--
Key: CASSANDRA-7859
URL: https://issues.apache.org/jira/browse/CASSANDRA-7859
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Labels: cql
Fix For: 2.1.2
Attachments: 7859-v1.txt
This is the follow-up to CASSANDRA-7857, to extend {{frozen}} to collections.
This will allow things like {{maptext, frozenmapint, int}} for
instance, as well as allowing {{frozen}} collections in PK columns.
Additionally (and that's alsmot a separate ticket but I figured we can start
discussing it here), we could decide that tuple is a frozen type by default.
This means that we would allow {{tupleint, text}} without needing to add
{{frozen}}, but we would require {{frozen}} for complex type inside tuple, so
{{tupleint, listtext}} would be rejected, but not {{tupleint,
frozenlisttext}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-8270:
---
Description:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include sub-slices we could effectively request
for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
Allow sub slices for composites
---
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server
side slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c 3; --return 1
select count(*) from foo where a = 'key' and b 2 and c 3; --error
select count(*) from foo where a = 'key' and c 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include sub-slices we could effectively request
for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200435#comment-14200435 ] Todd Nine commented on CASSANDRA-8257: -- [~brandon.williams] I would love to, where is it? Opscenter Agent does not properly download target cassandra cluster --- Key: CASSANDRA-8257 URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 Project: Cassandra Issue Type: Bug Components: Tools Environment: Opscenter 5.0.1, Cassandra 1.2.19 Reporter: Todd Nine Rather than re-post the issue, it is outlined here. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. Note that when omitting the target Cassandra cluster, using the same cluster as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-5493) Confusing output of CommandDroppedTasks
[
https://issues.apache.org/jira/browse/CASSANDRA-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-5493.
---
Resolution: Incomplete
Reproduced In: 1.2.11, 1.2.9 (was: 1.2.9, 1.2.11)
[~ondrej.cernos] please, reopen this ticket if further help is needed.
Confusing output of CommandDroppedTasks
---
Key: CASSANDRA-5493
URL: https://issues.apache.org/jira/browse/CASSANDRA-5493
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.3
Reporter: Ondřej Černoš
Priority: Minor
We have 2 DCs, 3 nodes in each, using EC2 support. We are debugging nodetool
repair problems (roughly 1 out of 2 attempts just freezes). We looked into
the MessagingServiceBean to see what is going on using jmxterm. See the
following:
{noformat}
#mbean = org.apache.cassandra.net:type=MessagingService:
CommandDroppedTasks = {
107.aaa.bbb.ccc = 0;
166.ddd.eee.fff = 124320;
10.ggg.hhh.iii = 0;
107.jjj.kkk.lll = 0;
166.mmm.nnn.ooo = 1336699;
166.ppp.qqq.rrr = 1329171;
10.sss.ttt.uuu = 0;
107.vvv.www.xxx = 0;
};
{noformat}
The problem with this output is it has 8 records. The node's neighbours (the
107 and 10 nodes) are mentioned twice in the output, once with their public
IPs and once with their private IPs. The nodes in remote DC (the 166 ones)
are reported only once. I am pretty sure this is a bug - the node should be
reported only with one of its addresses in all outputs from Cassandra and it
should be consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200440#comment-14200440 ] Brandon Williams commented on CASSANDRA-8257: - I am told you should use the Feedback button in the UI, or post on SO with an opscenter tag. Opscenter Agent does not properly download target cassandra cluster --- Key: CASSANDRA-8257 URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 Project: Cassandra Issue Type: Bug Components: Tools Environment: Opscenter 5.0.1, Cassandra 1.2.19 Reporter: Todd Nine Rather than re-post the issue, it is outlined here. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. Note that when omitting the target Cassandra cluster, using the same cluster as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200435#comment-14200435 ] Todd Nine edited comment on CASSANDRA-8257 at 11/6/14 4:53 PM: --- [~brandon.williams] I would love to, where is it? I'm assuming it's here. https://datastax.jira.com But I can't actually get in to create an issue. was (Author: tnine): [~brandon.williams] I would love to, where is it? Opscenter Agent does not properly download target cassandra cluster --- Key: CASSANDRA-8257 URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 Project: Cassandra Issue Type: Bug Components: Tools Environment: Opscenter 5.0.1, Cassandra 1.2.19 Reporter: Todd Nine Rather than re-post the issue, it is outlined here. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. Note that when omitting the target Cassandra cluster, using the same cluster as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200445#comment-14200445 ] Todd Nine commented on CASSANDRA-8257: -- Already done. The opscenter tag is on the issue in this ticket. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster Opscenter Agent does not properly download target cassandra cluster --- Key: CASSANDRA-8257 URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 Project: Cassandra Issue Type: Bug Components: Tools Environment: Opscenter 5.0.1, Cassandra 1.2.19 Reporter: Todd Nine Rather than re-post the issue, it is outlined here. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. Note that when omitting the target Cassandra cluster, using the same cluster as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7408) System hints corruption - dataSize ... would be larger than file
[
https://issues.apache.org/jira/browse/CASSANDRA-7408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200467#comment-14200467
]
Michael Shuler commented on CASSANDRA-7408:
---
If you get this error on the same sstable, try a {{nodetool scrub}} to see if
that helps.
System hints corruption - dataSize ... would be larger than file
Key: CASSANDRA-7408
URL: https://issues.apache.org/jira/browse/CASSANDRA-7408
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: RHEL 6.5
Cassandra 1.2.16
RF=3
Thrift
Reporter: Jeff Griffith
I've found several unresolved JIRA tickets related to SSTable corruption but
not sure if they apply to the case we are seeing in system/hints. We see
periodic exceptions such as:
{noformat}
dataSize of 144115248479299639 starting at 17209 would be larger than file
/home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
35542
{noformat}
Is there something we could possibly be doing from the application to cause
this sort of corruption? We also see it on some of our own column families
also some *negative* lengths which are presumably a similar corruption.
{noformat}
ERROR [HintedHandoff:57] 2014-06-17 17:08:04,690 CassandraDaemon.java (line
191) Exception in thread Thread[HintedHandoff:57,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
dataSize of 144115248479299639 starting at 17209 would be larger than file
/home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
35542
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:441)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:282)
at
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:90)
at
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:508)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException:
org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
dataSize of 144115248479299639 starting at 17209 would be larger than file
/home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
35542
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:437)
... 6 more
Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException:
java.io.IOException: dataSize of 144115248479299639 starting at 17209 would
be larger than file
/home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
35542
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.init(SSTableIdentityIterator.java:167)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.init(SSTableIdentityIterator.java:83)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.init(SSTableIdentityIterator.java:69)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:180)
at
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:155)
at
org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:142)
at
org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:38)
at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:145)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:122)
at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:96)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:145)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
at
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200472#comment-14200472
]
Michael Shuler commented on CASSANDRA-8248:
---
Repair is an expensive process. After repair has completed and the JVM has
garbage collected, what are you seeing? If things return to expected resource
use, this sounds like expected behavior.
Possible memory leak
-
Key: CASSANDRA-8248
URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
Project: Cassandra
Issue Type: Bug
Reporter: Alexander Sterligov
Sometimes during repair cassandra starts to consume more memory than expected.
Total amount of data on node is about 20GB.
Size of the data directory is 66GC because of snapshots.
Top reports:
{quote}
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
{quote}
At the /proc/15724/maps there are a lot of deleted file maps
{quote}
7f63a6102000-7f63a6332000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6332000-7f63a6562000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6562000-7f63a6792000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6792000-7f63a69c2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6e22000-7f63a7052000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7052000-7f63a7282000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7282000-7f63a74b2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a76e2000-7f63a7912000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7912000-7f63a7b42000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a81d2000-7f63a8402000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8402000-7f63a8622000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8622000-7f63a8842000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8842000-7f63a8a62000 r--s 08:21 9442763
[jira] [Updated] (CASSANDRA-7458) functional indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-7458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-7458:
--
Description:
Indexing information derived from the row can be powerful. For example, using
the hypothetical {{extract_date}} function,
{code}
create table ticks (
symbol text,
ticked_at datetime,
price int,
tags settext,
PRIMARY KEY (symbol, ticked_at)
);
CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
SELECT * FROM ticks_by_day WHERE extract_date(ticked_at) = '2014-5-13';
{code}
http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
was:
Indexing information derived from the row can be powerful. For example, using
the hypothetical {{extract_date}} function,
{code}
create table ticks (
symbol text,
ticked_at datetime,
price int,
tags settext,
PRIMARY KEY (symbol, ticked_at)
);
CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
{code}
http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
functional indexes
--
Key: CASSANDRA-7458
URL: https://issues.apache.org/jira/browse/CASSANDRA-7458
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Assignee: Mikhail Stepura
Fix For: 3.0
Indexing information derived from the row can be powerful. For example,
using the hypothetical {{extract_date}} function,
{code}
create table ticks (
symbol text,
ticked_at datetime,
price int,
tags settext,
PRIMARY KEY (symbol, ticked_at)
);
CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
SELECT * FROM ticks_by_day WHERE extract_date(ticked_at) = '2014-5-13';
{code}
http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7859) Extend freezing to collections
[
https://issues.apache.org/jira/browse/CASSANDRA-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200482#comment-14200482
]
Tyler Hobbs commented on CASSANDRA-7859:
bq. To reiter, collection values are sorted (even when used as cell values),
and so it felt fishy to me to consider that changing to a type that change the
internal sorting of those values is compatible
Right, I agree with that, and the current patch doesn't change that. I think
an example will clarify this. Take the type {{frozenlistint}}. If you use
that type for a clustering column, then entire lists will be compared (is [1,
2, 3] [2, 3, 4]? etc) to sort the cells. In this case, the sorting order of
{{int}} must be maintained when you alter the type. However, if you use
{{frozenlistint}} for a cell value, that sorting order no longer matters.
It would be safe to alter it to {{frozenlistbytes}}.
That does *not* apply to sets, because the items within a set should be sorted.
It also does not apply to map *keys*, but map *values* can be treated like
list values.
Does that clarify things? I was a little tired when I tried to explain it last
night.
bq. Maybe we should just remove the comment in that case.
Yup, already removed it in the last commit.
What do you think about my comment on 8264 and getRequestedColumns?
Extend freezing to collections
--
Key: CASSANDRA-7859
URL: https://issues.apache.org/jira/browse/CASSANDRA-7859
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Labels: cql
Fix For: 2.1.2
Attachments: 7859-v1.txt
This is the follow-up to CASSANDRA-7857, to extend {{frozen}} to collections.
This will allow things like {{maptext, frozenmapint, int}} for
instance, as well as allowing {{frozen}} collections in PK columns.
Additionally (and that's alsmot a separate ticket but I figured we can start
discussing it here), we could decide that tuple is a frozen type by default.
This means that we would allow {{tupleint, text}} without needing to add
{{frozen}}, but we would require {{frozen}} for complex type inside tuple, so
{{tupleint, listtext}} would be rejected, but not {{tupleint,
frozenlisttext}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7953) RangeTombstones not merging during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7953:
--
Assignee: Marcus Eriksson
RangeTombstones not merging during compaction
-
Key: CASSANDRA-7953
URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.1
Reporter: Marcus Olsson
Assignee: Marcus Eriksson
Priority: Minor
Labels: compaction, deletes, tombstone
Attachments: 0001-7953-v2.patch, CASSANDRA-7953-1.patch,
CASSANDRA-7953.patch
When performing a compaction on two sstables that contain the same
RangeTombstone with different timestamps, the tombstones are not merged in
the new sstable.
This has been tested using cassandra 2.1 with the following table:
{code}
CREATE TABLE test(
key text,
column text,
data text,
PRIMARY KEY(key, column)
);
{code}
And then doing the following:
{code}
INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the
sstable only contains tombstones during compaction it seems that the sstable
either gets removed or isn't created (but that could probably be a separate
JIRA issue).
INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts
are not actually needed, since the deletes create tombstones either way.
DELETE FROM test WHERE key=1 AND column=2;
nodetool flush
INSERT INTO test (key, column, data) VALUES (1, 2, 2);
DELETE FROM test WHERE key=1 AND column=2;
nodetool flush
nodetool compact
{code}
When checking with the SSTableExport tool two tombstones exists in the
compacted sstable. This can be repeated, resulting in more and more
tombstones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8244: - Reviewer: Aleksey Yeschenko Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner -- Key: CASSANDRA-8244 URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Currently some of the functionality of Token refers to StorageService.getPartitioner() to avoid needing an extra argument. This is in turn implicitly used by RowPosition and then Range, causing possible problems, for example when ranges on secondary indices are used in a murmur-partitioned database. These references should be removed to force explicit choice of partitioner by callers; alternatively, the Token interface could be changed to provide a reference to the partitioner that created it. (Note: the hidden reference to partitioner in serialization is a separate issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8025) AssertionError during trigger execution
[ https://issues.apache.org/jira/browse/CASSANDRA-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200493#comment-14200493 ] Michael Shuler commented on CASSANDRA-8025: --- [~alice] could you please provide a reproduction case, such as a schema, your trigger, a bit of data, and a query that results in your error, so someone can follow up? Thanks! AssertionError during trigger execution --- Key: CASSANDRA-8025 URL: https://issues.apache.org/jira/browse/CASSANDRA-8025 Project: Cassandra Issue Type: Bug Environment: Linux CentOS release 6.5 Cassandra version 2.1.0 Reporter: Alice I am trying to implement a trigger that, as a consequence of an insertion on a table, will update a counter table. I can get all of the informations from the ByteBuffer key, ColumnFamily update, but when I try to instanziate a new CellNameType, using as AbstractType a CounterColumnType, I get an Unexpected exception AssertionError: null -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8024) No boot finished or ready message anymore upon startup completion to CLI
[
https://issues.apache.org/jira/browse/CASSANDRA-8024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-8024.
---
Resolution: Not a Problem
It appears that you are starting manually with {{./bin/cassandra}} which sends
a small amount of stdout and detaches, logging normally. You can foreground
your startup, if you wish: {{./bin/cassandra -f}}
No boot finished or ready message anymore upon startup completion to CLI
Key: CASSANDRA-8024
URL: https://issues.apache.org/jira/browse/CASSANDRA-8024
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.0.10
Reporter: Karl Mueller
Priority: Trivial
This is trivial, but cassandra logs the following to its log:
...
INFO [main] 2014-09-29 23:10:35,793 CassandraDaemon.java (line 575) No
gossip backlog; proceeding
INFO [main] 2014-09-29 23:10:35,979 Server.java (line 156) Starting
listening for CQL clients on
kaos-cass00.sv.walmartlabs.com/10.93.12.10:9042...
INFO [main] 2014-09-29 23:10:36,048 ThriftServer.java (line 99) Using
TFramedTransport with a max frame size of 15728640 bytes.
However, on the command line I only see:
INFO 23:10:30,005 Compacted 4 sstables to
[/data2/data-cassandra/system/compactions_in_progress/system-compactions_in_progress-jb-67,].
1,333 bytes to 962 (~72% of original) in 32ms = 0.028670MB/s. 15 total
partitions merged to 12. Partition merge counts were {1:11, 2:2, }
INFO 23:10:35,793 No gossip backlog; proceeding
I would be nice if the Starting listening for.. or some other startup
complete message went to the command line STDOUT. There used to be one, I
think, but there isn't anymore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200506#comment-14200506 ] Tyler Hobbs commented on CASSANDRA-7826: bq. Where does this stand now that we've introduced frozen types? Frozen collections can be nested, and normal (multi-cell) collections can contain frozen collections. I believe [~tupshin] is more interested in having proper nested multi-cell collections, which would allow for updating individual elements in nested collections and (once we have the ability to do this in CQL) retrieving individual elements from nested collections. support arbitrary nesting of collection --- Key: CASSANDRA-7826 URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tupshin Harper Assignee: Tyler Hobbs Labels: ponies The inability to nest collections is one of the bigger data modelling limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8244: - Fix Version/s: 3.0 Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner -- Key: CASSANDRA-8244 URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Fix For: 3.0 Currently some of the functionality of Token refers to StorageService.getPartitioner() to avoid needing an extra argument. This is in turn implicitly used by RowPosition and then Range, causing possible problems, for example when ranges on secondary indices are used in a murmur-partitioned database. These references should be removed to force explicit choice of partitioner by callers; alternatively, the Token interface could be changed to provide a reference to the partitioner that created it. (Note: the hidden reference to partitioner in serialization is a separate issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8022) cqlsh hangs indefinitely within a Docker container connecting to itself with hostname
[ https://issues.apache.org/jira/browse/CASSANDRA-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200513#comment-14200513 ] Michael Shuler commented on CASSANDRA-8022: --- Without using hostnames, do container IP addresses work with cqlsh? If not, then it seems to be a host-guest network routing problem. If IPs work, then it's a DNS resolution problem. I would guess that $hostname on the host and in container resolve different IP addresses? cqlsh hangs indefinitely within a Docker container connecting to itself with hostname - Key: CASSANDRA-8022 URL: https://issues.apache.org/jira/browse/CASSANDRA-8022 Project: Cassandra Issue Type: Bug Components: Tools Environment: Ubuntu 14.04, running Docker, run inside a Ubuntu 14.04 container. Reporter: Matthew O'Riordan Labels: cqlsh I am unable to use the `cqlsh` tool within a Docker container running Cassandra. Previously I would use the Java Thrift based `cqlsh` tool as follows: ``` cqlsh --username cassandra --password whatever $(hostname) ``` When I run the `cqlsh` command after attaching to a running container (I use LXC containerisation that allows attaching to a running container and running a console), it simply hangs and never reports an error. With the `--debug` flag on, I get the following with: **cqlsh 4.1.1** ``` $ cqlsh --debug --username cassandra --password obfuscated $(hostname) Using CQL driver: module 'cql' from '/usr/share/cassandra/lib/cql-internal-only-1.4.1.zip/cql-1.4.1/cql/__init__.py' Using thrift lib: module 'thrift' from '/usr/share/cassandra/lib/thrift-python-internal-only-0.9.1.zip/thrift/__init__.py' ``` It then hangs in this state indefinitely, I have no errors from `cqlsh` and no errors in the Cassandra log. **cqlsh 5.0.1** ``` $ cqlsh --debug --username cassandra --passwor obfuscated $(hostname) Using CQL driver: module 'cassandra' from '/usr/share/cassandra/lib/cassandra-driver-internal-only-2.1.0.post.zip/cassandra-driver-2.1.0.post/cassandra/__init__.py' ``` It then also hangs in this state indefinitely, I have no errors from `cqlsh` and no errors in the Cassandra log. What's interesting, and quite confusing is that: * I can telnet within the container as follows `telnet $(hostname) 9042` and I get a socket. When trying to issue some commands, I see Protocol errors in the Cassandra log thus verifying that the port is indeed open on the host that resolves from $(hostname) * If I `cqlsh` from another container or another host to the Cassandra container it works just fine. * I have tried disabling authentication altogether and using the AllowAllAuthenticator, and I experience the same problem. * `nodetool` works fine In the mean time, I am forced to `cqlsh` from another container as a workaround. Happy to try and do anything require to diagnose the cause of this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8138) replace_address cannot find node to be replaced node after seed node restart
[
https://issues.apache.org/jira/browse/CASSANDRA-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8138:
--
Assignee: Brandon Williams
replace_address cannot find node to be replaced node after seed node restart
Key: CASSANDRA-8138
URL: https://issues.apache.org/jira/browse/CASSANDRA-8138
Project: Cassandra
Issue Type: Bug
Reporter: Oleg Anastasyev
Assignee: Brandon Williams
Attachments: ReplaceAfterSeedRestart.txt
If a node failed and a cluster was restarted (which is common case on massive
outages), replace_address fails with
{code}
Caused by: java.lang.RuntimeException: Cannot replace_address /172.19.56.97
because it doesn't exist in gossip
jvm 1|at
org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:472)
jvm 1|at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:724)
jvm 1|at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:686)
jvm 1|at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:562)
{code}
Although neccessary information is saved in system tables on seed nodes, it
is not loaded to gossip on seed node, so a replacement node cannot get this
info.
Attached patch loads all information from system tables to gossip with
generation 0 and fixes some bugs around this info on shadow gossip round.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8156) Support indexes on composite column components of COMPACT tables
[
https://issues.apache.org/jira/browse/CASSANDRA-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8156:
--
Assignee: Sylvain Lebresne
Support indexes on composite column components of COMPACT tables
Key: CASSANDRA-8156
URL: https://issues.apache.org/jira/browse/CASSANDRA-8156
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Denis Angilella
Assignee: Sylvain Lebresne
CASSANDRA-5125 added support of indexes on composite column components for
*non-compact* tables (see CASSANDRA-5125 comments for additional information).
This is a follow up for *compact* tables.
Using compact tables it is possible to CREATE INDEX on composite primary key
columns, but queries returns no results for the tests below.
{code:sql}
CREATE TABLE users2 (
userID uuid,
fname text,
zip int,
state text,
PRIMARY KEY ((userID, fname))
) WITH COMPACT STORAGE;
CREATE INDEX ON users2 (userID);
CREATE INDEX ON users2 (fname);
INSERT INTO users2 (userID, fname, zip, state) VALUES
(b3e3bc33-b237-4b55-9337-3d41de9a5649, 'John', 10007, 'NY');
-- the following queries returns 0 rows, instead of 1 expected
SELECT * FROM users2 WHERE fname='John';
SELECT * FROM users2 WHERE userid=b3e3bc33-b237-4b55-9337-3d41de9a5649;
SELECT * FROM users2 WHERE userid=b3e3bc33-b237-4b55-9337-3d41de9a5649 AND
fname='John';
-- dropping 2ndary indexes restore normal behavior
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8138) replace_address cannot find node to be replaced node after seed node restart
[
https://issues.apache.org/jira/browse/CASSANDRA-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200541#comment-14200541
]
Brandon Williams commented on CASSANDRA-8138:
-
I think I'd much rather say that the edge case of a node dying, and then a full
cluster restart (rolling would still work) is just not supported, rather than
make such invasive changes to support replacement under such strange and rare
conditions. If that happens, it's time to assassinate the node and bootstrap
another one.
replace_address cannot find node to be replaced node after seed node restart
Key: CASSANDRA-8138
URL: https://issues.apache.org/jira/browse/CASSANDRA-8138
Project: Cassandra
Issue Type: Bug
Reporter: Oleg Anastasyev
Assignee: Brandon Williams
Attachments: ReplaceAfterSeedRestart.txt
If a node failed and a cluster was restarted (which is common case on massive
outages), replace_address fails with
{code}
Caused by: java.lang.RuntimeException: Cannot replace_address /172.19.56.97
because it doesn't exist in gossip
jvm 1|at
org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:472)
jvm 1|at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:724)
jvm 1|at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:686)
jvm 1|at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:562)
{code}
Although neccessary information is saved in system tables on seed nodes, it
is not loaded to gossip on seed node, so a replacement node cannot get this
info.
Attached patch loads all information from system tables to gossip with
generation 0 and fixes some bugs around this info on shadow gossip round.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200544#comment-14200544
]
Alexander Sterligov commented on CASSANDRA-8248:
No, it doesn't return to expected resources after forcing GC.
Just in case:
tpstats:
{quote}
Pool NameActive Pending Completed Blocked All
time blocked
CounterMutationStage 0 0 0 0
0
ReadStage 0 06868466 0
0
RequestResponseStage 0 0 408799535 0
0
MutationStage 0 0 198570382 0
0
ReadRepairStage 0 0 46197 0
0
GossipStage 0 02150908 0
0
CacheCleanupExecutor 0 0 0 0
0
MigrationStage0 0126 0
0
ValidationExecutor0 0 72747 0
0
MemtableReclaimMemory 0 0 97987 0
0
InternalResponseStage 0 0 41583 0
0
AntiEntropyStage 0 0 630829 0
0
MiscStage 0 0 0 0
0
CommitLogArchiver 0 0 0 0
0
MemtableFlushWriter 0 0 97987 0
0
PendingRangeCalculator0 0 62 0
0
MemtablePostFlush 0 0 191444 0
0
CompactionExecutor0 04131622 0
0
AntiEntropySessions 0 0512 0
0
HintedHandoff 0 0421 0
0
Message type Dropped
RANGE_SLICE 0
READ_REPAIR 0
PAGED_RANGE 0
BINARY 0
READ 0
MUTATION 30193
_TRACE 0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
{quote}
compactionstats:
{quote}
pending tasks: 0
Active compaction remaining time :n/a
{quote}
GC was just finished:
{quote}
INFO [Service Thread] 2014-11-06 21:25:05,233 GCInspector.java:88 -
ConcurrentMarkSweep GC in 15275ms. CMS Old Gen: 3596661592 - 3596726792; Par
Eden Space: 1371921168 - 0;
{quote}
It still uses ~51g of RAM.
Possible memory leak
-
Key: CASSANDRA-8248
URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
Project: Cassandra
Issue Type: Bug
Reporter: Alexander Sterligov
Sometimes during repair cassandra starts to consume more memory than expected.
Total amount of data on node is about 20GB.
Size of the data directory is 66GC because of snapshots.
Top reports:
{quote}
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
{quote}
At the /proc/15724/maps there are a lot of deleted file maps
{quote}
7f63a6102000-7f63a6332000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6332000-7f63a6562000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6562000-7f63a6792000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6792000-7f63a69c2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6e22000-7f63a7052000 r--s 08:21 9442763
[jira] [Updated] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7083:
--
Assignee: Brandon Williams
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Fix For: 2.0.12
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8244: --- Reviewer: Joshua McKenzie (was: Aleksey Yeschenko) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner -- Key: CASSANDRA-8244 URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Fix For: 3.0 Currently some of the functionality of Token refers to StorageService.getPartitioner() to avoid needing an extra argument. This is in turn implicitly used by RowPosition and then Range, causing possible problems, for example when ranges on secondary indices are used in a murmur-partitioned database. These references should be removed to force explicit choice of partitioner by callers; alternatively, the Token interface could be changed to provide a reference to the partitioner that created it. (Note: the hidden reference to partitioner in serialization is a separate issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200557#comment-14200557 ] Joshua McKenzie commented on CASSANDRA-8244: Grabbing review as per IRC Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner -- Key: CASSANDRA-8244 URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Fix For: 3.0 Currently some of the functionality of Token refers to StorageService.getPartitioner() to avoid needing an extra argument. This is in turn implicitly used by RowPosition and then Range, causing possible problems, for example when ranges on secondary indices are used in a murmur-partitioned database. These references should be removed to force explicit choice of partitioner by callers; alternatively, the Token interface could be changed to provide a reference to the partitioner that created it. (Note: the hidden reference to partitioner in serialization is a separate issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7950) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names.
[ https://issues.apache.org/jira/browse/CASSANDRA-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-7950: -- Labels: lhf (was: ) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names. - Key: CASSANDRA-7950 URL: https://issues.apache.org/jira/browse/CASSANDRA-7950 Project: Cassandra Issue Type: Bug Environment: CentOS 5, 64bit, Oracle JDK 7, DSE Reporter: Eugene Priority: Minor Labels: lhf Fix For: 2.0.12 Attachments: nodetool-examples.txt When running these commands: nodetool compactionstats nodetool compactionhistory The output can be difficult to grok due to long keyspace names, column family names, and long values. I have attached an example. It's difficult for both humans and grep/sed/awk/perl to read. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4511) Secondary index support for CQL3 collections
[
https://issues.apache.org/jira/browse/CASSANDRA-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200569#comment-14200569
]
Lex Lythius commented on CASSANDRA-4511:
Note that CQL for Cassandra 2.x documentation (page 48) still says:
{quote}Currently, you cannot create an index on a column of type map, set, or
list.{quote}
whereas CREATE INDEX syntax is accurate:
{code}
CREATE CUSTOM INDEX IF NOT EXISTS index_name
ON keyspace_name.table_name ( KEYS ( column_name ) )
(USING class_name) (WITH OPTIONS = map)
{code}
Secondary index support for CQL3 collections
-
Key: CASSANDRA-4511
URL: https://issues.apache.org/jira/browse/CASSANDRA-4511
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.1 beta1
Attachments: 4511.txt
We should allow to 2ndary index on collections. A typical use case would be
to add a 'tag setString' to say a user profile and to query users based on
what tag they have.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200579#comment-14200579
]
Alex Liu commented on CASSANDRA-7083:
-
+1
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Fix For: 2.0.12
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8226) Cannot call a column 'timestamp'
[
https://issues.apache.org/jira/browse/CASSANDRA-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura resolved CASSANDRA-8226.
Resolution: Duplicate
Fix Version/s: (was: 2.1.3)
2.1.2
Reproduced In: 2.1.1
Will be fixed by CASSANDRA-8262
Cannot call a column 'timestamp'
Key: CASSANDRA-8226
URL: https://issues.apache.org/jira/browse/CASSANDRA-8226
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: [cqlsh 5.0.1 | Cassandra 2.1.1 | CQL spec 3.2.0 | Native
protocol v3]
Reporter: Dave Cunningham
Assignee: Mikhail Stepura
Priority: Minor
Labels: cqlsh
Fix For: 2.1.2
{code}
create table test(date text, timestamp timeuuid, stuff text, PRIMARY
KEY(date, timestamp));
insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
select * from test where date = '20141030' order by timestamp limit 1;
Traceback (most recent call last):
File bin/cqlsh, line 861, in onecmd
self.handle_statement(st, statementtext)
File bin/cqlsh, line 901, in handle_statement
return self.handle_parse_error(cmdword, tokens, parsed, srcstr)
File bin/cqlsh, line 910, in handle_parse_error
return self.perform_statement(cqlruleset.cql_extract_orig(tokens, srcstr))
File bin/cqlsh, line 935, in perform_statement
result = self.perform_simple_statement(stmt)
File bin/cqlsh, line 968, in perform_simple_statement
self.print_result(rows, self.parse_for_table_meta(statement.query_string))
File bin/cqlsh, line 946, in parse_for_table_meta
parsed = cqlruleset.cql_parse(query_string)[1]
IndexError: list index out of range
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8271) Remove SSTableDeletingTask
Joshua McKenzie created CASSANDRA-8271: -- Summary: Remove SSTableDeletingTask Key: CASSANDRA-8271 URL: https://issues.apache.org/jira/browse/CASSANDRA-8271 Project: Cassandra Issue Type: Improvement Reporter: Joshua McKenzie Assignee: Joshua McKenzie Priority: Minor Fix For: 3.0 With CASSANDRA-4050 and CASSANDRA-6993, our out-of-order deletion problems w/regards to Windows are resolved and the only outstanding reason to have SSTableDeletingTask would be for support of non-sun VM's w/regards to mmap'ed files. As this is no longer a big concern in the Cassandra ecosystem (non-sun VM's), we should remove SSTableDeletingTask. The one caveat is that if we want to revisit mmap'ed I/O on Windows in the future we may need to re-use this type of delayed deletion approach due to Windows not allowing deletion of hard linked files w/memory-mapped segments in the original file, but CASSANDRA-5863 would obviate that concern. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8019) Windows Unit tests and Dtests erroring due to sstable deleting task error
[
https://issues.apache.org/jira/browse/CASSANDRA-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200590#comment-14200590
]
Joshua McKenzie commented on CASSANDRA-8019:
Discussed this offline - I'm going to pursue formalizing the relationship
between SSTableScanners and SSTableReaders w/expectations of ordering, as we
don't want to close out SSTR w/scanners still open to them (in general
principle, though it doesn't seem to be hurting anything at this time).
Currently in the 2.1 branch, Windows' intolerance of deletion of files w/open
filehandles gives us a temporary canary in a coal mine on these specific
ordering issues but we shouldn't rely on that, and that goes away in 3.0 anyway.
My gut feeling is that codifying the expectation of release/deletion ordering
is best served by _aggressive_v1 but I'm loathe to add *more* users of the ref
counting mechanism on SSTR as it's been such a pain point for us.
I've also created CASSANDRA-8271 to cover SSTableDeletingTask removal in 3.0.
Windows Unit tests and Dtests erroring due to sstable deleting task error
-
Key: CASSANDRA-8019
URL: https://issues.apache.org/jira/browse/CASSANDRA-8019
Project: Cassandra
Issue Type: Bug
Environment: Windows 7
Reporter: Philip Thompson
Assignee: Joshua McKenzie
Labels: windows
Fix For: 2.1.2
Attachments: 8019_aggressive_v1.txt, 8019_conservative_v1.txt,
8019_v2.txt
Currently a large number of dtests and unit tests are erroring on windows
with the following error in the node log:
{code}
ERROR [NonPeriodicTasks:1] 2014-09-29 11:05:04,383
SSTableDeletingTask.java:89 - Unable to delete
c:\\users\\username\\appdata\\local\\temp\\dtest-vr6qgw\\test\\node1\\data\\system\\local-7ad54392bcdd35a684174e047860b377\\system-local-ka-4-Data.db
(it will be removed on server restart; we'll also retry after GC)\n
{code}
git bisect points to the following commit:
{code}
0e831007760bffced8687f51b99525b650d7e193 is the first bad commit
commit 0e831007760bffced8687f51b99525b650d7e193
Author: Benedict Elliott Smith bened...@apache.org
Date: Fri Sep 19 18:17:19 2014 +0100
Fix resource leak in event of corrupt sstable
patch by benedict; review by yukim for CASSANDRA-7932
:100644 100644 d3ee7d99179dce03307503a8093eb47bd0161681
f55e5d27c1c53db3485154cd16201fc5419f32df M CHANGES.txt
:04 04 194f4c0569b6be9cc9e129c441433c5c14de7249
3c62b53b2b2bd4b212ab6005eab38f8a8e228923 M src
:04 04 64f49266e328b9fdacc516c52ef1921fe42e994f
de2ca38232bee6d2a6a5e068ed9ee0fbbc5aaebe M test
{code}
You can reproduce this by running simple_bootstrap_test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200596#comment-14200596
]
Brandon Williams commented on CASSANDRA-7083:
-
Can you fix the code style problems? Brace on newline is missing in several
places, and the indentation is off everywhere.
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Fix For: 2.0.12
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200604#comment-14200604 ] Shawn Kumar commented on CASSANDRA-7217: I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482cmetric=op_rateoperation=1_writesmoothing=1show_aggregates=truexmin=0xmax=279.07ymin=0ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last and it seems like there is a noticeable drop. Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads - Key: CASSANDRA-7217 URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benedict Assignee: Shawn Kumar Labels: performance, triaged Fix For: 2.1.2 This is obviously bad. Let's figure out why it's happening and put a stop to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200604#comment-14200604 ] Shawn Kumar edited comment on CASSANDRA-7217 at 11/6/14 6:32 PM: - I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482cmetric=op_rateoperation=1_writesmoothing=1show_aggregates=truexmin=0xmax=279.07ymin=0ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1000 threads. was (Author: shawn.kumar): I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482cmetric=op_rateoperation=1_writesmoothing=1show_aggregates=truexmin=0xmax=279.07ymin=0ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last and it seems like there is a noticeable drop. Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads - Key: CASSANDRA-7217 URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benedict Assignee: Shawn Kumar Labels: performance, triaged Fix For: 2.1.2 This is obviously bad. Let's figure out why it's happening and put a stop to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4511) Secondary index support for CQL3 collections
[ https://issues.apache.org/jira/browse/CASSANDRA-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200624#comment-14200624 ] Kristine Hahn commented on CASSANDRA-4511: -- Thanks, [~lexlythius]. http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/collection_type_r.html Secondary index support for CQL3 collections - Key: CASSANDRA-4511 URL: https://issues.apache.org/jira/browse/CASSANDRA-4511 Project: Cassandra Issue Type: Improvement Affects Versions: 1.2.0 beta 1 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 2.1 beta1 Attachments: 4511.txt We should allow to 2ndary index on collections. A typical use case would be to add a 'tag setString' to say a user profile and to query users based on what tag they have. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8252) dtests that involve topology changes should verify system.peers on all nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-8252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-8252: --- Assignee: Shawn Kumar dtests that involve topology changes should verify system.peers on all nodes Key: CASSANDRA-8252 URL: https://issues.apache.org/jira/browse/CASSANDRA-8252 Project: Cassandra Issue Type: Test Reporter: Brandon Williams Assignee: Shawn Kumar This is especially true for replace where I've discovered it's wrong in 1.2.19, which is sad because now it's too late to fix. We've had a lot of problems with incorrect/null system.peers, so after any topology change we should verify it on every live node when everything is finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200604#comment-14200604 ] Shawn Kumar edited comment on CASSANDRA-7217 at 11/6/14 6:46 PM: - I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482cmetric=op_rateoperation=1_writesmoothing=1show_aggregates=truexmin=0xmax=279.07ymin=0ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1250 threads. was (Author: shawn.kumar): I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482cmetric=op_rateoperation=1_writesmoothing=1show_aggregates=truexmin=0xmax=279.07ymin=0ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1000 threads. Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads - Key: CASSANDRA-7217 URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benedict Assignee: Shawn Kumar Labels: performance, triaged Fix For: 2.1.2 This is obviously bad. Let's figure out why it's happening and put a stop to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200685#comment-14200685
]
Brandon Williams commented on CASSANDRA-7083:
-
Also this doesn't compile:
{noformat}
src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java:115: error:
variable password is already defined in constructor
CqlRecordWriter(Configuration)
[javac] String password =
ConfigHelper.getOutputKeyspacePassword(conf);
[javac] ^
{noformat}
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Fix For: 2.0.12
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200697#comment-14200697 ] Tyler Hobbs commented on CASSANDRA-8221: [~rnamboodiri] sorry for the slow response. The patch looks good, except for a whitespace nitpick: please use a space in between method parameters. With that fixed, please go ahead and attach a patch for the 2.0 branch as well. Thanks! Specify keyspace in error message when streaming fails due to missing replicas -- Key: CASSANDRA-8221 URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Rajanarayanan Thottuvaikkatumana Priority: Trivial Labels: lhf Fix For: 2.0.12, 2.1.2 Attachments: cassandra-2.1.1-8221.txt When there aren't sufficient live replicas for streaming (during bootstrap, etc), you'll get an error message like unable to find sufficient sources for streaming range. It would be helpful to include the keyspace that this failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200699#comment-14200699
]
Michael Shuler commented on CASSANDRA-8248:
---
Anything interesting in thread and heap dumps? If you are only looking at the
value top gives you, your initial post had 55G resident - you just posted 51G
(assume this to also be resident). I don't see where the leak is.
Possible memory leak
-
Key: CASSANDRA-8248
URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
Project: Cassandra
Issue Type: Bug
Reporter: Alexander Sterligov
Sometimes during repair cassandra starts to consume more memory than expected.
Total amount of data on node is about 20GB.
Size of the data directory is 66GC because of snapshots.
Top reports:
{quote}
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
{quote}
At the /proc/15724/maps there are a lot of deleted file maps
{quote}
7f63a6102000-7f63a6332000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6332000-7f63a6562000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6562000-7f63a6792000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6792000-7f63a69c2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6e22000-7f63a7052000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7052000-7f63a7282000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7282000-7f63a74b2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a76e2000-7f63a7912000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7912000-7f63a7b42000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a81d2000-7f63a8402000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8402000-7f63a8622000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8622000-7f63a8842000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8842000-7f63a8a62000 r--s
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200707#comment-14200707
]
Michael Shuler commented on CASSANDRA-8248:
---
How much physical RAM is in the machine and what are your heap configs?
Possible memory leak
-
Key: CASSANDRA-8248
URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
Project: Cassandra
Issue Type: Bug
Reporter: Alexander Sterligov
Sometimes during repair cassandra starts to consume more memory than expected.
Total amount of data on node is about 20GB.
Size of the data directory is 66GC because of snapshots.
Top reports:
{quote}
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
{quote}
At the /proc/15724/maps there are a lot of deleted file maps
{quote}
7f63a6102000-7f63a6332000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6332000-7f63a6562000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6562000-7f63a6792000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6792000-7f63a69c2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a6e22000-7f63a7052000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7052000-7f63a7282000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7282000-7f63a74b2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a76e2000-7f63a7912000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7912000-7f63a7b42000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a81d2000-7f63a8402000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8402000-7f63a8622000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8622000-7f63a8842000 r--s 08:21 9442763
/ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
(deleted)
7f63a8842000-7f63a8a62000 r--s 08:21 9442763
[jira] [Commented] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
[
https://issues.apache.org/jira/browse/CASSANDRA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200725#comment-14200725
]
Tyler Hobbs commented on CASSANDRA-7124:
[~rnamboodiri] that sounds correct to me. There will probably be some overlap
for each of the cleanup/compact/etc async operations, so you will probably want
to make a generic Runner class that CleanupRunner, CompactionRunner, etc can
inherit from.
Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
Key: CASSANDRA-7124
URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Rajanarayanan Thottuvaikkatumana
Priority: Minor
Labels: lhf
Fix For: 3.0
If {{nodetool cleanup}} or some other long-running operation takes too long
to complete, you'll see an error like the one in CASSANDRA-2126, so you can't
tell if the operation completed successfully or not. CASSANDRA-4767 fixed
this for repairs with JMX notifications. We should do something similar for
nodetool cleanup, compact, decommission, move, relocate, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200845#comment-14200845 ] Jonathan Ellis commented on CASSANDRA-7826: --- Right, we want both really. Do you want to create a separate ticket for frozen nested collections in 2.x? support arbitrary nesting of collection --- Key: CASSANDRA-7826 URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tupshin Harper Assignee: Tyler Hobbs Labels: ponies The inability to nest collections is one of the bigger data modelling limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200853#comment-14200853 ] Tupshin Harper commented on CASSANDRA-7826: --- Correct interpretation of my end goal. I'm neutral on the need/benefit of doing frozen/nested in 2.x. Personally I'd be OK deferring full nesting of collections until unfrozen nested support in 3.0. support arbitrary nesting of collection --- Key: CASSANDRA-7826 URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tupshin Harper Assignee: Tyler Hobbs Labels: ponies The inability to nest collections is one of the bigger data modelling limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 65f799008 - 6b3f3e960
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/cassandra-2.0
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie liang...@apache.org
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs ty...@datastax.com
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format(Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default,
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug(loaded network topology {},
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.EntryInetAddress, String[] entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(:).append(Arrays.toString(entry.getValue())).append(,
);
+logger.debug(Loaded network topology from property file: {},
StringUtils.removeEnd(sb.toString(), , ));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
[1/2] cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 9685622b3 - ffe69c3d7
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/cassandra-2.1
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie liang...@apache.org
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs ty...@datastax.com
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format(Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default,
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug(loaded network topology {},
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.EntryInetAddress, String[] entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(:).append(Arrays.toString(entry.getValue())).append(,
);
+logger.debug(Loaded network topology from property file: {},
StringUtils.removeEnd(sb.toString(), , ));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ffe69c3d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ffe69c3d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ffe69c3d Branch: refs/heads/cassandra-2.1 Commit: ffe69c3d7dd3b529c90bdd30356d991998289284 Parents: 9685622 6b3f3e9 Author: Tyler Hobbs ty...@datastax.com Authored: Thu Nov 6 14:31:55 2014 -0600 Committer: Tyler Hobbs ty...@datastax.com Committed: Thu Nov 6 14:31:55 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/CHANGES.txt -- diff --cc CHANGES.txt index 89db48e,8a7697b..c1cc083 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,6 +1,35 @@@ -2.0.12: +2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) ++Merged from 2.0: + * Improve PropertyFileSnitch logging (CASSANDRA-8183) - * Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164) + +2.1.2 + * (cqlsh) parse_for_table_meta errors out on queries with undefined + grammars (CASSANDRA-8262) + * (cqlsh) Fix SELECT ... TOKEN() function broken in C* 2.1.1 (CASSANDRA-8258) + * Fix Cassandra crash when running on JDK8 update 40 (CASSANDRA-8209) + * Optimize partitioner tokens (CASSANDRA-8230) + * Improve compaction of repaired/unrepaired sstables (CASSANDRA-8004) + * Make cache serializers pluggable (CASSANDRA-8096) + * Fix issues with CONTAINS (KEY) queries on secondary indexes + (CASSANDRA-8147) + * Fix read-rate tracking of sstables for some queries (CASSANDRA-8239) + * Fix default timestamp in QueryOptions (CASSANDRA-8246) + * Set socket timeout when reading remote version (CASSANDRA-8188) + * Refactor how we track live size (CASSANDRA-7852) + * Make sure unfinished compaction files are removed (CASSANDRA-8124) + * Fix shutdown when run as Windows service (CASSANDRA-8136) + * Fix DESCRIBE TABLE with custom indexes (CASSANDRA-8031) + * Fix race in RecoveryManagerTest (CASSANDRA-8176) + * Avoid IllegalArgumentException while sorting sstables in + IndexSummaryManager (CASSANDRA-8182) + * Shutdown JVM on file descriptor exhaustion (CASSANDRA-7579) + * Add 'die' policy for commit log and disk failure (CASSANDRA-7927) + * Fix installing as service on Windows (CASSANDRA-8115) + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144) + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109) +Merged from 2.0: * Correctly handle non-text column names in cql3 (CASSANDRA-8178) * Fix deletion for indexes on primary key columns (CASSANDRA-8206) * Add 'nodetool statusgossip' (CASSANDRA-8125) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java --
[jira] [Commented] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200858#comment-14200858
]
Yuki Morishita commented on CASSANDRA-8211:
---
+1
Overlapping sstables in L1+
---
Key: CASSANDRA-8211
URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
Project: Cassandra
Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
0001-Avoid-overlaps-in-L1.patch
Seems we have a bug that can create overlapping sstables in L1:
{code}
WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
2, SSTableReader(path='sstable') [DecoratedKey(2838397575996053472, 00
10066059b210066059b210400100),
DecoratedKey(5516674013223138308,
001000ff2d161000ff2d160
00010400100)] overlaps
SSTableReader(path='sstable') [DecoratedKey(2839992722300822584,
0010
00229ad21000229ad210400100),
DecoratedKey(5532836928694021724,
0010034b05a610034b05a6100
000400100)]. This could be caused by a bug in
Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
from another node into the data directory. Sending back to L0. If
you didn't drop in sstables, and have not yet run scrub, you should do so
since you may also have rows out-of-order within an sstable
{code}
Which might manifest itself during compaction with this exception:
{code}
ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
(line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
java.lang.RuntimeException: Last written key
DecoratedKey(5516674013223138308,
001000ff2d161000ff2d1610400100)
= current key DecoratedKey(2839992722300822584,
001000229ad21000229ad210400100)
writing into sstable
{code}
since we use LeveledScanner when compacting (the backing sstable scanner
might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a67980e8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a67980e8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a67980e8 Branch: refs/heads/trunk Commit: a67980e8ee8a6bc72bb78044ee2a1b20a3121a2a Parents: b4cbb28 ffe69c3 Author: Tyler Hobbs ty...@datastax.com Authored: Thu Nov 6 14:32:23 2014 -0600 Committer: Tyler Hobbs ty...@datastax.com Committed: Thu Nov 6 14:32:23 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a67980e8/CHANGES.txt --
[1/3] cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/trunk b4cbb28f3 - a67980e8e
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/trunk
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie liang...@apache.org
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs ty...@datastax.com
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format(Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default,
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug(loaded network topology {},
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.EntryInetAddress, String[] entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(:).append(Arrays.toString(entry.getValue())).append(,
);
+logger.debug(Loaded network topology from property file: {},
StringUtils.removeEnd(sb.toString(), , ));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ffe69c3d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ffe69c3d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ffe69c3d Branch: refs/heads/trunk Commit: ffe69c3d7dd3b529c90bdd30356d991998289284 Parents: 9685622 6b3f3e9 Author: Tyler Hobbs ty...@datastax.com Authored: Thu Nov 6 14:31:55 2014 -0600 Committer: Tyler Hobbs ty...@datastax.com Committed: Thu Nov 6 14:31:55 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/CHANGES.txt -- diff --cc CHANGES.txt index 89db48e,8a7697b..c1cc083 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,6 +1,35 @@@ -2.0.12: +2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) ++Merged from 2.0: + * Improve PropertyFileSnitch logging (CASSANDRA-8183) - * Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164) + +2.1.2 + * (cqlsh) parse_for_table_meta errors out on queries with undefined + grammars (CASSANDRA-8262) + * (cqlsh) Fix SELECT ... TOKEN() function broken in C* 2.1.1 (CASSANDRA-8258) + * Fix Cassandra crash when running on JDK8 update 40 (CASSANDRA-8209) + * Optimize partitioner tokens (CASSANDRA-8230) + * Improve compaction of repaired/unrepaired sstables (CASSANDRA-8004) + * Make cache serializers pluggable (CASSANDRA-8096) + * Fix issues with CONTAINS (KEY) queries on secondary indexes + (CASSANDRA-8147) + * Fix read-rate tracking of sstables for some queries (CASSANDRA-8239) + * Fix default timestamp in QueryOptions (CASSANDRA-8246) + * Set socket timeout when reading remote version (CASSANDRA-8188) + * Refactor how we track live size (CASSANDRA-7852) + * Make sure unfinished compaction files are removed (CASSANDRA-8124) + * Fix shutdown when run as Windows service (CASSANDRA-8136) + * Fix DESCRIBE TABLE with custom indexes (CASSANDRA-8031) + * Fix race in RecoveryManagerTest (CASSANDRA-8176) + * Avoid IllegalArgumentException while sorting sstables in + IndexSummaryManager (CASSANDRA-8182) + * Shutdown JVM on file descriptor exhaustion (CASSANDRA-7579) + * Add 'die' policy for commit log and disk failure (CASSANDRA-7927) + * Fix installing as service on Windows (CASSANDRA-8115) + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144) + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109) +Merged from 2.0: * Correctly handle non-text column names in cql3 (CASSANDRA-8178) * Fix deletion for indexes on primary key columns (CASSANDRA-8206) * Add 'nodetool statusgossip' (CASSANDRA-8125) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java --
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200865#comment-14200865 ] Tyler Hobbs commented on CASSANDRA-7826: bq. Do you want to create a separate ticket for frozen nested collections in 2.x? I suppose I wasn't very clear -- CASSANDRA-7859 supports frozen nested collections as-is, so there's no need for a separate ticket. support arbitrary nesting of collection --- Key: CASSANDRA-7826 URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tupshin Harper Assignee: Tyler Hobbs Labels: ponies The inability to nest collections is one of the bigger data modelling limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: 8236.txt
Patch to add a new ApplicationState, RPC_READY, which is a boolean that
defaults to false at startup. Once the native server is started, it then flips
this to true. If the remote node is older, we preserve the behavior of
CASSANDRA-7510 and give our best effort to notify clients when the server is
ready.
Note that while this didn't break anything, I didn't actually test it with a
native client.
Delay node up and node added notifications until native protocol server
is started
--
Key: CASSANDRA-8236
URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Fix For: 3.0
Attachments: 8236.txt
As discussed in CASSANDRA-7510, there is still a gap between when a node up
or node added notification may be sent to native protocol clients (in
response to a gossip event) and when the native protocol server is ready to
serve requests.
Everything in between the call to {{StorageService.instance.initServer()}}
and creation of the native server in {{CassandraDaemon.setup()}} contributes
to this delay, but waiting for Gossip to settle introduces the biggest delay.
We may need to introduce a STARTING gossip state for the period inbetween,
which is why this is scheduled for 3.0. If there's a better option, though,
it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: (was: 8236.txt)
Delay node up and node added notifications until native protocol server
is started
--
Key: CASSANDRA-8236
URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Fix For: 3.0
Attachments: 8236.txt
As discussed in CASSANDRA-7510, there is still a gap between when a node up
or node added notification may be sent to native protocol clients (in
response to a gossip event) and when the native protocol server is ready to
serve requests.
Everything in between the call to {{StorageService.instance.initServer()}}
and creation of the native server in {{CassandraDaemon.setup()}} contributes
to this delay, but waiting for Gossip to settle introduces the biggest delay.
We may need to introduce a STARTING gossip state for the period inbetween,
which is why this is scheduled for 3.0. If there's a better option, though,
it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: 8236.txt
Delay node up and node added notifications until native protocol server
is started
--
Key: CASSANDRA-8236
URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Fix For: 3.0
Attachments: 8236.txt
As discussed in CASSANDRA-7510, there is still a gap between when a node up
or node added notification may be sent to native protocol clients (in
response to a gossip event) and when the native protocol server is ready to
serve requests.
Everything in between the call to {{StorageService.instance.initServer()}}
and creation of the native server in {{CassandraDaemon.setup()}} contributes
to this delay, but waiting for Gossip to settle introduces the biggest delay.
We may need to introduce a STARTING gossip state for the period inbetween,
which is why this is scheduled for 3.0. If there's a better option, though,
it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200941#comment-14200941
]
Brandon Williams commented on CASSANDRA-8236:
-
We could also flip this back to false if the native server is stopped (via
nodetool or the like) but I'm not sure if onDown is really the correct
notification to send for that.
Delay node up and node added notifications until native protocol server
is started
--
Key: CASSANDRA-8236
URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Fix For: 3.0
Attachments: 8236.txt
As discussed in CASSANDRA-7510, there is still a gap between when a node up
or node added notification may be sent to native protocol clients (in
response to a gossip event) and when the native protocol server is ready to
serve requests.
Everything in between the call to {{StorageService.instance.initServer()}}
and creation of the native server in {{CassandraDaemon.setup()}} contributes
to this delay, but waiting for Gossip to settle introduces the biggest delay.
We may need to introduce a STARTING gossip state for the period inbetween,
which is why this is scheduled for 3.0. If there's a better option, though,
it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7826) support non-frozen, nested collections
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-7826: -- Component/s: API Fix Version/s: 3.0 Summary: support non-frozen, nested collections (was: support arbitrary nesting of collection) okay, edited title of this ticket to reflect that it's specifically for non-frozen collections, and tagged it 3.0 support non-frozen, nested collections -- Key: CASSANDRA-7826 URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Tupshin Harper Assignee: Tyler Hobbs Labels: ponies Fix For: 3.0 The inability to nest collections is one of the bigger data modelling limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8272) 2ndary indexes can return stale data
Sylvain Lebresne created CASSANDRA-8272:
---
Summary: 2ndary indexes can return stale data
Key: CASSANDRA-8272
URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
When replica return 2ndary index results, it's possible for a single replica to
return a stale result and that result will be sent back to the user,
potentially failing the CL contract.
For instance, consider 3 replicas A, B and C, and the following situation:
{noformat}
CREATE TABLE test (k int PRIMARY KEY, v text);
CREATE INDEX ON test(v);
INSERT INTO test(k, v) VALUES (0, 'foo');
{noformat}
with every replica up to date. Now, suppose that the following queries are done
at {{QUORUM}}:
{noformat}
UPDATE test SET v = 'bar' WHERE k = 0;
SELECT * FROM test WHERE v = 'foo';
{noformat}
then, if A and B acknowledge the insert but C respond to the read before having
applied the insert, then the now stale result will be returned (since C will
return it and A or B will return nothing).
A potential solution would be that when we read a tombstone in the index (and
provided we make the index inherit the gcGrace of it's parent CF), instead of
skipping that tombstone, we'd insert in the result a corresponding range
tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8273) Allow filtering queries can return stale data
Sylvain Lebresne created CASSANDRA-8273:
---
Summary: Allow filtering queries can return stale data
Key: CASSANDRA-8273
URL: https://issues.apache.org/jira/browse/CASSANDRA-8273
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Data filtering is done replica side. That means that a single replica with
stale data may make the whole query return that stale data.
For instance, consider 3 replicas A, B and C, and the following situation:
{noformat}
CREATE TABLE test (k int PRIMARY KEY, v1 text, v2 int);
CREATE INDEX ON test(v1);
INSERT INTO test(k, v1, v2) VALUES (0, 'foo', 1);
{noformat}
with every replica up to date. Now, suppose that the following queries are done
at {{QUORUM}}:
{noformat}
UPDATE test SET v2 = 2 WHERE k = 0;
SELECT * FROM test WHERE v1 = 'foo' AND v2 = 1;
{noformat}
then, if A and B acknowledge the insert but C respond to the read before having
applied the insert, then the now stale result will be returned. Let's note that
this is a problem related to filtering, not 2ndary indexes.
This issue share similarity with CASSANDRA-8272 but contrarily to that former
issue, I'm not sure how to fix it. Obviously, moving the filtering to the
coordinator would remove that problem, but doing so would, on top of not being
trivial to implmenent, have serious performance impact since we can't know in
advance how much data will be filtered and we may have to redo query to replica
multiple times.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200974#comment-14200974
]
Tyler Hobbs commented on CASSANDRA-7970:
There's one problem that I forgot about: the JSON spec only accepts strings for
map keys.
I think that leaves us with two options:
# Accept a string version of anything (including ints, floats, etc). For
example, if we're expecting a float when decoding JSON and the JSON decoder
returns a String, we should try to create a float from it. For {{toJson()}},
we would convert non-strings to strings when they're map keys.
# Don't allow {{toJson()}} and {{fromJson()}} to be used on maps with
non-string keys. If we have transform functions at some point, those could be
used to support these kinds of maps.
Option 2 would be my preference.
JSON support for CQL
Key: CASSANDRA-7970
URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
Project: Cassandra
Issue Type: Bug
Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Fix For: 3.0
JSON is popular enough that not supporting it is becoming a competitive
weakness. We can add JSON support in a way that is compatible with our
performance goals by *mapping* JSON to an existing schema: one JSON documents
maps to one CQL row.
Thus, it is NOT a goal to support schemaless documents, which is a misfeature
[1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
document from a service or a user into a CQL row, with all the validation
that entails.
Since we are not looking to support schemaless documents, we will not be
adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
map the JSON to UDT, collections, and primitive CQL types.
Here's how this might look:
{code}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
);
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses maptext, address
);
INSERT INTO users JSON
{‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
“city”: “Austin”,
“zip_code”: 78747,
“phones”: [2101234567]}}};
SELECT JSON id, address FROM users;
{code}
(We would also want to_json and from_json functions to allow mapping a single
column's worth of data. These would not require extra syntax.)
[1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
[2] https://blog.compose.io/schema-less-is-usually-a-lie/
[3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200982#comment-14200982
]
Aleksey Yeschenko commented on CASSANDRA-7970:
--
bq. Option 2 would be my preference.
Likewise.
JSON support for CQL
Key: CASSANDRA-7970
URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
Project: Cassandra
Issue Type: Bug
Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Fix For: 3.0
JSON is popular enough that not supporting it is becoming a competitive
weakness. We can add JSON support in a way that is compatible with our
performance goals by *mapping* JSON to an existing schema: one JSON documents
maps to one CQL row.
Thus, it is NOT a goal to support schemaless documents, which is a misfeature
[1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
document from a service or a user into a CQL row, with all the validation
that entails.
Since we are not looking to support schemaless documents, we will not be
adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
map the JSON to UDT, collections, and primitive CQL types.
Here's how this might look:
{code}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
);
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses maptext, address
);
INSERT INTO users JSON
{‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
“city”: “Austin”,
“zip_code”: 78747,
“phones”: [2101234567]}}};
SELECT JSON id, address FROM users;
{code}
(We would also want to_json and from_json functions to allow mapping a single
column's worth of data. These would not require extra syntax.)
[1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
[2] https://blog.compose.io/schema-less-is-usually-a-lie/
[3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200981#comment-14200981
]
Joshua McKenzie commented on CASSANDRA-7970:
Option 2 seems consistent from a JSON ecosystem perspective as well.
JSON support for CQL
Key: CASSANDRA-7970
URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
Project: Cassandra
Issue Type: Bug
Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Fix For: 3.0
JSON is popular enough that not supporting it is becoming a competitive
weakness. We can add JSON support in a way that is compatible with our
performance goals by *mapping* JSON to an existing schema: one JSON documents
maps to one CQL row.
Thus, it is NOT a goal to support schemaless documents, which is a misfeature
[1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
document from a service or a user into a CQL row, with all the validation
that entails.
Since we are not looking to support schemaless documents, we will not be
adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
map the JSON to UDT, collections, and primitive CQL types.
Here's how this might look:
{code}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
);
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses maptext, address
);
INSERT INTO users JSON
{‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
“city”: “Austin”,
“zip_code”: 78747,
“phones”: [2101234567]}}};
SELECT JSON id, address FROM users;
{code}
(We would also want to_json and from_json functions to allow mapping a single
column's worth of data. These would not require extra syntax.)
[1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
[2] https://blog.compose.io/schema-less-is-usually-a-lie/
[3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-8272:
--
Fix Version/s: 2.1.3
Tagging 2.1.3 rather than earlier based on the reasoning that since this has
been around since 2i was added in 0.7 with nobody hitting it in the wild, it
must not be a very common occurrence.
2ndary indexes can return stale data
Key: CASSANDRA-8272
URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Fix For: 2.1.3
When replica return 2ndary index results, it's possible for a single replica
to return a stale result and that result will be sent back to the user,
potentially failing the CL contract.
For instance, consider 3 replicas A, B and C, and the following situation:
{noformat}
CREATE TABLE test (k int PRIMARY KEY, v text);
CREATE INDEX ON test(v);
INSERT INTO test(k, v) VALUES (0, 'foo');
{noformat}
with every replica up to date. Now, suppose that the following queries are
done at {{QUORUM}}:
{noformat}
UPDATE test SET v = 'bar' WHERE k = 0;
SELECT * FROM test WHERE v = 'foo';
{noformat}
then, if A and B acknowledge the insert but C respond to the read before
having applied the insert, then the now stale result will be returned (since
C will return it and A or B will return nothing).
A potential solution would be that when we read a tombstone in the index (and
provided we make the index inherit the gcGrace of it's parent CF), instead of
skipping that tombstone, we'd insert in the result a corresponding range
tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200992#comment-14200992
]
Jonathan Ellis commented on CASSANDRA-7970:
---
I think we should apply Postel's principle here. If we have a mapint, text
and someone gives us {'1': 'foo'} we should turn it into an integer rather than
rejecting it. Especially since, as you point out, there's really no other
option absent transform functions that we're not implementing initially.
JSON support for CQL
Key: CASSANDRA-7970
URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
Project: Cassandra
Issue Type: Bug
Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Fix For: 3.0
JSON is popular enough that not supporting it is becoming a competitive
weakness. We can add JSON support in a way that is compatible with our
performance goals by *mapping* JSON to an existing schema: one JSON documents
maps to one CQL row.
Thus, it is NOT a goal to support schemaless documents, which is a misfeature
[1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
document from a service or a user into a CQL row, with all the validation
that entails.
Since we are not looking to support schemaless documents, we will not be
adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
map the JSON to UDT, collections, and primitive CQL types.
Here's how this might look:
{code}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
);
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses maptext, address
);
INSERT INTO users JSON
{‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
“city”: “Austin”,
“zip_code”: 78747,
“phones”: [2101234567]}}};
SELECT JSON id, address FROM users;
{code}
(We would also want to_json and from_json functions to allow mapping a single
column's worth of data. These would not require extra syntax.)
[1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
[2] https://blog.compose.io/schema-less-is-usually-a-lie/
[3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: (was: cassandra-2.1.1-8221.txt) Specify keyspace in error message when streaming fails due to missing replicas -- Key: CASSANDRA-8221 URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Rajanarayanan Thottuvaikkatumana Priority: Trivial Labels: lhf Fix For: 2.0.12, 2.1.2 When there aren't sufficient live replicas for streaming (during bootstrap, etc), you'll get an error message like unable to find sufficient sources for streaming range. It would be helpful to include the keyspace that this failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: cassandra-2.1.1-8221.txt Attaching the patch for 2.1.1 after making the intendation changes. Thanks Specify keyspace in error message when streaming fails due to missing replicas -- Key: CASSANDRA-8221 URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Rajanarayanan Thottuvaikkatumana Priority: Trivial Labels: lhf Fix For: 2.0.12, 2.1.2 Attachments: cassandra-2.1.1-8221.txt When there aren't sufficient live replicas for streaming (during bootstrap, etc), you'll get an error message like unable to find sufficient sources for streaming range. It would be helpful to include the keyspace that this failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: cassandra-2.0.11-8221.txt Attached the patch for the 2.0.11 branch Specify keyspace in error message when streaming fails due to missing replicas -- Key: CASSANDRA-8221 URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Rajanarayanan Thottuvaikkatumana Priority: Trivial Labels: lhf Fix For: 2.0.12, 2.1.2 Attachments: cassandra-2.0.11-8221.txt, cassandra-2.1.1-8221.txt When there aren't sufficient live replicas for streaming (during bootstrap, etc), you'll get an error message like unable to find sufficient sources for streaming range. It would be helpful to include the keyspace that this failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201067#comment-14201067 ] Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-8221: - Attached patches for both the 2.0 and 2.1 branches. Thanks Specify keyspace in error message when streaming fails due to missing replicas -- Key: CASSANDRA-8221 URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Rajanarayanan Thottuvaikkatumana Priority: Trivial Labels: lhf Fix For: 2.0.12, 2.1.2 Attachments: cassandra-2.0.11-8221.txt, cassandra-2.1.1-8221.txt When there aren't sufficient live replicas for streaming (during bootstrap, etc), you'll get an error message like unable to find sufficient sources for streaming range. It would be helpful to include the keyspace that this failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201074#comment-14201074
]
Henning Kropp commented on CASSANDRA-7083:
--
Looking in current trunk it seems like it has already been fixed.
{{AbstractColumnFamilyOutputFormat}} has the login function and the client is
correctly logged in. Looks good to me.
By the way it seems like your commit: 4b8bb86e272d8ddab62f671d5109e02c32c07728
:)
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Fix For: 2.0.12
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams resolved CASSANDRA-7083.
-
Resolution: Duplicate
Fix Version/s: (was: 2.0.12)
Reproduced In: 2.0.5, 1.2.16 (was: 1.2.16, 2.0.5)
Well that simplifies things :)
Authentication Support for CqlRecordWriter
--
Key: CASSANDRA-7083
URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Henning Kropp
Assignee: Brandon Williams
Labels: authentication, pig
Attachments: auth_cql.patch
The {{CqlRecordWriter}} seems not to support authentication. When the
keyspace in Cassandra is to set to use authentication our Pig job fails with,
when credentials are provided using the URI ('cql://username:password...):
{code}
java.lang.RuntimeException: InvalidRequestException(why:You have not logged
in)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
at
org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: InvalidRequestException(why:You have not logged in)
at
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
at
org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
at
org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
... 7 more
{code}
If not supplied in the URI but as only in the {{JobConf}} the exception is:
{code}
Output Location Validation Failed for: 'cql://...' More info to follow:
InvalidRequestException(why:You have not logged in)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
{code}
Which let to the finding, that authentication is correctly supplied for
{{CqlStorage}} but not for the {{CqlRecordWriter}}.
Maybe it would make sense to put the authentication part into
{{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
username and password in the conf would need to be set from the URI. If so
the {{ConfigHelper}} has all the information to authenticate and already
returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8010) cassandra-stress needs better docs for rate options
[ https://issues.apache.org/jira/browse/CASSANDRA-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-8010: -- Labels: lhf (was: ) cassandra-stress needs better docs for rate options --- Key: CASSANDRA-8010 URL: https://issues.apache.org/jira/browse/CASSANDRA-8010 Project: Cassandra Issue Type: Bug Components: Documentation website, Examples, Tools Reporter: Matt Stump Priority: Minor Labels: lhf It's not obvious how to use the rate option. I wasn't able to figure it out via the source, or from the docs. I kept trying to do -rate= or -threads=. I had to search confluence for usage examples. Need something like this in the docs: -rate threads=900 -rate threads=900 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7973) cqlsh connect error member_descriptor' object is not callable
[
https://issues.apache.org/jira/browse/CASSANDRA-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201179#comment-14201179
]
Michael Shuler commented on CASSANDRA-7973:
---
I wonder if we need to s/python (= 2.5)/python (= 2.7)/ in debian/control and
document python requirement to 2.7+ somewhere for tar installs?
cqlsh connect error member_descriptor' object is not callable
---
Key: CASSANDRA-7973
URL: https://issues.apache.org/jira/browse/CASSANDRA-7973
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.0
Reporter: Digant Modha
Priority: Minor
Labels: cqlsh
When using cqlsh (Cassandra 2.1.0) with ssl, python 2.6.9. I get Connection
error: ('Unable to connect to any servers', {...:
TypeError('member_descriptor' object is not callable,)})
I am able to connect from another machine using python 2.7.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
