[jira] [Updated] (CASSANDRA-4349) PFS should give a friendlier error message when a node has not been configured
[ https://issues.apache.org/jira/browse/CASSANDRA-4349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4349: -- Attachment: 4349.txt Patch attached. > PFS should give a friendlier error message when a node has not been configured > -- > > Key: CASSANDRA-4349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4349 > Project: Cassandra > Issue Type: Bug >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.1.2 > > Attachments: 4349.txt > > > see CASSANDRA-4345 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4345. --- Resolution: Duplicate Split into CASSANDRA-4348 and CASSANDRA-4349. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > Labels: datastax_qa > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4348) node should refuse to bootstrap if told to use a token that already exists in the ring
Jonathan Ellis created CASSANDRA-4348: - Summary: node should refuse to bootstrap if told to use a token that already exists in the ring Key: CASSANDRA-4348 URL: https://issues.apache.org/jira/browse/CASSANDRA-4348 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jonathan Ellis Assignee: Brandon Williams Fix For: 1.1.2 See CASSAANDRA-4345 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4349) PFS should give a friendlier error message when a node has not been configured
Jonathan Ellis created CASSANDRA-4349: - Summary: PFS should give a friendlier error message when a node has not been configured Key: CASSANDRA-4349 URL: https://issues.apache.org/jira/browse/CASSANDRA-4349 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.1.2 see CASSANDRA-4345 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296076#comment-13296076 ] Jonathan Ellis commented on CASSANDRA-4345: --- Looks like we have a check for "token exists in the ring" for move, but not for bootstrap. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > Labels: datastax_qa > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3885) Support multiple ranges in SliceQueryFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296050#comment-13296050 ] Vijay commented on CASSANDRA-3885: -- {quote} So I could regenerate the binary messages, but I'm confused on what SerializationsTest is actually testing. {quote} I always thought we have /cassandra/test/data/serialization/x.x if you want to test the older versions. LGTM +1 nit: setStart() method should have a assert which will check if there is only one Slice. kind of affects: {code} // As soon as we'd done our first call, we want to reset the start column if we're paging if (isPaging) ((SliceQueryFilter)initialFilter()).setStart(ByteBufferUtil.EMPTY_BYTE_BUFFER); {code} PS: I was actually doing a parallel work without the prefetch queue and thought of sharing. (https://github.com/Vijay2win/cassandra/commit/31ca6fd9e1bafc1f4d8dfe858929586637bffdef#L3L18) > Support multiple ranges in SliceQueryFilter > --- > > Key: CASSANDRA-3885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3885 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 3885-v2.txt, CASSANDRA-3885.patch, CASSANDRA-3885.patch, > CASSANDRA-3885.patch, CASSANDRA-3885.patch, CASSANDRA-3885.patch > > > This is logically a subtask of CASSANDRA-2710, but Jira doesn't allow > sub-sub-tasks. > We need to support multiple ranges in a SliceQueryFilter, and we want > querying them to be efficient, i.e., one pass through the row to get all of > the ranges, rather than one pass per range. > Supercolumns are irrelevant since the goal is to replace them anyway. Ignore > supercolumn-related code or rip it out, whichever is easier. > This is ONLY dealing with the storage engine part, not the StorageProxy and > Command intra-node messages or the Thrift or CQL client APIs. Thus, a unit > test should be added to ColumnFamilyStoreTest to demonstrate that it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: quell ant runtime warnings while executing target build-test
Updated Branches: refs/heads/trunk 7398e9363 -> 7030d1e23 quell ant runtime warnings while executing target build-test Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7030d1e2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7030d1e2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7030d1e2 Branch: refs/heads/trunk Commit: 7030d1e233aa7bcbef24fb3190e04bb819c3dc7d Parents: 7398e93 Author: Dave Brosius Authored: Fri Jun 15 20:36:53 2012 -0400 Committer: Dave Brosius Committed: Fri Jun 15 20:36:53 2012 -0400 -- build.xml |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7030d1e2/build.xml -- diff --git a/build.xml b/build.xml index 2f005f7..c5778ff 100644 --- a/build.xml +++ b/build.xml @@ -997,7 +997,8 @@ + destdir="${test.classes}" + includeantruntime="false">
[jira] [Commented] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296042#comment-13296042 ] Jonathan Ellis commented on CASSANDRA-4321: --- cassandra-1.1 branch > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTracker.java:560) > at > org.apache.cassandra.db.D
[jira] [Commented] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296040#comment-13296040 ] Al Tobey commented on CASSANDRA-4321: - What SHA / tag should these patches apply against? I've tried trunk, 1.1.1 and 1.1.0 and can't get a clean apply. I'll try a manual merge tomorrow. > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > o
[jira] [Commented] (CASSANDRA-4346) Hive autocreate C* tables/other app instead of CLI create CF - need to restart C* CLI to see new CF's created in existing Keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-4346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295994#comment-13295994 ] Jonathan Ellis commented on CASSANDRA-4346: --- specifically, you can simply issue a describe to force a refresh > Hive autocreate C* tables/other app instead of CLI create CF - need to > restart C* CLI to see new CF's created in existing Keyspaces > --- > > Key: CASSANDRA-4346 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4346 > Project: Cassandra > Issue Type: Bug >Reporter: Alex Liu >Priority: Minor > > Cassandra CliClient class keep a local cache of > private final Map keyspacesMap = new HashMap(); > which is refreshed each time add new keyspace, login, add new cf through the > CLI, any other meta data changes not made by CLI don't refresh the local > cache. > we can add one new command to CLI, refrech the meta data of the keyspace > or remove the local cache, so that any other app changes the meta data will > show in CLI real time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4346) Hive autocreate C* tables/other app instead of CLI create CF - need to restart C* CLI to see new CF's created in existing Keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-4346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-4346. --- Resolution: Duplicate done in CASSANDRA-4052 > Hive autocreate C* tables/other app instead of CLI create CF - need to > restart C* CLI to see new CF's created in existing Keyspaces > --- > > Key: CASSANDRA-4346 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4346 > Project: Cassandra > Issue Type: Bug >Reporter: Alex Liu >Priority: Minor > > Cassandra CliClient class keep a local cache of > private final Map keyspacesMap = new HashMap(); > which is refreshed each time add new keyspace, login, add new cf through the > CLI, any other meta data changes not made by CLI don't refresh the local > cache. > we can add one new command to CLI, refrech the meta data of the keyspace > or remove the local cache, so that any other app changes the meta data will > show in CLI real time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4347) IP change of node requires assassinate to really remove old IP
Karl Mueller created CASSANDRA-4347: --- Summary: IP change of node requires assassinate to really remove old IP Key: CASSANDRA-4347 URL: https://issues.apache.org/jira/browse/CASSANDRA-4347 Project: Cassandra Issue Type: Bug Affects Versions: 1.0.10 Environment: RHEL6, 64bit Reporter: Karl Mueller Priority: Minor In changing the IP addresses of nodes one-by-one, the node successfully moves itself and its token. Everything works properly. However, the node which had its IP changed (but NOT other nodes in the ring) continues to have some type of state associated with the old IP and produces log messages like this: INFO [GossipStage:1] 2012-06-15 15:25:01,490 Gossiper.java (line 838) Node /10.12.9.157 is now part of the cluster INFO [GossipStage:1] 2012-06-15 15:25:01,490 Gossiper.java (line 804) InetAddress /10.12.9.157 is now UP INFO [GossipStage:1] 2012-06-15 15:25:01,491 StorageService.java (line 1017) Nodes /10.12.9.157 and dev-cass01.sv.walmartlabs.com/10.93.15.11 have the same token 113427455640312821154458202477256070484. Ignoring /10.12.9.157 INFO [GossipTasks:1] 2012-06-15 15:25:11,373 Gossiper.java (line 818) InetAddress /10.12.9.157 is now dead. INFO [GossipTasks:1] 2012-06-15 15:25:32,380 Gossiper.java (line 632) FatClient /10.12.9.157 has been silent for 3ms, removing from gossip INFO [GossipStage:1] 2012-06-15 15:26:32,490 Gossiper.java (line 838) Node /10.12.9.157 is now part of the cluster INFO [GossipStage:1] 2012-06-15 15:26:32,491 Gossiper.java (line 804) InetAddress /10.12.9.157 is now UP INFO [GossipStage:1] 2012-06-15 15:26:32,491 StorageService.java (line 1017) Nodes /10.12.9.157 and dev-cass01.sv.walmartlabs.com/10.93.15.11 have the same token 113427455640312821154458202477256070484. Ignoring /10.12.9.157 INFO [GossipTasks:1] 2012-06-15 15:26:42,402 Gossiper.java (line 818) InetAddress /10.12.9.157 is now dead. INFO [GossipTasks:1] 2012-06-15 15:27:03,410 Gossiper.java (line 632) FatClient /10.12.9.157 has been silent for 3ms, removing from gossip INFO [GossipStage:1] 2012-06-15 15:28:04,533 Gossiper.java (line 838) Node /10.12.9.157 is now part of the cluster Other nodes do NOT have the old IP showing up in logs. It's only the node that moved. The old IP doesn't show up in ring anywhere or in any other fashion. The cluster seems to be fully operational, so I think it's just a cleanup issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4346) Hive autocreate C* tables/other app instead of CLI create CF - need to restart C* CLI to see new CF's created in existing Keyspaces
Alex Liu created CASSANDRA-4346: --- Summary: Hive autocreate C* tables/other app instead of CLI create CF - need to restart C* CLI to see new CF's created in existing Keyspaces Key: CASSANDRA-4346 URL: https://issues.apache.org/jira/browse/CASSANDRA-4346 Project: Cassandra Issue Type: Bug Reporter: Alex Liu Priority: Minor Cassandra CliClient class keep a local cache of private final Map keyspacesMap = new HashMap(); which is refreshed each time add new keyspace, login, add new cf through the CLI, any other meta data changes not made by CLI don't refresh the local cache. we can add one new command to CLI, refrech the meta data of the keyspace or remove the local cache, so that any other app changes the meta data will show in CLI real time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of "FAQ" by TylerHobbs
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "FAQ" page has been changed by TylerHobbs: http://wiki.apache.org/cassandra/FAQ?action=diff&rev1=147&rev2=148 Comment: Add a few client-specific details to the #iter_world faq entry == How can I iterate over all the rows in a ColumnFamily? == Simple but slow: Use get_range_slices, start with the empty string, and after each call use the last key read as the start key in the next iteration. + Most clients support an easy way to do this. For example, [[http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get_range|pycassa's get_range()]], and [[http://thobbs.github.com/phpcassa/api/class-phpcassa.ColumnFamily.html#_get_range|phpcassa's get_range()]] return an iterator that fetches the next batch of rows automatically. Hector has an [[https://github.com/zznate/hector-examples/blob/master/src/main/java/com/riptano/cassandra/hector/example/PaginateGetRangeSlices.java|example of how to do this]]. + Better: use HadoopSupport. <>
[jira] [Updated] (CASSANDRA-3337) Create a way to obliterate nodes from the ring
[ https://issues.apache.org/jira/browse/CASSANDRA-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3337: Summary: Create a way to obliterate nodes from the ring (was: Create a 'killtoken' command) > Create a way to obliterate nodes from the ring > -- > > Key: CASSANDRA-3337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3337 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Labels: gossip > Fix For: 1.0.6 > > Attachments: 3337.txt > > > Sometimes you just want a token gone: no re-replication, nothing, just excise > it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295827#comment-13295827 ] Joaquin Casares edited comment on CASSANDRA-4345 at 6/15/12 7:08 PM: - I was able to reproduce the original node getting kicked out as well while auto_bootstrap set to true on the new, clean machine. was (Author: j.casares): I was able to reproduce this error as well. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > Labels: datastax_qa > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For mo
[jira] [Commented] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295863#comment-13295863 ] Brandon Williams commented on CASSANDRA-4345: - If you mean the NPE, that is no surprise. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > Labels: datastax_qa > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295827#comment-13295827 ] Joaquin Casares commented on CASSANDRA-4345: I was able to reproduce this error as well. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > Labels: datastax_qa > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3855) RemoveDeleted dominates compaction time for large sstable counts
[ https://issues.apache.org/jira/browse/CASSANDRA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295818#comment-13295818 ] Sylvain Lebresne commented on CASSANDRA-3855: - I'll precise that I try to do a quick test to see if I could reproduce "back in the days" but wasn't really able to reproduce something similar to the attached hprof log. I didn't wait up to 100,000,000 keys though. > RemoveDeleted dominates compaction time for large sstable counts > > > Key: CASSANDRA-3855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3855 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.0 >Reporter: Stu Hood >Assignee: Yuki Morishita > Labels: compaction, deletes, leveled > Attachments: with-cleaning-java.hprof.txt > > > With very large numbers of sstables (2000+ generated by a `bin/stress -n > 100,000,000` run with LeveledCompactionStrategy), > PrecompactedRow.removeDeletedAndOldShards dominates compaction runtime, such > that commenting it out takes compaction throughput from 200KB/s to 12MB/s. > Stack attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3885) Support multiple ranges in SliceQueryFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3885: Attachment: 3885-v2.txt Patch needed rebase. Rebased version attached. I've also pushed it at https://github.com/pcmanus/cassandra/tree/3885. > Support multiple ranges in SliceQueryFilter > --- > > Key: CASSANDRA-3885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3885 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 3885-v2.txt, CASSANDRA-3885.patch, CASSANDRA-3885.patch, > CASSANDRA-3885.patch, CASSANDRA-3885.patch, CASSANDRA-3885.patch > > > This is logically a subtask of CASSANDRA-2710, but Jira doesn't allow > sub-sub-tasks. > We need to support multiple ranges in a SliceQueryFilter, and we want > querying them to be efficient, i.e., one pass through the row to get all of > the ranges, rather than one pass per range. > Supercolumns are irrelevant since the goal is to replace them anyway. Ignore > supercolumn-related code or rip it out, whichever is easier. > This is ONLY dealing with the storage engine part, not the StorageProxy and > Command intra-node messages or the Thrift or CQL client APIs. Thus, a unit > test should be added to ColumnFamilyStoreTest to demonstrate that it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3885) Support multiple ranges in SliceQueryFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3885: Attachment: (was: 3885-v2.txt) > Support multiple ranges in SliceQueryFilter > --- > > Key: CASSANDRA-3885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3885 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 3885-v2.txt, CASSANDRA-3885.patch, CASSANDRA-3885.patch, > CASSANDRA-3885.patch, CASSANDRA-3885.patch, CASSANDRA-3885.patch > > > This is logically a subtask of CASSANDRA-2710, but Jira doesn't allow > sub-sub-tasks. > We need to support multiple ranges in a SliceQueryFilter, and we want > querying them to be efficient, i.e., one pass through the row to get all of > the ranges, rather than one pass per range. > Supercolumns are irrelevant since the goal is to replace them anyway. Ignore > supercolumn-related code or rip it out, whichever is easier. > This is ONLY dealing with the storage engine part, not the StorageProxy and > Command intra-node messages or the Thrift or CQL client APIs. Thus, a unit > test should be added to ColumnFamilyStoreTest to demonstrate that it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295810#comment-13295810 ] Sylvain Lebresne commented on CASSANDRA-3647: - I've rebased and changed the syntax to match what's above. The new branch is at https://github.com/pcmanus/cassandra/commits/3647-2. To sum up the new syntax: * lists: {noformat} UPDATE L = L + [ 'a' , 'b' ] WHERE ... // Appends to list UPDATE L = [ 'a' , 'b' ] + L WHERE ... // Prepends to list UPDATE L[1] = 'c' WHERE ...// Sets by idx UPDATE L = L - [ 'a', 'b' ] WHERE ... // Remove all occurences of value 'a' and 'b' in list DELETE L[1] WHERE ... // Deletes by idx {noformat} * sets: {noformat} UPDATE S = S + { 'a', 'b' } WHERE ... // Adds to set UPDATE S = S - { 'a', 'b' } WHERE ... // Remove values 'a' and 'b' from set {noformat} * maps: {noformat} UPDATE M['a'] = 'c' WHERE ... // Put key,value UPDATE M = M + { 'a' : 'c' } WHERE ... // Equivalent to previous DELETE M['a'] WHERE ...// Remove value for key 'a' {noformat} A few remarks: * We could rename list -> array. I figured one reason to keep list could be to emphasize that there is no predefined size. But I'm good with array. * There is no support, for maps, of {noformat} UPDATE M = M - { 'a' : 'c' } WHERE ... {noformat} or some other syntax to remove a element of a map by value. The reason is that I don't think we can implement that correctly due to concurrency. > Support set and map value types in CQL > -- > > Key: CASSANDRA-3647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3647 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql > Fix For: 1.2 > > > Composite columns introduce the ability to have arbitrarily nested data in a > Cassandra row. We should expose this through CQL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[4/5] git commit: rename store -> cfs
rename store -> cfs Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55d5c041 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55d5c041 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55d5c041 Branch: refs/heads/trunk Commit: 55d5c041382a3185387def648f6a7c7a76847f75 Parents: a4fab90 Author: Jonathan Ellis Authored: Thu Jun 14 18:40:05 2012 -0500 Committer: Jonathan Ellis Committed: Thu Jun 14 18:40:05 2012 -0500 -- .../cassandra/db/compaction/CompactionsTest.java | 114 +++--- 1 files changed, 57 insertions(+), 57 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/55d5c041/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java -- diff --git a/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java b/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java index 3916669..4f87c86 100644 --- a/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java +++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java @@ -65,13 +65,13 @@ public class CompactionsTest extends SchemaLoader { // this test does enough rows to force multiple block indexes to be used Table table = Table.open(TABLE1); -ColumnFamilyStore store = table.getColumnFamilyStore("Standard1"); +ColumnFamilyStore cfs = table.getColumnFamilyStore("Standard1"); final int ROWS_PER_SSTABLE = 10; final int SSTABLES = DatabaseDescriptor.getIndexInterval() * 3 / ROWS_PER_SSTABLE; // disable compaction while flushing -store.disableAutoCompaction(); +cfs.disableAutoCompaction(); long maxTimestampExpected = Long.MIN_VALUE; Set inserted = new HashSet(); @@ -87,18 +87,18 @@ public class CompactionsTest extends SchemaLoader rm.apply(); inserted.add(key); } -store.forceBlockingFlush(); -assertMaxTimestamp(store, maxTimestampExpected); -assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(store).size()); +cfs.forceBlockingFlush(); +assertMaxTimestamp(cfs, maxTimestampExpected); +assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(cfs).size()); } -forceCompactions(store); +forceCompactions(cfs); -assertEquals(inserted.size(), Util.getRangeSlice(store).size()); +assertEquals(inserted.size(), Util.getRangeSlice(cfs).size()); // make sure max timestamp of compacted sstables is recorded properly after compaction. -assertMaxTimestamp(store, maxTimestampExpected); -store.truncate(); +assertMaxTimestamp(cfs, maxTimestampExpected); +cfs.truncate(); } @@ -106,13 +106,13 @@ public class CompactionsTest extends SchemaLoader public void testSuperColumnCompactions() throws IOException, ExecutionException, InterruptedException { Table table = Table.open(TABLE1); -ColumnFamilyStore store = table.getColumnFamilyStore("Super1"); +ColumnFamilyStore cfs = table.getColumnFamilyStore("Super1"); final int ROWS_PER_SSTABLE = 10; final int SSTABLES = DatabaseDescriptor.getIndexInterval() * 3 / ROWS_PER_SSTABLE; //disable compaction while flushing -store.disableAutoCompaction(); +cfs.disableAutoCompaction(); long maxTimestampExpected = Long.MIN_VALUE; Set inserted = new HashSet(); @@ -131,47 +131,47 @@ public class CompactionsTest extends SchemaLoader rm.apply(); inserted.add(key); } -store.forceBlockingFlush(); -assertMaxTimestamp(store, maxTimestampExpected); -assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(store, superColumn).size()); +cfs.forceBlockingFlush(); +assertMaxTimestamp(cfs, maxTimestampExpected); +assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(cfs, superColumn).size()); } -forceCompactions(store); +forceCompactions(cfs); -assertEquals(inserted.size(), Util.getRangeSlice(store, superColumn).size()); +assertEquals(inserted.size(), Util.getRangeSlice(cfs, superColumn).size()); // make sure max timestamp of compacted sstables is recorded properly after compaction. -assertMaxTimestamp(store, maxTimestampExpected); +assertMaxTimestamp(cfs, maxTimestampExpected); } -public void assertMaxTimestamp(ColumnFamilyStore store, long maxTimestampExpected) +public void assertMaxTimestamp(ColumnFam
[5/5] git commit: rename store -> cfs
rename store -> cfs Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55d5c041 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55d5c041 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55d5c041 Branch: refs/heads/cassandra-1.1 Commit: 55d5c041382a3185387def648f6a7c7a76847f75 Parents: a4fab90 Author: Jonathan Ellis Authored: Thu Jun 14 18:40:05 2012 -0500 Committer: Jonathan Ellis Committed: Thu Jun 14 18:40:05 2012 -0500 -- .../cassandra/db/compaction/CompactionsTest.java | 114 +++--- 1 files changed, 57 insertions(+), 57 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/55d5c041/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java -- diff --git a/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java b/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java index 3916669..4f87c86 100644 --- a/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java +++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java @@ -65,13 +65,13 @@ public class CompactionsTest extends SchemaLoader { // this test does enough rows to force multiple block indexes to be used Table table = Table.open(TABLE1); -ColumnFamilyStore store = table.getColumnFamilyStore("Standard1"); +ColumnFamilyStore cfs = table.getColumnFamilyStore("Standard1"); final int ROWS_PER_SSTABLE = 10; final int SSTABLES = DatabaseDescriptor.getIndexInterval() * 3 / ROWS_PER_SSTABLE; // disable compaction while flushing -store.disableAutoCompaction(); +cfs.disableAutoCompaction(); long maxTimestampExpected = Long.MIN_VALUE; Set inserted = new HashSet(); @@ -87,18 +87,18 @@ public class CompactionsTest extends SchemaLoader rm.apply(); inserted.add(key); } -store.forceBlockingFlush(); -assertMaxTimestamp(store, maxTimestampExpected); -assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(store).size()); +cfs.forceBlockingFlush(); +assertMaxTimestamp(cfs, maxTimestampExpected); +assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(cfs).size()); } -forceCompactions(store); +forceCompactions(cfs); -assertEquals(inserted.size(), Util.getRangeSlice(store).size()); +assertEquals(inserted.size(), Util.getRangeSlice(cfs).size()); // make sure max timestamp of compacted sstables is recorded properly after compaction. -assertMaxTimestamp(store, maxTimestampExpected); -store.truncate(); +assertMaxTimestamp(cfs, maxTimestampExpected); +cfs.truncate(); } @@ -106,13 +106,13 @@ public class CompactionsTest extends SchemaLoader public void testSuperColumnCompactions() throws IOException, ExecutionException, InterruptedException { Table table = Table.open(TABLE1); -ColumnFamilyStore store = table.getColumnFamilyStore("Super1"); +ColumnFamilyStore cfs = table.getColumnFamilyStore("Super1"); final int ROWS_PER_SSTABLE = 10; final int SSTABLES = DatabaseDescriptor.getIndexInterval() * 3 / ROWS_PER_SSTABLE; //disable compaction while flushing -store.disableAutoCompaction(); +cfs.disableAutoCompaction(); long maxTimestampExpected = Long.MIN_VALUE; Set inserted = new HashSet(); @@ -131,47 +131,47 @@ public class CompactionsTest extends SchemaLoader rm.apply(); inserted.add(key); } -store.forceBlockingFlush(); -assertMaxTimestamp(store, maxTimestampExpected); -assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(store, superColumn).size()); +cfs.forceBlockingFlush(); +assertMaxTimestamp(cfs, maxTimestampExpected); +assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(cfs, superColumn).size()); } -forceCompactions(store); +forceCompactions(cfs); -assertEquals(inserted.size(), Util.getRangeSlice(store, superColumn).size()); +assertEquals(inserted.size(), Util.getRangeSlice(cfs, superColumn).size()); // make sure max timestamp of compacted sstables is recorded properly after compaction. -assertMaxTimestamp(store, maxTimestampExpected); +assertMaxTimestamp(cfs, maxTimestampExpected); } -public void assertMaxTimestamp(ColumnFamilyStore store, long maxTimestampExpected) +public void assertMaxTimestamp(C
[1/5] git commit: Merge branch 'cassandra-1.1' into trunk
Updated Branches: refs/heads/cassandra-1.1 a4fab90af -> f74ed12a7 refs/heads/trunk 8cd09 -> 7398e9363 Merge branch 'cassandra-1.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7398e936 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7398e936 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7398e936 Branch: refs/heads/trunk Commit: 7398e9363ae99220d195d30eea3f3479485618f9 Parents: 8cd f74ed12 Author: Jonathan Ellis Authored: Fri Jun 15 12:27:40 2012 -0500 Committer: Jonathan Ellis Committed: Fri Jun 15 12:27:40 2012 -0500 -- CHANGES.txt|1 + .../cassandra/db/compaction/LeveledManifest.java | 60 ++-- .../cassandra/db/compaction/CompactionsTest.java | 114 +++--- 3 files changed, 103 insertions(+), 72 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7398e936/CHANGES.txt -- diff --cc CHANGES.txt index 2eb1b57,693b03b..7064eb0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,28 -1,5 +1,29 @@@ +1.2-dev + * update MS protocol with a version handshake + broadcast address id + (CASSANDRA-4311) + * multithreaded hint replay (CASSANDRA-4189) + * add inter-node message compression (CASSANDRA-3127) + * enforce 1m min keycache for auto (CASSANDRA-4306) + * remove COPP (CASSANDRA-2479) + * Track tombstone expiration and compact when tombstone content is + higher than a configurable threshold, default 20% (CASSANDRA-3442) + * update MurmurHash to version 3 (CASSANDRA-2975) + * (CLI) track elapsed time for `delete' operation (CASSANDRA-4060) + * (CLI) jline version is bumped to 1.0 to properly support + 'delete' key function (CASSANDRA-4132) + * Save IndexSummary into new SSTable 'Summary' component (CASSANDRA-2392) + * Add support for range tombstones (CASSANDRA-3708) + * Improve MessagingService efficiency (CASSANDRA-3617) + * Avoid ID conflicts from concurrent schema changes (CASSANDRA-3794) + * Set thrift HSHA server thread limit to unlimet by default (CASSANDRA-4277) + * Avoids double serialization of CF id in RowMutation messages + (CASSANDRA-4293) + * fix Summary component and caches to use correct partitioner (CASSANDRA-4289) + * stream compressed sstables directly with java nio (CASSANDRA-4297) + + 1.1.2 + * fix bug in sstable blacklisting with LCS (CASSANDRA-4343) * LCS no longer promotes tiny sstables out of L0 (CASSANDRA-4341) * skip tombstones during hint replay (CASSANDRA-4320) * fix NPE in compactionstats (CASSANDRA-4318) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7398e936/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7398e936/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java -- diff --cc test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java index 1476b4a,4f87c86..2b134d1 --- a/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java +++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java @@@ -85,69 -87,20 +85,69 @@@ public class CompactionsTest extends Sc rm.apply(); inserted.add(key); } - store.forceBlockingFlush(); - assertMaxTimestamp(store, maxTimestampExpected); - assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(store).size()); + cfs.forceBlockingFlush(); + assertMaxTimestamp(cfs, maxTimestampExpected); + assertEquals(inserted.toString(), inserted.size(), Util.getRangeSlice(cfs).size()); } - forceCompactions(store); + forceCompactions(cfs); - assertEquals(inserted.size(), Util.getRangeSlice(store).size()); + assertEquals(inserted.size(), Util.getRangeSlice(cfs).size()); // make sure max timestamp of compacted sstables is recorded properly after compaction. - assertMaxTimestamp(store, maxTimestampExpected); - store.truncate(); + assertMaxTimestamp(cfs, maxTimestampExpected); + cfs.truncate(); } +/** + * Test to see if sstable has enough expired columns, it is compacted itself. + */ +@Test +public void testSingleSSTableCompaction() throws Exception +{ +Table table = Table.open(TABLE1); +ColumnFamilyStore store = table.getColumnFamilyStore("Standard1"); +store.clearUnsafe(); +store.metadata.gcGraceSeconds(1); + store.setCompactionStrategyClass(SizeTieredCo
[2/5] git commit: fix bug in sstable blacklisting with LCS patch by jbellis; reviewed by yukim for CASSANDRA-4343
fix bug in sstable blacklisting with LCS patch by jbellis; reviewed by yukim for CASSANDRA-4343 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f74ed12a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f74ed12a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f74ed12a Branch: refs/heads/cassandra-1.1 Commit: f74ed12a70ba0e59efef20a2207ef6bf4df0ae04 Parents: 55d5c04 Author: Jonathan Ellis Authored: Thu Jun 14 20:09:21 2012 -0500 Committer: Jonathan Ellis Committed: Fri Jun 15 12:25:59 2012 -0500 -- CHANGES.txt|1 + .../cassandra/db/compaction/LeveledManifest.java | 60 +++ 2 files changed, 46 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f74ed12a/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index b0e667d..693b03b 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.1.2 + * fix bug in sstable blacklisting with LCS (CASSANDRA-4343) * LCS no longer promotes tiny sstables out of L0 (CASSANDRA-4341) * skip tombstones during hint replay (CASSANDRA-4320) * fix NPE in compactionstats (CASSANDRA-4318) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f74ed12a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java index 4ed5fac..a53d519 100644 --- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java +++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java @@ -277,15 +277,10 @@ public class LeveledManifest if (score > 1.001 || (i == 0 && sstables.size() > 1)) { Collection candidates = getCandidatesFor(i); - if (logger.isDebugEnabled()) logger.debug("Compaction candidates for L{} are {}", i, toString(candidates)); - -// check if have any SSTables marked as suspected, -// saves us filter time when no SSTables are suspects -return hasSuspectSSTables(candidates) -? filterSuspectSSTables(candidates) -: candidates; +if (!candidates.isEmpty()) +return candidates; } } @@ -386,6 +381,10 @@ public class LeveledManifest // 2. At most MAX_COMPACTING_L0 sstables will be compacted at once // 3. If total candidate size is less than maxSSTableSizeInMB, we won't bother compacting with L1, //and the result of the compaction will stay in L0 instead of being promoted (see promote()) +// +// Note that we ignore suspect-ness of L1 sstables here, since if an L1 sstable is suspect we're +// basically screwed, since we expect all or most L0 sstables to overlap with each L1 sstable. +// So if an L1 sstable is suspect we can't do much besides try anyway and hope for the best. Set candidates = new HashSet(); Set remaining = new HashSet(generations[0]); List ageSortedSSTables = new ArrayList(generations[0]); @@ -395,9 +394,14 @@ public class LeveledManifest if (candidates.contains(sstable)) continue; -List newCandidates = overlapping(sstable, remaining); -candidates.addAll(newCandidates); -remaining.removeAll(newCandidates); +for (SSTableReader newCandidate : overlapping(sstable, remaining)) +{ +if (!newCandidate.isMarkedSuspect()) +{ +candidates.add(newCandidate); +remaining.remove(newCandidate); +} +} if (candidates.size() > MAX_COMPACTING_L0) { @@ -421,14 +425,40 @@ public class LeveledManifest // for non-L0 compactions, pick up where we left off last time Collections.sort(generations[level], SSTable.sstableComparator); -for (SSTableReader sstable : generations[level]) +int start = 0; // handles case where the prior compaction touched the very last range +for (int i = 0; i < generations[level].size(); i++) { -// the first sstable that is > than the marked +SSTableReader sstable = generations[level].get(i); if (sstable.first.compareTo(lastCompactedKeys[level]) > 0) -return overlapping(sstable, generations[(le
[3/5] git commit: fix bug in sstable blacklisting with LCS patch by jbellis; reviewed by yukim for CASSANDRA-4343
fix bug in sstable blacklisting with LCS patch by jbellis; reviewed by yukim for CASSANDRA-4343 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f74ed12a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f74ed12a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f74ed12a Branch: refs/heads/trunk Commit: f74ed12a70ba0e59efef20a2207ef6bf4df0ae04 Parents: 55d5c04 Author: Jonathan Ellis Authored: Thu Jun 14 20:09:21 2012 -0500 Committer: Jonathan Ellis Committed: Fri Jun 15 12:25:59 2012 -0500 -- CHANGES.txt|1 + .../cassandra/db/compaction/LeveledManifest.java | 60 +++ 2 files changed, 46 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f74ed12a/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index b0e667d..693b03b 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.1.2 + * fix bug in sstable blacklisting with LCS (CASSANDRA-4343) * LCS no longer promotes tiny sstables out of L0 (CASSANDRA-4341) * skip tombstones during hint replay (CASSANDRA-4320) * fix NPE in compactionstats (CASSANDRA-4318) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f74ed12a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java index 4ed5fac..a53d519 100644 --- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java +++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java @@ -277,15 +277,10 @@ public class LeveledManifest if (score > 1.001 || (i == 0 && sstables.size() > 1)) { Collection candidates = getCandidatesFor(i); - if (logger.isDebugEnabled()) logger.debug("Compaction candidates for L{} are {}", i, toString(candidates)); - -// check if have any SSTables marked as suspected, -// saves us filter time when no SSTables are suspects -return hasSuspectSSTables(candidates) -? filterSuspectSSTables(candidates) -: candidates; +if (!candidates.isEmpty()) +return candidates; } } @@ -386,6 +381,10 @@ public class LeveledManifest // 2. At most MAX_COMPACTING_L0 sstables will be compacted at once // 3. If total candidate size is less than maxSSTableSizeInMB, we won't bother compacting with L1, //and the result of the compaction will stay in L0 instead of being promoted (see promote()) +// +// Note that we ignore suspect-ness of L1 sstables here, since if an L1 sstable is suspect we're +// basically screwed, since we expect all or most L0 sstables to overlap with each L1 sstable. +// So if an L1 sstable is suspect we can't do much besides try anyway and hope for the best. Set candidates = new HashSet(); Set remaining = new HashSet(generations[0]); List ageSortedSSTables = new ArrayList(generations[0]); @@ -395,9 +394,14 @@ public class LeveledManifest if (candidates.contains(sstable)) continue; -List newCandidates = overlapping(sstable, remaining); -candidates.addAll(newCandidates); -remaining.removeAll(newCandidates); +for (SSTableReader newCandidate : overlapping(sstable, remaining)) +{ +if (!newCandidate.isMarkedSuspect()) +{ +candidates.add(newCandidate); +remaining.remove(newCandidate); +} +} if (candidates.size() > MAX_COMPACTING_L0) { @@ -421,14 +425,40 @@ public class LeveledManifest // for non-L0 compactions, pick up where we left off last time Collections.sort(generations[level], SSTable.sstableComparator); -for (SSTableReader sstable : generations[level]) +int start = 0; // handles case where the prior compaction touched the very last range +for (int i = 0; i < generations[level].size(); i++) { -// the first sstable that is > than the marked +SSTableReader sstable = generations[level].get(i); if (sstable.first.compareTo(lastCompactedKeys[level]) > 0) -return overlapping(sstable, generations[(level + 1)
[jira] [Commented] (CASSANDRA-4338) Experiment with direct buffer in SequentialWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295771#comment-13295771 ] Jonathan Ellis commented on CASSANDRA-4338: --- Using direct buffers for RAR and CRAR may also help avoid heap fragmentation. > Experiment with direct buffer in SequentialWriter > - > > Key: CASSANDRA-4338 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4338 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Yuki Morishita >Priority: Minor > Fix For: 1.2 > > > Using a direct buffer instead of a heap-based byte[] should let us avoid a > copy into native memory when we flush the buffer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4343) fix sstable blacklisting for LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295759#comment-13295759 ] Yuki Morishita commented on CASSANDRA-4343: --- +1 (I didn't see any test failure without patch, though.) > fix sstable blacklisting for LCS > > > Key: CASSANDRA-4343 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4343 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.0 >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: compaction > Fix For: 1.1.2 > > Attachments: 4343.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4304) Add bytes-limit clause to queries
[ https://issues.apache.org/jira/browse/CASSANDRA-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295743#comment-13295743 ] Christian Spriegel commented on CASSANDRA-4304: --- Brandon, thank you for your feedback. I also see the need for these operator-limits. But I think they should be implemented in addition to client-specified limits as proposed by me. Here is why: # A operator-limit should throw an exception if too much data is loaded (maybe not an exception but some kind of flag in the result). If the server would silently reduce the amount of results, then the client would not know if there simply is no more data or if it was limited due to size. Think of some client asking for fixed-size batches for some processing - the operator would silently break the application by turning on the size limit. # More important (to me): I have different queries that expect different batch-sizes. Therefore I need the application to be able to control the result size. For example: mobile devices need smaller batches than a backend system that calls our middleware. Is there any reason not to have a client-limit? I agree, that adding another limit parameter does not look nice. In thrift we could reuse the the existing limit parameter and use the negative value range for byte limits :-). In cql/cli a new keyword might be nicer though. ... but I digress. Any thoughts? I dont know if it helps, but I would be willing to contribute. > Add bytes-limit clause to queries > - > > Key: CASSANDRA-4304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4304 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Christian Spriegel > Fix For: 1.2 > > Attachments: TestImplForSlices.patch > > > Idea is to add a second limit clause to (slice)queries. This would allow easy > loading of batches, even if content is variable sized. > Imagine the following use case: > You want to load a batch of XMLs, where each is between 100bytes and 5MB > large. > Currently you can load either > - a large number of XMLs, but risk OOMs or timeouts > or > - a small number of XMLs, and do too many queries where each query usually > retrieves very little data. > With cassandra being able to limit by size and not just count, we could do a > single query which would never OOM but always return a decent amount of data > -- with no extra overhead for multiple queries. > Few thoughts from my side: > - The limit should be a soft limit, not a hard limit. Therefore it will > always return at least one row/column, even if that one large than the limit > specifies. > - HintedHandoffManager:303 is already doing a > InMemoryCompactionLimit/averageColumnSize to avoid OOM. It could then simply > use the new limit clause :-) > - A bytes-limit on a range- or indexed-query should always return a complete > row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295737#comment-13295737 ] Jonathan Ellis commented on CASSANDRA-4345: --- While we could improve the error message, PFS shouldn't just Make Something Up when given an unknown node to replicate to. > New (bootstrapping) Supplant Healthy Nodes > -- > > Key: CASSANDRA-4345 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.9 >Reporter: Benjamin Coverston > > Copied a config from an existing node and fired up a new node, which happily > inserted itself at token 0 of a running ring. The surprising and worrisome > part is that EVERY node started throwing: > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > --- > Resulting in: > INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node > /192.168.88.48 is now part of the cluster > INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) > InetAddress /192.168.88.48 is now UP > INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) > Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 > .48 is the new owner > WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) > Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 > INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line > 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live > bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) > Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) > INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) > Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) > ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) > Internal error processing get_slice > java.lang.NullPointerException > at > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) > at > org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) > at > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) > at > org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) > at > org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) > at > org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) > at > org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) > at > org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) > at > org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) > at > org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) > at > org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) > at > org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) > at > org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4345) New (bootstrapping) Supplant Healthy Nodes
Benjamin Coverston created CASSANDRA-4345: - Summary: New (bootstrapping) Supplant Healthy Nodes Key: CASSANDRA-4345 URL: https://issues.apache.org/jira/browse/CASSANDRA-4345 Project: Cassandra Issue Type: Improvement Affects Versions: 1.0.9 Reporter: Benjamin Coverston Copied a config from an existing node and fired up a new node, which happily inserted itself at token 0 of a running ring. The surprising and worrisome part is that EVERY node started throwing: java.lang.NullPointerException ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,948 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException ERROR [RPC-Thread:205427] 2012-06-14 19:16:31,949 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException ERROR [RPC-Thread:205459] 2012-06-14 19:16:31,952 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException --- Resulting in: INFO [GossipStage:1] 2012-06-14 18:24:37,472 Gossiper.java (line 838) Node /192.168.88.48 is now part of the cluster INFO [GossipStage:1] 2012-06-14 18:24:37,473 Gossiper.java (line 804) InetAddress /192.168.88.48 is now UP INFO [GossipStage:1] 2012-06-14 18:24:37,473 StorageService.java (line 1008) Nodes /192.168.88.48 and /192.168.88.70 have the same token 0. /192.168.88 .48 is the new owner WARN [GossipStage:1] 2012-06-14 18:24:37,474 TokenMetadata.java (line 135) Token 0 changing ownership from /192.168.88.70 to /192.168.88.48 INFO [GossipStage:1] 2012-06-14 18:24:37,475 ColumnFamilyStore.java (line 705) Enqueuing flush of Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) INFO [FlushWriter:1272] 2012-06-14 18:24:37,475 Memtable.java (line 246) Writing Memtable-LocationInfo@961917618(20/25 serialized/live bytes, 1 ops) INFO [FlushWriter:1272] 2012-06-14 18:24:37,492 Memtable.java (line 283) Completed flushing /cass_ssd/system/LocationInfo-hc-23-Data.db (74 bytes) ERROR [RPC-Thread:200943] 2012-06-14 18:24:38,007 Cassandra.java (line 3041) Internal error processing get_slice java.lang.NullPointerException at org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:104) at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:122) at org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:93) at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:100) at org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:2002) at org.apache.cassandra.service.StorageService.getLiveNaturalEndpoints(StorageService.java:1996) at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:604) at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:564) at org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:128) at org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:283) at org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:365) at org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:326) at org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:3033) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631) at org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:105) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4304) Add bytes-limit clause to queries
[ https://issues.apache.org/jira/browse/CASSANDRA-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295722#comment-13295722 ] Brandon Williams commented on CASSANDRA-4304: - I think I do like the idea of limiting by bytes instead of count, as CASSANDRA-3911 does. However, I think that ticket has the right approach in that it should be the operator that defines that limit, not clients, since they will still have the ability to abuse it and OOM the server, and OOM is the operator's problem. > Add bytes-limit clause to queries > - > > Key: CASSANDRA-4304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4304 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Christian Spriegel > Fix For: 1.2 > > Attachments: TestImplForSlices.patch > > > Idea is to add a second limit clause to (slice)queries. This would allow easy > loading of batches, even if content is variable sized. > Imagine the following use case: > You want to load a batch of XMLs, where each is between 100bytes and 5MB > large. > Currently you can load either > - a large number of XMLs, but risk OOMs or timeouts > or > - a small number of XMLs, and do too many queries where each query usually > retrieves very little data. > With cassandra being able to limit by size and not just count, we could do a > single query which would never OOM but always return a decent amount of data > -- with no extra overhead for multiple queries. > Few thoughts from my side: > - The limit should be a soft limit, not a hard limit. Therefore it will > always return at least one row/column, even if that one large than the limit > specifies. > - HintedHandoffManager:303 is already doing a > InMemoryCompactionLimit/averageColumnSize to avoid OOM. It could then simply > use the new limit clause :-) > - A bytes-limit on a range- or indexed-query should always return a complete > row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4321: Attachment: (was: 0002-Scrub-detects-and-repair-outOfOrder-rows.txt) > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTracker.java:560) > at > org.apache.cassandra.db.DataTracke
[jira] [Updated] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4321: Attachment: (was: 0001-Change-Range-Bounds-in-LeveledManifest.overlapping.txt) > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTracker.java:560) > at > org.apache.cassandra.db
[jira] [Updated] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4321: Attachment: 0003-Create-standalone-scrub-v3.txt 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt bq. Tried the patch but the server still doesn't start. Right. So the problem is, as you noticed, that there is really no way to start the server and having it load a broken sstable, which means there is no way to run scrub on it. Even without assertions, we rely on interval trees which breaks if the sstable first key is not before the last one. After having look a bit more closely on that problem, I think the cleaner way to solve this is to provide a way to run scrub "offline", which allows to skip the interval trees. So attaching a 3rd patch that provide that. It adds a new binary 'sstablescrub' that takes as argument a keyspace name and column family name and scrub the relevant sstables, and does this without breaking if the sstable have some out of order keys. I kind of think that having an offline scrub is not a bad idea anyway. With that, you should be able to stop the node, run 'sstablescrub ksname cfname' and then restart the node and you should be good to go. > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.a
[jira] [Updated] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4321: Attachment: (was: 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt) > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v3.txt, > 0003-Create-standalone-scrub-v3.txt, ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTracker.java:560) > at > org.apache.cassandra
[jira] [Updated] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4321: Attachment: (was: 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v2.txt) > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping-v3.txt, > 0001-Change-Range-Bounds-in-LeveledManifest.overlapping.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows.txt, > ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTracker.java:560) > at
[cassandra-jdbc] 5 new revisions pushed by wfs...@gmail.com on 2012-06-15 02:41 GMT
5 new revisions: Revision: f97b06dd798c Author: Rick Shaw Date: Thu Jun 14 18:56:51 2012 Log: Remove the log4j.properties file from src/main/resources... http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=f97b06dd798c Revision: 5a690f77f8e7 Author: Rick Shaw Date: Thu Jun 14 13:49:16 2012 Log: Show more information in error messages from Server http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=5a690f77f8e7 Revision: 4930d2526457 Author: Rick Shaw Date: Thu Jun 14 13:50:55 2012 Log: Add test for Issue #33 http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=4930d2526457 Revision: 320b789c5fb1 Author: Rick Shaw Date: Thu Jun 14 18:57:35 2012 Log: Merge branch 'issue-33' into v1.1.0 http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=320b789c5fb1 Revision: af085cd94571 Author: Rick Shaw Date: Thu Jun 14 19:29:00 2012 Log: Update dependencies in build.xml and pom.xml to version 1.1.1 of C* http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=af085cd94571 == Revision: f97b06dd798c Author: Rick Shaw Date: Thu Jun 14 18:56:51 2012 Log: Remove the log4j.properties file from src/main/resources log4j file was getting put into the jar overriding client usage http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=f97b06dd798c Deleted: /src/main/resources/log4j.properties === --- /src/main/resources/log4j.propertiesMon Nov 21 14:37:51 2011 +++ /dev/null @@ -1,8 +0,0 @@ -# Test Log4J Properties File - -log4j.rootLogger=WARN, stdout -log4j.logger.org.apache.cassandra.cql.jdbc=INFO - -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout -log4j.appender.stdout.layout.ConversionPattern=%-6r %-5p [%-30c{3}] = %m%n == Revision: 5a690f77f8e7 Author: Rick Shaw Date: Thu Jun 14 13:49:16 2012 Log: Show more information in error messages from Server http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=5a690f77f8e7 Modified: /src/main/java/org/apache/cassandra/cql/jdbc/CassandraStatement.java === --- /src/main/java/org/apache/cassandra/cql/jdbc/CassandraStatement.java Wed Dec 21 21:16:09 2011 +++ /src/main/java/org/apache/cassandra/cql/jdbc/CassandraStatement.java Thu Jun 14 13:49:16 2012 @@ -176,7 +176,7 @@ } catch (InvalidRequestException e) { -throw new SQLSyntaxErrorException(e.getWhy()); +throw new SQLSyntaxErrorException(e.getWhy()+"\n'"+sql+"'",e); } catch (UnavailableException e) { @@ -184,7 +184,7 @@ } catch (TimedOutException e) { -throw new SQLTransientConnectionException(e.getMessage()); +throw new SQLTransientConnectionException(e); } catch (SchemaDisagreementException e) { @@ -192,7 +192,7 @@ } catch (TException e) { -throw new SQLNonTransientConnectionException(e.getMessage()); +throw new SQLNonTransientConnectionException(e); } } == Revision: 4930d2526457 Author: Rick Shaw Date: Thu Jun 14 13:50:55 2012 Log: Add test for Issue #33. o also add a bit of tidying up. http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=4930d2526457 Modified: /src/test/java/org/apache/cassandra/cql/jdbc/JdbcRegressionTest.java === --- /src/test/java/org/apache/cassandra/cql/jdbc/JdbcRegressionTest.java Sat Mar 24 18:43:56 2012 +++ /src/test/java/org/apache/cassandra/cql/jdbc/JdbcRegressionTest.java Thu Jun 14 13:50:55 2012 @@ -48,7 +48,10 @@ public static void setUpBeforeClass() throws Exception { Class.forName("org.apache.cassandra.cql.jdbc.CassandraDriver"); -con = DriverManager.getConnection(String.format("jdbc:cassandra://%s:%d/%s",HOST,PORT,"system")); +String URL = String.format("jdbc:cassandra://%s:%d/%s",HOST,PORT,"system"); +System.out.println("Connection URL = '"+URL +"'"); + +con = DriverManager.getConnection(URL); Statement stmt = con.createStatement(); // Drop Keyspace @@ -56,10 +59,12 @@ try { stmt.execute(dropKS);} catch (Exception e){/* Exception on DROP is OK */} - + // Create KeySpace String createKS = String.format("CREATE KEYSPACE %s WITH strategy_class = SimpleStrategy AND strategy_options:replication_factor = 1;",KEYSPACE); +System.out.println("createKS = '"+c
[wiki.cassandra-jdbc] push by wfs...@gmail.com - Fix typo on 2012-06-15 03:10 GMT
Revision: 7b8878b2816f Author: wfshaw Date: Thu Jun 14 20:10:27 2012 Log: Fix typo http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=7b8878b2816f&repo=wiki Modified: /Branch_Versions.wiki === --- /Branch_Versions.wiki Thu Jun 14 20:08:11 2012 +++ /Branch_Versions.wiki Thu Jun 14 20:10:27 2012 @@ -12,6 +12,6 @@ ||*Branch Name* || *Status* || *Description* || ||v1.0.5|| _inactive_|| Older version that follows the Cassandra 1.0.x releases. Only "critical" update will be made || ||v1.1.0|| _active_ || Version that follows the Cassandra 1.1.0 release. (Server side PreparedStatement support added here)| | -||v1.1.1|| _active_ || Version that follows the Cassandra 1.1.0 release. (Current Stable Release) || +||v1.1.1|| _active_ || Version that follows the Cassandra 1.1.1 release. (Current Stable Release) || ||master|| _active_ || Copy of the Current stable release. ("master" is the default branch)|| ||trunk || _active_ || New work that is following Cassandra 1.2 is committed here. Latest development work...||
[wiki.cassandra-jdbc] push by wfs...@gmail.com - Update to show new branches on 2012-06-15 03:08 GMT
Revision: 21bc62dab500 Author: wfshaw Date: Thu Jun 14 20:08:11 2012 Log: Update to show new branches http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/source/detail?r=21bc62dab500&repo=wiki Modified: /Branch_Versions.wiki === --- /Branch_Versions.wiki Sat Mar 24 19:34:56 2012 +++ /Branch_Versions.wiki Thu Jun 14 20:08:11 2012 @@ -5,9 +5,13 @@ {{{cassandra-jdbc}}} manages a number of branches. This wiki page should be kept up to date to document the names and descriptions of each of the branches. +See the "Source/Changes" tab of the branch of interest to see the change history for that branch = Details = ||*Branch Name* || *Status* || *Description* || -||trunk || _active_ || New work that is following Cassandra 1.1.0 is committed here. Latest development work (1.1-dev-SNAPSHOT)|| -||v1.0.5|| _active_ || Current version that follows the Cassandra 1.0.x releases || +||v1.0.5|| _inactive_|| Older version that follows the Cassandra 1.0.x releases. Only "critical" update will be made || +||v1.1.0|| _active_ || Version that follows the Cassandra 1.1.0 release. (Server side PreparedStatement support added here)| | +||v1.1.1|| _active_ || Version that follows the Cassandra 1.1.0 release. (Current Stable Release) || +||master|| _active_ || Copy of the Current stable release. ("master" is the default branch)|| +||trunk || _active_ || New work that is following Cassandra 1.2 is committed here. Latest development work...||
[jira] [Commented] (CASSANDRA-3885) Support multiple ranges in SliceQueryFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295647#comment-13295647 ] David Alves commented on CASSANDRA-3885: thanks for reviewing and extending the patch to the rest of the system Sylvain. PS: wrt to code style I've made changes to the imports to meet code style, but I was already using tjake's cassandra eclipse profile... > Support multiple ranges in SliceQueryFilter > --- > > Key: CASSANDRA-3885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3885 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 3885-v2.txt, CASSANDRA-3885.patch, CASSANDRA-3885.patch, > CASSANDRA-3885.patch, CASSANDRA-3885.patch, CASSANDRA-3885.patch > > > This is logically a subtask of CASSANDRA-2710, but Jira doesn't allow > sub-sub-tasks. > We need to support multiple ranges in a SliceQueryFilter, and we want > querying them to be efficient, i.e., one pass through the row to get all of > the ranges, rather than one pass per range. > Supercolumns are irrelevant since the goal is to replace them anyway. Ignore > supercolumn-related code or rip it out, whichever is easier. > This is ONLY dealing with the storage engine part, not the StorageProxy and > Command intra-node messages or the Thrift or CQL client APIs. Thus, a unit > test should be added to ColumnFamilyStoreTest to demonstrate that it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4344) Windows tools don't work and litter the environment
[ https://issues.apache.org/jira/browse/CASSANDRA-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holger Hoffstätte updated CASSANDRA-4344: - Attachment: stress.patch sstablemeta.patch Trivial fixes for tools to consistently find the right classes and not litter the environment. > Windows tools don't work and litter the environment > --- > > Key: CASSANDRA-4344 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4344 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 1.1.1 > Environment: any Windows, any JDK >Reporter: Holger Hoffstätte > Attachments: sstablemeta.patch, stress.patch > > > On Windows the tools either don't work at all (cassandra-stress) and/or > litter the shell environment (cassandra-stress & sstablemetadata) by > repeatedly appending the same information to variables, eventually running > out of space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4344) Windows tools don't work and litter the environment
Holger Hoffstätte created CASSANDRA-4344: Summary: Windows tools don't work and litter the environment Key: CASSANDRA-4344 URL: https://issues.apache.org/jira/browse/CASSANDRA-4344 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.1.1 Environment: any Windows, any JDK Reporter: Holger Hoffstätte On Windows the tools either don't work at all (cassandra-stress) and litter the shell environment (cassandra-stress & sstablemetadata) by repeatedly appending the same information to variables, eventually running out of space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4344) Windows tools don't work and litter the environment
[ https://issues.apache.org/jira/browse/CASSANDRA-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holger Hoffstätte updated CASSANDRA-4344: - Description: On Windows the tools either don't work at all (cassandra-stress) and/or litter the shell environment (cassandra-stress & sstablemetadata) by repeatedly appending the same information to variables, eventually running out of space. was: On Windows the tools either don't work at all (cassandra-stress) and litter the shell environment (cassandra-stress & sstablemetadata) by repeatedly appending the same information to variables, eventually running out of space. > Windows tools don't work and litter the environment > --- > > Key: CASSANDRA-4344 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4344 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 1.1.1 > Environment: any Windows, any JDK >Reporter: Holger Hoffstätte > > On Windows the tools either don't work at all (cassandra-stress) and/or > litter the shell environment (cassandra-stress & sstablemetadata) by > repeatedly appending the same information to variables, eventually running > out of space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3885) Support multiple ranges in SliceQueryFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-3885: Attachment: 3885-v2.txt Ok, I like the last patch much more and I've convinced myself that this prefetched thing is a good idea with multiple slices. I believe this can be simpler a tiny bit further though. Typically the {{readBackward}} business is a tad confusing and since only SimpleBlockFetcher really cares about it, it can be moved there. I also think the switch of slice/lookup of index block can be cleaned by doing both together. I'm attaching a v2 that does those things, a few other cleanups, adds a bunch of comments and also adds two optimizations: * Since we always know the order on which we read columns, we can remember when we've "entered" a slice to avoid a bunch of comparisons until we "leave" it. * When we switch to the next slice, we must reuse the binary search to locate where that new slice starts rather than going to the next block, otherwise we may end up reading lots of unnecessary blocks. I'll note that imho, with those optimizations, SimpleSliceReader (not to be confused with SimpleBlockFetcher) isn't really useful anymore, but we probably want to make sure by benchmarking it. Anyway, outside of IndexedSliceReader, there was a few problems (that are all solved by the attached v2): * We needed to deal with multiple slices for the memtable iterator too, otherwise we'll end up returning the wrong columns. * We needed to correctly serialize multiple slices for the inter-node protocol. I'll note that with this patch, db.SerializationsTest is broken, but that is no mystery: it's trying to read old messages using the current protocol version. So I could regenerate the binary messages, but I'm confused on what SerializationsTest is actually testing. I though it was making sure we don't break backward compatibility. If we regenerate the binary message at each release we're not testing that at all. ps: @david, it seems your editor is splitting lines where it shouldn't and is reordering imports in a way that don't respect http://wiki.apache.org/cassandra/CodeStyle. > Support multiple ranges in SliceQueryFilter > --- > > Key: CASSANDRA-3885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3885 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 3885-v2.txt, CASSANDRA-3885.patch, CASSANDRA-3885.patch, > CASSANDRA-3885.patch, CASSANDRA-3885.patch, CASSANDRA-3885.patch > > > This is logically a subtask of CASSANDRA-2710, but Jira doesn't allow > sub-sub-tasks. > We need to support multiple ranges in a SliceQueryFilter, and we want > querying them to be efficient, i.e., one pass through the row to get all of > the ranges, rather than one pass per range. > Supercolumns are irrelevant since the goal is to replace them anyway. Ignore > supercolumn-related code or rip it out, whichever is easier. > This is ONLY dealing with the storage engine part, not the StorageProxy and > Command intra-node messages or the Thrift or CQL client APIs. Thus, a unit > test should be added to ColumnFamilyStoreTest to demonstrate that it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira