date:20110404


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2415:
--

  Component/s: Core
 Priority: Minor  (was: Major)
Fix Version/s: 0.7.5
 Assignee: Brandon Williams

 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 :~$  INFO [GossipStage:3] 2011-03-23 00:38:24,859 StorageService.java
 (line 745) Nodes /10.0.0.2 and /10.0.0.3 have the same token 
 1909554714494251628118265338228798.  /10.0.0.2 is the new owner
  WARN [GossipStage:3] 2011-03-23 00:38:24,859 TokenMetadata.java (line
 115) Token

svn commit: r1088612 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java

2011-04-04 Thread jbellis

Author: jbellis
Date: Mon Apr  4 13:21:28 2011
New Revision: 1088612

URL: http://svn.apache.org/viewvc?rev=1088612view=rev
Log:
add IntegerType to CliUserHelp
patch by Joaquin Casares; reviewed by Pavel Yaskevich for CASSANDRA-2414

Modified:

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java

Modified: 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java?rev=1088612r1=1088611r2=1088612view=diff
==
--- 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliUserHelp.java
 Mon Apr  4 13:21:28 2011
@@ -55,7 +55,7 @@ public class CliUserHelp {
 {{
 put(ColumnFamilyArgument.COLUMN_TYPE, Super or Standard);
 put(ColumnFamilyArgument.COMMENT, Human-readable column family 
description. Any string is acceptable);
-put(ColumnFamilyArgument.COMPARATOR, The class used as a comparator 
when sorting column names.\n  Valid options include: AsciiType, 
BytesType, LexicalUUIDType,\n  LongType, TimeUUIDType, and 
UTF8Type);
+put(ColumnFamilyArgument.COMPARATOR, The class used as a comparator 
when sorting column names.\n  Valid options include: AsciiType, 
BytesType, LexicalUUIDType,\n  LongType, IntegerType, 
TimeUUIDType, and UTF8Type);
 put(ColumnFamilyArgument.SUBCOMPARATOR, Comparator for sorting 
subcolumn names, for Super columns only);
 put(ColumnFamilyArgument.MEMTABLE_OPERATIONS, Flush memtables after 
this many operations (in millions));
 put(ColumnFamilyArgument.MEMTABLE_THROUGHPUT, ... or after this many 
MB have been written);

[jira] [Resolved] (CASSANDRA-2414) IntegerType isn't noted in cli


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2414.
---

Resolution: Fixed

committed

 IntegerType isn't noted in cli
 --

 Key: CASSANDRA-2414
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2414
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 0.7.0
Reporter: Joaquin Casares
Assignee: Joaquin Casares
Priority: Trivial
 Fix For: 0.7.5

 Attachments: 2414.diff


 The cli's help doesn't list all comparator types.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2411) log why a SSTable is being deleted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015402#comment-13015402
 ] 

Jonathan Ellis commented on CASSANDRA-2411:
---

Maybe logging this at info at all is a mistake; it feels like we're leaking too 
much of the implementation out.  Should we change it to .debug as well as 
adding an explanation?

 log why a SSTable is being deleted
 --

 Key: CASSANDRA-2411
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2411
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.7.4
Reporter: Robert Coli
Priority: Minor

 ./src/java/org/apache/cassandra/io/sstable/SSTable.java: 147
 Has logger.info(Deleted  + desc); .
 This combined with the JVM not being able to delete files until a GC has run 
 means that restarting a node usually prompts a flood of log messages like :
 Deleted /mnt/var/cassandra/data/Digg/UserActivity-10658-Data.db
 I believe that I am not the only operator who reads a log upon restart that 
 says I deleted your data files and becomes concerned.
 Now, personally, I have read the code and understand the conditions under 
 which these files are deleted and so no longer get frightened. For new 
 operators and people who may feel less comfortable reading the code, 
 specifying WHY the file has been deleted (Deleted filename because it was 
 obsolete and marked for deletion as a result of compaction.) seems likely to 
 reduce blood pressure and operator stress levels. :)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Attachment: v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt
v1-0002-update-thrift-generated-code.txt
v1-0001-upgrade-thrift-jar.txt

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0001-upgrade-thrift-jar.txt, v1-0002-update-thrift-generated-code.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Attachment: v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt
v1-0002-update-thrift-generated-code.txt
v1-0001-upgrade-thrift-jar.txt

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0001-upgrade-thrift-jar.txt, v1-0002-update-thrift-generated-code.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Attachment: (was: v1-0001-upgrade-thrift-jar.txt)

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Attachment: (was: v1-0002-update-thrift-generated-code.txt)

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Attachment: (was: 
v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt)

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2265) Fix distributed tests after updating Guava to 08

2011-04-04 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015421#comment-13015421
 ] 

Hudson commented on CASSANDRA-2265:
---

Integrated in Cassandra-0.7 #418 (See 
[https://hudson.apache.org/hudson/job/Cassandra-0.7/418/])


 Fix distributed tests after updating Guava to 08
 

 Key: CASSANDRA-2265
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Affects Versions: 0.8
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Priority: Blocker
 Fix For: 0.8

 Attachments: 2265-partial.diff, CASSANDRA-2265.patch


 Distributed tests fail to start cluster:
 {code}
 [junit] java.lang.NoSuchMethodError: 
 com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
 [junit]   at 
 org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
 [junit]   at 
 org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
 [junit]   at 
 org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
 [junit]   at 
 org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
 [junit]   at $Proxy58.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 [junit]   at 
 org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
 [junit]   at $Proxy59.objectExists(Unknown Source)
 [junit]   at 
 org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
 [junit]   at 
 org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
 [junit]   at 
 org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
 [junit]   at 
 org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
 [junit]   at 
 org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
 [junit]   at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Reviewer: tjake  (was: jake)

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2412) Upgrade to thrift 0.6


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Dusbabek updated CASSANDRA-2412:
-

Reviewer: jake

 Upgrade to thrift 0.6
 -

 Key: CASSANDRA-2412
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2412
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Gary Dusbabek
Assignee: Gary Dusbabek
Priority: Trivial
  Labels: thrift
 Fix For: 0.8

 Attachments: v1-0001-upgrade-thrift-jar.txt, 
 v1-0002-update-thrift-generated-code.txt, 
 v1-0003-Adjust-CustomTThreadPoolServer-for-thrift-updates.txt




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets

[
https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015446#comment-13015446
]

Sylvain Lebresne commented on CASSANDRA-2191:
-

Here's some number of remarks on this. I wrote those on a plane so I did not
had access to what was previously said above, and I'm too lazy to edit those
back (and anyway there little intersections and nothing is outdated I think).

Comments:
* An unfortunate consequence of this is that the submission of a compaction
can now fail, in particular cleanup or major ones (more precisely, cleanup
compactions will leave some sstable unclean and major ones won't be major and
thus won't clean all tombstones). It will be a pain for users. Not only should
they check the log once they summit one of those compactions to see what is
left to do. It's also inefficient: for cleanups, you'll have to submit a new
one which will redo a full cleanup compaction (and those are not as efficient
as they could be). For major compaction, it's even worse. Anyway, grabbing the
write lock instead of the read lock for those compaction should be good enough.
* The different compactions weren't wrote to be run in parallel initially
We need a way to deactivate this if something goes wrong. It could
either be a flag that makes everyone acquire the write lock, or an
option to make the compaction executor mono-threaded.
* Related to the point above, I think that the cache writing tasks were fed
to the compaction executor to make sure we never do 2 cache writing at the same
time. We need to restore that property somehow.
* There's a number of TODO in the code. I am not a fan of leaving TODOs
unless there is a really good reason to, there is JIRA tickets for that. In
particular, I do not agree with the fact that flusherLock and compactionLock
should be replaced by a list of sstables (at least this is not as simple as
this is put).
* Let's not use clearUnsafe() to initialize DataTracker given its name (but
I'm fine with renaming it say init() and CFStore.clearUnsafe() calls init()).
* The patch about closing the sstableScanners should be another ticket for
traceability sake.

Some more minor remarks:
* On the JMX reporting: I think it is ok to remove the methods instead of
deprecating(it will be in a major release and there no indication that this is
deprecated for the user anyway). Also, since looking at a list of strings in
jconsole is a bit of a pain, it could be nice to at least expose the active
count, and maybe a sum over all running compactions of bytesCompacted/toCompact
(it would less ugly to expose a MBean for each CompactionInfo, but I'm not sure
how easy/efficient that would be).
* About ACompactionInfo, we use ICompactionInfo for interface, but
AbstractCompationInfo for abstract classes. Let's keep it at that for coherence
of the code base sake.
* The compacting set field in CompactionManager is dead code. So is
MIN_COMPACTION_THRESHOLD in CompactionsSet.
* I don't find the docString for markCompacting very clear (it kinds of
suggest markCompacting it be in a finally block, and it's unclear that this is
this method that does the marking.
* I would prefer adding forwarding methods in CFStore for mark/unmark since
that's what we've done so far (instead of exporting DataTracker).
* In validationCompaction, the try would ideally be the next thing after the
markCompacting().
* I'd prefer moving toString(CompactionInfo) in ICompactionInfo (or
AbstractCompactionInfo). Also, since this is for JMX reporting, adding the
object hashcode will confuse users more than anything else.
* In compactionExecutor, I suppose the HashMap with Object value is just a
way to deal with the absence of an IdentityHashSet. If so, for clarity I would
prefer using Collections.newSetFromMap(new IdentityHashMap()). Or really just a
set should do it as it would make no sense to redefine the CopmactionInfo
equality to be anything else than reference equality.

Multithread across compaction buckets
-

Key: CASSANDRA-2191
URL: https://issues.apache.org/jira/browse/CASSANDRA-2191
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Stu Hood
Assignee: Stu Hood
Priority: Critical
Labels: compaction
Fix For: 0.8

Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt,
0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt,
0003-Expose-multiple-compactions-via-JMX-and-deprecate-sing.txt,
0004-Try-harder-to-close-scanners-in-compaction-close.txt

This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and
reasoning are different enough to open a separate issue.
The problem with compactions currently

[jira] [Commented] (CASSANDRA-2156) Compaction Throttling

[
https://issues.apache.org/jira/browse/CASSANDRA-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015469#comment-13015469
]

Sylvain Lebresne commented on CASSANDRA-2156:
-

I think this will quite a useful patch.

Dividing the total compaction rate by the number of active compaction to
determine each given active compaction rate may be a bit coarse-grained in some
situations, but it's also probably good enough and I'm fine letting that as
further improvement if it happens that it needs to be improved.

Also, we may want ultimately to throttle cleanup compaction too and maybe have
a specific rate for validation compaction. But I'm fine having it as another
ticket.

A few comments:
* A MB is 1024 * 1024 bytes, and a ms is 1000 seconds. I think the definition
of CompactionIterator.THROTTLE_BYTES_PER_MS takes liberties with standard units
:).
* We should really allow 0 for the compaction rate to deactivate throttling
(and that should really throttle() completely), if only because bugs exist.
* To have compaction rate changeable live would be pretty cool and it's super
easy (an AtomicInteger for THROTTLE_BYTES_PER_MS with some jmx call in
CompactionManager to change it should be enough), so let's do it now.
* In theory, there is a risk of division by 0 because targetBytesPerMs can be
0. Granted this is more than unlikely given that the minimum value for THROTTLE
is 1024, but nevertheless, let's be on the safe side.
* In the same idea, excessBytes can be negative. Pretty sure sleep just
assumes that any negative number is 0, but it would be better to actually check
for all those limit case.
* I'd also be in favor of having the logging in changes of targetByteInMS at
debug level. Because there'll be one message each time you start a compaction
and n messages each time the number of active compaction change and we'll print
them even though we doesn't throttle anything, so it will be noise for most
people. Anyway, really no big deal.

Compaction Throttling
-

Key: CASSANDRA-2156
URL: https://issues.apache.org/jira/browse/CASSANDRA-2156
Project: Cassandra
Issue Type: New Feature
Reporter: Stu Hood
Assignee: Stu Hood
Fix For: 0.8

Attachments:
0005-Throttle-total-compaction-to-a-configurable-throughput.txt,
for-0.6-0001-Throttle-compaction-to-a-fixed-throughput.txt,
for-0.6-0002-Make-compaction-throttling-configurable.txt

Compaction is currently relatively bursty: we compact as fast as we can, and
then we wait for the next compaction to be possible (hurry up and wait).
Instead, to properly amortize compaction, you'd like to compact exactly as
fast as you need to to keep the sstable count under control.
For every new level of compaction, you need to increase the rate that you
compact at: a rule of thumb that we're testing on our clusters is to
determine the maximum number of buckets a node can support (aka, if the 15th
bucket holds 750 GB, we're not going to have more than 15 buckets), and then
multiply the flush throughput by the number of buckets to get a minimum
compaction throughput to maintain your sstable count.
Full explanation: for a min compaction threshold of {{T}}, the bucket at
level {{N}} can contain {{SsubN = T^N}} 'units' (unit == memtable's worth of
data on disk). Every time a new unit is added, it has a {{1/SsubN}} chance of
causing the bucket at level N to fill. If the bucket at level N fills, it
causes {{SsubN}} units to be compacted. So, for each active level in your
system you have {{SubN * 1 / SsubN}}, or {{1}} amortized unit to compact any
time a new unit is added.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1034) Remove assumption that Key to Token is one-to-one


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1034:


Attachment: 
0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch

Attaching v2 for my second patch. There was some failure in the unit tests for 
getRestrictedRanges. This fix and improves those test and fix a small bug 
related to the handling of the minimum value for DecoratedKey.

 Remove assumption that Key to Token is one-to-one
 -

 Key: CASSANDRA-1034
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1034
 Project: Cassandra
  Issue Type: Bug
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Generify-AbstractBounds.patch, 
 0001-Make-range-accept-both-Token-and-DecoratedKey.patch, 
 0002-LengthPartitioner.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one.patch, 1034_v1.txt


 get_range_slices assumes that Tokens do not collide and converts a KeyRange 
 to an AbstractBounds. For RandomPartitioner, this assumption isn't safe, and 
 would lead to a very weird heisenberg.
 Converting AbstractBounds to use a DecoratedKey would solve this, because the 
 byte[] key portion of the DecoratedKey can act as a tiebreaker. 
 Alternatively, we could make DecoratedKey extend Token, and then use 
 DecoratedKeys in places where collisions are unacceptable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-786) RPM Packages

2011-04-04 Thread Nick Bailey (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015484#comment-13015484
]

Nick Bailey commented on CASSANDRA-786:
---

Yes, open a new ticket.

First though, it may be that some of the build dependencies are provided by the
epel repository. You should enable that.

RPM Packages

Key: CASSANDRA-786
URL: https://issues.apache.org/jira/browse/CASSANDRA-786
Project: Cassandra
Issue Type: Improvement
Components: Contrib
Reporter: Daniel Lundin
Assignee: Nick Bailey
Priority: Minor
Labels: distribution, packaging, rpm
Fix For: 0.7 beta 2

Attachments: 768-update-spec-for-trunk.diff, 786-adjust-jars.patch,
apache-cassandra.spec, cassandra.spec, cassandra.spec

RPM packages (and debs of course) would be nice,especially now that cassandra
is maturing and gaining more interest.
Lowering the threshold for getting cassandra running and getting started is
also important.
I think the RabbitMQ project has an admirable Download and install
experience, not to mention the rather cute 2 min guarantee. Definitely a
good inspiration.
I've been studying Cloudera's Hadoop packages, which are very nice, and
really appreciate the separate packages for configuration.
This allows easy deployment of node configuration to a cluster.
I'll have a spec file for building RHEL5 / CentOS packages ready for review
and attached here in a bit.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2409) Add INSERT support to CQL

2011-04-04 Thread Eric Evans (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans updated CASSANDRA-2409:
--

Attachment: v1-0001-CASSANDRA-2409-CQL-INSERT-implementation.txt

 Add INSERT support to CQL
 -

 Key: CASSANDRA-2409
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2409
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.8

 Attachments: CASSANDRA-2409.patch, 
 v1-0001-CASSANDRA-2409-CQL-INSERT-implementation.txt


 There are two reasons to support INSERT:
 - It helps new users feel comfortable (everyone's first statement will be to 
 try to INSERT something, we should make that a positive experience instead of 
 slapping them)
 - Even though it is synonymous with update it is still useful in your code to 
 have both to help communicate intent, similar to choosing good variable names
 The only downside is explaining how INSERT isn't a true insert because it 
 doesn't error out if the row already exists -- but we already have to explain 
 that same concept for UPDATE; the cognitive load is extremely minor.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2409) Add INSERT support to CQL

2011-04-04 Thread Eric Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015523#comment-13015523
 ] 

Eric Evans commented on CASSANDRA-2409:
---

Attached is an updated version of the patch that refactors things a bit.  The 
biggest problem here is that we need to raise Thrift exceptions.

Also included is some minimal system tests.

 Add INSERT support to CQL
 -

 Key: CASSANDRA-2409
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2409
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.8

 Attachments: CASSANDRA-2409.patch, 
 v1-0001-CASSANDRA-2409-CQL-INSERT-implementation.txt


 There are two reasons to support INSERT:
 - It helps new users feel comfortable (everyone's first statement will be to 
 try to INSERT something, we should make that a positive experience instead of 
 slapping them)
 - Even though it is synonymous with update it is still useful in your code to 
 have both to help communicate intent, similar to choosing good variable names
 The only downside is explaining how INSERT isn't a true insert because it 
 doesn't error out if the row already exists -- but we already have to explain 
 that same concept for UPDATE; the cognitive load is extremely minor.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2409) Add INSERT support to CQL

2011-04-04 Thread Pavel Yaskevich (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015547#comment-13015547
 ] 

Pavel Yaskevich commented on CASSANDRA-2409:


Can you please explane a part about exceptions, what should be done?

 Add INSERT support to CQL
 -

 Key: CASSANDRA-2409
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2409
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.8

 Attachments: CASSANDRA-2409.patch, 
 v1-0001-CASSANDRA-2409-CQL-INSERT-implementation.txt


 There are two reasons to support INSERT:
 - It helps new users feel comfortable (everyone's first statement will be to 
 try to INSERT something, we should make that a positive experience instead of 
 slapping them)
 - Even though it is synonymous with update it is still useful in your code to 
 have both to help communicate intent, similar to choosing good variable names
 The only downside is explaining how INSERT isn't a true insert because it 
 doesn't error out if the row already exists -- but we already have to explain 
 that same concept for UPDATE; the cognitive load is extremely minor.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2409) Add INSERT support to CQL

2011-04-04 Thread Pavel Yaskevich (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015567#comment-13015567
 ] 

Pavel Yaskevich commented on CASSANDRA-2409:


Oh, yeah, I've seen it, thats why I wrote you to change exception... I 
misunderstood your previous comment and I thought that there are still problems 
with exceptions left...

+1 on your patch.

 Add INSERT support to CQL
 -

 Key: CASSANDRA-2409
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2409
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 0.8

 Attachments: CASSANDRA-2409.patch, 
 v1-0001-CASSANDRA-2409-CQL-INSERT-implementation.txt


 There are two reasons to support INSERT:
 - It helps new users feel comfortable (everyone's first statement will be to 
 try to INSERT something, we should make that a positive experience instead of 
 slapping them)
 - Even though it is synonymous with update it is still useful in your code to 
 have both to help communicate intent, similar to choosing good variable names
 The only downside is explaining how INSERT isn't a true insert because it 
 doesn't error out if the row already exists -- but we already have to explain 
 that same concept for UPDATE; the cognitive load is extremely minor.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2387) Make it possible for pig to understand packed data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-2387:


Attachment: 2387-v6.txt

v6 handles returning a byte[] instead of a BB when there is a BytesType.

 Make it possible for pig to understand packed data
 --

 Key: CASSANDRA-2387
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Jeremy Hanna
  Labels: contrib, hadoop, pig
 Attachments: 2387-1.txt, 2387-v2.txt, 2387-v3.txt, 2387-v4.txt, 
 2387-v5.txt, 2387-v6.txt, loadstorecaster-patch.txt


 Packed values are throwing off pig. This ticket is to make it so pig can 
 interpret packed values. Originally we thought we could just use a 
 loadcaster.  However, the only way we know how we can do it now is to get the 
 schema through thrift and essentially perform the function of the loadcaster 
 in the getNext method.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1088766 - in /cassandra/trunk: src/java/org/apache/cassandra/cql/ test/system/

2011-04-04 Thread eevans

Author: eevans
Date: Mon Apr  4 20:05:19 2011
New Revision: 1088766

URL: http://svn.apache.org/viewvc?rev=1088766view=rev
Log:
CQL INSERT implementation

Patch by Pavel Yaskevich and eevans for CASSANDRA-2409

Modified:
cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g
cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java
cassandra/trunk/src/java/org/apache/cassandra/cql/StatementType.java
cassandra/trunk/src/java/org/apache/cassandra/cql/UpdateStatement.java
cassandra/trunk/test/system/test_cql.py

Modified: cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g?rev=1088766r1=1088765r2=1088766view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g Mon Apr  4 20:05:19 
2011
@@ -31,6 +31,8 @@ options {
 import java.util.Map;
 import java.util.HashMap;
 import java.util.Collections;
+import java.util.List;
+import java.util.ArrayList;
 import org.apache.cassandra.thrift.ConsistencyLevel;
 import org.apache.cassandra.thrift.InvalidRequestException;
 }
@@ -100,6 +102,7 @@ options {
 
 query returns [CQLStatement stmnt]
 : selectStatement   { $stmnt = new CQLStatement(StatementType.SELECT, 
$selectStatement.expr); }
+| insertStatement   { $stmnt = new CQLStatement(StatementType.INSERT, 
$insertStatement.expr); }
 | updateStatement endStmnt { $stmnt = new 
CQLStatement(StatementType.UPDATE, $updateStatement.expr); }
 | batchUpdateStatement { $stmnt = new 
CQLStatement(StatementType.BATCH_UPDATE, $batchUpdateStatement.expr); }
 | useStatement  { $stmnt = new CQLStatement(StatementType.USE, 
$useStatement.keyspace); }
@@ -176,6 +179,36 @@ whereClause returns [WhereClause clause]
 ;
 
 /**
+ * INSERT INTO
+ *CF
+ *(KEY, column, column, ...)
+ * VALUES
+ *(key, value, value, ...)
+ * (USING
+ *CONSISTENCY level)?;
+ *
+ * Consistency level is set to ONE by default
+ */
+insertStatement returns [UpdateStatement expr]
+: {
+  ConsistencyLevel cLevel = ConsistencyLevel.ONE;
+  MapTerm, Term columns = new HashMapTerm, Term();
+
+  ListTerm columnNames  = new ArrayListTerm();
+  ListTerm columnValues = new ArrayListTerm();
+  }
+  K_INSERT K_INTO columnFamily=( IDENT | STRING_LITERAL | INTEGER )
+  '(' K_KEY( ',' column_name=term  { 
columnNames.add($column_name.item); } )+ ')'
+K_VALUES
+  '(' key=term ( ',' column_value=term { 
columnValues.add($column_value.item); })+ ')'
+( K_USING K_CONSISTENCY K_LEVEL { cLevel = 
ConsistencyLevel.valueOf($K_LEVEL.text); } )?
+  endStmnt
+  {
+  return new UpdateStatement($columnFamily.text, cLevel, columnNames, 
columnValues, key);
+  }
+;
+
+/**
  * BEGIN BATCH [USING CONSISTENCY.LVL]
  * UPDATE CF SET name1 = value1 WHERE KEY = keyname1;
  * UPDATE CF SET name2 = value2 WHERE KEY = keyname2;
@@ -350,6 +383,7 @@ K_WHERE:   W H E R E;
 K_AND: A N D;
 K_KEY: K E Y;
 K_COLUMN:  C O L (U M N)?;
+K_INSERT:  I N S E R T;
 K_UPDATE:  U P D A T E;
 K_WITH:W I T H;
 K_ROW: R O W;
@@ -381,6 +415,8 @@ K_INDEX:   I N D E X;
 K_ON:  O N;
 K_DROP:D R O P;
 K_PRIMARY: P R I M A R Y;
+K_INTO:I N T O;
+K_VALUES:  V A L U E S;
 
 // Case-insensitive alpha characters
 fragment A: ('a'|'A');

Modified: cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java?rev=1088766r1=1088765r2=1088766view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java 
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cql/QueryProcessor.java Mon 
Apr  4 20:05:19 2011
@@ -545,7 +545,8 @@ public class QueryProcessor
 
 avroResult.rows = avroRows;
 return avroResult;
-
+
+case INSERT: // insert uses UpdateStatement
 case UPDATE:
 UpdateStatement update = (UpdateStatement)statement.statement;
 batchUpdate(clientState, Collections.singletonList(update), 
update.getConsistencyLevel());

Modified: cassandra/trunk/src/java/org/apache/cassandra/cql/StatementType.java
URL: 
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cql/StatementType.java?rev=1088766r1=1088765r2=1088766view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cql/StatementType.java 
(original)
+++

[jira] [Updated] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-1969:
--

Attachment: 0002-implement-SerializingCache.txt
0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt

 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCache.txt, 0002-implement-SerializingCacheV2.txt, 
 0003-New_Cache_Providers-1969.txt, 0003-add-ICache.isCopying-method.txt, 
 0004-Null-Check-and-duplicate-bb.txt, 0004-TestCase-1969.txt, 
 1969-rollup-and-config.txt, 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015606#comment-13015606
 ] 

Jonathan Ellis commented on CASSANDRA-1969:
---

Yes, using the raw Memory should work.  (You don't need the synchronization 
hack either, if you avoid BB entirely.)

I've rebased again, fixed some tests, and split it back up into two patches. At 
worst, we can commit 01 now and 02 for 0.8.1 or whenever JNA gets a release out.

One other problem: the configuration option (in patch 01) is causing a problem.

{noformat}
ant clean test -Dtest.name=DatabaseDescriptorTestBuildfile
[junit] Testcase: 
testKSMetaDataSerialization(org.apache.cassandra.config.DatabaseDescriptorTest):
  Caused an ERROR
[junit] java.lang.String cannot be cast to org.apache.avro.util.Utf8
[junit] java.lang.ClassCastException: java.lang.String cannot be cast to 
org.apache.avro.util.Utf8
[junit] at org.apache.avro.util.Utf8.compareTo(Utf8.java:27)
[junit] at 
org.apache.avro.generic.GenericData.compare(GenericData.java:523)
[junit] at 
org.apache.avro.specific.SpecificData.compare(SpecificData.java:190)
[junit] at 
org.apache.avro.generic.GenericData.compare(GenericData.java:517)
[junit] at 
org.apache.avro.specific.SpecificData.compare(SpecificData.java:190)
[junit] at 
org.apache.avro.generic.GenericData.compare(GenericData.java:494)
[junit] at 
org.apache.avro.specific.SpecificData.compare(SpecificData.java:190)
[junit] at 
org.apache.avro.generic.GenericData.compare(GenericData.java:508)
[junit] at 
org.apache.avro.specific.SpecificData.compare(SpecificData.java:190)
[junit] at 
org.apache.avro.generic.GenericData.compare(GenericData.java:494)
[junit] at 
org.apache.avro.specific.SpecificData.compare(SpecificData.java:190)
[junit] at 
org.apache.avro.specific.SpecificRecordBase.compareTo(SpecificRecordBase.java:45)
[junit] at 
org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:35)
[junit] at 
org.apache.cassandra.config.DatabaseDescriptorTest.serDe(DatabaseDescriptorTest.java:42)
[junit] at 
org.apache.cassandra.config.DatabaseDescriptorTest.testKSMetaDataSerialization(DatabaseDescriptorTest.java:66)
{noformat}


 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCache.txt, 0002-implement-SerializingCacheV2.txt, 
 0003-New_Cache_Providers-1969.txt, 0003-add-ICache.isCopying-method.txt, 
 0004-Null-Check-and-duplicate-bb.txt, 0004-TestCase-1969.txt, 
 1969-rollup-and-config.txt, 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, 
 JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
 POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2387) Make it possible for pig to understand packed data

2011-04-04 Thread Jeremy Hanna (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-2387:


Attachment: 2387-v7.patch

Fixing the write for BytesType. Also used BBU.getArray because it was doing 
funky array overwrite things otherwise.

 Make it possible for pig to understand packed data
 --

 Key: CASSANDRA-2387
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Jeremy Hanna
  Labels: contrib, hadoop, pig
 Attachments: 2387-1.txt, 2387-v2.txt, 2387-v3.txt, 2387-v4.txt, 
 2387-v5.txt, 2387-v6.txt, 2387-v7.patch, loadstorecaster-patch.txt


 Packed values are throwing off pig. This ticket is to make it so pig can 
 interpret packed values. Originally we thought we could just use a 
 loadcaster.  However, the only way we know how we can do it now is to get the 
 schema through thrift and essentially perform the function of the loadcaster 
 in the getNext method.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-04 Thread Jon Hermes (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Hermes updated CASSANDRA-1969:
--

Attachment: 1969-0001-v2.txt

The test failure was due to CFMetaData:335. 
Also, 0001-introduce...txt didn't apply cleanly due to import rebasing.

Both those changes are in 1969-0001-v2.txt (where the correct patch application 
is 0001-v2, then 0002...).

 Use BB for row cache - To Improve GC performance.
 -

 Key: CASSANDRA-1969
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Linux and Mac
Reporter: Vijay
Assignee: Vijay
Priority: Minor
 Attachments: 0001-Config-1969.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
 0002-implement-SerializingCache.txt, 0002-implement-SerializingCacheV2.txt, 
 0003-New_Cache_Providers-1969.txt, 0003-add-ICache.isCopying-method.txt, 
 0004-Null-Check-and-duplicate-bb.txt, 0004-TestCase-1969.txt, 
 1969-0001-v2.txt, 1969-rollup-and-config.txt, 1969_Cache_SVN_Patch.diff, 
 BB_Cache-1945.png, JMX-Cache-1945.png, Old_Cahce-1945.png, 
 POC-0001-Config-1945.txt, POC-0002-Update_existing-1945.txt, 
 POC-0003-New_Cache_Providers-1945.txt


 Java BB.allocateDirect() will allocate native memory out of the JVM and will 
 help reducing the GC pressure in the JVM with a large Cache.
 From some of the basic tests it shows around 50% improvement than doing a 
 normal Object cache.
 In addition this patch provide the users an option to choose 
 BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2415) Ec2Snitch changing tokens


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015633#comment-13015633
 ] 

Brandon Williams commented on CASSANDRA-2415:
-

What was the token set to in cassandra.yaml?  The snitch can't change the 
token, so this sounds like a normal bootstrap collision; either the token 
specified conflicted, or the bootstrap rules for automatic selection weren't 
adhered to (wait 2 mins between every bootstrap, etc.)

 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 :~$  INFO [GossipStage:3] 2011-03-23 00:38:24,859 StorageService.java
 (line 745) Nodes

[jira] [Updated] (CASSANDRA-1034) Remove assumption that Key to Token is one-to-one


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1034:


Attachment: (was: 
0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch)

 Remove assumption that Key to Token is one-to-one
 -

 Key: CASSANDRA-1034
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1034
 Project: Cassandra
  Issue Type: Bug
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Generify-AbstractBounds.patch, 
 0001-Make-range-accept-both-Token-and-DecoratedKey.patch, 
 0002-LengthPartitioner.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one.patch, 1034_v1.txt


 get_range_slices assumes that Tokens do not collide and converts a KeyRange 
 to an AbstractBounds. For RandomPartitioner, this assumption isn't safe, and 
 would lead to a very weird heisenberg.
 Converting AbstractBounds to use a DecoratedKey would solve this, because the 
 byte[] key portion of the DecoratedKey can act as a tiebreaker. 
 Alternatively, we could make DecoratedKey extend Token, and then use 
 DecoratedKeys in places where collisions are unacceptable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-1034) Remove assumption that Key to Token is one-to-one


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1034:


Attachment: 
0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch

Reattaching v2, previous had a stupid mistake, sorry about that.

 Remove assumption that Key to Token is one-to-one
 -

 Key: CASSANDRA-1034
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1034
 Project: Cassandra
  Issue Type: Bug
Reporter: Stu Hood
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.8

 Attachments: 0001-Generify-AbstractBounds.patch, 
 0001-Make-range-accept-both-Token-and-DecoratedKey.patch, 
 0002-LengthPartitioner.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one-v2.patch, 
 0002-Remove-assumption-that-token-and-keys-are-one-to-one.patch, 1034_v1.txt


 get_range_slices assumes that Tokens do not collide and converts a KeyRange 
 to an AbstractBounds. For RandomPartitioner, this assumption isn't safe, and 
 would lead to a very weird heisenberg.
 Converting AbstractBounds to use a DecoratedKey would solve this, because the 
 byte[] key portion of the DecoratedKey can act as a tiebreaker. 
 Alternatively, we could make DecoratedKey extend Token, and then use 
 DecoratedKeys in places where collisions are unacceptable.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1088800 - in /cassandra/branches/cassandra-0.7: contrib/pig/src/java/org/apache/cassandra/hadoop/pig/ src/java/org/apache/cassandra/db/marshal/ src/java/org/apache/cassandra/utils/

2011-04-04 Thread brandonwilliams

Author: brandonwilliams
Date: Mon Apr  4 21:53:09 2011
New Revision: 1088800

URL: http://svn.apache.org/viewvc?rev=1088800view=rev
Log:
Pig uses schema information to cast to/from native types.
Patch by Jeremy Hanna and brandonwilliams, reviewed by brandonwilliams
for CASSANDRA-2387

Modified:

cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/AbstractType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/AsciiType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/BytesType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/IntegerType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/LexicalUUIDType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/LocalByPartionerType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/LongType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/TimeUUIDType.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/marshal/UTF8Type.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java

cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java

Modified: 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1088800r1=1088799r2=1088800view=diff
==
--- 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 Mon Apr  4 21:53:09 2011
@@ -20,6 +20,9 @@ import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import org.apache.cassandra.config.ConfigurationException;
+import org.apache.cassandra.db.marshal.BytesType;
+import org.apache.cassandra.thrift.*;
 import org.apache.cassandra.utils.FBUtilities;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
@@ -27,9 +30,8 @@ import org.apache.commons.logging.LogFac
 import org.apache.cassandra.db.Column;
 import org.apache.cassandra.db.IColumn;
 import org.apache.cassandra.db.SuperColumn;
+import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.hadoop.*;
-import org.apache.cassandra.thrift.SlicePredicate;
-import org.apache.cassandra.thrift.SliceRange;
 import org.apache.cassandra.avro.Mutation;
 import org.apache.cassandra.avro.Deletion;
 import org.apache.cassandra.avro.ColumnOrSuperColumn;
@@ -44,6 +46,14 @@ import org.apache.pig.backend.executione
 import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit;
 import org.apache.pig.data.*;
 import org.apache.pig.impl.logicalLayer.FrontendException;
+import org.apache.pig.impl.util.UDFContext;
+import org.apache.thrift.TDeserializer;
+import org.apache.thrift.TException;
+import org.apache.thrift.TSerializer;
+import org.apache.thrift.transport.TFramedTransport;
+import org.apache.thrift.transport.TSocket;
+import org.apache.thrift.transport.TTransport;
+import org.apache.thrift.transport.TTransportException;
 
 /**
  * A LoadFunc wrapping ColumnFamilyInputFormat.
@@ -58,6 +68,8 @@ public class CassandraStorage extends Lo
 public final static String PIG_INITIAL_ADDRESS = PIG_INITIAL_ADDRESS;
 public final static String PIG_PARTITIONER = PIG_PARTITIONER;
 
+private static String UDFCONTEXT_SCHEMA_KEY = schema;
+
 private final static ByteBuffer BOUND = ByteBufferUtil.EMPTY_BYTE_BUFFER;
 private static final Log logger = 
LogFactory.getLog(CassandraStorage.class);
 
@@ -72,8 +84,8 @@ public class CassandraStorage extends Lo
 private RecordWriter writer;
 private int limit;
 
-public CassandraStorage() 
-{ 
+public CassandraStorage()
+{
 this(1024);
 }
 
@@ -100,19 +112,20 @@ public class CassandraStorage extends Lo
 if (!reader.nextKeyValue())
 return null;
 
+CfDef cfDef = getCfDef();
 ByteBuffer key = (ByteBuffer)reader.getCurrentKey();
 SortedMapByteBuffer,IColumn cf = 
(SortedMapByteBuffer,IColumn)reader.getCurrentValue();
 assert key != null  cf != null;
 
 // and wrap it in a tuple
-   Tuple tuple = TupleFactory.getInstance().newTuple(2);
+   Tuple tuple = TupleFactory.getInstance().newTuple(2);
 ArrayListTuple columns = new ArrayListTuple();
 tuple.set(0, new

[jira] [Updated] (CASSANDRA-1600) Merge get_indexed_slices with get_range_slices


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-1600:


Attachment: 
0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch

0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch

Attaching v2 based on top of the actual patch for CASSANDRA-1034. It also fixes 
the 2-3 nitpicks of my previous comments.

 Merge get_indexed_slices with get_range_slices
 --

 Key: CASSANDRA-1600
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1600
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Stu Hood
Assignee: Jonathan Ellis
 Fix For: 0.8

 Attachments: 
 0001-Add-optional-FilterClause-to-KeyRange-and-support-do-v2.patch, 
 0001-Add-optional-FilterClause-to-KeyRange-and-support-doin.txt, 
 0002-allow-get_range_slices-to-apply-filter-to-a-sequenti-v2.patch, 
 0002-allow-get_range_slices-to-apply-filter-to-a-sequential.txt


 From a comment on 1157:
 {quote}
 IndexClause only has a start key for get_indexed_slices, but it would seem 
 that the reasoning behind using 'KeyRange' for get_range_slices applies there 
 as well, since if you know the range you care about in the primary index, you 
 don't want to continue scanning until you exhaust 'count' (or the cluster).
 Since it would appear that get_indexed_slices would benefit from a KeyRange, 
 why not smash get_(range|indexed)_slices together, and make IndexClause an 
 optional field on KeyRange?
 {quote}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2387) Make it possible for pig to understand packed data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-2387.
-

   Resolution: Fixed
Fix Version/s: 0.7.5

Committed with a brace format change in BytesType, and a cast to Object added 
in FBU.getInstance to avoid a warning.

 Make it possible for pig to understand packed data
 --

 Key: CASSANDRA-2387
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2387
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Jeremy Hanna
  Labels: contrib, hadoop, pig
 Fix For: 0.7.5

 Attachments: 2387-1.txt, 2387-v2.txt, 2387-v3.txt, 2387-v4.txt, 
 2387-v5.txt, 2387-v6.txt, 2387-v7.patch, loadstorecaster-patch.txt


 Packed values are throwing off pig. This ticket is to make it so pig can 
 interpret packed values. Originally we thought we could just use a 
 loadcaster.  However, the only way we know how we can do it now is to get the 
 schema through thrift and essentially perform the function of the loadcaster 
 in the getNext method.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-2403) Backport AbstractType.compose from trunk


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-2403.
-

Resolution: Fixed

Solved by CASSANDRA-2387

 Backport AbstractType.compose from trunk
 

 Key: CASSANDRA-2403
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2403
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 It was added in CASSANDRA-2262, but is also useful for 0.7.x.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1088805 - /cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java

2011-04-04 Thread slebresne

Author: slebresne
Date: Mon Apr  4 22:15:39 2011
New Revision: 1088805

URL: http://svn.apache.org/viewvc?rev=1088805view=rev
Log:
Uncomment tests that were commented for no apparent reason

Modified:

cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java

Modified: 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java?rev=1088805r1=1088804r2=1088805view=diff
==
--- 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java
 Mon Apr  4 22:15:39 2011
@@ -97,8 +97,8 @@ public abstract class PartitionerTestCas
 @Test
 public void testMidpointWrapping()
 {
-//assertMidpoint(tok(b), tok(a), 16);
-//assertMidpoint(tok(bbb), tok(a), 16);
+assertMidpoint(tok(b), tok(a), 16);
+assertMidpoint(tok(bbb), tok(a), 16);
 }
 
 @Test

svn commit: r1088806 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ test/unit/org/apache/cassandra/dht/

2011-04-04 Thread slebresne

Author: slebresne
Date: Mon Apr  4 22:25:02 2011
New Revision: 1088806

URL: http://svn.apache.org/viewvc?rev=1088806view=rev
Log:
merge from 0.7

Modified:
cassandra/trunk/   (props changed)
cassandra/trunk/contrib/   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java
   (props changed)

cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
   (props changed)
cassandra/trunk/test/unit/org/apache/cassandra/dht/PartitionerTestCase.java

Propchange: cassandra/trunk/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Apr  4 22:25:02 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7:1026516-1087402
+/cassandra/branches/cassandra-0.7:1026516-1087402,1088805
 /cassandra/branches/cassandra-0.7.0:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3:774578-796573

Propchange: cassandra/trunk/contrib/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Apr  4 22:25:02 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009
-/cassandra/branches/cassandra-0.7/contrib:1026516-1087402
+/cassandra/branches/cassandra-0.7/contrib:1026516-1087402,1088805
 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654
 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689
 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Apr  4 22:25:02 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1087402
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1087402,1088805
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Apr  4 22:25:02 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1087402
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1087402,1088805
 
/cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654
 
/cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689
 
/incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198

Propchange: 
cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Mon Apr  4 22:25:02 2011
@@ -1,5 +1,5 @@
 
/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1052356,1052358-1053452,1053454,1053456-1081914,1083000
-/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1087402
+/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516-1087402,1088805

[jira] [Updated] (CASSANDRA-2384) Merge Mutation and CounterMutation thrift structure


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-2384:


Attachment: 0002-Merge-CounterMutation-and-Mutation-v2.patch
0001-Make-thrift-changes-v2.patch

Attaching rebased v2.
Would be nice to get some review on this before the freeze.

 Merge Mutation and CounterMutation thrift structure
 ---

 Key: CASSANDRA-2384
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2384
 Project: Cassandra
  Issue Type: Improvement
  Components: API
Affects Versions: 0.8
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 0.8

 Attachments: 0001-Make-thrift-changes-v2.patch, 
 0001-Make-thrift-changes.patch, 
 0002-Merge-CounterMutation-and-Mutation-v2.patch, 
 0002-Merge-CounterMutation-and-Mutation.patch

   Original Estimate: 2h
  Remaining Estimate: 2h

 Standard and counter mutation are in 2 different structures, which prevents 
 from doing a standard and counter mutation in the same batch_mutate (you have 
 to use batch_add for counters). It's probably a good idea to merge 
 CounterMutation and Mutation (and simply remove batch_add) to lift that 
 limitation.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1088835 - /cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

2011-04-04 Thread brandonwilliams

Author: brandonwilliams
Date: Tue Apr  5 00:34:56 2011
New Revision: 1088835

URL: http://svn.apache.org/viewvc?rev=1088835view=rev
Log:
Namepsace the udf context key to avoid accidental overwrites in the future

Modified:

cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java

Modified: 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL: 
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1088835r1=1088834r2=1088835view=diff
==
--- 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 (original)
+++ 
cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
 Tue Apr  5 00:34:56 2011
@@ -68,7 +68,7 @@ public class CassandraStorage extends Lo
 public final static String PIG_INITIAL_ADDRESS = PIG_INITIAL_ADDRESS;
 public final static String PIG_PARTITIONER = PIG_PARTITIONER;
 
-private static String UDFCONTEXT_SCHEMA_KEY = schema;
+private static String UDFCONTEXT_SCHEMA_KEY = cassandra.schema;
 
 private final static ByteBuffer BOUND = ByteBufferUtil.EMPTY_BYTE_BUFFER;
 private static final Log logger = 
LogFactory.getLog(CassandraStorage.class);

[jira] [Commented] (CASSANDRA-2415) Ec2Snitch changing tokens

2011-04-04 Thread Sasha Dolgy (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015713#comment-13015713
 ] 

Sasha Dolgy commented on CASSANDRA-2415:


a token was not set in the cassandra.yaml initially -- when nodes 3  4 were 
added, the collision errors appear:

cluster_name: 'FOO_Cluster'
initial_token:
auto_bootstrap: true
hinted_handoff_enabled: true
authenticator: org.apache.cassandra.auth.AllowAllAuthenticator
authority: org.apache.cassandra.auth.AllowAllAuthority
partitioner: org.apache.cassandra.dht.RandomPartitioner
data_file_directories:
- /mnt/cassandra/data
commitlog_directory: /mnt/cassandra/commitlog
saved_caches_directory: /mnt/cassandra/saved_caches
commitlog_rotation_threshold_in_mb: 128
commitlog_sync: periodic
commitlog_sync_period_in_ms: 1
seeds:
   - ec2-xx-xx-xx-xx.ap-southeast-1.compute.amazonaws.com
   - ec2-yy-yy-yy-yy.ap-southeast-1.compute.amazonaws.com
disk_access_mode: auto
concurrent_reads: 8
concurrent_writes: 32
sliced_buffer_size_in_kb: 64
storage_port: 7000
rpc_port: 9160
rpc_keepalive: true
thrift_framed_transport_size_in_mb: 15
thrift_max_message_length_in_mb: 16
snapshot_before_compaction: false
binary_memtable_throughput_in_mb: 256
column_index_size_in_kb: 64
in_memory_compaction_limit_in_mb: 64
rpc_timeout_in_ms: 1
endpoint_snitch: org.apache.cassandra.locator.Ec2Snitch
dynamic_snitch: true
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 60
dynamic_snitch_badness_threshold: 0.0
request_scheduler: org.apache.cassandra.scheduler.NoScheduler



 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token

[jira] [Commented] (CASSANDRA-2415) Ec2Snitch changing tokens


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015715#comment-13015715
 ] 

Brandon Williams commented on CASSANDRA-2415:
-

Were they added simultaneously?

 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 :~$  INFO [GossipStage:3] 2011-03-23 00:38:24,859 StorageService.java
 (line 745) Nodes /10.0.0.2 and /10.0.0.3 have the same token 
 1909554714494251628118265338228798.  /10.0.0.2 is the new owner
  WARN [GossipStage:3] 2011-03-23 00:38:24,859 TokenMetadata.java (line
 115) Token 1909554714494251628118265338228798 changing ownership

[jira] [Commented] (CASSANDRA-2415) Ec2Snitch changing tokens

2011-04-04 Thread Sasha Dolgy (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015718#comment-13015718
 ] 

Sasha Dolgy commented on CASSANDRA-2415:


no.  they were added one at a time and only once the previous node had fully 
joined the ring 100% and state in nodetool ring was Normal.  

problem was stopped by changing the endpoint_snitch to SimpleSnitch in the yaml 
on each node, restarting each node in sequence, waiting for it to resume normal 
operations (as per the system.log) before continuing with the next node.  have 
not seen the error since moving away from Ec2Snitch

 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

[jira] [Commented] (CASSANDRA-2415) Ec2Snitch changing tokens


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015719#comment-13015719
 ] 

Brandon Williams commented on CASSANDRA-2415:
-

If you switch back to ec2snitch does the problem return?

 Ec2Snitch changing tokens
 -

 Key: CASSANDRA-2415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2415
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.7.4
 Environment: Amazon EC2 -- 4 nodes
Reporter: Sasha Dolgy
Assignee: Brandon Williams
Priority: Minor
 Fix For: 0.7.5


 A new 4 node 0.7.4 cluster on Amazon EC2
 1.  Brought up the first node without issue with Ec2Snitch configured in the 
 cassandra.yaml.
 2.  Brought up a second node, with the first node defined as the seed.  No 
 visible issues.  
 3.  Brought up node 3 in the same manner.  Receiving errors as shown in the 
 output below.  
 Initially, I -did not- define tokens for the nodes. 
 When node 3 was brought online, I had this problem and manually moved the 
 tokens and did a nodetool move/repair/clean before getting on to node 4.
 The tokens for the 4 nodes:
 0
 1909554714494251628118265338228798
 56713727820156410577229101238628035242
 170141183460469231731687303715884105726
 When the 4th node comes online, with it's token set in the cassandra.yaml 
 (first one i did it for because of the errors I saw with node 3) ... 
 everything goes well at first in joining the ring, etc.then I see the 
 following error in the system.log:
 :~$  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 304) Started hinted handoff for
 endpoint /10.0.0.2
  INFO [HintedHandoff:1] 2011-03-23 00:37:24,298 HintedHandOffManager.java 
 (line 360) Finished hinted handoff of 0 rows
 to endpoint /10.0.0.2
  INFO [GossipStage:2] 2011-03-23 00:37:55,381 StorageService.java (line 702) 
 Node /10.0.0.2 state jump to bootstrap
 ERROR [GossipStage:2] 2011-03-23 00:37:55,381 
 DebuggableThreadPoolExecutor.java (line 103) Error in
 ThreadPoolExecutor java.lang.RuntimeException: Bootstrap Token collision 
 between /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798 
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 ERROR [GossipStage:2] 2011-03-23 00:37:55,382
 AbstractCassandraDaemon.java (line 112) Fatal exception in thread
 Thread[GossipStage:2,5,main]
 java.lang.RuntimeException: Bootstrap Token collision between
 /10.0.0.3 and /10.0.0.2 (token 1909554714494251628118265338228798
 at 
 org.apache.cassandra.locator.TokenMetadata.addBootstrapToken(TokenMetadata.java:143)
 at 
 org.apache.cassandra.service.StorageService.handleStateBootstrap(StorageService.java:706)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:648)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:772)
 at 
 org.apache.cassandra.gms.Gossiper.applyApplicationStateLocally(Gossiper.java:737)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:679)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:60)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 :~$  INFO [GossipStage:3] 2011-03-23 00:38:24,859 StorageService.java
 (line 745) Nodes /10.0.0.2 and /10.0.0.3 have the same token 
 1909554714494251628118265338228798.  /10.0.0.2 is the new owner
  WARN [GossipStage:3] 2011-03-23 00:38:24,859 TokenMetadata.java (line
 115) Token

[jira] [Commented] (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal


[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13015721#comment-13015721
 ] 

Sylvain Lebresne commented on CASSANDRA-2231:
-

The thing is this, if we agree that for actual values, then end-of-component 
(eoc) is always 0, I don't see what could be the use for the start or end of a 
query to have anything after a eoc != 0, since for any comparison of the start 
(resp. end) with an actual value, the comparaison will return as soon as we 
compare that eoc.

Taking your example of start = (100+1):(10+0), when compared to any value 
(x:0):(y:0), the comparison will either return when comparing x to 100, and if 
x==100, will stop when comparing the eoc (returning then that start  value, so 
that query would typically return all value whose first component is strictly 
greater than 100).

 Add CompositeType comparer to the comparers provided in 
 org.apache.cassandra.db.marshal
 ---

 Key: CASSANDRA-2231
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
 Project: Cassandra
  Issue Type: Improvement
  Components: Contrib
Affects Versions: 0.7.3
Reporter: Ed Anuff
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 0.7.5

 Attachments: CompositeType-and-DynamicCompositeType.patch, 
 edanuff-CassandraCompositeType-1e253c4.zip


 CompositeType is a custom comparer that makes it possible to create 
 comparable composite values out of the basic types that Cassandra currently 
 supports, such as Long, UUID, etc.  This is very useful in both the creation 
 of custom inverted indexes using columns in a skinny row, where each column 
 name is a composite value, and also when using Cassandra's built-in secondary 
 index support, where it can be used to encode the values in the columns that 
 Cassandra indexes.  One scenario for the usage of these is documented here: 
 http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
 contribution is attached and has been previously maintained on github here: 
 https://github.com/edanuff/CassandraCompositeType

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal