date:20130528


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668190#comment-13668190
 ] 

Marcus Eriksson commented on CASSANDRA-5383:


[~enigmacurry] - could you test this now that CASSANDRA-5388 is fixed?

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:507)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testRemoveUnifinishedCompactionLeftovers(ColumnFamilyStoreTest.java:1246)
 [junit] Caused by: java.io.IOException:

git commit: Allow preparing timestamp, ttl and limit in queries

2013-05-28 Thread slebresne

Updated Branches:
  refs/heads/trunk bc3597d35 - 524261f88


Allow preparing timestamp, ttl and limit in queries

patch by slebresne; reviewed by iamaleksey for CASSANDRA-4450


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/524261f8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/524261f8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/524261f8

Branch: refs/heads/trunk
Commit: 524261f88cd2adcd623de3604e735b282dd5caac
Parents: bc3597d
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Mon Apr 29 09:27:36 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Tue May 28 11:09:16 2013 +0200

--
 CHANGES.txt|1 +
 src/java/org/apache/cassandra/cql3/Attributes.java |  110 ++-
 src/java/org/apache/cassandra/cql3/Cql.g   |   35 +++--
 .../cassandra/cql3/statements/BatchStatement.java  |   22 ++--
 .../cassandra/cql3/statements/DeleteStatement.java |4 +-
 .../cql3/statements/ModificationStatement.java |   28 ++--
 .../cassandra/cql3/statements/SelectStatement.java |  103 +-
 .../cassandra/cql3/statements/UpdateStatement.java |8 +-
 .../cassandra/transport/messages/BatchMessage.java |2 +-
 9 files changed, 225 insertions(+), 88 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/524261f8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e233ba0..e630d23 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -53,6 +53,7 @@
  * Track max/min column names in sstables to be able to optimize slice
queries (CASSANDRA-5514)
  * Binary protocol: allow batching already prepared statements (CASSANDRA-4693)
+ * Allow preparing timestamp, ttl and limit in CQL3 queries (CASSANDRA-4450)
 
 1.2.6
  * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/524261f8/src/java/org/apache/cassandra/cql3/Attributes.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Attributes.java 
b/src/java/org/apache/cassandra/cql3/Attributes.java
index 62f98b2..511f34e 100644
--- a/src/java/org/apache/cassandra/cql3/Attributes.java
+++ b/src/java/org/apache/cassandra/cql3/Attributes.java
@@ -17,7 +17,13 @@
  */
 package org.apache.cassandra.cql3;
 
+import java.nio.ByteBuffer;
+import java.util.List;
+
 import org.apache.cassandra.db.ExpiringColumn;
+import org.apache.cassandra.db.marshal.Int32Type;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.MarshalException;
 import org.apache.cassandra.exceptions.InvalidRequestException;
 
 /**
@@ -26,15 +32,107 @@ import 
org.apache.cassandra.exceptions.InvalidRequestException;
  */
 public class Attributes
 {
-public Long timestamp;
-public int timeToLive;
+private final Term timestamp;
+private final Term timeToLive;
+
+public static Attributes none()
+{
+return new Attributes(null, null);
+}
+
+private Attributes(Term timestamp, Term timeToLive)
+{
+this.timestamp = timestamp;
+this.timeToLive = timeToLive;
+}
+
+public boolean isTimestampSet()
+{
+return timestamp != null;
+}
+
+public boolean isTimeToLiveSet()
+{
+return timeToLive != null;
+}
+
+public long getTimestamp(long now, ListByteBuffer variables) throws 
InvalidRequestException
+{
+if (timestamp == null)
+return now;
+
+ByteBuffer tval = timestamp.bindAndGet(variables);
+if (tval == null)
+throw new InvalidRequestException(Invalid null value of 
timestamp);
 
-public void validate() throws InvalidRequestException
+try
+{
+LongType.instance.validate(tval);
+}
+catch (MarshalException e)
+{
+throw new InvalidRequestException(Invalid timestamp value);
+}
+
+return LongType.instance.compose(tval);
+}
+
+public int getTimeToLive(ListByteBuffer variables) throws 
InvalidRequestException
 {
-if (timeToLive  0)
+if (timeToLive == null)
+return 0;
+
+ByteBuffer tval = timeToLive.bindAndGet(variables);
+if (tval == null)
+throw new InvalidRequestException(Invalid null value of TTL);
+
+try
+{
+Int32Type.instance.validate(tval);
+}
+catch (MarshalException e)
+{
+throw new InvalidRequestException(Invalid timestamp value);
+}
+
+int ttl = Int32Type.instance.compose(tval);
+if (ttl  0)
 throw new InvalidRequestException(A TTL must be greater or

[jira] [Updated] (CASSANDRA-5595) Add counters to TRACE for how much CASSANDRA-5514 helped a query


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-5595:
---

Attachment: 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch

both adds tracing and rewrites Tracing.trace to use varargs

 Add counters to TRACE for how much CASSANDRA-5514 helped a query
 

 Key: CASSANDRA-5595
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5595
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Jeremiah Jordan
Assignee: Marcus Eriksson
Priority: Minor
 Attachments: 
 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch


 It might be an interesting stat to keep track of how many sstables got 
 skipped by the comparisons added in CASSANDRA-5514.  Could be exposed in 
 TRACE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5149) Respect slice count even if column expire mid-request

2013-05-28 Thread Sylvain Lebresne (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-5149:


Fix Version/s: (was: 2.1)
   2.0

 Respect slice count even if column expire mid-request
 -

 Key: CASSANDRA-5149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5149
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Sylvain Lebresne
 Fix For: 2.0


 This is a follow-up of CASSANDRA-5099.
 If a column expire just while a slice query is performed, it is possible for 
 replicas to count said column as live but to have the coordinator seeing it 
 as dead when building the final result. The effect that the query might 
 return strictly less columns that the requested slice count even though there 
 is some live columns matching the slice predicate but not returned in the 
 result.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5149) Respect slice count even if column expire mid-request

2013-05-28 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668222#comment-13668222
 ] 

Sylvain Lebresne commented on CASSANDRA-5149:
-

Actually, now that I think about it, I think CASSANDRA-4415 is why I'd really 
rather have this in 2.0.

Currently, because of this, when you page a slice query, you cannot trust a 
given page to return strictly results than you've asked only if paging is done, 
because you could have result expiring mid-request and thus you pretty much can 
never know if the paging is really done or if some columns expired on you. 
CASSANDRA-5099 solves this by waiting until basically a query return an empty 
page. However:
# this is really correct. In theory, you could have *all* of the columns fetch 
by the current patch that have expired mid-request, while there's still some 
live columns that match what your are trying to page. Granted, with a large 
enough page size it's very unlikely but still.
# this means you'll *always* do one more query (and that's StorageProxy level 
queries, it's not cheap than would be needed if this ticket was fixed. And 
while having paged get_count being slow don't really make me shed tears, it 
bugs me quite a bit more in the context of CASSANDRA-4415.
# this complicate reasoning about the logic for CASSANDRA-4415 imo. It's much 
easier not to have to care about oh, what if a column expires mid-request, is 
that ok?.

Besides, I don't think fixing this is very complicated in practice. All we need 
is ship a 'queryServerTimestamp' with the read commands, and carry that down to 
the Column.isMarkedForDelete() method so it uses that instead of 
System.currentTimeMillis(). This might end up being a few lines of code to pass 
this timestamp down as parameter, but it's pretty trivial changes.

 Respect slice count even if column expire mid-request
 -

 Key: CASSANDRA-5149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5149
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Sylvain Lebresne
 Fix For: 2.0


 This is a follow-up of CASSANDRA-5099.
 If a column expire just while a slice query is performed, it is possible for 
 replicas to count said column as live but to have the coordinator seeing it 
 as dead when building the final result. The effect that the query might 
 return strictly less columns that the requested slice count even though there 
 is some live columns matching the slice predicate but not returned in the 
 result.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-4415) Add cursor API/auto paging to the native CQL protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reassigned CASSANDRA-4415:


Assignee: Sylvain Lebresne  (was: Aleksey Yeschenko)

Sorry. Misclicked (:

 Add cursor API/auto paging to the native CQL protocol
 -

 Key: CASSANDRA-4415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4415
 Project: Cassandra
  Issue Type: New Feature
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
  Labels: cql, protocol
 Fix For: 2.0


 The goal here would be to use a query paging mechanism to the CQL native 
 protocol. Typically the client/server with that would look something like 
 this:
 {noformat}
 C sends query to S.
 S sends N first rows matching the query + flag saying the response is not 
 complete
 C requests the next N rows
 S sends N next rows + flag saying whether there is more
 C requests the next N rows
 ...
 S sends last rows + flag saying there is no more result
 {noformat}
 The clear goal is for user to not have to worry about limiting queries and 
 doing manual paging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-4415) Add cursor API/auto paging to the native CQL protocol


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reassigned CASSANDRA-4415:


Assignee: Aleksey Yeschenko  (was: Sylvain Lebresne)

 Add cursor API/auto paging to the native CQL protocol
 -

 Key: CASSANDRA-4415
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4415
 Project: Cassandra
  Issue Type: New Feature
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Aleksey Yeschenko
  Labels: cql, protocol
 Fix For: 2.0


 The goal here would be to use a query paging mechanism to the CQL native 
 protocol. Typically the client/server with that would look something like 
 this:
 {noformat}
 C sends query to S.
 S sends N first rows matching the query + flag saying the response is not 
 complete
 C requests the next N rows
 S sends N next rows + flag saying whether there is more
 C requests the next N rows
 ...
 S sends last rows + flag saying there is no more result
 {noformat}
 The clear goal is for user to not have to worry about limiting queries and 
 doing manual paging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-5149) Respect slice count even if column expire mid-request


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reassigned CASSANDRA-5149:


Assignee: Aleksey Yeschenko

 Respect slice count even if column expire mid-request
 -

 Key: CASSANDRA-5149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5149
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Sylvain Lebresne
Assignee: Aleksey Yeschenko
 Fix For: 2.0


 This is a follow-up of CASSANDRA-5099.
 If a column expire just while a slice query is performed, it is possible for 
 replicas to count said column as live but to have the coordinator seeing it 
 as dead when building the final result. The effect that the query might 
 return strictly less columns that the requested slice count even though there 
 is some live columns matching the slice predicate but not returned in the 
 result.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5595) Add counters to TRACE for how much CASSANDRA-5514 helped a query


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668369#comment-13668369
 ] 

Jonathan Ellis commented on CASSANDRA-5595:
---

We actually don't want to use varargs in Tracing since it forces array 
allocation even if tracing is off.

 Add counters to TRACE for how much CASSANDRA-5514 helped a query
 

 Key: CASSANDRA-5595
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5595
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Jeremiah Jordan
Assignee: Marcus Eriksson
Priority: Minor
 Attachments: 
 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch


 It might be an interesting stat to keep track of how many sstables got 
 skipped by the comparisons added in CASSANDRA-5514.  Could be exposed in 
 TRACE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5595) Add counters to TRACE for how much CASSANDRA-5514 helped a query


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668375#comment-13668375
 ] 

Marcus Eriksson commented on CASSANDRA-5595:


hehe figured there was a reason

i'll fix

 Add counters to TRACE for how much CASSANDRA-5514 helped a query
 

 Key: CASSANDRA-5595
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5595
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Jeremiah Jordan
Assignee: Marcus Eriksson
Priority: Minor
 Attachments: 
 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch


 It might be an interesting stat to keep track of how many sstables got 
 skipped by the comparisons added in CASSANDRA-5514.  Could be exposed in 
 TRACE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5595) Add counters to TRACE for how much CASSANDRA-5514 helped a query


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-5595:
---

Attachment: 0001-trace-how-many-sstables-were-skipped-during-slice-du.patch

no more varargs, only trace

 Add counters to TRACE for how much CASSANDRA-5514 helped a query
 

 Key: CASSANDRA-5595
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5595
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Jeremiah Jordan
Assignee: Marcus Eriksson
Priority: Minor
 Attachments: 
 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch, 
 0001-trace-how-many-sstables-were-skipped-during-slice-du.patch


 It might be an interesting stat to keep track of how many sstables got 
 skipped by the comparisons added in CASSANDRA-5514.  Could be exposed in 
 TRACE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5383:


Tester: enigmacurry

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:507)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testRemoveUnifinishedCompactionLeftovers(ColumnFamilyStoreTest.java:1246)
 [junit] Caused by: java.io.IOException: Failed to delete

[jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668418#comment-13668418
 ] 

Alex Liu commented on CASSANDRA-5544:
-

To get the splits for the node, call thrift API 
client.describe_splits_ex(cfName, range.start_token, range.end_token, 
splitsize) it returns the split for that node.

where range.start_token and range.end_token is the start and end token of the 
node, and splitsize is 64 *1024

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: Screen Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668421#comment-13668421
 ] 

Ryan McGuire commented on CASSANDRA-5383:
-

[~krummas] I can't get this patch to apply on the current trunk (It did back in 
March when I first tried this.)

It does apply to cassandra-1.2, but it doesn't work there:

{code}
$ ant clean test -Dtest.name=ColumnFamilyStoreTest
[...]

[junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
[junit] Tests run: 27, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
9.162 sec
[junit]
[junit] Testcase: 
testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
Caused a
n ERROR
[junit] Failed to rename 
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ic-8-Index.db 
to build\te
st\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ic-9-Index.db
[junit] java.lang.RuntimeException: Failed to rename 
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standar
d1-ic-8-Index.db to 
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ic-9-Index.db
[junit] at 
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:155)
[junit] at 
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:139)
[junit] at 
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:409)
[junit] at 
org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:504)
[junit] at 
org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTes
t.java:925)
[junit] Caused by: java.nio.file.FileSystemException: 
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standa
rd1-ic-8-Index.db - 
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ic-9-Index.db:
 The process cannot
 access the file because it is being used by another process.
[junit]
[junit] at 
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
[junit] at 
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[junit] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
[junit] at 
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:286)
[junit] at java.nio.file.Files.move(Files.java:1345)
[junit] at 
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:169)
[junit] at 
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:151)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.ColumnFamilyStoreTest FAILED

{code}

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at

[jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668449#comment-13668449
 ] 

Alex Liu commented on CASSANDRA-5544:
-

[~shamim] I think you already found the answer, SET pig.noSplitCombination 
true, so Pig doesn't combine the small splits into one mapper. HBase internal 
code does it as well. I found that C*-1.2.1 update Pig from 0.9.0 version to 
0.10.0 version which may cause the behavior changes.

As far as number 4) and number 5) concerns, I think the empty maps/big maps are 
due to data skewness. If you can first print out the splits, then you can check 
the rows for each split.

I will add the following code to CassandraStorage.java

job.getConfiguration().setBoolean(pig.noSplitCombination, true);

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: Screen Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-5544:


Attachment: 5544.txt

I attached the patch.

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: 5544.txt, Screen Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5551) intersects the bounds not right

2013-05-28 Thread Robert Coli (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668467#comment-13668467
 ] 

Robert Coli commented on CASSANDRA-5551:


What impact does/did this bug have, on what code paths?

 intersects the bounds not right
 ---

 Key: CASSANDRA-5551
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5551
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: yangwei
Assignee: yangwei
Priority: Minor
 Fix For: 1.1.12, 1.2.5

 Attachments: 0001-correctly-intersects-the-bounds.patch


 intersecs the bound includes the left of bound instead of right

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3929) Support row size limits

2013-05-28 Thread Rick Branson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668470#comment-13668470
 ] 

Rick Branson commented on CASSANDRA-3929:
-

+1. After implementing this on top of stock Cassandra, I think pushing this 
down to storage is probably not all that advantageous. Would still like to see 
compaction more extensible, however.

 Support row size limits
 ---

 Key: CASSANDRA-3929
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3929
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor
  Labels: ponies
 Attachments: 3929_b.txt, 3929_c.txt, 3929_d.txt, 3929_e.txt, 
 3929_f.txt, 3929_g_tests.txt, 3929_g.txt, 3929.txt


 We currently support expiring columns by time-to-live; we've also had 
 requests for keeping the most recent N columns in a row.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5595) Add counters to TRACE for how much CASSANDRA-5514 helped a query


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668472#comment-13668472
 ] 

Jonathan Ellis commented on CASSANDRA-5595:
---

+1

 Add counters to TRACE for how much CASSANDRA-5514 helped a query
 

 Key: CASSANDRA-5595
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5595
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Jeremiah Jordan
Assignee: Marcus Eriksson
Priority: Minor
 Attachments: 
 0001-add-tracing-that-tracks-how-many-sstables-were-skipp.patch, 
 0001-trace-how-many-sstables-were-skipped-during-slice-du.patch


 It might be an interesting stat to keep track of how many sstables got 
 skipped by the comparisons added in CASSANDRA-5514.  Could be exposed in 
 TRACE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3929) Support row size limits


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668482#comment-13668482
 ] 

Jonathan Ellis commented on CASSANDRA-3929:
---

Fine in principle w/ making compaction more extensible.  Just need someone w/ a 
compaction extension to come along and show what extension points he needs. :)

 Support row size limits
 ---

 Key: CASSANDRA-3929
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3929
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor
  Labels: ponies
 Attachments: 3929_b.txt, 3929_c.txt, 3929_d.txt, 3929_e.txt, 
 3929_f.txt, 3929_g_tests.txt, 3929_g.txt, 3929.txt


 We currently support expiring columns by time-to-live; we've also had 
 requests for keeping the most recent N columns in a row.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5551) intersects the bounds not right


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668540#comment-13668540
 ] 

Jonathan Ellis commented on CASSANDRA-5551:
---

The impact is that LCS might not compact two sstables if one is entirely 
spanned by the start and end keys in another.

LCS prints out warnings if this happens, so you'd know if it were affecting you.

 intersects the bounds not right
 ---

 Key: CASSANDRA-5551
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5551
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: yangwei
Assignee: yangwei
Priority: Minor
 Fix For: 1.1.12, 1.2.5

 Attachments: 0001-correctly-intersects-the-bounds.patch


 intersecs the bound includes the left of bound instead of right

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-5383:
---

Attachment: 0001-CASSANDRA-5383-v2.patch

rebased patch, probably wont work either though

the error is quite strange - had a quick look and the files should all be 
closed before renaming...

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:507)
 [junit]   at

[jira] [Comment Edited] (CASSANDRA-5551) intersects the bounds not right


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668540#comment-13668540
 ] 

Jonathan Ellis edited comment on CASSANDRA-5551 at 5/28/13 6:51 PM:


Basically nil.  The effect is that cleanup will take not take the fast discard 
entire sstable path occasionally.

  was (Author: jbellis):
The impact is that LCS might not compact two sstables if one is entirely 
spanned by the start and end keys in another.

LCS prints out warnings if this happens, so you'd know if it were affecting you.
  
 intersects the bounds not right
 ---

 Key: CASSANDRA-5551
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5551
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: yangwei
Assignee: yangwei
Priority: Minor
 Fix For: 1.1.12, 1.2.5

 Attachments: 0001-correctly-intersects-the-bounds.patch


 intersecs the bound includes the left of bound instead of right

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[2/3] git commit: Add get commands for compaction/streaming throughput to nodetool. Patch by Michal Michalski, reviewed by brandonwilliams for CASSANDRA-5588

2013-05-28 Thread brandonwilliams

Add get commands for compaction/streaming throughput to nodetool.
Patch by Michal Michalski, reviewed by brandonwilliams for
CASSANDRA-5588


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/843dc796
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/843dc796
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/843dc796

Branch: refs/heads/trunk
Commit: 843dc796ea5e80b003f6429a329ec04a9f1bfff4
Parents: aaf18bd
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 28 13:51:47 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 28 13:51:47 2013 -0500

--
 src/java/org/apache/cassandra/tools/NodeCmd.java   |   40 ++-
 src/java/org/apache/cassandra/tools/NodeProbe.java |   18 ++-
 .../org/apache/cassandra/tools/NodeToolHelp.yaml   |6 ++
 3 files changed, 50 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/843dc796/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index dce9dba..bca0fcd 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -118,6 +118,8 @@ public class NodeCmd
 ENABLETHRIFT,
 FLUSH,
 GETCOMPACTIONTHRESHOLD,
+GETCOMPACTIONTHROUGHPUT,
+GETSTREAMTHROUGHPUT,
 GETENDPOINTS,
 GETSSTABLES,
 GOSSIPINFO,
@@ -721,6 +723,39 @@ public class NodeCmd
 outs.printf(%25s%10s%n, Active compaction remaining time : , 
remainingTime);
 }
 
+/**
+ * Print the compaction threshold
+ *
+ * @param outs the stream to write to
+ */
+public void printCompactionThreshold(PrintStream outs, String ks, String 
cf)
+{
+ColumnFamilyStoreMBean cfsProxy = probe.getCfsProxy(ks, cf);
+outs.println(Current compaction thresholds for  + ks + / + cf + : 
\n +
+  min =  + cfsProxy.getMinimumCompactionThreshold() + , 
 +
+  max =  + cfsProxy.getMaximumCompactionThreshold());
+}
+
+/**
+ * Print the compaction throughput
+ *
+ * @param outs the stream to write to
+ */
+public void printCompactionThroughput(PrintStream outs)
+{
+outs.println(Current compaction throughput:  + 
probe.getCompactionThroughput() +  MB/s);
+}
+
+/**
+ * Print the stream throughput
+ *
+ * @param outs the stream to write to
+ */
+public void printStreamThroughput(PrintStream outs)
+{
+outs.println(Current stream throughput:  + 
probe.getStreamThroughput() +  MB/s);
+}
+
 public void printColumnFamilyStats(PrintStream outs)
 {
 Map String, List ColumnFamilyStoreMBean cfstoreMap = new HashMap 
String, List ColumnFamilyStoreMBean();
@@ -1178,9 +1213,12 @@ public class NodeCmd
 
 case GETCOMPACTIONTHRESHOLD :
 if (arguments.length != 2) { 
badUse(getcompactionthreshold requires ks and cf args.); }
-probe.getCompactionThreshold(System.out, arguments[0], 
arguments[1]);
+nodeCmd.printCompactionThreshold(System.out, arguments[0], 
arguments[1]);
 break;
 
+case GETCOMPACTIONTHROUGHPUT : 
nodeCmd.printCompactionThroughput(System.out); break;
+case GETSTREAMTHROUGHPUT : 
nodeCmd.printStreamThroughput(System.out); break;
+
 case CFHISTOGRAMS :
 if (arguments.length != 2) { badUse(cfhistograms requires 
ks and cf args); }
 nodeCmd.printCfHistograms(arguments[0], arguments[1], 
System.out);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/843dc796/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index b2b9cf8..5db8f1c 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java
@@ -496,19 +496,6 @@ public class NodeProbe
 }
 
 /**
- * Get the compaction threshold
- *
- * @param outs the stream to write to
- */
-public void getCompactionThreshold(PrintStream outs, String ks, String cf)
-{
-ColumnFamilyStoreMBean cfsProxy = getCfsProxy(ks, cf);
-outs.println(Current compaction thresholds for  + ks + / + cf + : 
\n +
-  min =  + cfsProxy.getMinimumCompactionThreshold() + , 
 +
-  max =  +

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

2013-05-28 Thread brandonwilliams

Merge branch 'cassandra-1.2' into trunk

Conflicts:
src/java/org/apache/cassandra/tools/NodeCmd.java
src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5dd390b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5dd390b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5dd390b

Branch: refs/heads/trunk
Commit: e5dd390b96fb2ff6126b6d60de235fe0fb18af51
Parents: bef5209 843dc79
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 28 13:54:13 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 28 13:54:13 2013 -0500

--
 src/java/org/apache/cassandra/tools/NodeCmd.java   |   40 ++-
 src/java/org/apache/cassandra/tools/NodeProbe.java |   18 ++-
 .../org/apache/cassandra/tools/NodeToolHelp.yaml   |6 ++
 3 files changed, 50 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5dd390b/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --cc src/java/org/apache/cassandra/tools/NodeCmd.java
index 2e050a8,bca0fcd..2af02ea
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@@ -115,8 -118,8 +115,10 @@@ public class NodeCm
  ENABLETHRIFT,
  FLUSH,
  GETCOMPACTIONTHRESHOLD,
 +DISABLEAUTOCOMPACTION,
 +ENABLEAUTOCOMPACTION,
+ GETCOMPACTIONTHROUGHPUT,
+ GETSTREAMTHROUGHPUT,
  GETENDPOINTS,
  GETSSTABLES,
  GOSSIPINFO,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5dd390b/src/java/org/apache/cassandra/tools/NodeProbe.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5dd390b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml
--
diff --cc src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml
index c66c501,568aa20..6ec6d2e
--- a/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml
+++ b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml
@@@ -161,12 -161,12 +161,18 @@@ commands
- name: getcompactionthreshold keyspace cfname
  help: |
Print min and max compaction thresholds for a given column family
 +  - name: disableautocompaction [keyspace] [cfnames]
 +help: |
 +  Disable autocompaction for the given keyspace and column family
 +  - name: enableautocompaction [keyspace] [cfnames]
 +help: |
 +  Enable autocompaction
+   - name: getcompactionthroughput
+ help: |
+   Print the MB/s throughput cap for compaction in the system
+   - name: getstreamthroughput
+ help: |
+   Print the MB/s throughput cap for streaming in the system
- name: stop compaction_type
  help: |
Supported types are COMPACTION, VALIDATION, CLEANUP, SCRUB, INDEX_BUILD

[1/3] git commit: Add get commands for compaction/streaming throughput to nodetool. Patch by Michal Michalski, reviewed by brandonwilliams for CASSANDRA-5588

2013-05-28 Thread brandonwilliams

Updated Branches:
  refs/heads/cassandra-1.2 aaf18bd08 - 843dc796e
  refs/heads/trunk bef5209d6 - e5dd390b9


Add get commands for compaction/streaming throughput to nodetool.
Patch by Michal Michalski, reviewed by brandonwilliams for
CASSANDRA-5588


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/843dc796
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/843dc796
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/843dc796

Branch: refs/heads/cassandra-1.2
Commit: 843dc796ea5e80b003f6429a329ec04a9f1bfff4
Parents: aaf18bd
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue May 28 13:51:47 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue May 28 13:51:47 2013 -0500

--
 src/java/org/apache/cassandra/tools/NodeCmd.java   |   40 ++-
 src/java/org/apache/cassandra/tools/NodeProbe.java |   18 ++-
 .../org/apache/cassandra/tools/NodeToolHelp.yaml   |6 ++
 3 files changed, 50 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/843dc796/src/java/org/apache/cassandra/tools/NodeCmd.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeCmd.java 
b/src/java/org/apache/cassandra/tools/NodeCmd.java
index dce9dba..bca0fcd 100644
--- a/src/java/org/apache/cassandra/tools/NodeCmd.java
+++ b/src/java/org/apache/cassandra/tools/NodeCmd.java
@@ -118,6 +118,8 @@ public class NodeCmd
 ENABLETHRIFT,
 FLUSH,
 GETCOMPACTIONTHRESHOLD,
+GETCOMPACTIONTHROUGHPUT,
+GETSTREAMTHROUGHPUT,
 GETENDPOINTS,
 GETSSTABLES,
 GOSSIPINFO,
@@ -721,6 +723,39 @@ public class NodeCmd
 outs.printf(%25s%10s%n, Active compaction remaining time : , 
remainingTime);
 }
 
+/**
+ * Print the compaction threshold
+ *
+ * @param outs the stream to write to
+ */
+public void printCompactionThreshold(PrintStream outs, String ks, String 
cf)
+{
+ColumnFamilyStoreMBean cfsProxy = probe.getCfsProxy(ks, cf);
+outs.println(Current compaction thresholds for  + ks + / + cf + : 
\n +
+  min =  + cfsProxy.getMinimumCompactionThreshold() + , 
 +
+  max =  + cfsProxy.getMaximumCompactionThreshold());
+}
+
+/**
+ * Print the compaction throughput
+ *
+ * @param outs the stream to write to
+ */
+public void printCompactionThroughput(PrintStream outs)
+{
+outs.println(Current compaction throughput:  + 
probe.getCompactionThroughput() +  MB/s);
+}
+
+/**
+ * Print the stream throughput
+ *
+ * @param outs the stream to write to
+ */
+public void printStreamThroughput(PrintStream outs)
+{
+outs.println(Current stream throughput:  + 
probe.getStreamThroughput() +  MB/s);
+}
+
 public void printColumnFamilyStats(PrintStream outs)
 {
 Map String, List ColumnFamilyStoreMBean cfstoreMap = new HashMap 
String, List ColumnFamilyStoreMBean();
@@ -1178,9 +1213,12 @@ public class NodeCmd
 
 case GETCOMPACTIONTHRESHOLD :
 if (arguments.length != 2) { 
badUse(getcompactionthreshold requires ks and cf args.); }
-probe.getCompactionThreshold(System.out, arguments[0], 
arguments[1]);
+nodeCmd.printCompactionThreshold(System.out, arguments[0], 
arguments[1]);
 break;
 
+case GETCOMPACTIONTHROUGHPUT : 
nodeCmd.printCompactionThroughput(System.out); break;
+case GETSTREAMTHROUGHPUT : 
nodeCmd.printStreamThroughput(System.out); break;
+
 case CFHISTOGRAMS :
 if (arguments.length != 2) { badUse(cfhistograms requires 
ks and cf args); }
 nodeCmd.printCfHistograms(arguments[0], arguments[1], 
System.out);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/843dc796/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java 
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index b2b9cf8..5db8f1c 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java
@@ -496,19 +496,6 @@ public class NodeProbe
 }
 
 /**
- * Get the compaction threshold
- *
- * @param outs the stream to write to
- */
-public void getCompactionThreshold(PrintStream outs, String ks, String cf)
-{
-ColumnFamilyStoreMBean cfsProxy = getCfsProxy(ks, cf);
-outs.println(Current compaction thresholds for  + ks + / + cf + : 
\n +
-  min =  +

[jira] [Commented] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size

2013-05-28 Thread Rick Branson (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668584#comment-13668584
]

Rick Branson commented on CASSANDRA-5272:
-

There's a small spelling error eeach in the cassandra.yaml. Also, should we
add a line to NEWS to describe this behavior since it's going to change the
setting in place? (i.e. we will need to bump our throttle up a bunch when
rolling this out).

Hinted Handoff Throttle based on cluster size
-

Key: CASSANDRA-5272
URL: https://issues.apache.org/jira/browse/CASSANDRA-5272
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.2.0
Reporter: Rick Branson
Assignee: Jonathan Ellis
Priority: Minor
Labels: lhf
Fix For: 1.2.6

Attachments: 5272.txt

For a 12-node EC2 m1.xlarge cluster, restarting a node causes it to get
completely overloaded with the default 2-thread, 1024KB setting in 1.2.x.
This seemed to be a smaller problem when it was 6-nodes, but still required
us to abort handoffs. The old defaults in 1.1.x were WAY more conservative.
I've dropped this way down to 128KB on our production cluster which is really
conservative, but appears to have solved it. The default seems way too high
on any cluster that is non-trivial in size.
After putting some thought to this, it seems that this should really be based
on cluster size, making the throttle a target for how much write load a
single node can swallow. As the cluster grows, the amount of hints that can
be delivered by each other node in the cluster goes down, so the throttle
should self-adjust to take that into account.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5383:


Attachment: test_log.5383.patch_v2.log.txt

Uploaded test_log.5383.patch_v2.log.txt

This log is the entire test suite run with the v2 patch. Windows 7, java 
1.7.0_17 64bit

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 test_log.5383.patch_v2.log.txt


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:507)
 [junit]   at

[jira] [Comment Edited] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668585#comment-13668585
 ] 

Ryan McGuire edited comment on CASSANDRA-5383 at 5/28/13 7:32 PM:
--

Uploaded test_log.5383.patch_v2.log.txt

This log is the entire test suite run against trunk with the v2 patch. Windows 
7, java 1.7.0_17 64bit

  was (Author: enigmacurry):
Uploaded test_log.5383.patch_v2.log.txt

This log is the entire test suite run with the v2 patch. Windows 7, java 
1.7.0_17 64bit
  
 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 test_log.5383.patch_v2.log.txt


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5383:


Attachment: 5383_patch_v2_system.log

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 5383_patch_v2_system.log, test_log.5383.patch_v2.log.txt


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:507)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testRemoveUnifinishedCompactionLeftovers(ColumnFamilyStoreTest.java:1246)
 [junit] Caused by: java.io.IOException: Failed to delete

[jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task

2013-05-28 Thread Cyril Scetbon (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668593#comment-13668593
 ] 

Cyril Scetbon commented on CASSANDRA-5544:
--

AFAIK split combination is used to improve performance. Doesn't it mean the 
same for cassandra ?
And if performance decreases without split combination, will the performance 
decrease much more with vnodes ?

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: 5544.txt, Screen Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4872) Move manifest into sstable metadata

2013-05-28 Thread Rick Branson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668598#comment-13668598
 ] 

Rick Branson commented on CASSANDRA-4872:
-

Correct me if I'm wrong, but it looks like there's no longer a straightforward 
way to force a re-level now (say if you changed sstable_size_in_mb). Previously 
deleting the JSON manifest file would work.

 Move manifest into sstable metadata
 ---

 Key: CASSANDRA-4872
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4872
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.0

 Attachments: 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v1.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v2.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v3.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v4.patch, 
 0001-CASSANDRA-4872-wip-v6.patch, 0001-CASSANDRA-4872-wip-v7.patch, 
 4872-v5.txt


 Now that we have a metadata component it would be better to keep sstable 
 level there, than in a separate manifest.  With information per-sstable we 
 don't need to do a full re-level if there is corruption.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size

2013-05-28 Thread Brandon Williams (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668609#comment-13668609
]

Brandon Williams commented on CASSANDRA-5272:
-

Would it be better perhaps to throttle by the amount of live nodes? It won't
always be accurate, but if you lost a WAN link between DCs you'd be throttling
a lot more than needed at the time.

Hinted Handoff Throttle based on cluster size
-

Attachments: 5272.txt

[jira] [Commented] (CASSANDRA-4872) Move manifest into sstable metadata


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668622#comment-13668622
 ] 

Marcus Eriksson commented on CASSANDRA-4872:


CASSANDRA-5271 does that for you

 Move manifest into sstable metadata
 ---

 Key: CASSANDRA-4872
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4872
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Priority: Minor
 Fix For: 2.0

 Attachments: 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v1.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v2.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v3.patch, 
 0001-CASSANDRA-4872-move-sstable-level-into-sstable-metad-v4.patch, 
 0001-CASSANDRA-4872-wip-v6.patch, 0001-CASSANDRA-4872-wip-v7.patch, 
 4872-v5.txt


 Now that we have a metadata component it would be better to keep sstable 
 level there, than in a separate manifest.  With information per-sstable we 
 don't need to do a full re-level if there is corruption.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668633#comment-13668633
 ] 

Alex Liu commented on CASSANDRA-5544:
-

CassandraColumnInputFormat define the split size, so we don't want Pig to 
override it by combining splits. We can always tune the split size to tune the 
performance. Next step, we can open up a little bit so that Pig user can 
specify split size configuration.

Vnode hadoop performance generally decreases, we can do the split combination 
at Cassandra side to improve the performance, which could be another ticket.

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: 5544.txt, Screen Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-5544:


Attachment: 5544-1.txt

Version 2 patch is attached. It allows user to define PIG_INPUT_SPLIT_SIZE in 
the system env

 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Attachments: 5544-1.txt, 5544.txt, Screen Shot 2013-05-26 at 4.49.48 
 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5038) LZ4Compressor


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668725#comment-13668725
 ] 

Jonathan Ellis commented on CASSANDRA-5038:
---

Is there a valid reason to stick with Snappy now?  E.g. are there workloads 
where it would compress better?

 LZ4Compressor
 -

 Key: CASSANDRA-5038
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5038
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: T Jake Luciani
Assignee: Adrien Grand
Priority: Minor
 Fix For: 1.2.2

 Attachments: CASSANDRA-5038.patch, CASSANDRA-5038.patch, 
 CASSANDRA-5038.patch, LZ4Compressor.java, lz4-java.jar


 LZ4 is a new compression algo that's ~2x faster than Snappy.
 [~jpountz] has written a nice java port which includes a misc.Unsafe version 
 that performs = than our java snappy version.
 Details at http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast
 The nice thing is this should work with java7 and be more portable.
 We can also fallback the pure java impl

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3486) Node Tool command to stop repair


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3486:
--

Fix Version/s: (was: 2.0)
   2.1
 Assignee: (was: Vijay)

 Node Tool command to stop repair
 

 Key: CASSANDRA-3486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3486
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
 Environment: JVM
Reporter: Vijay
Priority: Minor
  Labels: nodetool, repair, stop
 Fix For: 2.1

 Attachments: 0001-stop-repair-3583.patch


 After CASSANDRA-1740, If the validation compaction is stopped then the repair 
 will hang. This ticket will allow users to kill the original repair.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3486) Node Tool command to stop repair


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3486:
--

Reviewer:   (was: slebresne)
  Labels: repair  (was: nodetool repair stop)

 Node Tool command to stop repair
 

 Key: CASSANDRA-3486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3486
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
 Environment: JVM
Reporter: Vijay
Priority: Minor
  Labels: repair
 Fix For: 2.1

 Attachments: 0001-stop-repair-3583.patch


 After CASSANDRA-1740, If the validation compaction is stopped then the repair 
 will hang. This ticket will allow users to kill the original repair.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5051) Allow automatic cleanup after gc_grace


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5051:
--

Fix Version/s: (was: 2.0)
   2.1

I think that creeps the scope enough that we should push this to 2.1, 
especially since Yuki is concurrently working on a new Streaming design.

 Allow automatic cleanup after gc_grace
 --

 Key: CASSANDRA-5051
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5051
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Brandon Williams
Assignee: Vijay
  Labels: vnodes
 Fix For: 2.1

 Attachments: 0001-5051-v4.patch, 0001-5051-v6.patch, 
 0001-5051-with-test-fixes.patch, 0001-CASSANDRA-5051.patch, 
 0002-5051-remove-upgradesstable.patch, 
 0002-5051-remove-upgradesstable-v4.patch, 0004-5051-additional-test-v4.patch, 
 5051-v2.txt


 When using vnodes, after adding a new node you have to run cleanup on all the 
 machines, because you don't know which are affected and chances are it was 
 most if not all of them.  As an alternative to this intensive process, we 
 could allow cleanup during compaction if the data is older than gc_grace (or 
 perhaps some other time period since people tend to use gc_grace hacks to get 
 rid of tombstones.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4784) Create separate sstables for each token range handled by a node

[
https://issues.apache.org/jira/browse/CASSANDRA-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-4784:
--

Fix Version/s: (was: 2.0)
2.1
Assignee: (was: Benjamin Coverston)

Create separate sstables for each token range handled by a node
---

Key: CASSANDRA-4784
URL: https://issues.apache.org/jira/browse/CASSANDRA-4784
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.2.0 beta 1
Reporter: sankalp kohli
Priority: Minor
Labels: perfomance
Fix For: 2.1

Attachments: 4784.patch

Currently, each sstable has data for all the ranges that node is handling. If
we change that and rather have separate sstables for each range that node is
handling, it can lead to some improvements.
Improvements
1) Node rebuild will be very fast as sstables can be directly copied over to
the bootstrapping node. It will minimize any application level logic. We can
directly use Linux native methods to transfer sstables without using CPU and
putting less pressure on the serving node. I think in theory it will be the
fastest way to transfer data.
2) Backup can only transfer sstables for a node which belong to its primary
keyrange.
3) ETL process can only copy one replica of data and will be much faster.
Changes:
We can split the writes into multiple memtables for each range it is
handling. The sstables being flushed from these can have details of which
range of data it is handling.
There will be no change I think for any reads as they work with interleaved
data anyway. But may be we can improve there as well?
Complexities:
The change does not look very complicated. I am not taking into account how
it will work when ranges are being changed for nodes.
Vnodes might make this work more complicated. We can also have a bit on each
sstable which says whether it is primary data or not.

[jira] [Resolved] (CASSANDRA-3512) Getting Started instructions don't work in README.txt - wrong version of jamm, wrong path


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3512.
---

   Resolution: Cannot Reproduce
Fix Version/s: (was: 2.0)

 Getting Started instructions don't work in README.txt - wrong version of 
 jamm, wrong path
 -

 Key: CASSANDRA-3512
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3512
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
 Environment: Ubuntu 11.04
Reporter: David Allsopp
Assignee: Dave Brosius
Priority: Minor

 Download latest release from 
 http://www.apache.org/dyn/closer.cgi?path=/cassandra/1.0.3/apache-cassandra-1.0.3-bin.tar.gz
 Unpack the tarball.
 Follow instructions in README.txt, concluding with:
 {noformat}
 dna@master:~/code/apache-cassandra-1.0.3$ bin/cassandra -f
 Error opening zip file or JAR manifest missing : /lib/jamm-0.2.1.jar
 Error occurred during initialization of VM
 agent library failed to init: instrument
 {noformat}
 Firstly, the version of jamm packaged with Cassandra 1.0.3 is jamm-0.2.5, not 
 jamm-0.2.1. 
 Both bin/cassandra.bat and conf/cassandra-env.sh reference jamm-0.2.5 so not 
 sure where jamm-0.2.1 is being referenced from - nothing obvious using grep.
 Secondly, /lib/jamm-0.2.1.jar is the wrong path - should be set relative to 
 working directory, not filesystem root
 (Incidentally, Cassandra v1.0.3 is still listed as unreleased on JIRA.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5538) Reduce Empty Map allocations in StorageProxy.sendToHintedEndpoints


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668751#comment-13668751
 ] 

Jonathan Ellis commented on CASSANDRA-5538:
---

/me throws up the [~dbrosius] signal

 Reduce Empty Map allocations in StorageProxy.sendToHintedEndpoints
 --

 Key: CASSANDRA-5538
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5538
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 2.0

 Attachments: 5538-2.txt, 5538.txt


 StorageProxy.sendToHintedEndpoints allocates HashMaps consistently that are 
 very often not used.
 See output: http://pastebin.com/jEaBxz1h
 Format is
 Type Date SourceLine CollectionSize NumBuckets NumBucketsUsed
 The snapshot is taken after the size of the collection hasn't changed for 5 
 seconds.
 Postpone creation of hashmap until you need it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-5062) Support CAS


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5062.
---

Resolution: Fixed
  Assignee: Jonathan Ellis

Subtasks complete; declaring victory.

 Support CAS
 ---

 Key: CASSANDRA-5062
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5062
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: half-baked commit 1.jpg, half-baked commit 2.jpg, 
 half-baked commit 3.jpg


 Strong consistency is not enough to prevent race conditions.  The classic 
 example is user account creation: we want to ensure usernames are unique, so 
 we only want to signal account creation success if nobody else has created 
 the account yet.  But naive read-then-write allows clients to race and both 
 think they have a green light to create.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5429) Update scrub and scrubtest for single-pass compaction format


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668754#comment-13668754
 ] 

Jonathan Ellis commented on CASSANDRA-5429:
---

Have you had a chance to start on this, Jason?

 Update scrub and scrubtest for single-pass compaction format
 

 Key: CASSANDRA-5429
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5429
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0
Reporter: Jonathan Ellis
Assignee: Jason Brown
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3734) Support native link w/o JNA in Java7


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3734:
--

Reviewer: iamaleksey
Assignee: Jason Brown  (was: Peter Schuller)

Let's get this crossed off for 2.0.  Can you review v3, [~iamaleksey]?

 Support native link w/o JNA in Java7
 

 Key: CASSANDRA-3734
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3734
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jason Brown
Priority: Minor
 Fix For: 2.0

 Attachments: 3734-v2.diff, 3734-v3.patch, CASSANDRA-3734-trunk-v1.txt


 Java7 provides native support for hard links: 
 http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#createLink(java.nio.file.Path,
  java.nio.file.Path)
 We should prefer this method when Java7 is the host.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668757#comment-13668757
]

Jonathan Ellis commented on CASSANDRA-5417:
---

Should we push this to 2.1?

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

CompositeType happens to be very useful and is now widely used: CQL3 heavily
rely on it, and super columns are now using it too internally. Besides,
CompositeType has been advised as a replacement of super columns on the
thrift side for a while, so it's safe to assume that it's generally used
there too.
CompositeType has initially been introduced as just another AbstractType.
Meaning that the storage engine has no nothing whatsoever of composites
being, well, composite. This has the following drawbacks:
* Because internally a composite value is handled as just a ByteBuffer, we
end up doing a lot of extra work. Typically, each time we compare 2 composite
value, we end up deserializing the components (which, while it doesn't copy
data per-se because we just slice the global ByteBuffer, still waste some cpu
cycles and allocate a bunch of ByteBuffer objects). And since compare can be
called *a lot*, this is likely not negligible.
* This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive
use of composites, and since it gets backs ByteBuffer from the internal
columns, it always have to check if it's actually a compositeType or not, and
then split it and pick the different parts it needs. It's only an API
problem, but having things exposed as composites directly would definitively
make thinks cleaner. In particular, in most cases, CQL3 don't care whether it
has a composite with only one component or a non-really-composite value, but
we still always distinguishes both cases. Lastly, if we do expose composites
more directly internally, it's not a lot more work to internalize better
the different parts of the cell name that CQL3 uses (what's the clustering
key, what's the actuall CQL3 column name, what's the collection element),
making things cleaner. Last but not least, there is currently a bunch of
places where methods take a ByteBuffer as argument and it's hard to know
whether it expects a cell name or a CQL3 column name. This is pretty error
prone.
* It makes it hard (or impossible) to do a number of performance
improvements. Consider CASSANDRA-4175, I'm not really sure how you can do it
properly (in memory) if cell names are just ByteBuffer (since CQL3 column
names are just one of the component in general). But we also miss
oportunities of sharing prefixes. If we were able to share prefixes of
composite names in memory we would 1) lower the memory footprint and 2)
potentially speed-up comparison (of the prefixes) by checking reference
equality first (also, doing prefix sharing on-disk, which is a separate
concern btw, might be easier to do if we do prefix sharing in memory).
So I suggest pushing CompositeType support inside the storage engine. What I
mean by that concretely would be change the internal {{Column.name}} from
ByteBuffer to some CellName type. A CellName would API-wise just be a list of
ByteBuffer. But in practice, we'd have a specific CellName implementation for
not-really-composite names, and the truly composite implementation will allow
some prefix sharing. From an external API however, nothing would change, we
would pack the composite as usual before sending it back to the client, but
at least internally, comparison won't have to deserialize the components
every time, and CQL3 code will be cleaner.

[jira] [Commented] (CASSANDRA-3112) Make repair fail when an unexpected error occurs


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668760#comment-13668760
 ] 

Jonathan Ellis commented on CASSANDRA-3112:
---

What is the scope of this ticket?  Should it be wontfixed or moved to 2.1?

 Make repair fail when an unexpected error occurs
 

 Key: CASSANDRA-3112
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3112
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
  Labels: repair
 Fix For: 2.0

 Attachments: 0003-Report-streaming-errors-back-to-repair-v4.patch, 
 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch


 CASSANDRA-2433 makes it so that nodetool repair will fail if a node 
 participating to repair dies before completing his part of the repair. This 
 handles most of the situation where repair was previously hanging, but repair 
 can still hang if an unexpected error occurs during either the merkle tree 
 creation (an on-disk corruption triggers an IOError say) or during streaming 
 (though I'm not sure what could make streaming failed outside of 'one of the 
 node died' (besides a bug)).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3734) Support native link w/o JNA in Java7


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668759#comment-13668759
 ] 

Aleksey Yeschenko commented on CASSANDRA-3734:
--

Sure.

 Support native link w/o JNA in Java7
 

 Key: CASSANDRA-3734
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3734
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jason Brown
Priority: Minor
 Fix For: 2.0

 Attachments: 3734-v2.diff, 3734-v3.patch, CASSANDRA-3734-trunk-v1.txt


 Java7 provides native support for hard links: 
 http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#createLink(java.nio.file.Path,
  java.nio.file.Path)
 We should prefer this method when Java7 is the host.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4495) Don't tie client side use of AbstractType to JDBC


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4495:
--

Assignee: (was: Sylvain Lebresne)

[~ardot] [~mfiguiere] [~urandom] I don't suppose I can interest one of you in 
this?

 Don't tie client side use of AbstractType to JDBC
 -

 Key: CASSANDRA-4495
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4495
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0


 We currently expose the AbstractType to java clients that want to reuse them 
 though the cql.jdbc.* classes. I think this shouldn't be tied to the JDBC 
 standard. JDBC was make for SQL DB, which Cassandra is not (CQL is not SQL 
 and will never be). Typically, there is a fair amount of the JDBC standard 
 that cannot be implemented with C*, and there is a number of specificity of 
 C* that are not in JDBC (typically the set and maps collections).
 So I propose to extract simple type classes with just a compose and decompose 
 method (but without ties to jdbc, which would allow all the jdbc specific 
 method those types have) in the purpose of exporting that in a separate jar 
 for clients (we could put that in a org.apache.cassandra.type package for 
 instance). We could then deprecate the jdbc classes with basically the same 
 schedule than CQL2.
 Let me note that this is *not* saying there shouldn't be a JDBC driver for 
 Cassandra.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5086) MultiSlice bug in SimpleBlockFetcher


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668774#comment-13668774
 ] 

Jonathan Ellis commented on CASSANDRA-5086:
---

Well, 4762 is going nowhere fast, should we push this to 2.1 as well?

 MultiSlice bug in SimpleBlockFetcher
 

 Key: CASSANDRA-5086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5086
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.0 beta 1
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Minor
 Fix For: 2.0

 Attachments: 5086.txt


 Related to CASSANDRA-3885 it works for most cases but logic when there is no 
 index entries is wrong

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3487) better repair session timeouts and retrys


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3487:
--

 Priority: Minor  (was: Major)
Affects Version/s: (was: 1.1.0)
Fix Version/s: (was: 2.0)
   2.1

 better repair session timeouts and retrys
 -

 Key: CASSANDRA-3487
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3487
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Vijay
Priority: Minor
 Fix For: 2.1


 It would be great if we can timeout a validation compaction which is taking 
 long or had an exception while doing a Validation. 
 Repair can gossip its status to all the other nodes, hence any node which is 
 waiting for response of a tree request to wait until it complete, if the 
 repair is not going to complete because of exception or because it is too 
 busy taking the incoming request we can timeout the user request.
 Bonus: By displaying the repair gossip via nodetool, user/script running the 
 request can have a better handle on whats going on in the cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4911) Lift limitation that order by columns must be selected for IN queries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4911:
--

Fix Version/s: (was: 2.0)
   2.0.1

 Lift limitation that order by columns must be selected for IN queries
 -

 Key: CASSANDRA-4911
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4911
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0.1


 This is the followup of CASSANDRA-4645. We should remove the limitation that 
 for IN queries, you must have columns on which you have an ORDER BY in the 
 select clause.
 For that, we'll need to automatically add the columns on which we have an 
 ORDER BY to the one queried internally, and remove it afterwards (once the 
 sorting is done) from the resultSet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-3225) Clean up CompactionManager / CompactionTask / CompactionInfo / CompactionInfo.Holder mess


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-3225:
--

Fix Version/s: (was: 2.0)

 Clean up CompactionManager / CompactionTask / CompactionInfo / 
 CompactionInfo.Holder mess
 -

 Key: CASSANDRA-3225
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3225
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jonathan Ellis
Priority: Minor

 These classes are too tightly coupled in unintuitive ways, causing problems 
 like CASSANDRA-3224.
 Ideally we would move this back to the classic executor model (tasks are 
 submitted to an executor, which runs them, and handles beforeExecute / 
 afterExecute without the task having to explicitly invoke them).
 In particular re 3224 this would allow moving the check for more compactions 
 to run to afterExecute.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-2737) CQL: support IF EXISTS extension for DROP commands (table, keyspace, index)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-2737:
--

Fix Version/s: (was: 2.0)
   2.1

 CQL: support IF EXISTS extension for DROP commands (table, keyspace, index)
 ---

 Key: CASSANDRA-2737
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2737
 Project: Cassandra
  Issue Type: New Feature
Affects Versions: 0.8.0
Reporter: Cathy Daw
Priority: Trivial
  Labels: cql
 Fix For: 2.1

 Attachments: 2737-concept-v1.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3112) Make repair fail when an unexpected error occurs

2013-05-28 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668807#comment-13668807
 ] 

Yuki Morishita commented on CASSANDRA-3112:
---

I'm working on CASSANDRA-5426 and this can be dup of that. CASSANDRA-5426 is 
targeting for 2.0.0 release.

 Make repair fail when an unexpected error occurs
 

 Key: CASSANDRA-3112
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3112
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
  Labels: repair
 Fix For: 2.0

 Attachments: 0003-Report-streaming-errors-back-to-repair-v4.patch, 
 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch


 CASSANDRA-2433 makes it so that nodetool repair will fail if a node 
 participating to repair dies before completing his part of the repair. This 
 handles most of the situation where repair was previously hanging, but repair 
 can still hang if an unexpected error occurs during either the merkle tree 
 creation (an on-disk corruption triggers an IOError say) or during streaming 
 (though I'm not sure what could make streaming failed outside of 'one of the 
 node died' (besides a bug)).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5596) CREATE TABLE error message swaps table and keyspace

2013-05-28 Thread Lex Lythius (JIRA)

Lex Lythius created CASSANDRA-5596:
--

 Summary: CREATE TABLE error message swaps table and keyspace
 Key: CASSANDRA-5596
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5596
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 1.2.0
 Environment: CQLSH 3.0 on Ubuntu Linux 12.04 LTS
Reporter: Lex Lythius
Priority: Trivial


When trying to create an existing table, CQLSH rightfully complains in a weird 
way:

USE blink;
CREATE TABLE tags ( tag ascii PRIMARY KEY, type ascii, label varchar, rel 
mapascii, ascii )

Bad Request: Cannot add already existing column family blink to keyspace 
tags

Keyspace and table names are swapped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-3112) Make repair fail when an unexpected error occurs


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-3112.
---

   Resolution: Duplicate
Fix Version/s: (was: 2.0)
 Assignee: (was: Sylvain Lebresne)

 Make repair fail when an unexpected error occurs
 

 Key: CASSANDRA-3112
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3112
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sylvain Lebresne
Priority: Minor
  Labels: repair
 Attachments: 0003-Report-streaming-errors-back-to-repair-v4.patch, 
 0004-Reports-validation-compaction-errors-back-to-repair-v4.patch


 CASSANDRA-2433 makes it so that nodetool repair will fail if a node 
 participating to repair dies before completing his part of the repair. This 
 handles most of the situation where repair was previously hanging, but repair 
 can still hang if an unexpected error occurs during either the merkle tree 
 creation (an on-disk corruption triggers an IOError say) or during streaming 
 (though I'm not sure what could make streaming failed outside of 'one of the 
 node died' (besides a bug)).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/3] git commit: fix parameter order in error message for #5596

2013-05-28 Thread jbellis

Updated Branches:
  refs/heads/cassandra-1.2 843dc796e - 27e8f87ff
  refs/heads/trunk e5dd390b9 - 34407424b


fix parameter order in error message for #5596


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27e8f87f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27e8f87f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27e8f87f

Branch: refs/heads/cassandra-1.2
Commit: 27e8f87ff14296c7381d1e7767e37f75072c916c
Parents: 843dc79
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue May 28 20:32:49 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Tue May 28 20:32:49 2013 -0500

--
 .../exceptions/AlreadyExistsException.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27e8f87f/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
--
diff --git 
a/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java 
b/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
index f023a07..4530568 100644
--- a/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
+++ b/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
@@ -31,7 +31,7 @@ public class AlreadyExistsException extends 
ConfigurationException
 
 public AlreadyExistsException(String ksName, String cfName)
 {
-this(ksName, cfName, String.format(Cannot add already existing column 
family \%s\ to keyspace \%s\, ksName, cfName));
+this(ksName, cfName, String.format(Cannot add already existing column 
family \%s\ to keyspace \%s\, cfName, ksName));
 }
 
 public AlreadyExistsException(String ksName)

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

2013-05-28 Thread jbellis

Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/34407424
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/34407424
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/34407424

Branch: refs/heads/trunk
Commit: 34407424b39297776c9db47b11f6fea34e6f17a2
Parents: e5dd390 27e8f87
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue May 28 20:32:55 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Tue May 28 20:32:55 2013 -0500

--
 .../exceptions/AlreadyExistsException.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--

[2/3] git commit: fix parameter order in error message for #5596

2013-05-28 Thread jbellis

fix parameter order in error message for #5596


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27e8f87f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27e8f87f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27e8f87f

Branch: refs/heads/trunk
Commit: 27e8f87ff14296c7381d1e7767e37f75072c916c
Parents: 843dc79
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue May 28 20:32:49 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Tue May 28 20:32:49 2013 -0500

--
 .../exceptions/AlreadyExistsException.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/27e8f87f/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
--
diff --git 
a/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java 
b/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
index f023a07..4530568 100644
--- a/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
+++ b/src/java/org/apache/cassandra/exceptions/AlreadyExistsException.java
@@ -31,7 +31,7 @@ public class AlreadyExistsException extends 
ConfigurationException
 
 public AlreadyExistsException(String ksName, String cfName)
 {
-this(ksName, cfName, String.format(Cannot add already existing column 
family \%s\ to keyspace \%s\, ksName, cfName));
+this(ksName, cfName, String.format(Cannot add already existing column 
family \%s\ to keyspace \%s\, cfName, ksName));
 }
 
 public AlreadyExistsException(String ksName)

[jira] [Resolved] (CASSANDRA-5596) CREATE TABLE error message swaps table and keyspace


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5596.
---

   Resolution: Fixed
Fix Version/s: 1.2.6
 Assignee: Jonathan Ellis

fixed in 27e8f87ff14296c7381d1e7767e37f75072c916c; thanks!

 CREATE TABLE error message swaps table and keyspace
 ---

 Key: CASSANDRA-5596
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5596
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 1.2.0
 Environment: CQLSH 3.0 on Ubuntu Linux 12.04 LTS
Reporter: Lex Lythius
Assignee: Jonathan Ellis
Priority: Trivial
 Fix For: 1.2.6


 When trying to create an existing table, CQLSH rightfully complains in a 
 weird way:
 USE blink;
 CREATE TABLE tags ( tag ascii PRIMARY KEY, type ascii, label varchar, rel 
 mapascii, ascii )
 Bad Request: Cannot add already existing column family blink to keyspace 
 tags
 Keyspace and table names are swapped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3734) Support native link w/o JNA in Java7

2013-05-28 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668942#comment-13668942
 ] 

Jason Brown commented on CASSANDRA-3734:


Let me know if you want me to rebase (probably necessary by now). Also, we can 
chuck the aforementioned and unused FileUtils.createTemporaryDirectory() 
method. If it's not used, let's not introduce (well-intentioned) baggage.

 Support native link w/o JNA in Java7
 

 Key: CASSANDRA-3734
 URL: https://issues.apache.org/jira/browse/CASSANDRA-3734
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jason Brown
Priority: Minor
 Fix For: 2.0

 Attachments: 3734-v2.diff, 3734-v3.patch, CASSANDRA-3734-trunk-v1.txt


 Java7 provides native support for hard links: 
 http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#createLink(java.nio.file.Path,
  java.nio.file.Path)
 We should prefer this method when Java7 is the host.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-3734) Support native link w/o JNA in Java7