date:20121005

Alexey Zotov created CASSANDRA-4768:
---

 Summary: Add separate max_hint_window_in_ms option for remote data 
centers
 Key: CASSANDRA-4768
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4768
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1.2
Reporter: Alexey Zotov
Assignee: Alexey Zotov
Priority: Minor


It would be nice to have possibility to configure hint window size for remote 
dc separately. It will allow to prevent accumulating of big amount of data for 
remote dc and long hints delivery as the result of it.

I suggest to add max_hint_window_for_remote_dc_in_ms option. 



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4768) Add separate max_hint_window_in_ms option for remote data centers


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zotov updated CASSANDRA-4768:


Attachment: cassandra-1.2-4768-remote_hint_window.txt
cassandra-1.1-4768-remote_hint_window.txt

 Add separate max_hint_window_in_ms option for remote data centers
 -

 Key: CASSANDRA-4768
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4768
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1.2
Reporter: Alexey Zotov
Assignee: Alexey Zotov
Priority: Minor
  Labels: configuration, hintedhandoff
 Attachments: cassandra-1.1-4768-remote_hint_window.txt, 
 cassandra-1.2-4768-remote_hint_window.txt


 It would be nice to have possibility to configure hint window size for remote 
 dc separately. It will allow to prevent accumulating of big amount of data 
 for remote dc and long hints delivery as the result of it.
 I suggest to add max_hint_window_for_remote_dc_in_ms option. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4768) Add separate max_hint_window_in_ms option for remote data centers


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zotov updated CASSANDRA-4768:


Fix Version/s: 1.2.0
   1.1.6

 Add separate max_hint_window_in_ms option for remote data centers
 -

 Key: CASSANDRA-4768
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4768
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1.2
Reporter: Alexey Zotov
Assignee: Alexey Zotov
Priority: Minor
  Labels: configuration, hintedhandoff
 Fix For: 1.1.6, 1.2.0

 Attachments: cassandra-1.1-4768-remote_hint_window.txt, 
 cassandra-1.2-4768-remote_hint_window.txt


 It would be nice to have possibility to configure hint window size for remote 
 dc separately. It will allow to prevent accumulating of big amount of data 
 for remote dc and long hints delivery as the result of it.
 I suggest to add max_hint_window_for_remote_dc_in_ms option. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (CASSANDRA-4710) High key hashing overhead for index scans when using RandomPartitioner

2012-10-05 Thread Yuki Morishita (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reopened CASSANDRA-4710:
---


When I was trying to reproduce CASSANDRA-4733, I stumbled upon following error.

{code}
ERROR [ValidationExecutor:2] 2012-10-04 15:24:43,440 CassandraDaemon.java (line 
132) Exception in thread Thread[ValidationExecutor:2,1,main]
java.lang.AssertionError: 113427529603963934725865253558964126270 is not 
contained in 
(56713727820156410577229101238628035242,113427455640312821154458202477256070484]
at 
org.apache.cassandra.service.AntiEntropyService$Validator.add(AntiEntropyService.java:345)
at 
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:727)
at 
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:66)
at 
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:451)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}

It turned out that the cause was SSTR#getPositionsForRanges returning unrelated 
section of file due to bug in SSTR#getPosition. getPosition was returning null 
when it should return position.

getPosition starts search for key from nearest sampled index up to index 
interval count.
The following check inside getPosition:

{code}
 while (!input.isEOF()  i  DatabaseDescriptor.getIndexInterval())
{code}

stops search for indexed position when it searches all indexes between index 
sampling intervals and method returns null.
But with the check above, when searching for key that is greater than the last 
key inside index interval but is less than next sampled index, the method 
returns null instead of the position.

I think the fix for this is changing  to =.

 High key hashing overhead for index scans when using RandomPartitioner
 --

 Key: CASSANDRA-4710
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4710
 Project: Cassandra
  Issue Type: Improvement
Reporter: Daniel Norberg
Assignee: Daniel Norberg
Priority: Minor
 Fix For: 1.2.0 beta 2

 Attachments: 
 0001-SSTableReader-compare-raw-key-when-scanning-index.patch


 For a workload where the dataset is completely in ram, the md5 hashing of the 
 keys during index scans becomes a bottleneck for reads when using 
 RandomPartitioner, according to profiling.
 Instead performing a raw key equals check in SSTableReader.getPosition() for 
 EQ operations improves throughput by some 30% for my workload (moving the 
 bottleneck elsewhere).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients of schema changes

[
https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470072#comment-13470072
]

Sylvain Lebresne commented on CASSANDRA-4684:
-

All I'm saying is that there is a lot of case where a client needs to access
the schema. And for some tools, like an eclipse plugin for instance, they would
need to access the schema all the time (to offer say completion or validation).

Now you don't need this to access the schema, you query the system table.
However, if you do need to access the schema often, how do you do that? Well,
either you query the database every damn time, and your plugin/tool/code will
be super slow. Or, more likely, you cache the schema client side and implement
some regular polling to refresh that cache. Which works, mostly, but has the
defaults of polling: should you poll often or not? If you poll too often it's
inefficient, if you poll not often enough you'll provide a bad user experience.
Don't get me wrong, I'm not pretending that this is the worst problem database
face today, but that is not far fetched either and all this ticket does is to
provide a better solution to that problem. So why wouldn't we give people a
better solution if we can? And we can very easily. It's not like this small
patch touch any sensible part of the code, or somehow makes parts of the code
unreadable (at least to me it seems like a very minor addition).

I also note that the patch don't even impose anything on clients implementors
since events are optional (and optional by type of events).

As for libpq, as I said, this patch is an optimization so the fact that libpq
don't support it is not a proof that it's useless either. And as a side note,
and while I'm not expert in libpq, it has an aynchronous notification mechanism
and I wouldn't be surprised that along with some simple trigger you can very
easily have schema change notifications (does the eclipse plugin uses that if
that does work? I don't know and frankly I don't care).

Binary protocol: inform clients of schema changes
-

Key: CASSANDRA-4684
URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.2.0 beta 2

Attachments: 0001-Return-schema-change-infos.txt,
0002-Add-migration-events.txt

It would be nice to inform clients when a schema change occurs as this would
allow said client to maintain the current state of the schema, which might be
useful/desirable. To allow that, we can:
# return that a query has changed the schema (instead of simply a 'void'
return), in the same spirit than CASSANDRA-3707.
# add events notification on schema change.
Just to be clear, the goal is only to inform that a change has occured, the
client would still have to query the system table to know the exact content
of the change, but at least it'll know when to do such query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4733) Last written key = current key exception when streaming

2012-10-05 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470075#comment-13470075
 ] 

Yuki Morishita commented on CASSANDRA-4733:
---

I still cannot reproduce above error, but I believe this was caused by the 
change in CASSANDRA-4710.
SSTableReader#getPositionsForRanges is used to determine the sections to 
transfer inside sstable, but the method returns incorrect sections for some 
cases.
In fact, system.log file that I got from Brandon showed that the node was 
trying to stream sections way bigger than actual sstable file size.

 Last written key = current key exception when streaming
 

 Key: CASSANDRA-4733
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4733
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0 beta 1
Reporter: Brandon Williams
Assignee: Yuki Morishita

 {noformat}
 ERROR 16:52:56,260 Exception in thread Thread[Streaming to 
 /10.179.111.137:1,5,main]
 java.lang.RuntimeException: java.io.IOException: Connection reset by peer
 at com.google.common.base.Throwables.propagate(Throwables.java:160)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcher.write0(Native Method)
 at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
 at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:72)
 at sun.nio.ch.IOUtil.write(IOUtil.java:43)
 at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
 at java.nio.channels.Channels.writeFullyImpl(Channels.java:59)
 at java.nio.channels.Channels.writeFully(Channels.java:81)
 at java.nio.channels.Channels.access$000(Channels.java:47)
 at java.nio.channels.Channels$1.write(Channels.java:155)
 at 
 com.ning.compress.lzf.ChunkEncoder.encodeAndWriteChunk(ChunkEncoder.java:133)
 at 
 com.ning.compress.lzf.LZFOutputStream.writeCompressedBlock(LZFOutputStream.java:203)
 at 
 com.ning.compress.lzf.LZFOutputStream.write(LZFOutputStream.java:97)
 at 
 org.apache.cassandra.streaming.FileStreamTask.write(FileStreamTask.java:218)
 at 
 org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:164)
 at 
 org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
 ... 3 more
 ERROR 16:53:03,951 Exception in thread Thread[Thread-11,5,main]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(113424593524874987650593774422007331058, 3036303936343535) = 
 current key DecoratedKey(59229538317742990547810678738983628664, 
 3036313133373139) writing into 
 /var/lib/cassandra/data/Keyspace1-Standard1-tmp-ia-95-Data.db
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:132)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:208)
 at 
 org.apache.cassandra.streaming.IncomingStreamReader.streamIn(IncomingStreamReader.java:164)
 at 
 org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:107)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:220)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:165)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:65)
 {noformat}
 I didn't do anything fancy here, just inserted about 6M keys at rf=2, then 
 ran repair and got this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-4769) Prevent parallel hint delivery to the node

Alexey Zotov created CASSANDRA-4769:
---

 Summary: Prevent parallel hint delivery to the node 
 Key: CASSANDRA-4769
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4769
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.1.2
Reporter: Alexey Zotov


It's actual only in case of the using a big enough cluster. After node's 
failure other nodes try to send hints to the restored node. So theoretically it 
can affect performance of restored node. 
I suggest to create some mechanism for synchronization of hints delivery 
processes to restored node.

Could you please explain how it can be implemented.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Fix support of collections in prepared statements

Updated Branches:
  refs/heads/trunk d54a93f2d - 8b00f3a25


Fix support of collections in prepared statements

patch by slebresne; reviewed by jbellis for CASSANDRA-4739


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8b00f3a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8b00f3a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8b00f3a2

Branch: refs/heads/trunk
Commit: 8b00f3a258fcc04f0350d4f46760eacacbfed3df
Parents: d54a93f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Oct 5 09:29:17 2012 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Oct 5 09:29:17 2012 +0200

--
 CHANGES.txt|1 +
 doc/native_protocol.spec   |   25 -
 .../cassandra/cql3/operations/ColumnOperation.java |   20 
 .../cassandra/cql3/operations/ListOperation.java   |   27 ++
 .../cassandra/cql3/operations/MapOperation.java|   25 +
 .../cassandra/cql3/operations/SetOperation.java|   28 ++
 .../cassandra/cql3/statements/UpdateStatement.java |   35 +-
 .../org/apache/cassandra/db/marshal/ListType.java  |   28 +++---
 .../org/apache/cassandra/db/marshal/MapType.java   |   39 ++-
 .../org/apache/cassandra/db/marshal/SetType.java   |   28 +++---
 10 files changed, 210 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b00f3a2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8d25639..868183e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,6 +19,7 @@
  * Add support for multiple column family outputs in CFOF (CASSANDRA-4208)
  * Support repairing only the local DC nodes (CASSANDRA-4747)
  * Use rpc_address for binary protocol and change default port (CASSANRA-4751)
+ * Fix use of collections in prepared statements (CASSANDRA-4739)
 
 
 1.2-beta1

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b00f3a2/doc/native_protocol.spec
--
diff --git a/doc/native_protocol.spec b/doc/native_protocol.spec
index 908..2e03a02 100644
--- a/doc/native_protocol.spec
+++ b/doc/native_protocol.spec
@@ -33,7 +33,8 @@ Table of Contents
 4.2.5.4. Prepared
   4.2.6. EVENT
   5. Compression
-  6. Error codes
+  6. Collection types
+  7. Error codes
 
 
 1. Overview
@@ -286,7 +287,7 @@ Table of Contents
   Indicates an error processing a request. The body of the message will be an
   error code ([int]) followed by a [string] error message. Then, depending on
   the exception, more content may follow. The error codes are defined in
-  Section 6, along with their additional content if any.
+  Section 7, along with their additional content if any.
 
 
 4.2.2. READY
@@ -452,7 +453,25 @@ Table of Contents
   flag (see Section 2.2) is set.
 
 
-6. Error codes
+6. Collection types
+
+  This section describe the serialization format for the collection types:
+  list, map and set. This serialization format is both useful to decode values
+  returned in RESULT messages but also to encode values for EXECUTE ones.
+
+  The serialization formats are:
+ List: a [short] n indicating the size of the list, followed by n elements.
+   Each element is [short bytes] representing the serialized element
+   value.
+ Map: a [short] n indicating the size of the map, followed by n entries.
+  Each entry is composed of two [short bytes] representing the key and
+  the value of the entry map.
+ Set: a [short] n indicating the size of the set, followed by n elements.
+  Each element is [short bytes] representing the serialized element
+  value.
+
+
+7. Error codes
 
   The supported error codes are described below:
 0xServer error: something unexpected happened. This indicates a

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b00f3a2/src/java/org/apache/cassandra/cql3/operations/ColumnOperation.java
--
diff --git a/src/java/org/apache/cassandra/cql3/operations/ColumnOperation.java 
b/src/java/org/apache/cassandra/cql3/operations/ColumnOperation.java
index e7086c1..0f4c1fc 100644
--- a/src/java/org/apache/cassandra/cql3/operations/ColumnOperation.java
+++ b/src/java/org/apache/cassandra/cql3/operations/ColumnOperation.java
@@ -27,6 +27,9 @@ import org.apache.cassandra.db.IColumn;
 import org.apache.cassandra.db.filter.QueryPath;
 import org.apache.cassandra.db.marshal.AbstractType;
 import org.apache.cassandra.db.marshal.CollectionType;
+import org.apache.cassandra.db.marshal.ListType;
+import

git commit: Store more informations in peers table

Updated Branches:
  refs/heads/trunk 8b00f3a25 - d5ec013ce


Store more informations in peers table

patch by slebresne; reviewed by jbellis for CASSANDRA-4351


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d5ec013c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d5ec013c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d5ec013c

Branch: refs/heads/trunk
Commit: d5ec013cee4f3d923d9618694716a265ab04fe1b
Parents: 8b00f3a
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Oct 5 09:34:51 2012 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Oct 5 09:34:51 2012 +0200

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/config/CFMetaData.java|   12 +-
 .../apache/cassandra/cql3/UntypedResultSet.java|6 +
 src/java/org/apache/cassandra/db/SystemTable.java  |  134 +++
 .../apache/cassandra/service/StorageService.java   |   19 ++
 5 files changed, 95 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ec013c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 868183e..342135f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -20,6 +20,7 @@
  * Support repairing only the local DC nodes (CASSANDRA-4747)
  * Use rpc_address for binary protocol and change default port (CASSANRA-4751)
  * Fix use of collections in prepared statements (CASSANDRA-4739)
+ * Store more information into peers table (CASSANDRA-4351)
 
 
 1.2-beta1

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ec013c/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 6abeb33..ef25d2a 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -158,13 +158,19 @@ public final class CFMetaData
  + AND COMMENT='hints 
awaiting delivery');
 
 public static final CFMetaData PeersCf = compile(12, CREATE TABLE  + 
SystemTable.PEERS_CF +  (
- + token_bytes blob 
PRIMARY KEY,
- + peer inet
+ + peer inet PRIMARY 
KEY,
+ + ring_id uuid,
+ + tokens setblob,
+ + schema_version 
uuid,
+ + release_version 
text,
+ + rpc_address inet,
+ + data_center text,
+ + rack text
  + ) WITH 
COMMENT='known peers in the cluster');
 
 public static final CFMetaData LocalCf = compile(13, CREATE TABLE  + 
SystemTable.LOCAL_CF +  (
  + key text PRIMARY 
KEY,
- + token_bytes blob,
+ + tokens setblob,
  + cluster_name text,
  + gossip_generation 
int,
  + bootstrapped text,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d5ec013c/src/java/org/apache/cassandra/cql3/UntypedResultSet.java
--
diff --git a/src/java/org/apache/cassandra/cql3/UntypedResultSet.java 
b/src/java/org/apache/cassandra/cql3/UntypedResultSet.java
index 203e4c1..ca3acf5 100644
--- a/src/java/org/apache/cassandra/cql3/UntypedResultSet.java
+++ b/src/java/org/apache/cassandra/cql3/UntypedResultSet.java
@@ -25,6 +25,7 @@ import java.util.HashMap;
 import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
 import java.util.UUID;
 
 import com.google.common.collect.AbstractIterator;
@@ -130,6 +131,11 @@ public class UntypedResultSet implements 
IterableUntypedResultSet.Row
 return DateType.instance.compose(data.get(column));
 }
 
+public T SetT getSet(String column, AbstractTypeT type)
+{
+return SetType.getInstance(type).compose(data.get(column));
+}
+

buildbot failure in ASF Buildbot on cassandra-trunk

2012-10-05 Thread buildbot

The Buildbot has detected a new failure on builder cassandra-trunk while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/1901

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] d5ec013cee4f3d923d9618694716a265ab04fe1b
Blamelist: Sylvain Lebresne sylv...@datastax.com

BUILD FAILED: failed shell

sincerely,
 -The Buildbot

[jira] [Commented] (CASSANDRA-4351) Consider storing more informations on peers in system tables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470087#comment-13470087
 ] 

Sylvain Lebresne commented on CASSANDRA-4351:
-

Committed 0001 but holding this open a little long to see if we decide 
something on the 2nd part.

bq. what if instead we change LongToken to/fromString to hex-encode with a 
constant width, the way CASSANDRA-4550 wanted? Of course then we'd need to 
switch it to unsigned comparison

That's not a bad idea (since we have no backward compatibility problem) so why 
not (and switching to unsigned comparison is probably not a big deal (though we 
do have to be careful about the fact that the minimum token shouldn't be a 
valid token, so tokens value will have to be in [1, 2^64-1])). That being said, 
I'm not sure about the instead in the sentence above. Was that to be 
understood as in addition to 0002?

 Consider storing more informations on peers in system tables 
 -

 Key: CASSANDRA-4351
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Sylvain Lebresne
Priority: Minor
 Fix For: 1.2.0 beta 2

 Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt


 Currently, the only thing we keep in system tables about other peers is their 
 token and IP addresses. We should probably also record the new ring_id, but 
 since CASSANDRA-4018 makes system table easily queriable, may it could be 
 worth adding some more information (basically most of what we gossip could be 
 a candidate (schema UUID, status, C* version, ...)) as a simple way to expose 
 the ring state to users (even if it's just a view of the ring state from 
 one specific node I believe it's still nice).
 Of course that means storing information that may not be absolutely needed by 
 the server, but I'm not sure there is much harm to that.
 Note that doing this cleanly may require changing the schema of current 
 system tables but as long as we do that in the 1.2 timeframe it's ok (since 
 the concerned system table 'local' and 'peers' are news anyway).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path

[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470088#comment-13470088
]

Sylvain Lebresne commented on CASSANDRA-4763:
-

bq. we require sstable files to be named according to the convention cassandra
is using?

We do, and so there is no need for the user to provide the keyspace name as
argument to the loader. That being said, we can lift the limitation that
sstables currently must be in a directory named after the keyspace (I think we
all agree here, just wanted to clarify).

SSTableLoader shouldn't get keyspace from path
--

Key: CASSANDRA-4763
URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.0 beta 1
Reporter: Nick Bailey
Priority: Minor

SSTableLoader currently gets the keyspace it is going to load to from the
path of the directoy of sstables it is loading. This isn't really documented
(or I didn't see it), but also isn't really a good way of doing it in general.
{noformat}
this.keyspace = directory.getParentFile().getName();
{noformat}
We should probably just let users pass the name in. If you are loading a
snapshot the file names will have the keyspace which is slightly better but
people manually creating their own sstables might not format them the same.

git commit: Fix comparison against IndexInterval in SSTR.getPosition()

Updated Branches:
  refs/heads/trunk d5ec013ce - 2a91a4818


Fix comparison against IndexInterval in SSTR.getPosition()


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2a91a481
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2a91a481
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2a91a481

Branch: refs/heads/trunk
Commit: 2a91a48181b269684d491d961a0c513bf81baf25
Parents: d5ec013
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Oct 5 10:24:37 2012 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Oct 5 10:24:37 2012 +0200

--
 .../apache/cassandra/io/sstable/SSTableReader.java |9 ++---
 1 files changed, 6 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a91a481/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index b89ee24..a67c1ab 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -758,15 +758,18 @@ public class SSTableReader extends SSTable
 
 // scan the on-disk index, starting at the nearest sampled position.
 // The check against IndexInterval is to be exit the loop in the EQ 
case when the key looked for is not present
-// (bloom filter false positive).
+// (bloom filter false positive). But note that for non-EQ cases, we 
might need to check the first key of the
+// next index position because the searched key can be greater the 
last key of the index interval checked if it
+// is lesser than the first key of next interval (and in that case we 
must return the position of the first key
+// of the next interval).
 int i = 0;
 IteratorFileDataInput segments = ifile.iterator(sampledPosition, 
INDEX_FILE_BUFFER_BYTES);
-while (segments.hasNext()  i  DatabaseDescriptor.getIndexInterval())
+while (segments.hasNext()  i = 
DatabaseDescriptor.getIndexInterval())
 {
 FileDataInput in = segments.next();
 try
 {
-while (!in.isEOF()  i  
DatabaseDescriptor.getIndexInterval())
+while (!in.isEOF()  i = 
DatabaseDescriptor.getIndexInterval())
 {
 i++;

[jira] [Commented] (CASSANDRA-4710) High key hashing overhead for index scans when using RandomPartitioner


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470100#comment-13470100
 ] 

Sylvain Lebresne commented on CASSANDRA-4710:
-

I agree, good catch. I went ahead a committed the fix (with some comment) in 
commit 2a91a48. Thanks Yuki.

 High key hashing overhead for index scans when using RandomPartitioner
 --

 Key: CASSANDRA-4710
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4710
 Project: Cassandra
  Issue Type: Improvement
Reporter: Daniel Norberg
Assignee: Daniel Norberg
Priority: Minor
 Fix For: 1.2.0 beta 2

 Attachments: 
 0001-SSTableReader-compare-raw-key-when-scanning-index.patch


 For a workload where the dataset is completely in ram, the md5 hashing of the 
 keys during index scans becomes a bottleneck for reads when using 
 RandomPartitioner, according to profiling.
 Instead performing a raw key equals check in SSTableReader.getPosition() for 
 EQ operations improves throughput by some 30% for my workload (moving the 
 bottleneck elsewhere).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-4710) High key hashing overhead for index scans when using RandomPartitioner


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-4710.
-

Resolution: Fixed

 High key hashing overhead for index scans when using RandomPartitioner
 --

 Key: CASSANDRA-4710
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4710
 Project: Cassandra
  Issue Type: Improvement
Reporter: Daniel Norberg
Assignee: Daniel Norberg
Priority: Minor
 Fix For: 1.2.0 beta 2

 Attachments: 
 0001-SSTableReader-compare-raw-key-when-scanning-index.patch


 For a workload where the dataset is completely in ram, the md5 hashing of the 
 keys during index scans becomes a bottleneck for reads when using 
 RandomPartitioner, according to profiling.
 Instead performing a raw key equals check in SSTableReader.getPosition() for 
 EQ operations improves throughput by some 30% for my workload (moving the 
 bottleneck elsewhere).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4710) High key hashing overhead for index scans when using RandomPartitioner

2012-10-05 Thread Daniel Norberg (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470105#comment-13470105
 ] 

Daniel Norberg commented on CASSANDRA-4710:
---

Good catch.

 High key hashing overhead for index scans when using RandomPartitioner
 --

 Key: CASSANDRA-4710
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4710
 Project: Cassandra
  Issue Type: Improvement
Reporter: Daniel Norberg
Assignee: Daniel Norberg
Priority: Minor
 Fix For: 1.2.0 beta 2

 Attachments: 
 0001-SSTableReader-compare-raw-key-when-scanning-index.patch


 For a workload where the dataset is completely in ram, the md5 hashing of the 
 keys during index scans becomes a bottleneck for reads when using 
 RandomPartitioner, according to profiling.
 Instead performing a raw key equals check in SSTableReader.getPosition() for 
 EQ operations improves throughput by some 30% for my workload (moving the 
 bottleneck elsewhere).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Fix SystemTableTest

Updated Branches:
  refs/heads/trunk 2a91a4818 - b7716c76b


Fix SystemTableTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7716c76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7716c76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7716c76

Branch: refs/heads/trunk
Commit: b7716c76b338ad8eea0a19822ec9ba99de1699b6
Parents: 2a91a48
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Oct 5 14:54:12 2012 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Oct 5 14:54:12 2012 +0200

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7716c76/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index e2ff161..ff7d81a 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -238,7 +238,7 @@ public class SystemTable
 continue;
 
 String req = UPDATE system.%s SET tokens = tokens - %s WHERE peer 
= '%s';
-processInternal(String.format(req, PEERS_CF, 
serializeTokens(toRemove), entry.getKey()));
+processInternal(String.format(req, PEERS_CF, 
serializeTokens(toRemove), entry.getKey().getHostAddress()));
 }
 forceBlockingFlush(PEERS_CF);
 }

git commit: (cql3) protect against null prepared variables (and avoid flooding the log on InvalidException errors)