date:20150325


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379400#comment-14379400
 ] 

Stefania commented on CASSANDRA-8236:
-

Tests are occasionally throwing following exception but I don't believe this is 
related:

{code}
[node2 ERROR] java.lang.RuntimeException: Error reading: 
org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
at 
org.apache.cassandra.metrics.ThreadPoolMetrics.getJmxMetric(ThreadPoolMetrics.java:134)
at org.apache.cassandra.utils.StatusLogger.log(StatusLogger.java:55)
at 
org.apache.cassandra.service.GCInspector.handleNotification(GCInspector.java:147)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor$ListenerWrapper.handleNotification(DefaultMBeanServerInterceptor.java:1754)
at 
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:156)
at 
sun.management.GarbageCollectorImpl.createGCNotification(GarbageCollectorImpl.java:150)
Caused by: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy5.getValue(Unknown Source)
at 
org.apache.cassandra.metrics.ThreadPoolMetrics.getJmxMetric(ThreadPoolMetrics.java:123)
... 5 more
Caused by: javax.management.InstanceNotFoundException: 
org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:643)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at 
javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
... 7 more
{code}

 Delay node up and node added notifications until native protocol server 
 is started
 --

 Key: CASSANDRA-8236
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Stefania
 Fix For: 3.0

 Attachments: 8236.txt


 As discussed in CASSANDRA-7510, there is still a gap between when a node up 
 or node added notification may be sent to native protocol clients (in 
 response to a gossip event) and when the native protocol server is ready to 
 serve requests.
 Everything in between the call to {{StorageService.instance.initServer()}} 
 and creation of the native server in {{CassandraDaemon.setup()}} contributes 
 to this delay, but waiting for Gossip to settle introduces the biggest delay.
 We may need to introduce a STARTING gossip state for the period inbetween, 
 which is why this is scheduled for 3.0.  If there's a better option, though, 
 it may make sense to put this in 2.1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8899) cqlsh - not able to get row count with select(*) for large table

2015-03-25 Thread Tommy Stendahl (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379410#comment-14379410
 ] 

Tommy Stendahl commented on CASSANDRA-8899:
---

I think I run in to this problem and I think I found the cause.
Using 2.1.2 I did {{select count\(*)}} on a large table in cqlsh and got the 
error {{errors={}, last_host=127.0.0.1}}. I then tried the same thing using 
2.1.3 and got the error {{OperationTimedOut: errors={}, last_host=127.0.0.1}}. 
Then I realized that what happens is that there is a timeout in cqlsh because 
the query takes very long time. This timeout is configurable, see  
[-CASSANDRA-7516-|https://issues.apache.org/jira/browse/CASSANDRA-7516] for 
details. The default timeout is 10s, I had to increase it to 30s.

 cqlsh - not able to get row count with select(*) for large table
 

 Key: CASSANDRA-8899
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8899
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.1.2 ubuntu12.04
Reporter: Jeff Liu
Assignee: Benjamin Lerer

  I'm getting errors when running a query that looks at a large number of rows.
 {noformat}
 cqlsh:events select count(*) from catalog;
  count
 ---
  1
 (1 rows)
 cqlsh:events select count(*) from catalog limit 11000;
  count
 ---
  11000
 (1 rows)
 cqlsh:events select count(*) from catalog limit 5;
 errors={}, last_host=127.0.0.1
 cqlsh:events 
 {noformat}
 We are not able to make the select * query to get row count.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7814) enable describe on indices


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379429#comment-14379429
 ] 

Stefania commented on CASSANDRA-7814:
-

Thanks for the detailed explanation [~thobbs].

I merged my python driver feature branch into my master (after pulling from the 
official driver repository) and then updated the patch with a zip file built on 
my python master branch by following your instructions:

https://github.com/apache/cassandra/commit/9abe6b451d715faca8ee792dba1379cedca39619

I hope that's OK.

 enable describe on indices
 --

 Key: CASSANDRA-7814
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7814
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: radha
Assignee: Stefania
Priority: Minor
 Fix For: 2.1.4


 Describe index should be supported, right now, the only way is to export the 
 schema and find what it really is before updating/dropping the index.
 verified in 
 [cqlsh 3.1.8 | Cassandra 1.2.18.1 | CQL spec 3.0.0 | Thrift protocol 19.36.2]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8425) Add full entry indexing capability for maps


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379481#comment-14379481
 ] 

Aleksey Yeschenko commented on CASSANDRA-8425:
--

It's actually already done, in CASSANDRA-8473.

 Add full entry indexing capability for maps
 ---

 Key: CASSANDRA-8425
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8425
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Lex Lythius
Priority: Minor

 Since C* 2.1 we're able to index map keys or map values and query them using 
 {{CONTAINS KEY}} and {{CONTAINS}} respectively.
 However, some use cases require being able to filter for specific key/value 
 combination. Syntax might be something along the lines of 
 {code:sql}
 SELECT * FROM table WHERE map['country'] = 'usa';
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
 {code}
 Of course, right now we can have the client refine the results from
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS { 'usa' };
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS KEY { 'country' };
 {code}
 but I believe this would add a good deal of flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8425) Add full entry indexing capability for maps


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-8425.
--
Resolution: Duplicate

 Add full entry indexing capability for maps
 ---

 Key: CASSANDRA-8425
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8425
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Lex Lythius
Priority: Minor

 Since C* 2.1 we're able to index map keys or map values and query them using 
 {{CONTAINS KEY}} and {{CONTAINS}} respectively.
 However, some use cases require being able to filter for specific key/value 
 combination. Syntax might be something along the lines of 
 {code:sql}
 SELECT * FROM table WHERE map['country'] = 'usa';
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
 {code}
 Of course, right now we can have the client refine the results from
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS { 'usa' };
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS KEY { 'country' };
 {code}
 but I believe this would add a good deal of flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7994) Commit logs on the fly compression

[
https://issues.apache.org/jira/browse/CASSANDRA-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379498#comment-14379498
]

Aleksey Yeschenko commented on CASSANDRA-7994:
--

Since changing something as major as that on 2.0, or even 2.1 at the time was a
no go, we went with CASSANDRA-6809 on trunk option.

I believe that some of the points raised by Oleg have been addressed there,
though not all.

I suggest we create separate follow up tickets for the remained of them (like
we did with CASSANDRA-8634), to be addressed in either 3.0 or 3.1, if you can
find some time for that, Oleg.

Will be closing this one as a duplicate for now.

Commit logs on the fly compression
---

Key: CASSANDRA-7994
URL: https://issues.apache.org/jira/browse/CASSANDRA-7994
Project: Cassandra
Issue Type: New Feature
Reporter: Oleg Anastasyev
Assignee: Oleg Anastasyev
Attachments: CompressedCommitLogs-7994.txt

This patch employs lz4 algo to comress commit logs. This could be useful to
conserve disk space either archiving commit logs for a long time or for
conserviing iops for use cases with often and large mutations updating the
same record.
The compression is performed on blocks of 64k, for better cross mutation
compression. CRC is computed on each 64k block, unlike original code
computing it on each individual mutation.
On one of our real production cluster this saved 2/3 of the space consumed by
commit logs. The replay is 20-30% slower for the same number of mutations.
While doing this, also refactored commit log reading code to CommitLogReader
class, which i believe makes code cleaner.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL

2015-03-25 Thread Benjamin Lerer (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379523#comment-14379523
 ] 

Benjamin Lerer commented on CASSANDRA-6237:
---

[~mambocab] Be carefull Jim the patch is not ready yet. Sorry if my comment was 
misleading. I just wanted to write done the current state of my work. 
For the moment I have focused on non conditional single statements. I have not 
tested it for conditional operations or batches. So I do not expect them to 
work. 
Once I am confident with the patch I will mark the ticket with patch available

 Allow range deletions in CQL
 

 Key: CASSANDRA-6237
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benjamin Lerer
Priority: Minor
  Labels: cql, docs
 Fix For: 3.0

 Attachments: CASSANDRA-6237.txt


 We uses RangeTombstones internally in a number of places, but we could expose 
 more directly too. Typically, given a table like:
 {noformat}
 CREATE TABLE events (
 id text,
 created_at timestamp,
 content text,
 PRIMARY KEY (id, created_at)
 )
 {noformat}
 we could allow queries like:
 {noformat}
 DELETE FROM events WHERE id='someEvent' AND created_at  'Jan 3, 2013';
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8357) ArrayOutOfBounds in cassandra-stress with inverted exponential distribution

2015-03-25 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379443#comment-14379443
 ] 

Jens Preußner commented on CASSANDRA-8357:
--

Hi! I ran the command from above again, adding -log level=verbose to it. I 
streamed the STDOUT to CASSANDRA-8357-2.1.3.log.txt and the STDERR to 
CASSANDRA-8357-2.1.3.errlog.txt
Hope this helps :)
Best!



 ArrayOutOfBounds in cassandra-stress with inverted exponential distribution
 ---

 Key: CASSANDRA-8357
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8357
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 6-node cassandra cluster (2.1.1) on debian.
Reporter: Jens Preußner
 Fix For: 2.1.4

 Attachments: CASSANDRA-8357-2.1.3.errlog.txt, 
 CASSANDRA-8357-2.1.3.log.txt


 When using the CQLstress example from GitHub 
 (https://github.com/apache/cassandra/blob/trunk/tools/cqlstress-example.yaml) 
 with an inverted exponential distribution in the insert-partitions field, 
 generated threads fail with
 Exception in thread Thread-20 java.lang.ArrayIndexOutOfBoundsException: 20 
 at 
 org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:307)
 See the gist https://gist.github.com/jenzopr/9edde53122554729c852 for the 
 typetest.yaml I used.
 The call was:
 cassandra-stress user profile=typetest.yaml ops\(insert=1\) -node $NODES



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8357) ArrayOutOfBounds in cassandra-stress with inverted exponential distribution

2015-03-25 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jens Preußner updated CASSANDRA-8357:
-
Reproduced In: 2.1.3, 2.1.1  (was: 2.1.1)
   Attachment: CASSANDRA-8357-2.1.3.log.txt
   CASSANDRA-8357-2.1.3.errlog.txt

 ArrayOutOfBounds in cassandra-stress with inverted exponential distribution
 ---

 Key: CASSANDRA-8357
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8357
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 6-node cassandra cluster (2.1.1) on debian.
Reporter: Jens Preußner
 Fix For: 2.1.4

 Attachments: CASSANDRA-8357-2.1.3.errlog.txt, 
 CASSANDRA-8357-2.1.3.log.txt


 When using the CQLstress example from GitHub 
 (https://github.com/apache/cassandra/blob/trunk/tools/cqlstress-example.yaml) 
 with an inverted exponential distribution in the insert-partitions field, 
 generated threads fail with
 Exception in thread Thread-20 java.lang.ArrayIndexOutOfBoundsException: 20 
 at 
 org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:307)
 See the gist https://gist.github.com/jenzopr/9edde53122554729c852 for the 
 typetest.yaml I used.
 The call was:
 cassandra-stress user profile=typetest.yaml ops\(insert=1\) -node $NODES



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8425) Add full entry indexing capability for maps

2015-03-25 Thread Marco Palladino (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379452#comment-14379452
 ] 

Marco Palladino commented on CASSANDRA-8425:


This would be a very useful feature. This suggested syntax looks very concise:

{code:sql}
SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
{code}

 Add full entry indexing capability for maps
 ---

 Key: CASSANDRA-8425
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8425
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Lex Lythius
Priority: Minor

 Since C* 2.1 we're able to index map keys or map values and query them using 
 {{CONTAINS KEY}} and {{CONTAINS}} respectively.
 However, some use cases require being able to filter for specific key/value 
 combination. Syntax might be something along the lines of 
 {code:sql}
 SELECT * FROM table WHERE map['country'] = 'usa';
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
 {code}
 Of course, right now we can have the client refine the results from
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS { 'usa' };
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS KEY { 'country' };
 {code}
 but I believe this would add a good deal of flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7807) Push notification when tracing completes for an operation

[
https://issues.apache.org/jira/browse/CASSANDRA-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379540#comment-14379540
]

Stefania commented on CASSANDRA-7807:
-

I completed the review of the latest commits and the code is in pretty good
shape, we are almost ready to commit.

I still have the following points to clarify, concerning requirements,
[~thobbs] can you give us your input since you have opened the ticket?
\\
* Is it OK to send the notification to the connection that originated the trace
request even if it has not registered for the event (unlike other events).
* Should other connections who have registered for this event receive
notifications at all? From the description above it seems not but just to be
extra sure.
* What should happen if the trace is started because of probabilistic tracing
rather than a request from the client - should the client still receive the
notification? What about sessions who registered (if applicable from previous
point)? I believe this is what Robert was not sure about in his comment
yesterday.

Regarding tests it's OK to defer to another ticket if the infrastructure is not
available, thank you Ariel for your input.

Push notification when tracing completes for an operation
-

Key: CASSANDRA-7807
URL: https://issues.apache.org/jira/browse/CASSANDRA-7807
Project: Cassandra
Issue Type: Sub-task
Components: Core
Reporter: Tyler Hobbs
Assignee: Robert Stupp
Priority: Minor
Labels: client-impacting, protocolv4
Fix For: 3.0

Attachments: 7807.txt

Tracing is an asynchronous operation, and drivers currently poll to determine
when the trace is complete (in a loop with sleeps). Instead, the server
could push a notification to the driver when the trace completes.
I'm guessing that most of the work for this will be around pushing
notifications to a single connection instead of all connections that have
registered listeners for a particular event type.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9034) AssertionError in SizeEstimatesRecorder

Stefania created CASSANDRA-9034:
---

 Summary: AssertionError in SizeEstimatesRecorder
 Key: CASSANDRA-9034
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9034
 Project: Cassandra
  Issue Type: Bug
 Environment: Trunk (52ddfe412a)
Reporter: Stefania
Priority: Minor
 Fix For: 3.0


One of the dtests of CASSANDRA-8236 
(https://github.com/stef1927/cassandra-dtest/tree/8236) raises the following 
exception unless I set {{-Dcassandra.size_recorder_interval=0}}:

{code}
ERROR [OptionalTasks:1] 2015-03-25 12:58:47,015 CassandraDaemon.java:179 - 
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.AssertionError: null
at 
org.apache.cassandra.service.StorageService.getLocalTokens(StorageService.java:2235)
 ~[main/:na]
at 
org.apache.cassandra.db.SizeEstimatesRecorder.run(SizeEstimatesRecorder.java:61)
 ~[main/:na]
at 
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:82)
 ~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_76]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
[na:1.7.0_76]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
 [na:1.7.0_76]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.7.0_76]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_76]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_76]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_76]
INFO  [RMI TCP Connection(2)-127.0.0.1] 2015-03-25 12:59:23,189 
StorageService.java:863 - Joining ring by operator request
{code}

The test is {{start_node_without_join_test}} in _pushed_notifications_test.py_ 
but starting a node that won't join the ring might be sufficient to reproduce 
the exception (I haven't tried though).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379647#comment-14379647
]

Benedict commented on CASSANDRA-8670:
-

bq. Do you want to see it commit by commit or should I squash it?

I don't mind, so long as it's easy to squash myself (it looks to be interleaved
with commits from elsewhere right now, is the difficulty)

Large columns + NIO memory pooling causes excessive direct memory usage
---

Key: CASSANDRA-8670
URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 3.0

Attachments: largecolumn_test.py

If you provide a large byte array to NIO and ask it to populate the byte
array from a socket it will allocate a thread local byte buffer that is the
size of the requested read no matter how large it is. Old IO wraps new IO for
sockets (but not files) so old IO is effected as well.
Even If you are using Buffered{Input | Output}Stream you can end up passing a
large byte array to NIO. The byte array read method will pass the array to
NIO directly if it is larger than the internal buffer.
Passing large cells between nodes as part of intra-cluster messaging can
cause the NIO pooled buffers to quickly reach a high watermark and stay
there. This ends up costing 2x the largest cell size because there is a
buffer for input and output since they are different threads. This is further
multiplied by the number of nodes in the cluster - 1 since each has a
dedicated thread pair with separate thread locals.
Anecdotally it appears that the cost is doubled beyond that although it isn't
clear why. Possibly the control connections or possibly there is some way in
which multiple
Need a workload in CI that tests the advertised limits of cells on a cluster.
It would be reasonable to ratchet down the max direct memory for the test to
trigger failures if a memory pooling issue is introduced. I don't think we
need to test concurrently pulling in a lot of them, but it should at least
work serially.
The obvious fix to address this issue would be to read in smaller chunks when
dealing with large values. I think small should still be relatively large (4
megabytes) so that code that is reading from a disk can amortize the cost of
a seek. It can be hard to tell what the underlying thing being read from is
going to be in some of the contexts where we might choose to implement
switching to reading chunks.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9035) Enhance hinted handoff YAML parameter

Stefania created CASSANDRA-9035:
---

 Summary: Enhance hinted handoff YAML parameter
 Key: CASSANDRA-9035
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9035
 Project: Cassandra
  Issue Type: Improvement
Reporter: Stefania
Assignee: Stefania
Priority: Minor
 Fix For: 3.0


As discussed in CASSANDRA-6157, at the moment we have a single parameter 
{{hinted_handoff_enabled}} that can be either a boolean or a csv list of 
enabled data centers.

We should have a boolean global {{hinted_handoff_enabled}} param plus a 
separate yaml list for the HH DC blacklist - 
{{hinted_handoff_disabled_datacenters}} to be checked when the global flag is 
on.

Backward compatibility with the existing approach should be kept.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8425) Add full entry indexing capability for maps


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379582#comment-14379582
 ] 

Aleksey Yeschenko commented on CASSANDRA-8425:
--

You won't, b/c it's only in the yet unreleased 3.0 for now.

 Add full entry indexing capability for maps
 ---

 Key: CASSANDRA-8425
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8425
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Lex Lythius
Priority: Minor

 Since C* 2.1 we're able to index map keys or map values and query them using 
 {{CONTAINS KEY}} and {{CONTAINS}} respectively.
 However, some use cases require being able to filter for specific key/value 
 combination. Syntax might be something along the lines of 
 {code:sql}
 SELECT * FROM table WHERE map['country'] = 'usa';
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
 {code}
 Of course, right now we can have the client refine the results from
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS { 'usa' };
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS KEY { 'country' };
 {code}
 but I believe this would add a good deal of flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: ninja-fix merge of CASSANDRA-6809

2015-03-25 Thread benedict

Repository: cassandra
Updated Branches:
  refs/heads/trunk 966ff2159 - 8a03181dd


ninja-fix merge of CASSANDRA-6809


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8a03181d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8a03181d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8a03181d

Branch: refs/heads/trunk
Commit: 8a03181dd58c02f4297f4918c7fdc80da5f63fb3
Parents: 966ff21
Author: blambov branimir.lam...@datastax.com
Authored: Wed Mar 25 10:35:05 2015 +
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Wed Mar 25 10:35:05 2015 +

--
 .../org/apache/cassandra/config/Config.java |  6 +-
 .../cassandra/config/DatabaseDescriptor.java|  4 +-
 .../cassandra/config/ParameterizedClass.java| 60 
 .../cassandra/config/ParametrizedClass.java | 60 
 .../config/YamlConfigurationLoader.java |  2 +-
 .../cassandra/db/commitlog/CommitLog.java   |  4 +-
 .../db/commitlog/CommitLogDescriptor.java   | 14 ++---
 .../io/compress/CompressionParameters.java  |  4 +-
 .../db/commitlog/CommitLogStressTest.java   | 10 ++--
 .../org/apache/cassandra/db/CommitLogTest.java  |  8 +--
 10 files changed, 87 insertions(+), 85 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8a03181d/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 25a9b31..1e69b26 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -64,7 +64,7 @@ public class Config
 public SetString hinted_handoff_enabled_by_dc = 
Sets.newConcurrentHashSet();
 public volatile Integer max_hint_window_in_ms = 3600 * 1000; // one hour
 
-public ParametrizedClass seed_provider;
+public ParameterizedClass seed_provider;
 public DiskAccessMode disk_access_mode = DiskAccessMode.auto;
 
 public DiskFailurePolicy disk_failure_policy = DiskFailurePolicy.ignore;
@@ -167,7 +167,9 @@ public class Config
 public Double commitlog_sync_batch_window_in_ms;
 public Integer commitlog_sync_period_in_ms;
 public int commitlog_segment_size_in_mb = 32;
-
+public ParameterizedClass commitlog_compression;
+public int commitlog_max_compression_buffers_in_pool = 3;
+ 
 @Deprecated
 public int commitlog_periodic_queue_size = -1;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8a03181d/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index c36c9e9..781dcfa 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -1077,12 +1077,12 @@ public class DatabaseDescriptor
 return conf.commitlog_directory;
 }
 
-public static ParametrizedClass getCommitLogCompression()
+public static ParameterizedClass getCommitLogCompression()
 {
 return conf.commitlog_compression;
 }
 
-public static void setCommitLogCompression(ParametrizedClass compressor)
+public static void setCommitLogCompression(ParameterizedClass compressor)
 {
 conf.commitlog_compression = compressor;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8a03181d/src/java/org/apache/cassandra/config/ParameterizedClass.java
--
diff --git a/src/java/org/apache/cassandra/config/ParameterizedClass.java 
b/src/java/org/apache/cassandra/config/ParameterizedClass.java
new file mode 100644
index 000..6b7af63
--- /dev/null
+++ b/src/java/org/apache/cassandra/config/ParameterizedClass.java
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing

[jira] [Commented] (CASSANDRA-8425) Add full entry indexing capability for maps

2015-03-25 Thread Marco Palladino (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379551#comment-14379551
 ] 

Marco Palladino commented on CASSANDRA-8425:


[~iamaleksey] Thanks for linking to the CASSANDRA-8473 issue. I couldn't find 
any documentation on this (Cassandra v2.1.3), could you please point me to the 
right direction?

 Add full entry indexing capability for maps
 ---

 Key: CASSANDRA-8425
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8425
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Lex Lythius
Priority: Minor

 Since C* 2.1 we're able to index map keys or map values and query them using 
 {{CONTAINS KEY}} and {{CONTAINS}} respectively.
 However, some use cases require being able to filter for specific key/value 
 combination. Syntax might be something along the lines of 
 {code:sql}
 SELECT * FROM table WHERE map['country'] = 'usa';
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS ENTRY { 'country': 'usa' };
 {code}
 Of course, right now we can have the client refine the results from
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS { 'usa' };
 {code}
 or
 {code:sql}
 SELECT * FROM table WHERE map CONTAINS KEY { 'country' };
 {code}
 but I believe this would add a good deal of flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

2015-03-25 Thread Branimir Lambov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379649#comment-14379649
 ] 

Branimir Lambov commented on CASSANDRA-6809:


Fix-up and rename 
[here|https://github.com/apache/cassandra/commit/9bd5aaa050ab3826c5f8339cbcdc85984adf9047],
 and 
[diff|https://github.com/apache/cassandra/commit/9bd5aaa050ab3826c5f8339cbcdc85984adf9047.diff].

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: docs-impacting, performance
 Fix For: 3.0

 Attachments: ComitLogStress.java, logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9037) Terminal UDFs evaluated at prepare time throw protocol version error

2015-03-25 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379776#comment-14379776
 ] 

Sam Tunnicliffe commented on CASSANDRA-9037:


We should use the version from {{ProtocolVersion.NEWEST_SUPPORTED}} to ensure 
we don't get ahead of the included driver. Unfortunately, C* uses a simple int 
to represent this version (in {{QueryOptions}} for instance) but the driver's 
{{ProtocolVersion}} doesn't expose the version number in this way so I've 
opened [JAVA-700|https://datastax-oss.atlassian.net/browse/JAVA-700] to do so.

[This branch|https://github.com/beobal/cassandra/tree/9037] includes a unit 
test  a SNAPSHOT driver jar build from [the JAVA-700 pull 
request|https://github.com/datastax/java-driver/pull/308]

 Terminal UDFs evaluated at prepare time throw protocol version error
 

 Key: CASSANDRA-9037
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9037
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0


 When a pure function with only terminal arguments (or with no arguments) is 
 used in a where clause, it's executed at prepare time and 
 {{Server.CURRENT_VERSION}} passed as the protocol version for serialization 
 purposes. For native functions, this isn't a problem, but UDFs use classes in 
 the bundled java-driver-core jar for (de)serialization of args and return 
 values. When {{Server.CURRENT_VERSION}} is greater than the highest version 
 supported by the bundled java driver the execution fails with the following 
 exception:
 {noformat}
 ERROR [SharedPool-Worker-1] 2015-03-24 18:10:59,391 QueryMessage.java:132 - 
 Unexpected error during query
 org.apache.cassandra.exceptions.FunctionExecutionException: execution of 
 'ks.overloaded[text]' failed: java.lang.IllegalArgumentException: No protocol 
 version matching integer version 4
 at 
 org.apache.cassandra.exceptions.FunctionExecutionException.create(FunctionExecutionException.java:35)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.udf.gen.Cksoverloaded_1.execute(Cksoverloaded_1.java)
  ~[na:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.executeInternal(FunctionCall.java:78)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.access$200(FunctionCall.java:34)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.execute(FunctionCall.java:176)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.prepare(FunctionCall.java:161)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.toTerm(SingleColumnRelation.java:108)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.newEQRestriction(SingleColumnRelation.java:143)
  ~[main/:na]
 at org.apache.cassandra.cql3.Relation.toRestriction(Relation.java:127) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.restrictions.StatementRestrictions.init(StatementRestrictions.java:126)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:787)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:740)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:246) 
 ~[main/:na]
 at 
 org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
  ~[main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:475)
  [main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:371)
  [main/:na]
 at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_71]
 at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
  [main/:na]
 at

[jira] [Resolved] (CASSANDRA-6809) Compressed Commit Log

2015-03-25 Thread Dave Brosius (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius resolved CASSANDRA-6809.
-
Resolution: Fixed

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: docs-impacting, performance
 Fix For: 3.0

 Attachments: ComitLogStress.java, logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8357) ArrayOutOfBounds in cassandra-stress with inverted exponential distribution


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8357:
---
Assignee: Benedict

 ArrayOutOfBounds in cassandra-stress with inverted exponential distribution
 ---

 Key: CASSANDRA-8357
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8357
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 6-node cassandra cluster (2.1.1) on debian.
Reporter: Jens Preußner
Assignee: Benedict
 Fix For: 2.1.4

 Attachments: CASSANDRA-8357-2.1.3.errlog.txt, 
 CASSANDRA-8357-2.1.3.log.txt


 When using the CQLstress example from GitHub 
 (https://github.com/apache/cassandra/blob/trunk/tools/cqlstress-example.yaml) 
 with an inverted exponential distribution in the insert-partitions field, 
 generated threads fail with
 Exception in thread Thread-20 java.lang.ArrayIndexOutOfBoundsException: 20 
 at 
 org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:307)
 See the gist https://gist.github.com/jenzopr/9edde53122554729c852 for the 
 typetest.yaml I used.
 The call was:
 cassandra-stress user profile=typetest.yaml ops\(insert=1\) -node $NODES



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9037) Terminal UDFs evaluated at prepare time throw protocol version error

2015-03-25 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-9037:
---
Reviewer: Tyler Hobbs

 Terminal UDFs evaluated at prepare time throw protocol version error
 

 Key: CASSANDRA-9037
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9037
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0


 When a pure function with only terminal arguments (or with no arguments) is 
 used in a where clause, it's executed at prepare time and 
 {{Server.CURRENT_VERSION}} passed as the protocol version for serialization 
 purposes. For native functions, this isn't a problem, but UDFs use classes in 
 the bundled java-driver-core jar for (de)serialization of args and return 
 values. When {{Server.CURRENT_VERSION}} is greater than the highest version 
 supported by the bundled java driver the execution fails with the following 
 exception:
 {noformat}
 ERROR [SharedPool-Worker-1] 2015-03-24 18:10:59,391 QueryMessage.java:132 - 
 Unexpected error during query
 org.apache.cassandra.exceptions.FunctionExecutionException: execution of 
 'ks.overloaded[text]' failed: java.lang.IllegalArgumentException: No protocol 
 version matching integer version 4
 at 
 org.apache.cassandra.exceptions.FunctionExecutionException.create(FunctionExecutionException.java:35)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.udf.gen.Cksoverloaded_1.execute(Cksoverloaded_1.java)
  ~[na:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.executeInternal(FunctionCall.java:78)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.access$200(FunctionCall.java:34)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.execute(FunctionCall.java:176)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.prepare(FunctionCall.java:161)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.toTerm(SingleColumnRelation.java:108)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.newEQRestriction(SingleColumnRelation.java:143)
  ~[main/:na]
 at org.apache.cassandra.cql3.Relation.toRestriction(Relation.java:127) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.restrictions.StatementRestrictions.init(StatementRestrictions.java:126)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:787)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:740)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:246) 
 ~[main/:na]
 at 
 org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
  ~[main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:475)
  [main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:371)
  [main/:na]
 at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_71]
 at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
  [main/:na]
 at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
 [main/:na]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: java.lang.IllegalArgumentException: No protocol version matching 
 integer version 4
 at 
 com.datastax.driver.core.ProtocolVersion.fromInt(ProtocolVersion.java:89) 
 ~[cassandra-driver-core-2.1.2.jar:na]
 at 
 org.apache.cassandra.cql3.functions.UDFunction.compose(UDFunction.java:177) 
 ~[main/:na]
 ... 25 common frames omitted
 {noformat}
 This is currently the case on trunk following the bump of 
 {{Server.CURRENT_VERSION}} to 4 by CASSANDRA-7660.



--
This message was sent by

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379897#comment-14379897
]

Ariel Weisberg commented on CASSANDRA-8670:
---

What tool are you using to review? github does a good job handling merge
commits and not show them as part of the diff and it doesn't show them as
individual commits.

You can also convert any github comparison to a single diff by adding .diff to
the URL. That's what I used to create a [new
branch|https://github.com/apache/cassandra/compare/trunk...aweisberg:C-8670-2?expand=1]

Large columns + NIO memory pooling causes excessive direct memory usage
---

Key: CASSANDRA-8670
URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 3.0

Attachments: largecolumn_test.py

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL

2015-03-25 Thread Jim Witschey (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379849#comment-14379849
 ] 

Jim Witschey commented on CASSANDRA-6237:
-

Ah, I see - my mistake. Thanks!

 Allow range deletions in CQL
 

 Key: CASSANDRA-6237
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benjamin Lerer
Priority: Minor
  Labels: cql, docs
 Fix For: 3.0

 Attachments: CASSANDRA-6237.txt


 We uses RangeTombstones internally in a number of places, but we could expose 
 more directly too. Typically, given a table like:
 {noformat}
 CREATE TABLE events (
 id text,
 created_at timestamp,
 content text,
 PRIMARY KEY (id, created_at)
 )
 {noformat}
 we could allow queries like:
 {noformat}
 DELETE FROM events WHERE id='someEvent' AND created_at  'Jan 3, 2013';
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO

2015-03-25 Thread Aaron Ploetz (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379863#comment-14379863
 ] 

Aaron Ploetz commented on CASSANDRA-8970:
-

[~thobbs] you're absolutely right.  Solving the interpretation of timestamps 
with {{COPY FROM}} is definitely the larger issue here.  My patch helps as a 
workaround to allow timestamps to be exported in a format that the current 
{{COPY FROM}} can understand.  But it becomes a nice to have if {{COPY FROM}} 
could interpret the default exported timestamps correctly.


 Allow custom time_format on cqlsh COPY TO
 -

 Key: CASSANDRA-8970
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Aaron Ploetz
Priority: Trivial
  Labels: cqlsh
 Fix For: 2.1.4

 Attachments: CASSANDRA-8970.patch

   Original Estimate: 4h
  Remaining Estimate: 4h

 When executing a COPY TO from cqlsh, the user is currently has no control 
 over the format of exported timestamp columns.  If the user has indicated a 
 {{time_format}} in their cqlshrc file, that format will be used.  Otherwise, 
 the system default format will be used.
 The problem comes into play when the timestamp format used on a COPY TO, is 
 not valid when the data is sent back into Cassandra with a COPY FROM.
 For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified 
 in their cqlshrc, COPY TO will format timestamp columns like this:
 {{userid|posttime|postcontent}}
 {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
 {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
 {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
 Executing a COPY FROM on that same file will produce an unable to coerce to 
 formatted date(long) error.
 Right now, the only way to change the way timestamps are formatted is to exit 
 cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.  
 The ability to specify a COPY option of TIME_FORMAT with a Python strftime 
 format, would allow the user to quickly alter the timestamp format for 
 export, without reconfiguring cqlsh.
 {{aploetz@cqlsh:stackoverflow COPY posts1 TO '/home/aploetz/posts1.csv' WITH 
 DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9026) Garbage in the sstable of the secondary index

2015-03-25 Thread Charles (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379884#comment-14379884
 ] 

Charles commented on CASSANDRA-9026:


Yes, but it does not work.
The unknown key and value is still in the sstable of secondary index.

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9026) Garbage in the sstable of the secondary index

2015-03-25 Thread Charles (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Charles updated CASSANDRA-9026:
---
Comment: was deleted

(was: Yes, but it does not work.
The unknown key and value is still in the sstable of secondary index.)

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9026) Garbage in the sstable of the secondary index

2015-03-25 Thread Charles (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379883#comment-14379883
 ] 

Charles commented on CASSANDRA-9026:


Yes, but it does not work.
The unknown key and value is still in the sstable of secondary index.

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started

2015-03-25 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379788#comment-14379788
 ] 

Yuki Morishita commented on CASSANDRA-8236:
---

This is addressed in CASSANDRA-9019.

 Delay node up and node added notifications until native protocol server 
 is started
 --

 Key: CASSANDRA-8236
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Stefania
 Fix For: 3.0

 Attachments: 8236.txt


 As discussed in CASSANDRA-7510, there is still a gap between when a node up 
 or node added notification may be sent to native protocol clients (in 
 response to a gossip event) and when the native protocol server is ready to 
 serve requests.
 Everything in between the call to {{StorageService.instance.initServer()}} 
 and creation of the native server in {{CassandraDaemon.setup()}} contributes 
 to this delay, but waiting for Gossip to settle introduces the biggest delay.
 We may need to introduce a STARTING gossip state for the period inbetween, 
 which is why this is scheduled for 3.0.  If there's a better option, though, 
 it may make sense to put this in 2.1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Lower log from error to debug when schema pull can't be executed

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 b296c55f9 - 9625910a5


Lower log from error to debug when schema pull can't be executed

Patch by Tyler Hobbs; reviewed by Aleksey Yeschenko for CASSANDRA-9032


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9625910a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9625910a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9625910a

Branch: refs/heads/cassandra-2.0
Commit: 9625910a533715e24211038ddd2776f5ec74ceae
Parents: b296c55
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:39:49 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:39:49 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e17e82f..293dc55 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.14:
+ * Lower logging level from ERROR to DEBUG when a scheduled schema pull
+   cannot be completed due to a node being down (CASSANDRA-9032)
  * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/src/java/org/apache/cassandra/service/MigrationTask.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationTask.java 
b/src/java/org/apache/cassandra/service/MigrationTask.java
index 0944c55..290465e 100644
--- a/src/java/org/apache/cassandra/service/MigrationTask.java
+++ b/src/java/org/apache/cassandra/service/MigrationTask.java
@@ -59,7 +59,7 @@ class MigrationTask extends WrappedRunnable
 
 if (!FailureDetector.instance.isAlive(endpoint))
 {
-logger.error(Can't send migration request: node {} is down., 
endpoint);
+logger.debug(Can't send schema pull request: node {} is down., 
endpoint);
 return;
 }

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/14327e4b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/14327e4b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/14327e4b

Branch: refs/heads/cassandra-2.1
Commit: 14327e4b9e120c1446573bfd121eabd14f16dc1a
Parents: f9c57a5 9625910
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:41:04 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:41:04 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/14327e4b/CHANGES.txt
--
diff --cc CHANGES.txt
index a8eda63,293dc55..9bc314d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,62 -1,6 +1,64 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add

[1/2] cassandra git commit: Lower log from error to debug when schema pull can't be executed

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 f9c57a597 - 14327e4b9


Lower log from error to debug when schema pull can't be executed

Patch by Tyler Hobbs; reviewed by Aleksey Yeschenko for CASSANDRA-9032


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9625910a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9625910a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9625910a

Branch: refs/heads/cassandra-2.1
Commit: 9625910a533715e24211038ddd2776f5ec74ceae
Parents: b296c55
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:39:49 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:39:49 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e17e82f..293dc55 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.14:
+ * Lower logging level from ERROR to DEBUG when a scheduled schema pull
+   cannot be completed due to a node being down (CASSANDRA-9032)
  * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/src/java/org/apache/cassandra/service/MigrationTask.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationTask.java 
b/src/java/org/apache/cassandra/service/MigrationTask.java
index 0944c55..290465e 100644
--- a/src/java/org/apache/cassandra/service/MigrationTask.java
+++ b/src/java/org/apache/cassandra/service/MigrationTask.java
@@ -59,7 +59,7 @@ class MigrationTask extends WrappedRunnable
 
 if (!FailureDetector.instance.isAlive(endpoint))
 {
-logger.error(Can't send migration request: node {} is down., 
endpoint);
+logger.debug(Can't send schema pull request: node {} is down., 
endpoint);
 return;
 }

[jira] [Commented] (CASSANDRA-7807) Push notification when tracing completes for an operation

[
https://issues.apache.org/jira/browse/CASSANDRA-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380109#comment-14380109
]

Tyler Hobbs commented on CASSANDRA-7807:

bq. Is it OK to send the notification to the connection that originated the
trace request even if it has not registered for the event (unlike other events).

I would definitely prefer to only send the event if the connection registered
for it. I'm not sure if all of the drivers will properly handle receiving an
event type they didn't register for. Also, this should only be allowed on v4
protocol connections (it looks like the patch doesn't enforce this yet).

bq. Should other connections who have registered for this event receive
notifications at all?

No. Only the connection that executed the traced query should get a
notification.

bq. What should happen if the trace is started because of probabilistic tracing
rather than a request from the client - should the client still receive the
notification?

I don't think so. Probabilistic tracing is useful for getting an overview of
how all of your queries are performing (usually to find performance problems).
The user will generally look through the tracing sessions manually afterward,
rather than synchronously fetching them through the driver. Plus, the driver
isn't aware ahead of time that the query it's executing may be traced.

Push notification when tracing completes for an operation
-

Attachments: 7807.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] cassandra git commit: Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 495ae9c7e - 7c98030a7
  refs/heads/cassandra-2.1 86a380294 - 768f83d25
  refs/heads/trunk c15ac661d - ec9d0d1b7


Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Patch by Jerimiah Jordan, reviewed by brandonwilliams for CASSANDRA-7533


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c98030a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c98030a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c98030a

Branch: refs/heads/cassandra-2.0
Commit: 7c98030a7d4551780377cfa6b3442868b62b4b20
Parents: 495ae9c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:32:57 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:32:57 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bcc59fc..fecc7e3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
  * (cqlsh) Allow increasing CSV field size limit through

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index 579b6ee..5687138 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -47,7 +47,7 @@ import org.cliffc.high_scale_lib.NonBlockingHashSet;
 public class CommitLogReplayer
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogReplayer.class);
-private static final int MAX_OUTSTANDING_REPLAY_COUNT = 1024;
+private static final int MAX_OUTSTANDING_REPLAY_COUNT = 
Integer.getInteger(cassandra.commitlog_max_outstanding_replay_count, 1024);
 
 private final SetKeyspace keyspacesRecovered;
 private final ListFuture? futures;

[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9c57a59
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9c57a59
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9c57a59

Branch: refs/heads/trunk
Commit: f9c57a597ab738d13273dd61cdad6a5068e7561d
Parents: 768f83d b296c55
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:11:25 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:11:25 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9c57a59/CHANGES.txt
--
diff --cc CHANGES.txt
index f4d4008,e17e82f..a8eda63
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,64 -1,8 +1,65 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables

[2/6] cassandra git commit: Fix MOVED_NODE client event

Fix MOVED_NODE client event

Patch by Stefania, reviewed by brandonwilliams for CASSANDRA-8516


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b296c55f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b296c55f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b296c55f

Branch: refs/heads/cassandra-2.1
Commit: b296c55f956c6ef07c8330dc28ef8c351e5bcfe2
Parents: 7c98030
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:10:17 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:10:17 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fecc7e3..e17e82f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 622380e..d997459 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1618,7 +1618,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (!localTokensToRemove.isEmpty())
 SystemKeyspace.updateLocalTokens(Collections.TokenemptyList(), 
localTokensToRemove);
 
-if (isMoving)
+if (isMoving || operationMode == Mode.MOVING)
 {
 tokenMetadata.removeFromMoving(endpoint);

[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e1302e7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e1302e7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e1302e7

Branch: refs/heads/trunk
Commit: 1e1302e7756d8b1cbfa41557508992e98665d0f5
Parents: ec9d0d1 f9c57a5
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:11:39 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:11:39 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e1302e7/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e1302e7/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 996791e,7c1d2a2..a75c08c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1753,11 -1650,15 +1753,11 @@@ public class StorageService extends Not
  if (!localTokensToRemove.isEmpty())
  SystemKeyspace.updateLocalTokens(Collections.TokenemptyList(), 
localTokensToRemove);
  
- if (isMoving)
+ if (isMoving || operationMode == Mode.MOVING)
  {
  tokenMetadata.removeFromMoving(endpoint);
 -
 -if (!isClientMode)
 -{
 -for (IEndpointLifecycleSubscriber subscriber : 
lifecycleSubscribers)
 -subscriber.onMove(endpoint);
 -}
 +for (IEndpointLifecycleSubscriber subscriber : 
lifecycleSubscribers)
 +subscriber.onMove(endpoint);
  }
  else
  {

[1/6] cassandra git commit: Fix MOVED_NODE client event

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 7c98030a7 - b296c55f9
  refs/heads/cassandra-2.1 768f83d25 - f9c57a597
  refs/heads/trunk ec9d0d1b7 - 1e1302e77


Fix MOVED_NODE client event

Patch by Stefania, reviewed by brandonwilliams for CASSANDRA-8516


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b296c55f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b296c55f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b296c55f

Branch: refs/heads/cassandra-2.0
Commit: b296c55f956c6ef07c8330dc28ef8c351e5bcfe2
Parents: 7c98030
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:10:17 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:10:17 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fecc7e3..e17e82f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 622380e..d997459 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1618,7 +1618,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (!localTokensToRemove.isEmpty())
 SystemKeyspace.updateLocalTokens(Collections.TokenemptyList(), 
localTokensToRemove);
 
-if (isMoving)
+if (isMoving || operationMode == Mode.MOVING)
 {
 tokenMetadata.removeFromMoving(endpoint);

[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9c57a59
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9c57a59
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9c57a59

Branch: refs/heads/cassandra-2.1
Commit: f9c57a597ab738d13273dd61cdad6a5068e7561d
Parents: 768f83d b296c55
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:11:25 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:11:25 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9c57a59/CHANGES.txt
--
diff --cc CHANGES.txt
index f4d4008,e17e82f..a8eda63
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,64 -1,8 +1,65 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables

[jira] [Commented] (CASSANDRA-9026) Garbage in the sstable of the secondary index


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380011#comment-14380011
 ] 

Philip Thompson commented on CASSANDRA-9026:


Re-reading this ticket, I'm not sure what the bug here is. You're seeing data 
appear that you didn't write, but doesn't appear to be corruption?

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9039) CommitLog compressed configuration not run in several unit tests

Ariel Weisberg created CASSANDRA-9039:
-

 Summary: CommitLog compressed configuration not run in several 
unit tests
 Key: CASSANDRA-9039
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9039
 Project: Cassandra
  Issue Type: Bug
Reporter: Ariel Weisberg


CommitLogTest, RecoveryManagerTest, RecoveryManagerTruncateTest, 
RecoveryManager2Test, RecoveryManager3Test

Some or all of these should be run with a commit log configured to use 
compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9040) o.a.c.net.* has no unit test coverage and no coverage with compression

Ariel Weisberg created CASSANDRA-9040:
-

 Summary: o.a.c.net.* has no unit test coverage and no coverage 
with compression
 Key: CASSANDRA-9040
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9040
 Project: Cassandra
  Issue Type: Bug
Reporter: Ariel Weisberg


I think the unit test issue is minor since this code is validated as part of 
running  everything on a regular basis.

I think we do need a quick test that shows that the database starts and runs 
data correctly with compression enabled and that the config options for none, 
intradc, all work.

Further validation would be done in a harness test that validates the kitchen 
sink against the different compression configurations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7410:

Reviewer: Piotr Kołaczkowski  (was: Brandon Williams)

 Pig support for BulkOutputFormat as a parameter in url
 --

 Key: CASSANDRA-7410
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Priority: Minor
 Fix For: 2.0.14

 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 
 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, 
 CASSANDRA-7410-v2-2.1-branch.txt


 Add BulkOutputFormat support in Pig url



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8576) Primary Key Pushdown For Hadoop


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8576:

Reviewer: Piotr Kołaczkowski  (was: Brandon Williams)

 Primary Key Pushdown For Hadoop
 ---

 Key: CASSANDRA-8576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8576
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Russell Alexander Spitzer
Assignee: Alex Liu
 Fix For: 2.1.4

 Attachments: 8576-2.1-branch.txt, 8576-trunk.txt


 I've heard reports from several users that they would like to have predicate 
 pushdown functionality for hadoop (Hive in particular) based services. 
 Example usecase
 Table with wide partitions, one per customer
 Application team has HQL they would like to run on a single customer
 Currently time to complete scales with number of customers since Input Format 
 can't pushdown primary key predicate
 Current implementation requires a full table scan (since it can't recognize 
 that a single partition was specified)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9032) Reduce logging level for MigrationTask abort due to down node from ERROR to INFO


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380058#comment-14380058
 ] 

Tyler Hobbs commented on CASSANDRA-9032:


I would guess that the difference is due to how quickly the tests are running.  
This particular task is scheduled with a 60s delay, so if the trunk tests are, 
for example, taking longer to stop nodes, that might explain why we're seeing 
it there.

If there _is_ a problem with schema migrations (or something else that might 
cause this), then hopefully tests that are more specific to that will catch it. 
 Right now it's too hard to tell where we actually have problems on trunk due 
to unrelated failures like this, so the alternative is basically disabling 
those tests for now.

 Reduce logging level for MigrationTask abort due to down node from ERROR to 
 INFO
 

 Key: CASSANDRA-9032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9032
 Project: Cassandra
  Issue Type: Improvement
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.4, 2.0.14

 Attachments: 9032.txt


 A lot of the dtests are failing during Jenkins runs due to the following 
 error message in the logs:
 {noformat}
 ERROR [MigrationStage:1] 2015-03-24 20:02:03,464 MigrationTask.java:62 - 
 Can't send migration request: node /127.0.0.3 is down.\n]
 {noformat}
 This log message happens when a schema pull is scheduled, but the target 
 endpoint is down when the scheduled task actually runs.  The failing dtests 
 generally stop a node as part of the test, which results in this.
 I believe the log message should be moved from ERROR to INFO (or perhaps even 
 DEBUG).  This isn't an unexpected type of problem (nodes go down all the 
 time), and it's not actionable by the user.  This would also have the nice 
 side effect of fixing the dtests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8561) Tombstone log warning does not log partition key

2015-03-25 Thread Lyuben Todorov (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lyuben Todorov updated CASSANDRA-8561:
--
Attachment: trunk-1427288702-8561-v3.diff
cassandra-2.1-1427290549-8561-v3.diff

v3 should fix the things [~iamaleksey] pointed out:
- No yaml / -D option added
- fixed the rebase mistake for 2.1
- using String.format to display first 512 chars of the key and the slices
- added logger.isWarnEnabled() to allow for skipping the warn code path.
- import fix in trunk. 

 Tombstone log warning does not log partition key
 

 Key: CASSANDRA-8561
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8561
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Datastax DSE 4.5
Reporter: Jens Rantil
Assignee: Lyuben Todorov
  Labels: logging
 Fix For: 2.1.4

 Attachments: cassandra-2.1-1427196372-8561-v2.diff, 
 cassandra-2.1-1427290549-8561-v3.diff, cassandra-2.1-8561.diff, 
 cassandra-2.1-head-1427124485-8561.diff, 
 cassandra-trunk-head-1427125869-8561.diff, trunk-1427195046-8561-v2.diff, 
 trunk-1427288702-8561-v3.diff


 AFAIK, the tombstone warning in system.log does not contain the primary key. 
 See: https://gist.github.com/JensRantil/44204676f4dbea79ea3a
 Including it would help a lot in diagnosing why the (CQL) row has so many 
 tombstones.
 Let me know if I have misunderstood something.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8808) CQLSSTableWriter: close does not work + more than one table throws ex

2015-03-25 Thread Sebastian YEPES FERNANDEZ (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380046#comment-14380046
 ] 

Sebastian YEPES FERNANDEZ commented on CASSANDRA-8808:
--

Any change this could be included in the  2.1.4 branch?

 CQLSSTableWriter: close does not work + more than one table throws ex
 -

 Key: CASSANDRA-8808
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8808
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sebastian YEPES FERNANDEZ
Assignee: Benjamin Lerer
  Labels: cql
 Fix For: 2.1.4, 2.0.14

 Attachments: CASSANDRA-8808-2.0-V2.txt, CASSANDRA-8808-2.0.txt, 
 CASSANDRA-8808-2.1-V2.txt, CASSANDRA-8808-2.1.txt, 
 CASSANDRA-8808-trunk-V2.txt, CASSANDRA-8808-trunk.txt


 I have encountered the following two issues:
  - When closing the CQLSSTableWriter it just hangs the process and does 
 nothing. (https://issues.apache.org/jira/browse/CASSANDRA-8281)
  - When writing more than one table throws ex. 
 (https://issues.apache.org/jira/browse/CASSANDRA-8251)
 These issue can be reproduced with the following code:
 {code:title=test.java|borderStyle=solid}
 import org.apache.cassandra.config.Config;
 import org.apache.cassandra.io.sstable.CQLSSTableWriter;
 public static void main(String[] args) {
   Config.setClientMode(true);
   CQLSSTableWriter w1 = CQLSSTableWriter.builder()
 .inDirectory(/tmp/kspc/t1)
 .forTable(CREATE TABLE kspc.t1 ( id  int, PRIMARY KEY (id));)
 .using(INSERT INTO kspc.t1 (id) VALUES ( ? );)
 .build();
   CQLSSTableWriter w2 = CQLSSTableWriter.builder()
 .inDirectory(/tmp/kspc/t2)
 .forTable(CREATE TABLE kspc.t2 ( id  int, PRIMARY KEY (id));)
 .using(INSERT INTO kspc.t2 (id) VALUES ( ? );)
 .build();
   try {
 w1.addRow(1);
 w2.addRow(1);
 w1.close();
 w2.close();
   } catch (Exception e) {
 System.out.println(e);
   }
 }
 {code}
 {code:title=The error|borderStyle=solid}
 Exception in thread main java.lang.ExceptionInInitializerError
 at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:324)
 at org.apache.cassandra.db.Keyspace.init(Keyspace.java:277)
 at org.apache.cassandra.db.Keyspace.open(Keyspace.java:119)
 at org.apache.cassandra.db.Keyspace.open(Keyspace.java:96)
 at 
 org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:101)
 at 
 org.apache.cassandra.io.sstable.CQLSSTableWriter.rawAddRow(CQLSSTableWriter.java:226)
 at 
 org.apache.cassandra.io.sstable.CQLSSTableWriter.addRow(CQLSSTableWriter.java:145)
 at 
 org.apache.cassandra.io.sstable.CQLSSTableWriter.addRow(CQLSSTableWriter.java:120)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at 
 org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:189)
 at 
 org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
 at 
 org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
 at 
 org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
 at 
 org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
 at 
 com.allthingsmonitoring.utils.BulkDataLoader.main(BulkDataLoader.groovy:415)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1053)
 at 
 org.apache.cassandra.db.ColumnFamilyStore.clinit(ColumnFamilyStore.java:85)
 ... 18 more
 {code}
 I have just tested the in the cassandra-2.1 branch and the issue still 
 persists.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9026) Garbage in the sstable of the secondary index

2015-03-25 Thread Carl Yeksigian (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380062#comment-14380062
 ] 

Carl Yeksigian commented on CASSANDRA-9026:
---

Looks like it would be fixed by CASSANDRA-5174, but that's for 3.0.

There isn't a good way to fix the underlying index data, because we don't have 
a command which will drop the index sstables and rebuild them before 3.0. You 
can drop the index and recreate, but there will be some time when the index 
isn't created, and when the index is being built.

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9032) Reduce logging level for MigrationTask abort due to down node from ERROR to INFO


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380063#comment-14380063
 ] 

Philip Thompson commented on CASSANDRA-9032:


Okay. I agree that the proposed solution is better than disabling the tests.

 Reduce logging level for MigrationTask abort due to down node from ERROR to 
 INFO
 

 Key: CASSANDRA-9032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9032
 Project: Cassandra
  Issue Type: Improvement
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.4, 2.0.14

 Attachments: 9032.txt


 A lot of the dtests are failing during Jenkins runs due to the following 
 error message in the logs:
 {noformat}
 ERROR [MigrationStage:1] 2015-03-24 20:02:03,464 MigrationTask.java:62 - 
 Can't send migration request: node /127.0.0.3 is down.\n]
 {noformat}
 This log message happens when a schema pull is scheduled, but the target 
 endpoint is down when the scheduled task actually runs.  The failing dtests 
 generally stop a node as part of the test, which results in this.
 I believe the log message should be moved from ERROR to INFO (or perhaps even 
 DEBUG).  This isn't an unexpected type of problem (nodes go down all the 
 time), and it's not actionable by the user.  This would also have the nice 
 side effect of fixing the dtests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ec9d0d1b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ec9d0d1b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ec9d0d1b

Branch: refs/heads/trunk
Commit: ec9d0d1b7e535a4d8dda2ef0601f3467c8a7e0fe
Parents: c15ac66 768f83d
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:36:55 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:36:55 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec9d0d1b/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec9d0d1b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--

[jira] [Updated] (CASSANDRA-9036) disk full when running cleanup (on a far from full disk)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9036:
---
Assignee: Marcus Eriksson

 disk full when running cleanup (on a far from full disk)
 --

 Key: CASSANDRA-9036
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9036
 Project: Cassandra
  Issue Type: Bug
Reporter: Erik Forsberg
Assignee: Marcus Eriksson

 I'm trying to run cleanup, but get this:
 {noformat}
  INFO [CompactionExecutor:18] 2015-03-25 10:29:16,355 CompactionManager.java 
 (line 564) Cleaning up 
 SSTableReader(path='/cassandra/production/Data_daily/production-Data_daily-jb-4345750-Data.db')
 ERROR [CompactionExecutor:18] 2015-03-25 10:29:16,664 CassandraDaemon.java 
 (line 199) Exception in thread Thread[CompactionExecutor:18,1,main]
 java.io.IOException: disk full
 at 
 org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:567)
 at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:63)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$5.perform(CompactionManager.java:281)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:225)
 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {noformat}
 Now that's odd, since:
 * Disk has some 680G left
 * The sstable it's trying to cleanup is far less than 680G:
 {noformat}
 # ls -lh *4345750*
 -rw-r--r-- 1 cassandra cassandra  64M Mar 21 04:42 
 production-Data_daily-jb-4345750-CompressionInfo.db
 -rw-r--r-- 1 cassandra cassandra 219G Mar 21 04:42 
 production-Data_daily-jb-4345750-Data.db
 -rw-r--r-- 1 cassandra cassandra 503M Mar 21 04:42 
 production-Data_daily-jb-4345750-Filter.db
 -rw-r--r-- 1 cassandra cassandra  42G Mar 21 04:42 
 production-Data_daily-jb-4345750-Index.db
 -rw-r--r-- 1 cassandra cassandra 5.9K Mar 21 04:42 
 production-Data_daily-jb-4345750-Statistics.db
 -rw-r--r-- 1 cassandra cassandra  81M Mar 21 04:42 
 production-Data_daily-jb-4345750-Summary.db
 -rw-r--r-- 1 cassandra cassandra   79 Mar 21 04:42 
 production-Data_daily-jb-4345750-TOC.txt
 {noformat}
 Sure, it's large, but it's not 680G. 
 No other compactions are running on that server. I'm getting this on 12 / 56 
 servers right now. 
 Could it be some bug in the calculation of the expected size of the new 
 sstable, perhaps? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8236) Delay node up and node added notifications until native protocol server is started


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1438#comment-1438
 ] 

Brandon Williams commented on CASSANDRA-8236:
-

I think this looks pretty good, but I'd like to see some dead state protection, 
just to be cautious, because I don't want dead states to trigger this.

 Delay node up and node added notifications until native protocol server 
 is started
 --

 Key: CASSANDRA-8236
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Stefania
 Fix For: 3.0

 Attachments: 8236.txt


 As discussed in CASSANDRA-7510, there is still a gap between when a node up 
 or node added notification may be sent to native protocol clients (in 
 response to a gossip event) and when the native protocol server is ready to 
 serve requests.
 Everything in between the call to {{StorageService.instance.initServer()}} 
 and creation of the native server in {{CassandraDaemon.setup()}} contributes 
 to this delay, but waiting for Gossip to settle introduces the biggest delay.
 We may need to introduce a STARTING gossip state for the period inbetween, 
 which is why this is scheduled for 3.0.  If there's a better option, though, 
 it may make sense to put this in 2.1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/6] cassandra git commit: Fix MOVED_NODE client event

Fix MOVED_NODE client event

Patch by Stefania, reviewed by brandonwilliams for CASSANDRA-8516


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b296c55f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b296c55f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b296c55f

Branch: refs/heads/trunk
Commit: b296c55f956c6ef07c8330dc28ef8c351e5bcfe2
Parents: 7c98030
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 10:10:17 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 10:10:17 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fecc7e3..e17e82f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b296c55f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 622380e..d997459 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1618,7 +1618,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (!localTokensToRemove.isEmpty())
 SystemKeyspace.updateLocalTokens(Collections.TokenemptyList(), 
localTokensToRemove);
 
-if (isMoving)
+if (isMoving || operationMode == Mode.MOVING)
 {
 tokenMetadata.removeFromMoving(endpoint);

[jira] [Updated] (CASSANDRA-7533) Let MAX_OUTSTANDING_REPLAY_COUNT be configurable


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7533:

Reproduced In: 2.0.9, 2.0.6  (was: 2.0.6, 2.0.9)
Fix Version/s: 2.1.4

 Let MAX_OUTSTANDING_REPLAY_COUNT be configurable
 

 Key: CASSANDRA-7533
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7533
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremiah Jordan
Assignee: Jeremiah Jordan
Priority: Minor
 Fix For: 2.1.4, 2.0.14

 Attachments: 0001-CASSANDRA-7533.txt


 There are some workloads where commit log replay will run into contention 
 issues with multiple things updating the same partition.  Through some 
 testing it was found that lowering CommitLogReplayer.java 
 MAX_OUTSTANDING_REPLAY_COUNT can help with this issue.
 The calculations added in CASSANDRA-6655 are one such place things get 
 bottlenecked.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/6] cassandra git commit: Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Patch by Jerimiah Jordan, reviewed by brandonwilliams for CASSANDRA-7533


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c98030a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c98030a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c98030a

Branch: refs/heads/cassandra-2.1
Commit: 7c98030a7d4551780377cfa6b3442868b62b4b20
Parents: 495ae9c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:32:57 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:32:57 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bcc59fc..fecc7e3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
  * (cqlsh) Allow increasing CSV field size limit through

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index 579b6ee..5687138 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -47,7 +47,7 @@ import org.cliffc.high_scale_lib.NonBlockingHashSet;
 public class CommitLogReplayer
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogReplayer.class);
-private static final int MAX_OUTSTANDING_REPLAY_COUNT = 1024;
+private static final int MAX_OUTSTANDING_REPLAY_COUNT = 
Integer.getInteger(cassandra.commitlog_max_outstanding_replay_count, 1024);
 
 private final SetKeyspace keyspacesRecovered;
 private final ListFuture? futures;

[3/6] cassandra git commit: Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Allow overriding MAX_OUTSTANDING_REPLAY_COUNT

Patch by Jerimiah Jordan, reviewed by brandonwilliams for CASSANDRA-7533


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c98030a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c98030a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c98030a

Branch: refs/heads/trunk
Commit: 7c98030a7d4551780377cfa6b3442868b62b4b20
Parents: 495ae9c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:32:57 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:32:57 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index bcc59fc..fecc7e3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
  * (cqlsh) Allow increasing CSV field size limit through

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c98030a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index 579b6ee..5687138 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -47,7 +47,7 @@ import org.cliffc.high_scale_lib.NonBlockingHashSet;
 public class CommitLogReplayer
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CommitLogReplayer.class);
-private static final int MAX_OUTSTANDING_REPLAY_COUNT = 1024;
+private static final int MAX_OUTSTANDING_REPLAY_COUNT = 
Integer.getInteger(cassandra.commitlog_max_outstanding_replay_count, 1024);
 
 private final SetKeyspace keyspacesRecovered;
 private final ListFuture? futures;

[jira] [Commented] (CASSANDRA-8110) Make streaming backwards compatible


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379984#comment-14379984
 ] 

Aleksey Yeschenko commented on CASSANDRA-8110:
--

[~yukim] would CASSANDRA-8928 make it easier?

 Make streaming backwards compatible
 ---

 Key: CASSANDRA-8110
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8110
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
Assignee: Yuki Morishita
 Fix For: 3.0


 To be able to seamlessly upgrade clusters we need to make it possible to 
 stream files between nodes with different StreamMessage.CURRENT_VERSION



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9032) Reduce logging level for MigrationTask abort due to down node from ERROR to INFO


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379983#comment-14379983
 ] 

Philip Thompson commented on CASSANDRA-9032:


Are we unconcerned that this has begun happening with incredible frequency on 
trunk, but is not happening in 2.0 or 2.1?

 Reduce logging level for MigrationTask abort due to down node from ERROR to 
 INFO
 

 Key: CASSANDRA-9032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9032
 Project: Cassandra
  Issue Type: Improvement
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.4, 2.0.14

 Attachments: 9032.txt


 A lot of the dtests are failing during Jenkins runs due to the following 
 error message in the logs:
 {noformat}
 ERROR [MigrationStage:1] 2015-03-24 20:02:03,464 MigrationTask.java:62 - 
 Can't send migration request: node /127.0.0.3 is down.\n]
 {noformat}
 This log message happens when a schema pull is scheduled, but the target 
 endpoint is down when the scheduled task actually runs.  The failing dtests 
 generally stop a node as part of the test, which results in this.
 I believe the log message should be moved from ERROR to INFO (or perhaps even 
 DEBUG).  This isn't an unexpected type of problem (nodes go down all the 
 time), and it's not actionable by the user.  This would also have the nice 
 side effect of fixing the dtests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9038) Atomic batches and single row atomicity appear to have no test coverage

Ariel Weisberg created CASSANDRA-9038:
-

 Summary: Atomic batches and single row atomicity appear to have no 
test coverage
 Key: CASSANDRA-9038
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9038
 Project: Cassandra
  Issue Type: Bug
Reporter: Ariel Weisberg


Leaving the solution to this up to the assignee. It seems like this is a 
guarantee that should be checked.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/768f83d2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/768f83d2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/768f83d2

Branch: refs/heads/cassandra-2.1
Commit: 768f83d2561372be8bbc9aeb42560c77c5c51999
Parents: 86a3802 7c98030
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:36:40 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:36:40 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/768f83d2/CHANGES.txt
--
diff --cc CHANGES.txt
index 02bd7b0,fecc7e3..f4d4008
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,63 -1,7 +1,64 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + *

[jira] [Updated] (CASSANDRA-9033) Upgrading from 2.1.1 to 2.1.3 with LCS and many sstable files makes nodes unresponsive


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9033:
---
Assignee: Marcus Eriksson

 Upgrading from 2.1.1 to 2.1.3 with LCS  and many sstable files makes nodes 
 unresponsive
 ---

 Key: CASSANDRA-9033
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9033
 Project: Cassandra
  Issue Type: Bug
 Environment: * Ubuntu 14.04.2 - Linux ip-10-0-2-122 3.13.0-46-generic 
 #79-Ubuntu SMP Tue Mar 10 20:06:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
 * EC2 m2-xlarge instances [4cpu, 16GB RAM, 1TB storage on 3 platters]
 * 12 nodes running a mix of 2.1.1 and 2.1.3
 * 8GB stack size with offheap objects
Reporter: Brent Haines
Assignee: Marcus Eriksson
Priority: Blocker
 Attachments: cassandra-env.sh, cassandra.yaml, system.log.1.zip


 We have an Event Log table using LCS that has grown fast. There are more than 
 100K sstable files that are around 1KB. Increasing compactors and adjusting 
 compaction throttling upward doesn't make a difference. It has been running 
 great though until we upgraded to 2.1.3. Those nodes needed more RAM for the 
 stack (12 GB) to even have a prayer of responding to queries. They bog down 
 and become unresponsive. There are no GC messages that I can see, and no 
 compaction either. 
 The only work-around I have found is to decommission, blow away the big CF 
 and rejoin. That happens in about 20 minutes and everything is freaking happy 
 again. The size of the files is more like what I'd expect as well. 
 Our schema: 
 {code}
 cqlsh describe columnfamily data.stories
 CREATE TABLE data.stories (
 id timeuuid PRIMARY KEY,
 action_data timeuuid,
 action_name text,
 app_id timeuuid,
 app_instance_id timeuuid,
 data maptext, text,
 objects settimeuuid,
 time_stamp timestamp,
 user_id timeuuid
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = 'Stories represent the timeline and are placed in the 
 dashboard for the brand manager to see'
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.1
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 cqlsh 
 {code}
 There were no log entries that stood out. It pretty much consisted of x is 
 down x is up repeated ad infinitum. I have attached the zipped system.log 
 that has the situation after the upgrade and then after I stopped, removed 
 system, system_traces, OpsCenter, and data/stories-/* and restarted. 
 It has rejoined the cluster now and is busy read-repairing to recover its 
 data.
 On another note, we see a lot of this during repair now (on all the nodes): 
 {code}
 ERROR [AntiEntropySessions:5] 2015-03-24 20:03:10,207 RepairSession.java:303 
 - [repair #c5043c40-d260-11e4-a2f2-8bb3e2bbdb35] session completed with the 
 following error
 java.io.IOException: Failed during snapshot creation.
 at 
 org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:344)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
 at 
 org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:146) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
 at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172) 
 ~[guava-16.0.jar:na]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_55]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_55]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
 ERROR [AntiEntropySessions:5] 2015-03-24 20:03:10,208 
 CassandraDaemon.java:167 - Exception in thread 
 Thread[AntiEntropySessions:5,5,RMI Runtime]
 java.lang.RuntimeException: java.io.IOException: Failed during snapshot 
 creation.
 at com.google.common.base.Throwables.propagate(Throwables.java:160) 
 ~[guava-16.0.jar:na]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 ~[na:1.7.0_55]
 at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
 ~[na:1.7.0_55]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_55]

[1/3] cassandra git commit: Lower log from error to debug when schema pull can't be executed

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1e1302e77 - 843934548


Lower log from error to debug when schema pull can't be executed

Patch by Tyler Hobbs; reviewed by Aleksey Yeschenko for CASSANDRA-9032


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9625910a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9625910a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9625910a

Branch: refs/heads/trunk
Commit: 9625910a533715e24211038ddd2776f5ec74ceae
Parents: b296c55
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:39:49 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:39:49 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index e17e82f..293dc55 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
 2.0.14:
+ * Lower logging level from ERROR to DEBUG when a scheduled schema pull
+   cannot be completed due to a node being down (CASSANDRA-9032)
  * Fix MOVED_NODE client event (CASSANDRA-8516)
  * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9625910a/src/java/org/apache/cassandra/service/MigrationTask.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationTask.java 
b/src/java/org/apache/cassandra/service/MigrationTask.java
index 0944c55..290465e 100644
--- a/src/java/org/apache/cassandra/service/MigrationTask.java
+++ b/src/java/org/apache/cassandra/service/MigrationTask.java
@@ -59,7 +59,7 @@ class MigrationTask extends WrappedRunnable
 
 if (!FailureDetector.instance.isAlive(endpoint))
 {
-logger.error(Can't send migration request: node {} is down., 
endpoint);
+logger.debug(Can't send schema pull request: node {} is down., 
endpoint);
 return;
 }

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84393454
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84393454
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84393454

Branch: refs/heads/trunk
Commit: 8439345487cff581e3c37110bfcefeaa542a72c3
Parents: 1e1302e 14327e4
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:41:25 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:41:25 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/84393454/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/84393454/src/java/org/apache/cassandra/service/MigrationTask.java
--

[jira] [Resolved] (CASSANDRA-9026) Garbage in the sstable of the secondary index


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-9026.

Resolution: Duplicate

As Carl said, it does look like this is fixed in 3.0, and will not be 
backported. Until then, an alternate solution is to drop and rebuild the index.

 Garbage in the sstable of the secondary index
 -

 Key: CASSANDRA-9026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9026
 Project: Cassandra
  Issue Type: Bug
 Environment: CentOS 6.5
Reporter: Charles

 We create a secondary index for a specific column.
 After running a few months, by using sstable2json tool, we found there are 
 some unknown keys which we've never assigned are stored in the sstable of the 
 secondary index. The key value is incremental hexbyte, for example, 55012157, 
 55012158.
 It can not be removed by nodetool repair/compact/cleanup.
 When the sstable of the secondary index become larger, the read performance 
 is dropped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/14327e4b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/14327e4b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/14327e4b

Branch: refs/heads/trunk
Commit: 14327e4b9e120c1446573bfd121eabd14f16dc1a
Parents: f9c57a5 9625910
Author: Tyler Hobbs ty...@datastax.com
Authored: Wed Mar 25 10:41:04 2015 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Mar 25 10:41:04 2015 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/service/MigrationTask.java | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/14327e4b/CHANGES.txt
--
diff --cc CHANGES.txt
index a8eda63,293dc55..9bc314d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,62 -1,6 +1,64 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add nodetool

[Cassandra Wiki] Trivial Update of JmxSecurity by JakeLuciani

2015-03-25 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The JmxSecurity page has been changed by JakeLuciani:
https://wiki.apache.org/cassandra/JmxSecurity?action=diffrev1=6rev2=7

  at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:1099)
  }}} 
  
+ 
+ == JMX SSL ==
+ 
+ Setting up SSL for JMX is more involved, a good resource for this is [[here | 
https://www.lullabot.com/blog/article/monitor-java-jmx]]
+

[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/768f83d2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/768f83d2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/768f83d2

Branch: refs/heads/trunk
Commit: 768f83d2561372be8bbc9aeb42560c77c5c51999
Parents: 86a3802 7c98030
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Mar 25 09:36:40 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Mar 25 09:36:40 2015 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/768f83d2/CHANGES.txt
--
diff --cc CHANGES.txt
index 02bd7b0,fecc7e3..f4d4008
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,63 -1,7 +1,64 @@@
 -2.0.14:
 +2.1.4
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress

[jira] [Assigned] (CASSANDRA-7032) Improve vnode allocation


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-7032:
---

Assignee: Brandon Williams  (was: Branimir Lambov)

 Improve vnode allocation
 

 Key: CASSANDRA-7032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Brandon Williams
  Labels: performance, vnodes
 Fix For: 3.0

 Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java, 
 TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java, 
 TestVNodeAllocation.java


 It's been known for a little while that random vnode allocation causes 
 hotspots of ownership. It should be possible to improve dramatically on this 
 with deterministic allocation. I have quickly thrown together a simple greedy 
 algorithm that allocates vnodes efficiently, and will repair hotspots in a 
 randomly allocated cluster gradually as more nodes are added, and also 
 ensures that token ranges are fairly evenly spread between nodes (somewhat 
 tunably so). The allocation still permits slight discrepancies in ownership, 
 but it is bound by the inverse of the size of the cluster (as opposed to 
 random allocation, which strangely gets worse as the cluster size increases). 
 I'm sure there is a decent dynamic programming solution to this that would be 
 even better.
 If on joining the ring a new node were to CAS a shared table where a 
 canonical allocation of token ranges lives after running this (or a similar) 
 algorithm, we could then get guaranteed bounds on the ownership distribution 
 in a cluster. This will also help for CASSANDRA-6696.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9041) Allow a PerRowSecondaryIndex to perform it's index operation before the underlying data has been mutated

2015-03-25 Thread Bryn Cooke (JIRA)

Bryn Cooke created CASSANDRA-9041:
-

 Summary: Allow a PerRowSecondaryIndex to perform it's index 
operation before the underlying data has been mutated
 Key: CASSANDRA-9041
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9041
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Bryn Cooke


The current PerRowSecondaryIndex receives its index event after the call to 
BTree.update. This means that it is impossible to write an index that eagerly 
removes stale index entries.

It would be great to have some sort of preIndex method that gets called with 
the key and column family.

In addition a postIndex method that is guaranteed to get called regardless if 
the actual BTree operation succeeds or not would allow cleanup in the event of 
error. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9041) Allow a PerRowSecondaryIndex to perform its index operation before the underlying data has been mutated

2015-03-25 Thread Bryn Cooke (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryn Cooke updated CASSANDRA-9041:
--
Summary: Allow a PerRowSecondaryIndex to perform its index operation before 
the underlying data has been mutated  (was: Allow a PerRowSecondaryIndex to 
perform it's index operation before the underlying data has been mutated)

 Allow a PerRowSecondaryIndex to perform its index operation before the 
 underlying data has been mutated
 ---

 Key: CASSANDRA-9041
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9041
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Bryn Cooke

 The current PerRowSecondaryIndex receives its index event after the call to 
 BTree.update. This means that it is impossible to write an index that eagerly 
 removes stale index entries.
 It would be great to have some sort of preIndex method that gets called with 
 the key and column family.
 In addition a postIndex method that is guaranteed to get called regardless if 
 the actual BTree operation succeeds or not would allow cleanup in the event 
 of error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of GettingStarted by JonathanEllis

2015-03-25 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The GettingStarted page has been changed by JonathanEllis:
https://wiki.apache.org/cassandra/GettingStarted?action=diffrev1=104rev2=105

  This document aims to provide a few easy to follow steps to take the 
first-time user from installation, to running single node Cassandra, and 
overview to configure multinode cluster. Cassandra is meant to run on a cluster 
of nodes, but will run equally well on a single machine. This is a handy way of 
getting familiar with the software while avoiding the complexities of a larger 
system.
  
  == Step 0: Prerequisites and Connecting to the Community ==
- Cassandra requires the most stable version of Java 7 you can deploy, 
preferably the 
[[http://www.oracle.com/technetwork/java/javase/downloads/index.html|Oracle/Sun 
JVM]].  Cassandra also runs on OpenJDK and the IBM JVM.  Because Cassandra 
requires Java 7, it will NOT run on JRockit.
+ Cassandra requires the most stable version of Java 7 or 8 you can deploy, 
preferably the 
[[http://www.oracle.com/technetwork/java/javase/downloads/index.html|Oracle/Sun 
JVM]].  Cassandra also runs on OpenJDK and the IBM JVM.  (It will NOT run on 
JRockit, which is only compatible with Java 6.)
  
  The best way to ensure you always have up to date information on the project, 
releases, stability, bugs, and features is to subscribe to the users mailing 
list ([[mailto:user-subscr...@cassandra.apache.org|subscription required]]) and 
participate in the #cassandra channel on 
[[http://webchat.freenode.net/?channels=#cassandra|IRC]].

[jira] [Commented] (CASSANDRA-7032) Improve vnode allocation

2015-03-25 Thread Branimir Lambov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380174#comment-14380174
 ] 

Branimir Lambov commented on CASSANDRA-7032:


Patch is up for review 
[here|https://github.com/apache/cassandra/compare/trunk...blambov:7032-vnode-assignment].
 It gives the option to specify a allocate_tokens_keyspace when bringing up a 
node. The node's tokens are then allocated to optimize the load distribution 
for the replication strategy of that keyspace.

The allocation is currently restricted to Murmur3Partitioner and SimpleStrategy 
or NetworkTopologyStrategy (is there anything else we need to support?). With 
the latter it cannot deal with cases where the number of racks in the dc is 
more than one but less than the replication factor, which should not be a 
common case.

There are a couple of things still left to do or explore, possibly in separate 
patches:
- add a dtest starting several nodes with allocation
- run a cstar_perf to see if it could show improvement for RF 2 in a 3-node 
cluster
- optimization of the selection for the first RF nodes in the cluster to 
guarantee good distribution later (see 
ReplicationAwareTokenAllocator.testNewCluster)
- (if deemed worthwhile) multiple different replication factors in one 
datacentre; the current code works ok when asked to allocate alternatingly but 
this could be improved if we consider all relevant strategies in parallel

 Improve vnode allocation
 

 Key: CASSANDRA-7032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Branimir Lambov
  Labels: performance, vnodes
 Fix For: 3.0

 Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java, 
 TestVNodeAllocation.java, TestVNodeAllocation.java, TestVNodeAllocation.java, 
 TestVNodeAllocation.java


 It's been known for a little while that random vnode allocation causes 
 hotspots of ownership. It should be possible to improve dramatically on this 
 with deterministic allocation. I have quickly thrown together a simple greedy 
 algorithm that allocates vnodes efficiently, and will repair hotspots in a 
 randomly allocated cluster gradually as more nodes are added, and also 
 ensures that token ranges are fairly evenly spread between nodes (somewhat 
 tunably so). The allocation still permits slight discrepancies in ownership, 
 but it is bound by the inverse of the size of the cluster (as opposed to 
 random allocation, which strangely gets worse as the cluster size increases). 
 I'm sure there is a decent dynamic programming solution to this that would be 
 even better.
 If on joining the ring a new node were to CAS a shared table where a 
 canonical allocation of token ranges lives after running this (or a similar) 
 algorithm, we could then get guaranteed bounds on the ownership distribution 
 in a cluster. This will also help for CASSANDRA-6696.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8993) EffectiveIndexInterval calculation is incorrect


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380252#comment-14380252
 ] 

Benedict commented on CASSANDRA-8993:
-

* Might it be easier to filter the list of readers earlier, in 
{{redistributeSummaries(ListSSTableReader compacting, ListSSTableReader 
nonCompacting, long memoryPoolBytes)}} ?
* Could you clarify SSTableReader Line 875: why do we check BASE_SAMPLING_LEVEL 
entries?

 EffectiveIndexInterval calculation is incorrect
 ---

 Key: CASSANDRA-8993
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8993
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Blocker
 Fix For: 2.1.4

 Attachments: 8993-2.1.txt, 8993.txt


 I'm not familiar enough with the calculation itself to understand why this is 
 happening, but see discussion on CASSANDRA-8851 for the background. I've 
 introduced a test case to look for this during downsampling, but it seems to 
 pass just fine, so it may be an artefact of upgrading.
 The problem was, unfortunately, not manifesting directly because it would 
 simply result in a failed lookup. This was only exposed when early opening 
 used firstKeyBeyond, which does not use the effective interval, and provided 
 the result to getPosition().
 I propose a simple fix that ensures a bug here cannot break correctness. 
 Perhaps [~thobbs] can follow up with an investigation as to how it actually 
 went wrong?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9042) Make trunk always releasable

Ariel Weisberg created CASSANDRA-9042:
-

 Summary: Make trunk always releasable
 Key: CASSANDRA-9042
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9042
 Project: Cassandra
  Issue Type: Task
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5239) Asynchronous (non-blocking) StorageProxy

[
https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380520#comment-14380520
]

Ariel Weisberg commented on CASSANDRA-5239:
---

Sorry about resurrecting this I just happened to be passing by.

By incremental processing are you talking about something like streaming a
large result set?

Guava listenable futures might be insufficient for that, but we could roll our
own for asynchronous activities that have that kind of lifecycle. That is
plumbing we would have to build in one form or another either way. It wouldn't
be particular useful outside of using a familiar idiom because all the existing
code that works with Futures would materialize the entire thing on get().

Starting with plain listenable futures and then retrofitting a new kind of
future doesn't seem like it's much better/worse then what is there from an
incremental processing implementation perspective. It might even just be
something like an async iterator of plain listenable futures.

Asynchronous (non-blocking) StorageProxy

Key: CASSANDRA-5239
URL: https://issues.apache.org/jira/browse/CASSANDRA-5239
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 2.0 beta 1
Reporter: Vijay
Labels: performance
Fix For: 3.0

Problem Statement:
Currently we have rpc_min_threads, rpc_max_threads/
native_transport_min_threads/native_transport_max_threads all of the
threads in the TPE are blocking and takes resources, the threads are mostly
sleeping. Increasing the Context switch costs.
Details:
We should change StorageProxy methods to provide a callback which contains
the location where the results has to be written. When the response arrive
StorageProxy callback can write the results directly into the connection.
Timeouts can be handled in the same way.
Fixing Netty should be trivial with some refactor in the storage proxy
(currently it is one method call for sending the request and waiting) we need
callback.
Fixing Thrift may be harder because thrift calls the method and expects a
return value. We might need to write a custom Codec on Netty for thrift
support, which can potentially do callbacks (A Custom codec may be similar to
http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
but we dont know details about it). Another option is to update thrift to
have a callback.
FYI, The motivation for this ticket is from another project which i am
working on with similar Proxy (blocking Netty transport) and making it Async
gave us 2x throughput improvement.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8993) EffectiveIndexInterval calculation is incorrect


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8993:
---
Attachment: 8993-2.1-v2.txt

 EffectiveIndexInterval calculation is incorrect
 ---

 Key: CASSANDRA-8993
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8993
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Blocker
 Fix For: 2.1.4

 Attachments: 8993-2.1-v2.txt, 8993-2.1.txt, 8993.txt


 I'm not familiar enough with the calculation itself to understand why this is 
 happening, but see discussion on CASSANDRA-8851 for the background. I've 
 introduced a test case to look for this during downsampling, but it seems to 
 pass just fine, so it may be an artefact of upgrading.
 The problem was, unfortunately, not manifesting directly because it would 
 simply result in a failed lookup. This was only exposed when early opening 
 used firstKeyBeyond, which does not use the effective interval, and provided 
 the result to getPosition().
 I propose a simple fix that ensures a bug here cannot break correctness. 
 Perhaps [~thobbs] can follow up with an investigation as to how it actually 
 went wrong?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6477) Global indexes

2015-03-25 Thread Carl Yeksigian (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380504#comment-14380504
]

Carl Yeksigian commented on CASSANDRA-6477:
---

I've pushed up a few updates to the branch. There was an issue with clustering
columns which [~philipthompson] pointed out, which has been resolved; an index
on a clustering column works properly now.

Currently, {{ALLOW FILTERING}} queries are disallowed. We can change that
behavior later, but it should be a follow-on ticket. Also disabled ins any
index including Collections, either as target or an included column. This
should also be added, but it will require a substantial rewrite, and makes
sense to hold off on until the major rewrite after CASSANDRA-8099.

[~pateljay3001]: I think it makes sense to wait until this gets committed.
Especially with the storage engine refactor, there may come significant changes
to the global indexes, so any work would need to be redone later.

It currently doesn't have any sort of read-repair because it doesn't work at
the level of mutation applications. It also doesn't have a proper repair built
in; we can use the {{Builder}} from the creation part to add a rebuild, but we
would have to drop the data currently stored to make sure we aren't returning
previous results.

Global indexes
--

Key: CASSANDRA-6477
URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Assignee: Carl Yeksigian
Labels: cql
Fix For: 3.0

Local indexes are suitable for low-cardinality data, where spreading the
index across the cluster is a Good Thing. However, for high-cardinality
data, local indexes require querying most nodes in the cluster even if only a
handful of rows is returned.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8993) EffectiveIndexInterval calculation is incorrect

[
https://issues.apache.org/jira/browse/CASSANDRA-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380553#comment-14380553
]

Tyler Hobbs commented on CASSANDRA-8993:

bq. Might it be easier to filter the list of readers earlier

Sure, there's no good reason not to do this earlier, so I've moved it to the
suggested location.

bq. Could you clarify SSTableReader Line 875: why do we check
BASE_SAMPLING_LEVEL entries?

I meant to explain this a little better, thanks for catching that. I've
improved the comment enough that it should hopefully clarify the reason.

My [branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-8993] has been
updated in addition to the v2 patch.

EffectiveIndexInterval calculation is incorrect
---

Key: CASSANDRA-8993
URL: https://issues.apache.org/jira/browse/CASSANDRA-8993
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Blocker
Fix For: 2.1.4

Attachments: 8993-2.1-v2.txt, 8993-2.1.txt, 8993.txt

I'm not familiar enough with the calculation itself to understand why this is
happening, but see discussion on CASSANDRA-8851 for the background. I've
introduced a test case to look for this during downsampling, but it seems to
pass just fine, so it may be an artefact of upgrading.
The problem was, unfortunately, not manifesting directly because it would
simply result in a failed lookup. This was only exposed when early opening
used firstKeyBeyond, which does not use the effective interval, and provided
the result to getPosition().
I propose a simple fix that ensures a bug here cannot break correctness.
Perhaps [~thobbs] can follow up with an investigation as to how it actually
went wrong?

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9037) Terminal UDFs evaluated at prepare time throw protocol version error


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380757#comment-14380757
 ] 

Tyler Hobbs commented on CASSANDRA-9037:


Overall the patch looks good, just a couple of comments:
* There are some (apparently) unrelated changes to CqlRecordReader included
* Can you add a test case that uses literal function arguments (especially 
collections)?

 Terminal UDFs evaluated at prepare time throw protocol version error
 

 Key: CASSANDRA-9037
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9037
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0


 When a pure function with only terminal arguments (or with no arguments) is 
 used in a where clause, it's executed at prepare time and 
 {{Server.CURRENT_VERSION}} passed as the protocol version for serialization 
 purposes. For native functions, this isn't a problem, but UDFs use classes in 
 the bundled java-driver-core jar for (de)serialization of args and return 
 values. When {{Server.CURRENT_VERSION}} is greater than the highest version 
 supported by the bundled java driver the execution fails with the following 
 exception:
 {noformat}
 ERROR [SharedPool-Worker-1] 2015-03-24 18:10:59,391 QueryMessage.java:132 - 
 Unexpected error during query
 org.apache.cassandra.exceptions.FunctionExecutionException: execution of 
 'ks.overloaded[text]' failed: java.lang.IllegalArgumentException: No protocol 
 version matching integer version 4
 at 
 org.apache.cassandra.exceptions.FunctionExecutionException.create(FunctionExecutionException.java:35)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.udf.gen.Cksoverloaded_1.execute(Cksoverloaded_1.java)
  ~[na:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.executeInternal(FunctionCall.java:78)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall.access$200(FunctionCall.java:34)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.execute(FunctionCall.java:176)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.functions.FunctionCall$Raw.prepare(FunctionCall.java:161)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.toTerm(SingleColumnRelation.java:108)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.SingleColumnRelation.newEQRestriction(SingleColumnRelation.java:143)
  ~[main/:na]
 at org.apache.cassandra.cql3.Relation.toRestriction(Relation.java:127) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.restrictions.StatementRestrictions.init(StatementRestrictions.java:126)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:787)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:740)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488)
  ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) 
 ~[main/:na]
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:246) 
 ~[main/:na]
 at 
 org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
  ~[main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:475)
  [main/:na]
 at 
 org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:371)
  [main/:na]
 at 
 io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
  [netty-all-4.0.23.Final.jar:4.0.23.Final]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_71]
 at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
  [main/:na]
 at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
 [main/:na]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: java.lang.IllegalArgumentException: No protocol version matching 
 integer version 4
 at 
 com.datastax.driver.core.ProtocolVersion.fromInt(ProtocolVersion.java:89) 
 ~[cassandra-driver-core-2.1.2.jar:na]
 at

[jira] [Updated] (CASSANDRA-8359) Make DTCS consider removing SSTables much more frequently

[
https://issues.apache.org/jira/browse/CASSANDRA-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Philip Thompson updated CASSANDRA-8359:
---
Reviewer: Marcus Eriksson

Make DTCS consider removing SSTables much more frequently
-

Key: CASSANDRA-8359
URL: https://issues.apache.org/jira/browse/CASSANDRA-8359
Project: Cassandra
Issue Type: Improvement
Reporter: Björn Hegerfors
Assignee: Björn Hegerfors
Priority: Minor
Attachments: cassandra-2.0-CASSANDRA-8359.txt

When I run DTCS on a table where every value has a TTL (always the same TTL),
SSTables are completely expired, but still stay on disk for much longer than
they need to. I've applied CASSANDRA-8243, but it doesn't make an apparent
difference (probably because the subject SSTables are purged via compaction
anyway, if not by directly dropping them).
Disk size graphs show clearly that tombstones are only removed when the
oldest SSTable participates in compaction. In the long run, size on disk
continually grows bigger. This should not have to happen. It should easily be
able to stay constant, thanks to DTCS separating the expired data from the
rest.
I think checks for whether SSTables can be dropped should happen much more
frequently. This is something that probably only needs to be tweaked for
DTCS, but perhaps there's a more general place to put this. Anyway, my
thinking is that DTCS should, on every call to getNextBackgroundTask, check
which SSTables can be dropped. It would be something like a call to
CompactionController.getFullyExpiredSSTables with all non-compactingSSTables
sent in as compacting and all other SSTables sent in as overlapping. The
returned SSTables, if any, are then added to whichever set of SSTables that
DTCS decides to compact. Then before the compaction happens, Cassandra is
going to make another call to CompactionController.getFullyExpiredSSTables,
where it will see that it can just drop them.
This approach has a bit of redundancy in that it needs to call
CompactionController.getFullyExpiredSSTables twice. To avoid that, the code
path for deciding SSTables to drop would have to be changed.
(Side tracking a little here: I'm also thinking that tombstone compactions
could be considered more often in DTCS. Maybe even some kind of multi-SSTable
tombstone compaction involving the oldest couple of SSTables...)

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9023) 2.0.13 write timeouts on driver


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380842#comment-14380842
 ] 

Philip Thompson commented on CASSANDRA-9023:


So your problems are probably related to the FileNotFoundException. I see a 
similar UnavailableException in testing after deleting an sstable. Has this 
problem occurred more than the once with this node/sstable?

 2.0.13 write timeouts on driver
 ---

 Key: CASSANDRA-9023
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9023
 Project: Cassandra
  Issue Type: Bug
 Environment: For testing using only Single node 
 hardware configuration as follows:
 cpu :
 CPU(s):16
 On-line CPU(s) list:   0-15
 Thread(s) per core:2
 Core(s) per socket:8
 Socket(s): 1
 NUMA node(s):  1
 Vendor ID: GenuineIntel
 CPU MHz:   2000.174
 L1d cache: 32K
 L1i cache: 32K
 L2 cache:  256K
 L3 cache:  20480K
 NUMA node0 CPU(s): 0-15
 OS:
 Linux version 2.6.32-504.8.1.el6.x86_64 (mockbu...@c6b9.bsys.dev.centos.org) 
 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) 
 Disk: There only single disk in Raid i think space is 500 GB used is 5 GB
Reporter: anishek
 Attachments: out_system.log


 Initially asked @ 
 http://www.mail-archive.com/user@cassandra.apache.org/msg41621.html
 Was suggested to post here. 
 If any more details are required please let me know 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9039) CommitLog compressed configuration not run in several unit tests


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9039:
---
Issue Type: Test  (was: Bug)

 CommitLog compressed configuration not run in several unit tests
 

 Key: CASSANDRA-9039
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9039
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg

 CommitLogTest, RecoveryManagerTest, RecoveryManagerTruncateTest, 
 RecoveryManager2Test, RecoveryManager3Test
 Some or all of these should be run with a commit log configured to use 
 compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9040) o.a.c.net.* has no unit test coverage and no coverage with compression


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9040:
---
Issue Type: Test  (was: Bug)

 o.a.c.net.* has no unit test coverage and no coverage with compression
 --

 Key: CASSANDRA-9040
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9040
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg

 I think the unit test issue is minor since this code is validated as part of 
 running  everything on a regular basis.
 I think we do need a quick test that shows that the database starts and runs 
 data correctly with compression enabled and that the config options for none, 
 intradc, all work.
 Further validation would be done in a harness test that validates the kitchen 
 sink against the different compression configurations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8359) Make DTCS consider removing SSTables much more frequently

2015-03-25 Thread Ryan McGuire (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ryan McGuire updated CASSANDRA-8359:

Tester: Shawn Kumar

Make DTCS consider removing SSTables much more frequently
-

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9038) Atomic batches and single row atomicity appear to have no test coverage


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-9038:
---
Issue Type: Test  (was: Bug)

 Atomic batches and single row atomicity appear to have no test coverage
 ---

 Key: CASSANDRA-9038
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9038
 Project: Cassandra
  Issue Type: Test
Reporter: Ariel Weisberg

 Leaving the solution to this up to the assignee. It seems like this is a 
 guarantee that should be checked.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8917) Upgrading from 2.0.9 to 2.1.3 with 3 nodes, CL = quorum causes exceptions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380848#comment-14380848
 ] 

Philip Thompson commented on CASSANDRA-8917:


I don't believe the number of seed nodes is related to the issue. Have you run 
into the problem again?

 Upgrading from 2.0.9 to 2.1.3 with 3 nodes, CL = quorum causes exceptions
 -

 Key: CASSANDRA-8917
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8917
 Project: Cassandra
  Issue Type: Bug
 Environment: C* 2.0.9, Centos 6.5, Java 1.7.0_72, spring data 
 cassandra 1.1.1, cassandra java driver 2.0.9
Reporter: Gary Ogden
 Fix For: 2.1.4

 Attachments: b_output.log, jersey_error.log, node1-cassandra.yaml, 
 node1-system.log, node2-cassandra.yaml, node2-system.log, 
 node3-cassandra.yaml, node3-system.log


 We have java apps running on glassfish that read/write to our 3 node cluster 
 running on 2.0.9. 
 we have the CL set to quorum for all reads and writes.
 When we started to upgrade the first node and did the sstable upgrade on that 
 node, we started getting this error on reads and writes:
 com.datastax.driver.core.exceptions.UnavailableException: Not enough replica 
 available for query at consistency QUORUM (2 required but only 1 alive)
 How is that possible when we have 3 nodes total, and there was 2 that were up 
 and it's saying we can't get the required CL?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8993) EffectiveIndexInterval calculation is incorrect

[
https://issues.apache.org/jira/browse/CASSANDRA-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381053#comment-14381053
]

Benedict commented on CASSANDRA-8993:
-

bq. that it should hopefully clarify the reason.

Unfortunately not, at least for me. I am afraid I don't really follow the
Downsampling logic, and the comment actually confused me further. I tried to
print out the various static method states in Downsampling to visualise them
better, and it still doesn't make much sense to me, so I'll outline my
confusion and we can decide either to help me understand, or to perhaps get
[~jbellis] (the original reviewer iirc) to step in. I should note that, either
way, I expect the test you've introduced to be sound for solving this problem,
so if this takes too long we can roll a version without further ceremony, but I
think it may be an unnecessarily lengthy test on startup for old format files,
which could be onerous.

bq. Unfortunately, the first entry to be dropped is the entry at index
(BASE_SAMPLING_LEVEL - 1), so we need to check a full set of
BASE_SAMPLING_LEVEL entries.

If I print out the original indices and effective intervals, it seems that
at the first downsampling level (64) alternating summary entries are dropped
(and again for each extra level), up to dropping all but each 128th (beginning
with zero), so the first half of the comment doesn't seem to match with the
behaviour I see in a way I understand. If the behaviour matches what I see, and
not the comment, then it seems we could safely just check the first two
expected intervals and if they mismatch we've downsampled and need to
rebuild. This would translate into always just checking 2 * minIndexInterval
records in the index, or 1/64th the data.

Further confusion to understanding Downsampling as a whole stems from the
permission of a -1 index into getEffectiveIndexIntervalAfterIndex without
explanation, and the fact that every effective interval is the same despite
there being multiple avenues for calculating it (it would be much clearer if it
were just minIndexInterval * (BASE_SAMPLING_LEVEL / samplingLevel).

I apologise if I'm just being a bit dimwitted.

EffectiveIndexInterval calculation is incorrect
---

Attachments: 8993-2.1-v2.txt, 8993-2.1.txt, 8993.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-5239) Asynchronous (non-blocking) StorageProxy

[
https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380940#comment-14380940
]

Benedict edited comment on CASSANDRA-5239 at 3/25/15 10:46 PM:
---

bq. are you talking about something like streaming a large result set?

Right. But the definition of large is important. Really I'm talking about
ensuring all queries require a fixed number of resources (or thereabouts), and
that returning the entire result can be done without significant extra overhead
nor breaking isolation, so that even relatively modest resultsets can be
streamed.

bq. doesn't seem like it's much better/worse

Exactly my point: as such it doesn't seem to be an intervening step, so using
it as one isn't a good justification for this patch. Either way, this is IMO
one of the more pressing concerns to address in the near future, and my view is
we should decide what our approximate end goal looks like concretely, and base
any intervening states on that.

was (Author: benedict):
bq. doesn't seem like it's much better/worse

Asynchronous (non-blocking) StorageProxy

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9028) Optimize LIMIT execution to mitigate need for a full partition scan

2015-03-25 Thread jonathan lacefield (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381123#comment-14381123
]

jonathan lacefield edited comment on CASSANDRA-9028 at 3/26/15 12:38 AM:
-

Hi Sylvain,

My apologies if my statement isn't specifically correct. We thought this
because of some behavior observed with a user. The behavior observed was that
we executed 2 types of queries with tracing enabled and noticed that the LIMIT
query touched all sstables for which the particular partition existed.
However, the clustering key specific query only touched the individual
sstable that contained the partition and clusterin key value for which was
queried.

I have recreated this behavior using the following, and attached, examples.
test.ddl - simple table with a partition key and clustering column
trace.out - output of tracing the 2 types of queries (with queries)
Data.*.json - output of each json file.

If my statement is incorrect, will you please help clarify the internals
that are impacting these results?

The goal of this enhancement request is that each query would preform, from a
latency perspective, the same. My thinking is that the tracing should appear
the same for the performance to be the same.

was (Author: jlacefie):
Hi Sylvain,

My apologies if my statement isn't specifically correct. We thought this
because of some behavior observed with a user. The behavior observed was that
we executed 2 types of queries with tracing enabled and notices that the LIMIT
query touched all sstables for which the particular partition existed.
However, the clustering key specific query only touched the individual
sstable that contained the partition and clusterin key value for which was
queried.

If my statement is incorrect, will you please help clarify the internals
that are impacting these results? The goal of this enhancement is that each
query would preform, from a latency perspective, the same. My thinking is that
the tracing should appear the same for the performance to be the same.

Optimize LIMIT execution to mitigate need for a full partition scan
---

Key: CASSANDRA-9028
URL: https://issues.apache.org/jira/browse/CASSANDRA-9028
Project: Cassandra
Issue Type: Improvement
Components: API, Core
Reporter: jonathan lacefield
Attachments: Data.1.json, Data.2.json, Data.3.json, test.ddl,
tracing.out

Currently, a SELECT statement for a single Partition Key that contains a
LIMIT X clause will fetch an entire partition from a node and place the
partition into memory prior to applying the limit clause and returning
results to be served to the client via the coordinator.
This JIRA is to request an optimization for the CQL LIMIT clause to avoid the
entire partition retrieval step, and instead only retrieve the components to
satisfy the LIMIT condition.
Ideally, any LIMIT X would avoid the need to retrieve a full partition. This
may not be possible though. As a compromise, it would still be incredibly
beneficial if a LIMIT 1 clause could be optimized to only retrieve the
latest item. Ideally a LIMIT 1 would operationally behave the same way
as a Clustering Key WHERE clause where the latest, i.e. LIMIT 1 field, col
value was specified.
We can supply some trace results to help show the difference between 2
different queries that preform the same logical function if desired.
For example, a query that returns the latest value for a clustering col
where QUERY 1 uses a LIMIT 1 clause and QUERY 2 uses a WHERE clustering col
= latest value

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9028) Optimize LIMIT execution to mitigate need for a full partition scan

2015-03-25 Thread jonathan lacefield (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381123#comment-14381123
]

jonathan lacefield commented on CASSANDRA-9028:
---

Hi Sylvain,

My apologies if my statement isn't specifically correct. We thought this
because of some behavior observed with a user. The behavior observed was that
we executed 2 types of queries with tracing enabled and notices that the LIMIT
query touched all sstables for which the particular partition existed.
However, the clustering key specific query only touched the individual
sstable that contained the partition and clusterin key value for which was
queried.

Optimize LIMIT execution to mitigate need for a full partition scan
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9028) Optimize LIMIT execution to mitigate need for a full partition scan

2015-03-25 Thread jonathan lacefield (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jonathan lacefield updated CASSANDRA-9028:
--
Attachment: Data.3.json
Data.2.json
Data.1.json
tracing.out
test.ddl

 Optimize LIMIT execution to mitigate need for a full partition scan
 ---

 Key: CASSANDRA-9028
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9028
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: jonathan lacefield
 Attachments: Data.1.json, Data.2.json, Data.3.json, test.ddl, 
 tracing.out


 Currently, a SELECT statement for a single Partition Key that contains a 
 LIMIT X clause will fetch an entire partition from a node and place the 
 partition into memory prior to applying the limit clause and returning 
 results to be served to the client via the coordinator.
 This JIRA is to request an optimization for the CQL LIMIT clause to avoid the 
 entire partition retrieval step, and instead only retrieve the components to 
 satisfy the LIMIT condition.
 Ideally, any LIMIT X would avoid the need to retrieve a full partition.  This 
 may not be possible though.  As a compromise, it would still be incredibly 
 beneficial if a LIMIT 1 clause could be optimized to only retrieve the 
 latest item.  Ideally a LIMIT 1 would operationally behave the same way 
 as a Clustering Key WHERE clause where the latest, i.e. LIMIT 1 field, col 
 value was specified.
 We can supply some trace results to help show the difference between 2 
 different queries that preform the same logical function if desired.
   For example, a query that returns the latest value for a clustering col 
 where QUERY 1 uses a LIMIT 1 clause and QUERY 2 uses a WHERE clustering col 
 = latest value



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9043) Improve COPY command to work with Counter columns

2015-03-25 Thread Sebastian Estevez (JIRA)

Sebastian Estevez created CASSANDRA-9043:


 Summary: Improve COPY command to work with Counter columns
 Key: CASSANDRA-9043
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9043
 Project: Cassandra
  Issue Type: New Feature
Reporter: Sebastian Estevez


Noticed today that the copy command doesn't work with counter column tables.

This makes sense given that we need to use UPDATE instead of INSERT with 
counters.

Given that we're making improvements in the COPY command in 3.0 with 
CASSANDRA-7405, can we also tweak it to work with counters?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7807) Push notification when tracing completes for an operation


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381163#comment-14381163
 ] 

Stefania commented on CASSANDRA-7807:
-

Thanks Tyler. 

So Robert these are the last few things:
# Check if the channel is registered with the tracker
# Check protocol version is 4
# Only send notification when client requested it

It should be possible to test these points using {{SimpleClient}}. For 
probabilistic tracing you can set the probability to one by calling 
{{StorageService.setTraceProbability()}}.

 Push notification when tracing completes for an operation
 -

 Key: CASSANDRA-7807
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7807
 Project: Cassandra
  Issue Type: Sub-task
  Components: Core
Reporter: Tyler Hobbs
Assignee: Robert Stupp
Priority: Minor
  Labels: client-impacting, protocolv4
 Fix For: 3.0

 Attachments: 7807.txt


 Tracing is an asynchronous operation, and drivers currently poll to determine 
 when the trace is complete (in a loop with sleeps).  Instead, the server 
 could push a notification to the driver when the trace completes.
 I'm guessing that most of the work for this will be around pushing 
 notifications to a single connection instead of all connections that have 
 registered listeners for a particular event type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8561) Tombstone log warning does not log partition key

[
https://issues.apache.org/jira/browse/CASSANDRA-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379731#comment-14379731
]

Aleksey Yeschenko commented on CASSANDRA-8561:
--

Firstly, you've rebased incorrectly, overriding the changes of CASSANDRA-8559.

Secondly, you worry about key length here, while slices formatted could take
just as much, or much less, of the log space.

I'd just leave it be as is, except make {{boolean warnTombstones}} also take
{{logger.isWarnEnabled}} into account. Altogether this should give people a way
to at least disable the code path entirely by disabling logger preferences for
{{SliceQueryFilter}}.

500 feels extremely arbitrary to me. It's not even a nice round number. If you
do decide to truncate output, after all, pick a round one - maybe 256 or 512 -
and truncate the entire log message, not just the key.

Adding yet another config option, yaml or -D, is the last thing we need. Pick a
hardcoded one or don't truncate at all, one of the two.

Tombstone log warning does not log partition key

Key: CASSANDRA-8561
URL: https://issues.apache.org/jira/browse/CASSANDRA-8561
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Datastax DSE 4.5
Reporter: Jens Rantil
Assignee: Lyuben Todorov
Labels: logging
Fix For: 2.1.4

Attachments: cassandra-2.1-1427196372-8561-v2.diff,
cassandra-2.1-8561.diff, cassandra-2.1-head-1427124485-8561.diff,
cassandra-trunk-head-1427125869-8561.diff, trunk-1427195046-8561-v2.diff

AFAIK, the tombstone warning in system.log does not contain the primary key.
See: https://gist.github.com/JensRantil/44204676f4dbea79ea3a
Including it would help a lot in diagnosing why the (CQL) row has so many
tombstones.
Let me know if I have misunderstood something.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8561) Tombstone log warning does not log partition key

[
https://issues.apache.org/jira/browse/CASSANDRA-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379731#comment-14379731
]

Aleksey Yeschenko edited comment on CASSANDRA-8561 at 3/25/15 11:45 AM:

Firstly, you've rebased incorrectly, overriding the changes of CASSANDRA-8559.

Secondly, you worry about key length here, while slices formatted could take
just as much, or much more, of the log space.

Adding yet another config option, yaml or -D, is the last thing we need. Pick a
hardcoded one or don't truncate at all, one of the two.

was (Author: iamaleksey):
Firstly, you've rebased incorrectly, overriding the changes of CASSANDRA-8559.

Secondly, you worry about key length here, while slices formatted could take
just as much, or much less, of the log space.

Adding yet another config option, yaml or -D, is the last thing we need. Pick a
hardcoded one or don't truncate at all, one of the two.

Tombstone log warning does not log partition key

Attachments: cassandra-2.1-1427196372-8561-v2.diff,
cassandra-2.1-8561.diff, cassandra-2.1-head-1427124485-8561.diff,
cassandra-trunk-head-1427125869-8561.diff, trunk-1427195046-8561-v2.diff

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9037) Terminal UDFs evaluated at prepare time throw protocol version error

2015-03-25 Thread Sam Tunnicliffe (JIRA)

Sam Tunnicliffe created CASSANDRA-9037:
--

 Summary: Terminal UDFs evaluated at prepare time throw protocol 
version error
 Key: CASSANDRA-9037
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9037
 Project: Cassandra
  Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0


When a pure function with only terminal arguments (or with no arguments) is 
used in a where clause, it's executed at prepare time and 
{{Server.CURRENT_VERSION}} passed as the protocol version for serialization 
purposes. For native functions, this isn't a problem, but UDFs use classes in 
the bundled java-driver-core jar for (de)serialization of args and return 
values. When {{Server.CURRENT_VERSION}} is greater than the highest version 
supported by the bundled java driver the execution fails with the following 
exception:

{noformat}
ERROR [SharedPool-Worker-1] 2015-03-24 18:10:59,391 QueryMessage.java:132 - 
Unexpected error during query
org.apache.cassandra.exceptions.FunctionExecutionException: execution of 
'ks.overloaded[text]' failed: java.lang.IllegalArgumentException: No protocol 
version matching integer version 4
at 
org.apache.cassandra.exceptions.FunctionExecutionException.create(FunctionExecutionException.java:35)
 ~[main/:na]
at 
org.apache.cassandra.cql3.udf.gen.Cksoverloaded_1.execute(Cksoverloaded_1.java) 
~[na:na]
at 
org.apache.cassandra.cql3.functions.FunctionCall.executeInternal(FunctionCall.java:78)
 ~[main/:na]
at 
org.apache.cassandra.cql3.functions.FunctionCall.access$200(FunctionCall.java:34)
 ~[main/:na]
at 
org.apache.cassandra.cql3.functions.FunctionCall$Raw.execute(FunctionCall.java:176)
 ~[main/:na]
at 
org.apache.cassandra.cql3.functions.FunctionCall$Raw.prepare(FunctionCall.java:161)
 ~[main/:na]
at 
org.apache.cassandra.cql3.SingleColumnRelation.toTerm(SingleColumnRelation.java:108)
 ~[main/:na]
at 
org.apache.cassandra.cql3.SingleColumnRelation.newEQRestriction(SingleColumnRelation.java:143)
 ~[main/:na]
at org.apache.cassandra.cql3.Relation.toRestriction(Relation.java:127) 
~[main/:na]
at 
org.apache.cassandra.cql3.restrictions.StatementRestrictions.init(StatementRestrictions.java:126)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:787)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:740)
 ~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:246) 
~[main/:na]
at 
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
 ~[main/:na]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:475)
 [main/:na]
at 
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:371)
 [main/:na]
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at 
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
 [netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_71]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
 [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.lang.IllegalArgumentException: No protocol version matching 
integer version 4
at 
com.datastax.driver.core.ProtocolVersion.fromInt(ProtocolVersion.java:89) 
~[cassandra-driver-core-2.1.2.jar:na]
at 
org.apache.cassandra.cql3.functions.UDFunction.compose(UDFunction.java:177) 
~[main/:na]
... 25 common frames omitted
{noformat}

This is currently the case on trunk following the bump of 
{{Server.CURRENT_VERSION}} to 4 by CASSANDRA-7660.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5239) Asynchronous (non-blocking) StorageProxy

[
https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380940#comment-14380940
]

Benedict commented on CASSANDRA-5239:
-

bq. doesn't seem like it's much better/worse

Asynchronous (non-blocking) StorageProxy

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException

2015-03-25 Thread Carl Yeksigian (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-8978:
--
Attachment: 8978-2.1.txt

This is caused because of a race condition in the CQLSSTableWriter. If we are 
writing to a single partition, we create a new column family to write into and 
send the previous one to the writer thread, but continue to use the old column 
family until we are done with the current partition.

This adds a call to check whether the row needs to be added, and if needed will 
sync at that point which is after an insert has completed is finished.

 CQLSSTableWriter causes ArrayIndexOutOfBoundsException
 --

 Key: CASSANDRA-8978
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8978
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 3.8.0-42-generic #62~precise1-Ubuntu SMP Wed Jun 4 
 22:04:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
 java version 1.8.0_20
 Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
 Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
Reporter: Thomas Borg Salling
Assignee: Carl Yeksigian
 Fix For: 2.1.4

 Attachments: 8978-2.1.txt


 On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
 load via sstableloader, occassionally I get the sporadic error shown below.
 I can run the exact same job again - and it will succeed or fail with the 
 same error at another location in the input stream. The error is appears to 
 occur randomly - with the same input it may occur never, early or late in 
 the run with no apparent logic or system.
 I use five instances of CQLSSTableWriter in the application (to write 
 redundantly to five different tables). But these instances do not exist at 
 the same time; and thus never used concurrently.
 {code}
 09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
 processing directory, 369582175 packets was converted from /nas1/
 Exception in thread main java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)
 at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
 at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
 at dk.dma.ais.store.Main.main(Main.java:34)
 Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
 at 
 org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
 at 
 org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
 at 
 org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
 at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
 at 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}
 So far I overcome this problem by simply retrying with another run of the 
 application in attempt to generate the sstables. But this is a rather time 
 consuming and shaky approach - and I feel a bit uneasy relying on the 
 produced sstables, though their contents appear to be correct when I sample 
 them with cqlsh 'select' after load into Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8993) EffectiveIndexInterval calculation is incorrect