[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991433#comment-14991433
 ] 

Sylvain Lebresne commented on CASSANDRA-10365:
--

bq. If anything (new) is failing, I'll fix it all today

Well, there is a lot of failures. Some are due to tests needing to be upgraded 
following the change of driver version on cassci. But at least for the thrift 
tests (that are all failing), it's due to a genuine bug introduced by this 
patch. Thrift dtests are failing with the following stack:
{noformat}
[...]
Caused by: org.apache.cassandra.exceptions.InvalidRequestException: Counters 
are not allowed inside collections: map
at 
org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:469) 
~[main/:na]
at 
org.apache.cassandra.schema.CQLTypeParser.parse(CQLTypeParser.java:55) 
~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.createColumnFromRow(SchemaKeyspace.java:1012)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.lambda$fetchColumns$245(SchemaKeyspace.java:996)
 ~[main/:na]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_66]
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchColumns(SchemaKeyspace.java:996)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:950) 
~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:929) 
~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:892)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesOnly(SchemaKeyspace.java:884)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1274)
 ~[main/:na]
at 
org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1253)
 ~[main/:na]
at 
org.apache.cassandra.service.MigrationManager$1.runMayThrow(MigrationManager.java:491)
 ~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_66]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_66]
{noformat}
The reason is that super columns uses maps internally, and we allow counters in 
super columns. We do indeed not allow counters in collections in CQL, but it's 
not a problem internally and we shouldn't throw in {{CQL3Type.Raw.prepare}} in 
that case anymore. Not entirely sure where to move that check however, so the 
simplest solution might be to use a {{prepareInternal}} version using by schema 
handling that disable this check.

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10534) CompressionInfo not being fsynced on close

[Study Hsueh (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991380#comment-14991380
 ] 

Study Hsueh commented on CASSANDRA-10534:
-

This bug also happened in 2.1.10.

> CompressionInfo not being fsynced on close
> --
>
> Key: CASSANDRA-10534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10534
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Stefania
> Fix For: 2.1.x
>
>
> I was seeing SSTable corruption due to a CompressionInfo.db file of size 0, 
> this happened multiple times in our testing with hard node reboots. After 
> some investigation it seems like these file is not being fsynced, and that 
> can potentially lead to data corruption. I am working with version 2.1.9.
> I checked for fsync calls using strace, and found them happening for all but 
> the following components: CompressionInfo, TOC.txt and digest.sha1. All of 
> these but the CompressionInfo seem tolerable. Also a quick look through the 
> code did not reveal any fsync calls. Moreover, I suspect the commit  
> 4e95953f29d89a441dfe06d3f0393ed7dd8586df 
> (https://github.com/apache/cassandra/commit/4e95953f29d89a441dfe06d3f0393ed7dd8586df#diff-b7e48a1398e39a936c11d0397d5d1966R344)
>  has caused the regression, which removed the line
> {noformat}
>  getChannel().force(true);
> {noformat}
> from CompressionMetadata.Writer.close.
> Following is the trace I saw in system.log:
> {noformat}
> INFO  [SSTableBatchOpen:1] 2015-09-29 19:24:39,170 SSTableReader.java:478 - 
> Opening 
> /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-13368
>  (79 bytes)
> ERROR [SSTableBatchOpen:1] 2015-09-29 19:24:39,177 FileUtils.java:447 - 
> Exiting forcefully due to file system exception on startup, disk failure 
> policy "stop"
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:131)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:752) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:703) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:491) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:387) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:534) 
> ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_80]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> [na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_80]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> Caused by: java.io.EOFException: null
> at 
> java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340) 
> ~[na:1.7.0_80]
> at java.io.DataInputStream.readUTF(DataInputStream.java:589) 
> ~[na:1.7.0_80]
> at java.io.DataInputStream.readUTF(DataInputStream.java:564) 
> ~[na:1.7.0_80]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:106)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> ... 14 common frames omitted
> {noformat}
> Following is the result of ls on the data directory of a corrupted SSTable 
> after the hard reboot:
> {noformat}
> $ ls -l 
> /var/lib/cassandra/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/
> total 60
> -rw-r--r-- 1 cassandra cassandra 0 Oct 15 09:31 
> system-sstable_activity-ka-1-CompressionInfo.db
> -rw-r--r-- 1 cassandra cassandra  9740 Oct 15 09:31 
> system-sstable_activity-ka-1-Data.db
> -rw-r--r-- 1 cassandra cassandra 0 

[jira] [Comment Edited] (CASSANDRA-9304) COPY TO improvements

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991279#comment-14991279
 ] 

Stefania edited comment on CASSANDRA-9304 at 11/5/15 9:11 AM:
--

Thank you for your input. 

Regarding version support for Windows, fine for 2.2+ but for completeness I'll 
point out that the only obstacle left in 2.1 is the name of the file (_cqlsh_ 
-> _cqlsh.py_).

Regarding the problem with pipes, I've replaced pipes with queues so we don't 
need to deal with the low level platform specific details. Queues can also be 
safely used from the callback threads, which was not the case for pipes.

Regarding the problem with the driver, -I haven't tested in 2.2 but I don't 
think it matters which version since- I verified the problem applies to 2.2 as 
well, yesterday I was using the latest cassandra-test driver version, today I 
used 2.7.2. The column type is the same, {{cassandra.cqltypes.BytesType}}, the 
method called from {{recv_result_rows()}} is the same, {{>}} but 
{{cls.serialize}} in {{from_binary}} is a lambda for the case that works and 
the default implementation {{CassandraType.deserialize}} for  the case that 
does not work. I don't know where the lambda comes from but I noticed there is 
a cython deserialize for {{BytesType}} in deserializers.pyx. I don't know how 
cython works but if this is picked up in the normal case then the problem is 
again with the way multiprocessing imports modules. 

The problem can be solved by adding a deserialize implementation to BytesType, 
like it's done for other types:

{code}
Stefi@Lila MINGW64 ~/git/cstar/python-driver ((2.7.2))
$ git diff
diff --git a/cassandra/cqltypes.py b/cassandra/cqltypes.py
index f39d28b..eb8d3b6 100644
--- a/cassandra/cqltypes.py
+++ b/cassandra/cqltypes.py
@@ -350,6 +350,10 @@ class BytesType(_CassandraType):
 def serialize(val, protocol_version):
 return six.binary_type(val)

+@staticmethod
+def deserialize(byts, protocol_version):
+return bytearray(byts)
+

 class DecimalType(_CassandraType):
 typename = 'decimal'
{code}

If this is not enough and you want to debug some more [~aholmber], you can use 
the 2.1 patch attached. I'm still working on the 2.2. merge. You need to 
generate a table with a blob, I used cassandra-stress. Then run {{COPY 
 TO 'anyfile';}} from cqlsh and this should result in a Unicode 
decode error on Windows because the blob is received as a string. If you prefer 
me to test things for you, that works too.



was (Author: stefania):
Thank you for your input. 

Regarding version support for Windows, fine for 2.2+ but for completeness I'll 
point out that the only obstacle left in 2.1 is the name of the file (_cqlsh_ 
-> _cqlsh.py_).

Regarding the problem with pipes, I've replaced pipes with queues so we don't 
need to deal with the low level platform specific details. Queues can also be 
safely used from the callback threads, which was not the case for pipes.

Regarding the problem with the driver, I haven't tested in 2.2 but I don't 
think it matters which version since yesterday I was using the latest 
cassandra-test driver version. Today I used 2.7.2. The column type is the same, 
{{cassandra.cqltypes.BytesType}}, the method called from {{recv_result_rows()}} 
is the same, {{>}} but {{cls.serialize}} in {{from_binary}} is 
a lambda for the case that works and the default implementation 
{{CassandraType.deserialize}} for  the case that does not work. I don't know 
where the lambda comes from but I noticed there is a cython deserialize for 
{{BytesType}} in deserializers.pyx. I don't know how cython works but if this 
is picked up in the normal case then the problem is again with the way 
multiprocessing imports modules. 

The problem can be solved by adding a deserialize implementation to BytesType, 
like it's done for other types:

{code}
Stefi@Lila MINGW64 ~/git/cstar/python-driver ((2.7.2))
$ git diff
diff --git a/cassandra/cqltypes.py b/cassandra/cqltypes.py
index f39d28b..eb8d3b6 100644
--- a/cassandra/cqltypes.py
+++ b/cassandra/cqltypes.py
@@ -350,6 +350,10 @@ class BytesType(_CassandraType):
 def serialize(val, protocol_version):
 return six.binary_type(val)

+@staticmethod
+def deserialize(byts, protocol_version):
+return bytearray(byts)
+

 class DecimalType(_CassandraType):
 typename = 'decimal'
{code}

If this is not enough and you want to debug some more [~aholmber], you can use 
the 2.1 patch attached. I'm still working on the 2.2. merge. You need to 
generate a table with a blob, I used cassandra-stress. Then run {{COPY 
 TO 'anyfile';}} from cqlsh and this should result in a Unicode 
decode error on Windows because the blob is received as a string. If you prefer 
me to test things for you, that works too.


> COPY TO improvements
> 
>
> Key: CASSANDRA-9304
>  

[jira] [Commented] (CASSANDRA-10513) Update cqlsh for new driver execution API

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991415#comment-14991415
 ] 

Sylvain Lebresne commented on CASSANDRA-10513:
--

I'll note for the record that this ticket has never been about changing the 
driver version used by dtests, only by cqlsh, so it's not an oversight of this 
ticket. So I suggest tracking the current failure of dtests on the ticket that 
required this change of drivers in the dtests, namely CASSANDRA-10365. I'm fine 
if Paulo want to take a shot at fixing the tests though, as long as it's done 
ASAP.

> Update cqlsh for new driver execution API
> -
>
> Key: CASSANDRA-10513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10513
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.2.4, 3.0.0
>
> Attachments: 10513-2.1.txt, 10513-2.2.txt, 10513-3.0-fix.patch, 
> 10513.txt
>
>
> The 3.0 Python driver will have a few tweaks to the execution API. We'll need 
> to update cqlsh in a couple of minor ways.
> [Results are always returned as an iterable 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-368]
> [Trace data is always attached to the 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-318] (instead of 
> being attached to a statement)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10650) Fix INITCOND after 10365

[Alexandre Dutra (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991437#comment-14991437
 ] 

Alexandre Dutra commented on CASSANDRA-10650:
-

Updated Java driver [here|https://github.com/datastax/java-driver/pull/467]. 
Note that the driver doesn't depend anymore on jgrapht.

> Fix INITCOND after 10365
> 
>
> Key: CASSANDRA-10650
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10650
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0
>
>
> CASSANDRA-10365 changed UDA's _initcond_ type to text. That required 
> CASSANDRA-10617 to be able to transform the serialized data to a valid CQL3 
> literal.
> This ticket is about to fix the _initcond_ thing broken in CASSANDRA-10365 
> after CASSANDRA-10617.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10584) reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a ClassCastException

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991367#comment-14991367
 ] 

Sylvain Lebresne commented on CASSANDRA-10584:
--

bq. Yikes... the pull request for CASSANDRA-9602 was never merged.

Has it been now? If not, can you point to which PR you're talking about so we 
can merge it?

> reads with EACH_QUORUM  on keyspace with SimpleTopologyStrategy throw a 
> ClassCastException
> --
>
> Key: CASSANDRA-10584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10584
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0
>
>
> I think this may be a regression introduced w/ [CASSANDRA-9602].  Starting 
> with C* 3.0.0-rc2 an error is returned when querying a keyspace with 
> {{SimpleTopologyStrategy}} using EACH_QUORUM CL:
> {noformat}
> cqlsh> create keyspace test with replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test (k int PRIMARY KEY, i int);
> cqlsh> consistency EACH_QUORUM;
> Consistency level set to EACH_QUORUM.
> cqlsh> select * from test.test;
> ServerError:  message="java.lang.ClassCastException: 
> org.apache.cassandra.locator.SimpleStrategy cannot be cast to 
> org.apache.cassandra.locator.NetworkTopologyStrategy">
> {noformat}
> The exception yielded in the system logs:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-10-23 13:02:15,405 ErrorMessage.java:336 - 
> Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy 
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForEachQuorum(ConsistencyLevel.java:227)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:188)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:180)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1795)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1762)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149) 
> ~[guava-18.0.jar:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1814)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1799)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1925)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1892)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager$PagerIterator.hasNext(AbstractQueryPager.java:99)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:327)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:205)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:236) 
> ~[main/:na]
> at 
> 

[jira] [Commented] (CASSANDRA-9304) COPY TO improvements

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991364#comment-14991364
 ] 

Stefania commented on CASSANDRA-9304:
-

The [2.2 patch|https://github.com/stef1927/cassandra/tree/9304-2.2] is also 
available now. It applies without conflict to 3.0. 

CI:

http://cassci.datastax.com/job/stef1927-9304-2.1-dtest/
http://cassci.datastax.com/job/stef1927-9304-2.2-dtest/
http://cassci.datastax.com/job/stef1927-9304-3.0-dtest/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9304-2.2-windows-dtest_win32/



> COPY TO improvements
> 
>
> Key: CASSANDRA-9304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9304
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Minor
>  Labels: cqlsh
> Fix For: 3.x, 2.1.x, 2.2.x
>
>
> COPY FROM has gotten a lot of love.  COPY TO not so much.  One obvious 
> improvement could be to parallelize reading and writing (write one page of 
> data while fetching the next).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10652) Tracing prevents startup after upgrading

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne reassigned CASSANDRA-10652:


Assignee: Sylvain Lebresne

> Tracing prevents startup after upgrading
> 
>
> Key: CASSANDRA-10652
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10652
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0
>
>
> After upgrading from 2.1 to 3.0, the {{system_traces.sessions}} table is not 
> properly upgraded to include the {{client}} column added in CASSANDRA-8162. 
> Just upgrading from a clean 2.2 install to 3.0 won't show this error because 
> the column was included in 2.2, it just doesn't break the queries in that 
> release.
> The errors I get when querying {{system_traces.sessions}}:
> {noformat}
> java.lang.RuntimeException: java.lang.IllegalStateException: 
> [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2350)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.lang.IllegalStateException: [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:465) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
> 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c42fcf57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c42fcf57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c42fcf57

Branch: refs/heads/trunk
Commit: c42fcf570904964298c2c0ec9fffab6aa0cc90db
Parents: 2a4b8f1 9940892
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 14:35:34 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 14:35:34 2015 +

--
 bin/cqlsh.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--

cassandra git commit: Fix cqlsh type handling after driver metadata api change

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 454f32dcb -> 9940892c3


Fix cqlsh type handling after driver metadata api change

patch by Paulo Motta; reviewed by Aleksey Yeschenko


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9940892c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9940892c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9940892c

Branch: refs/heads/cassandra-3.0
Commit: 9940892c3c89e1780cfbda72bfe8c19bd4b1e36a
Parents: 454f32d
Author: Paulo Motta 
Authored: Thu Nov 5 06:13:58 2015 -0800
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 14:35:01 2015 +

--
 bin/cqlsh.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9940892c/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 725f6e9..23d 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2394,7 +2394,7 @@ class ImportProcess(multiprocessing.Process):
 table_meta = new_cluster.metadata.keyspaces[self.ks].tables[self.cf]
 
 pk_cols = [col.name for col in table_meta.primary_key]
-cqltypes = [table_meta.columns[name].typestring for name in 
self.columns]
+cqltypes = [table_meta.columns[name].cql_type for name in self.columns]
 pk_indexes = [self.columns.index(col.name) for col in 
table_meta.primary_key]
 query = 'INSERT INTO %s.%s (%s) VALUES (%%s)' % (
 protect_name(table_meta.keyspace_name),

[1/2] cassandra git commit: Fix cqlsh type handling after driver metadata api change

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2a4b8f10e -> c42fcf570


Fix cqlsh type handling after driver metadata api change

patch by Paulo Motta; reviewed by Aleksey Yeschenko


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9940892c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9940892c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9940892c

Branch: refs/heads/trunk
Commit: 9940892c3c89e1780cfbda72bfe8c19bd4b1e36a
Parents: 454f32d
Author: Paulo Motta 
Authored: Thu Nov 5 06:13:58 2015 -0800
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 14:35:01 2015 +

--
 bin/cqlsh.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9940892c/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 725f6e9..23d 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -2394,7 +2394,7 @@ class ImportProcess(multiprocessing.Process):
 table_meta = new_cluster.metadata.keyspaces[self.ks].tables[self.cf]
 
 pk_cols = [col.name for col in table_meta.primary_key]
-cqltypes = [table_meta.columns[name].typestring for name in 
self.columns]
+cqltypes = [table_meta.columns[name].cql_type for name in self.columns]
 pk_indexes = [self.columns.index(col.name) for col in 
table_meta.primary_key]
 query = 'INSERT INTO %s.%s (%s) VALUES (%%s)' % (
 protect_name(table_meta.keyspace_name),

[jira] [Commented] (CASSANDRA-10242) Validate rack information on startup

[Marcus Olsson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991605#comment-14991605
 ] 

Marcus Olsson commented on CASSANDRA-10242:
---

I have a similar issue [here|CASSANDRA-9474], with the difference that the 
issue is for both the rack and dc validation. I think we should consider 
implementing the data center validation like it was done for racks here.

> Validate rack information on startup
> 
>
> Key: CASSANDRA-10242
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10242
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc2, 2.2.4, 2.1.12
>
>
> Moving to a new rack means that different data should be stored on a node.  
> We already persist rack information in a system table; we should fail startup 
> if this doesn't match what the snitch thinks it should be.  (Either the 
> snitch is wrong, and needs to be fixed, or the machine has been moved and 
> needs to be rebootstrapped.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-10551) Investigate JMX auth using JMXMP & SASL

[Jan Karlsson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Karlsson reassigned CASSANDRA-10551:


Assignee: Jan Karlsson

> Investigate JMX auth using JMXMP & SASL
> ---
>
> Key: CASSANDRA-10551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10551
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Jan Karlsson
> Fix For: 3.x
>
>
> (broken out from CASSANDRA-10091)
> We should look into whether using 
> [JMXMP|https://meteatamel.wordpress.com/2012/02/13/jmx-rmi-vs-jmxmp/] would 
> enable JMX authentication using SASL. If so, could we then define a custom 
> SaslServer which wraps a SaslNegotiator instance provided by the configured 
> IAuthenticator. 
> An intial look at the 
> [JMXMP|http://docs.oracle.com/cd/E19698-01/816-7609/6mdjrf873/] docs, 
> particularly section *11.4.2 SASL Provider*, suggests this might be feasible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[slebresne]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/65000b3c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/65000b3c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/65000b3c

Branch: refs/heads/trunk
Commit: 65000b3c2d508c6f972022607fb14b27d7d8a11d
Parents: be2cf1a 4a00438
Author: Sylvain Lebresne 
Authored: Thu Nov 5 11:16:05 2015 +0100
Committer: Sylvain Lebresne 
Committed: Thu Nov 5 11:16:05 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/UnfilteredDeserializer.java| 45 +++-
 .../columniterator/AbstractSSTableIterator.java |  4 +-
 .../columniterator/SSTableReversedIterator.java | 17 ++--
 4 files changed, 51 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/65000b3c/CHANGES.txt
--
diff --cc CHANGES.txt
index 4fcf4e9,1ff2fdb..14b6007
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * Fix reading of legacy sstables (CASSANDRA-10590)
   * Use CQL type names in schema metadata tables (CASSANDRA-10365)
   * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
   * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)

cassandra git commit: Don't account for unconsumed data when checking if past sstable index

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 242b9738f -> 4a00438a2


Don't account for unconsumed data when checking if past sstable index

patch by slebresne; reviewed by blambov for CASSANDRA-10590


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a00438a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a00438a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a00438a

Branch: refs/heads/cassandra-3.0
Commit: 4a00438a27cfa5bcd29cdcd9b30eb6d69e836a1f
Parents: 242b973
Author: Sylvain Lebresne 
Authored: Wed Oct 28 12:21:36 2015 +0100
Committer: Sylvain Lebresne 
Committed: Thu Nov 5 11:07:09 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/UnfilteredDeserializer.java| 45 +++-
 .../columniterator/AbstractSSTableIterator.java |  4 +-
 .../columniterator/SSTableReversedIterator.java | 17 ++--
 4 files changed, 51 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a00438a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5fdeae5..1ff2fdb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix reading of legacy sstables (CASSANDRA-10590)
  * Use CQL type names in schema metadata tables (CASSANDRA-10365)
  * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
  * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a00438a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 52de159..66f6b71 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -29,6 +29,7 @@ import org.slf4j.LoggerFactory;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.io.util.DataInputPlus;
+import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.net.MessagingService;
 
 /**
@@ -107,6 +108,20 @@ public abstract class UnfilteredDeserializer
  */
 public abstract void skipNext() throws IOException;
 
+
+/**
+ * For the legacy layout deserializer, we have to deal with the fact that 
a row can span multiple index blocks and that
+ * the call to hasNext() reads the next element upfront. We must take that 
into account when we check in AbstractSSTableIterator if
+ * we're past the end of an index block boundary as that check expect to 
account for only consumed data (that is, if hasNext has
+ * been called and made us cross an index boundary but neither readNext() 
or skipNext() as yet been called, we shouldn't consider
+ * the index block boundary crossed yet).
+ *
+ * TODO: we don't care about this for the current file format because a 
row can never span multiple index blocks (further, hasNext()
+ * only just basically read 2 bytes from disk in that case). So once we 
drop backward compatibility with pre-3.0 sstable, we should
+ * remove this.
+ */
+public abstract long bytesReadForUnconsumedData();
+
 private static class CurrentDeserializer extends UnfilteredDeserializer
 {
 private final ClusteringPrefix.Deserializer clusteringDeserializer;
@@ -216,6 +231,13 @@ public abstract class UnfilteredDeserializer
 isReady = false;
 isDone = false;
 }
+
+public long bytesReadForUnconsumedData()
+{
+// In theory, hasNext() does consume 2-3 bytes, but we don't care 
about this for the current file format so returning
+// 0 to mean "do nothing".
+return 0;
+}
 }
 
 public static class OldFormatDeserializer extends UnfilteredDeserializer
@@ -233,8 +255,8 @@ public abstract class UnfilteredDeserializer
 // The Unfiltered as read from the old format input
 private final UnfilteredIterator iterator;
 
-// Tracks which tombstone are opened at any given point of the 
deserialization. Note that this
-// is directly populated by UnfilteredIterator.
+// The position in the input after the last data consumption 
(readNext/skipNext).
+private long lastConsumedPosition;
 
 private OldFormatDeserializer(CFMetaData metadata,
   DataInputPlus in,
@@ -245,6 +267,7 @@ public abstract class 

cassandra git commit: Fix handling counters in supercolumns when parsing schema

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 4a00438a2 -> 18e2fca8b


Fix handling counters in supercolumns when parsing schema

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-10365


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18e2fca8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18e2fca8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18e2fca8

Branch: refs/heads/cassandra-3.0
Commit: 18e2fca8b47f9e2dc84ad43c957f0d226dd883d7
Parents: 4a00438
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 11:01:51 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 11:18:38 2015 +

--
 .../org/apache/cassandra/cql3/CQL3Type.java | 22 +---
 .../apache/cassandra/schema/CQLTypeParser.java  |  2 +-
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 3 files changed, 21 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/cql3/CQL3Type.java
--
diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java 
b/src/java/org/apache/cassandra/cql3/CQL3Type.java
index 7f5afa6..fde3fab 100644
--- a/src/java/org/apache/cassandra/cql3/CQL3Type.java
+++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java
@@ -17,10 +17,8 @@
  */
 package org.apache.cassandra.cql3;
 
-import java.nio.ByteBuffer;
 import java.util.ArrayList;
 import java.util.List;
-import java.util.Set;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -355,6 +353,11 @@ public interface CQL3Type
 
 public abstract CQL3Type prepare(String keyspace, Types udts) throws 
InvalidRequestException;
 
+public CQL3Type prepareInternal(String keyspace, Types udts) throws 
InvalidRequestException
+{
+return prepare(keyspace, udts);
+}
+
 public boolean referencesUserType(String name)
 {
 return false;
@@ -461,11 +464,24 @@ public interface CQL3Type
 
 public CQL3Type prepare(String keyspace, Types udts) throws 
InvalidRequestException
 {
+return prepare(keyspace, udts, false);
+}
+
+public CQL3Type prepareInternal(String keyspace, Types udts)
+{
+return prepare(keyspace, udts, true);
+}
+
+public CQL3Type prepare(String keyspace, Types udts, boolean 
isInternal) throws InvalidRequestException
+{
 assert values != null : "Got null values type for a 
collection";
 
 if (!frozen && values.supportsFreezing() && !values.frozen)
 throw new InvalidRequestException("Non-frozen collections 
are not allowed inside collections: " + this);
-if (values.isCounter())
+
+// we represent Thrift supercolumns as maps, internally, and 
we do allow counters in supercolumns. Thus,
+// for internal type parsing (think schema) we have to make an 
exception and allow counters as (map) values
+if (values.isCounter() && !isInternal)
 throw new InvalidRequestException("Counters are not 
allowed inside collections: " + this);
 
 if (keys != null)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/schema/CQLTypeParser.java
--
diff --git a/src/java/org/apache/cassandra/schema/CQLTypeParser.java 
b/src/java/org/apache/cassandra/schema/CQLTypeParser.java
index 87eebd7..ed68498 100644
--- a/src/java/org/apache/cassandra/schema/CQLTypeParser.java
+++ b/src/java/org/apache/cassandra/schema/CQLTypeParser.java
@@ -52,7 +52,7 @@ public final class CQLTypeParser
 if (udt != null)
 return udt;
 
-return parseRaw(unparsed).prepare(keyspace, userTypes).getType();
+return parseRaw(unparsed).prepareInternal(keyspace, 
userTypes).getType();
 }
 
 static CQL3Type.Raw parseRaw(String type)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/schema/Types.java
--
diff --git a/src/java/org/apache/cassandra/schema/Types.java 
b/src/java/org/apache/cassandra/schema/Types.java
index 0d6e36d..4f3d78c 100644
--- a/src/java/org/apache/cassandra/schema/Types.java
+++ b/src/java/org/apache/cassandra/schema/Types.java
@@ -274,7 +274,7 @@ public final class Types implements Iterable
 
 List preparedFieldTypes =
 fieldTypes.stream()
-  .map(t 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4effabc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4effabc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4effabc8

Branch: refs/heads/trunk
Commit: 4effabc8fe842224cd8e2a846274b617d2b4a9e6
Parents: 65000b3 18e2fca
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 11:20:06 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 11:20:06 2015 +

--
 .../org/apache/cassandra/cql3/CQL3Type.java | 22 +---
 .../apache/cassandra/schema/CQLTypeParser.java  |  2 +-
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 3 files changed, 21 insertions(+), 5 deletions(-)
--

[1/2] cassandra git commit: Fix handling counters in supercolumns when parsing schema

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 65000b3c2 -> 4effabc8f


Fix handling counters in supercolumns when parsing schema

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-10365


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18e2fca8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18e2fca8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18e2fca8

Branch: refs/heads/trunk
Commit: 18e2fca8b47f9e2dc84ad43c957f0d226dd883d7
Parents: 4a00438
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 11:01:51 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 11:18:38 2015 +

--
 .../org/apache/cassandra/cql3/CQL3Type.java | 22 +---
 .../apache/cassandra/schema/CQLTypeParser.java  |  2 +-
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 3 files changed, 21 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/cql3/CQL3Type.java
--
diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java 
b/src/java/org/apache/cassandra/cql3/CQL3Type.java
index 7f5afa6..fde3fab 100644
--- a/src/java/org/apache/cassandra/cql3/CQL3Type.java
+++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java
@@ -17,10 +17,8 @@
  */
 package org.apache.cassandra.cql3;
 
-import java.nio.ByteBuffer;
 import java.util.ArrayList;
 import java.util.List;
-import java.util.Set;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -355,6 +353,11 @@ public interface CQL3Type
 
 public abstract CQL3Type prepare(String keyspace, Types udts) throws 
InvalidRequestException;
 
+public CQL3Type prepareInternal(String keyspace, Types udts) throws 
InvalidRequestException
+{
+return prepare(keyspace, udts);
+}
+
 public boolean referencesUserType(String name)
 {
 return false;
@@ -461,11 +464,24 @@ public interface CQL3Type
 
 public CQL3Type prepare(String keyspace, Types udts) throws 
InvalidRequestException
 {
+return prepare(keyspace, udts, false);
+}
+
+public CQL3Type prepareInternal(String keyspace, Types udts)
+{
+return prepare(keyspace, udts, true);
+}
+
+public CQL3Type prepare(String keyspace, Types udts, boolean 
isInternal) throws InvalidRequestException
+{
 assert values != null : "Got null values type for a 
collection";
 
 if (!frozen && values.supportsFreezing() && !values.frozen)
 throw new InvalidRequestException("Non-frozen collections 
are not allowed inside collections: " + this);
-if (values.isCounter())
+
+// we represent Thrift supercolumns as maps, internally, and 
we do allow counters in supercolumns. Thus,
+// for internal type parsing (think schema) we have to make an 
exception and allow counters as (map) values
+if (values.isCounter() && !isInternal)
 throw new InvalidRequestException("Counters are not 
allowed inside collections: " + this);
 
 if (keys != null)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/schema/CQLTypeParser.java
--
diff --git a/src/java/org/apache/cassandra/schema/CQLTypeParser.java 
b/src/java/org/apache/cassandra/schema/CQLTypeParser.java
index 87eebd7..ed68498 100644
--- a/src/java/org/apache/cassandra/schema/CQLTypeParser.java
+++ b/src/java/org/apache/cassandra/schema/CQLTypeParser.java
@@ -52,7 +52,7 @@ public final class CQLTypeParser
 if (udt != null)
 return udt;
 
-return parseRaw(unparsed).prepare(keyspace, userTypes).getType();
+return parseRaw(unparsed).prepareInternal(keyspace, 
userTypes).getType();
 }
 
 static CQL3Type.Raw parseRaw(String type)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/18e2fca8/src/java/org/apache/cassandra/schema/Types.java
--
diff --git a/src/java/org/apache/cassandra/schema/Types.java 
b/src/java/org/apache/cassandra/schema/Types.java
index 0d6e36d..4f3d78c 100644
--- a/src/java/org/apache/cassandra/schema/Types.java
+++ b/src/java/org/apache/cassandra/schema/Types.java
@@ -274,7 +274,7 @@ public final class Types implements Iterable
 
 List preparedFieldTypes =
 fieldTypes.stream()
-  .map(t -> 

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991553#comment-14991553
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

bq. That's exactly what I had in mind so lgtm.

Thanks, pushed.

Going through other dtests now. Execution API was not the only thing to change. 
Because of this ticket, metadata API had changes to (to type representations at 
least), but it's all straightforward.

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991648#comment-14991648
 ] 

Robert Stupp commented on CASSANDRA-10365:
--

Some {{DROP (IF EXISTS) non_existing_keyspace.foo}} statements fail to execute 
(or raise an error).
Fix for that is in [this 
branch|https://github.com/snazy/cassandra/tree/10365-drop-if-exists] and 
cassia's working on [the 
utests|http://cassci.datastax.com/job/snazy-10365-drop-if-exists-testall/1/].

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10551) Investigate JMX auth using JMXMP & SASL

[Jan Karlsson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991610#comment-14991610
 ] 

Jan Karlsson commented on CASSANDRA-10551:
--

Changing to JMXMP seems to work from an implementation standpoint. However this 
will mean that current tools which are hardcoded to connect through RMI will 
have to be changed to function with JMXMP. I'm refering mostly to nodetool. 
i.e. eariler versions of nodetool will not be able to connect to the server.

What is more concerning is that some 3rd party tools like jconsole seem to lack 
the functionality to connect with Sasl profiles through jmxmp. I tried 
connecting with a [plain 
profile/mechanism|https://tools.ietf.org/html/rfc4616], but have not found a 
way to set a profile for jconsole.

> Investigate JMX auth using JMXMP & SASL
> ---
>
> Key: CASSANDRA-10551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10551
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Jan Karlsson
> Fix For: 3.x
>
>
> (broken out from CASSANDRA-10091)
> We should look into whether using 
> [JMXMP|https://meteatamel.wordpress.com/2012/02/13/jmx-rmi-vs-jmxmp/] would 
> enable JMX authentication using SASL. If so, could we then define a custom 
> SaslServer which wraps a SaslNegotiator instance provided by the configured 
> IAuthenticator. 
> An intial look at the 
> [JMXMP|http://docs.oracle.com/cd/E19698-01/816-7609/6mdjrf873/] docs, 
> particularly section *11.4.2 SASL Provider*, suggests this might be feasible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix NativeSSTableLoaderClient post CASSANDRA-10365

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4effabc8f -> 2a4b8f10e


Fix NativeSSTableLoaderClient post CASSANDRA-10365


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/454f32dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/454f32dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/454f32dc

Branch: refs/heads/trunk
Commit: 454f32dcb67dda3151da993dc74b5b0d0f5817b0
Parents: 18e2fca
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 13:02:03 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 13:02:03 2015 +

--
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 .../utils/NativeSSTableLoaderClient.java| 54 +++-
 2 files changed, 42 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/454f32dc/src/java/org/apache/cassandra/schema/Types.java
--
diff --git a/src/java/org/apache/cassandra/schema/Types.java 
b/src/java/org/apache/cassandra/schema/Types.java
index 4f3d78c..258be9f 100644
--- a/src/java/org/apache/cassandra/schema/Types.java
+++ b/src/java/org/apache/cassandra/schema/Types.java
@@ -237,7 +237,7 @@ public final class Types implements Iterable
 return Types.builder().add(types).build();
 }
 
-void add(String name, List fieldNames, List fieldTypes)
+public void add(String name, List fieldNames, List 
fieldTypes)
 {
 List rawFieldTypes =
 fieldTypes.stream()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/454f32dc/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 8dff532..cebee4a 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -30,7 +30,9 @@ import org.apache.cassandra.dht.*;
 import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.dht.Token.TokenFactory;
 import org.apache.cassandra.io.sstable.SSTableLoader;
+import org.apache.cassandra.schema.CQLTypeParser;
 import org.apache.cassandra.schema.SchemaKeyspace;
+import org.apache.cassandra.schema.Types;
 
 public class NativeSSTableLoaderClient extends SSTableLoader.Client
 {
@@ -79,9 +81,11 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 addRangeForEndpoint(range, endpoint.getAddress());
 }
 
-tables.putAll(fetchTablesMetadata(keyspace, session, partitioner));
+Types types = fetchTypes(keyspace, session);
+
+tables.putAll(fetchTables(keyspace, session, partitioner, types));
 // We only need the CFMetaData for the views, so we only load that.
-tables.putAll(fetchViewMetadata(keyspace, session, partitioner));
+tables.putAll(fetchViews(keyspace, session, partitioner, types));
 }
 }
 
@@ -96,6 +100,21 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 tables.put(cfm.cfName, cfm);
 }
 
+private static Types fetchTypes(String keyspace, Session session)
+{
+String query = String.format("SELECT * FROM %s.%s WHERE keyspace_name 
= ?", SchemaKeyspace.NAME, SchemaKeyspace.TYPES);
+
+Types.RawBuilder types = Types.rawBuilder(keyspace);
+for (Row row : session.execute(query, keyspace))
+{
+String name = row.getString("type_name");
+List fieldNames = row.getList("field_names", String.class);
+List fieldTypes = row.getList("field_types", String.class);
+types.add(name, fieldNames, fieldTypes);
+}
+return types.build();
+}
+
 /*
  * The following is a slightly simplified but otherwise duplicated version 
of
  * SchemaKeyspace.createTableFromTableRowAndColumnRows().
@@ -105,7 +124,7 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
  * Note: It is not safe for this class to use static methods from 
SchemaKeyspace (static final fields are ok)
  * as that triggers initialization of the class, which fails in client 
mode.
  */
-private static Map fetchTablesMetadata(String 
keyspace, Session session, IPartitioner partitioner)
+private static Map fetchTables(String keyspace, 
Session session, IPartitioner partitioner, Types types)
 {
 Map tables = new HashMap<>();
 String query = 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2a4b8f10
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2a4b8f10
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2a4b8f10

Branch: refs/heads/trunk
Commit: 2a4b8f10e9fe00dadebaebb9372ff15f7cc8c873
Parents: 4effabc 454f32d
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 13:03:16 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 13:03:16 2015 +

--
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 .../utils/NativeSSTableLoaderClient.java| 54 +++-
 2 files changed, 42 insertions(+), 14 deletions(-)
--

cassandra git commit: Fix NativeSSTableLoaderClient post CASSANDRA-10365

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 18e2fca8b -> 454f32dcb


Fix NativeSSTableLoaderClient post CASSANDRA-10365


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/454f32dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/454f32dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/454f32dc

Branch: refs/heads/cassandra-3.0
Commit: 454f32dcb67dda3151da993dc74b5b0d0f5817b0
Parents: 18e2fca
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 13:02:03 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 13:02:03 2015 +

--
 src/java/org/apache/cassandra/schema/Types.java |  2 +-
 .../utils/NativeSSTableLoaderClient.java| 54 +++-
 2 files changed, 42 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/454f32dc/src/java/org/apache/cassandra/schema/Types.java
--
diff --git a/src/java/org/apache/cassandra/schema/Types.java 
b/src/java/org/apache/cassandra/schema/Types.java
index 4f3d78c..258be9f 100644
--- a/src/java/org/apache/cassandra/schema/Types.java
+++ b/src/java/org/apache/cassandra/schema/Types.java
@@ -237,7 +237,7 @@ public final class Types implements Iterable
 return Types.builder().add(types).build();
 }
 
-void add(String name, List fieldNames, List fieldTypes)
+public void add(String name, List fieldNames, List 
fieldTypes)
 {
 List rawFieldTypes =
 fieldTypes.stream()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/454f32dc/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 8dff532..cebee4a 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -30,7 +30,9 @@ import org.apache.cassandra.dht.*;
 import org.apache.cassandra.dht.Token;
 import org.apache.cassandra.dht.Token.TokenFactory;
 import org.apache.cassandra.io.sstable.SSTableLoader;
+import org.apache.cassandra.schema.CQLTypeParser;
 import org.apache.cassandra.schema.SchemaKeyspace;
+import org.apache.cassandra.schema.Types;
 
 public class NativeSSTableLoaderClient extends SSTableLoader.Client
 {
@@ -79,9 +81,11 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 addRangeForEndpoint(range, endpoint.getAddress());
 }
 
-tables.putAll(fetchTablesMetadata(keyspace, session, partitioner));
+Types types = fetchTypes(keyspace, session);
+
+tables.putAll(fetchTables(keyspace, session, partitioner, types));
 // We only need the CFMetaData for the views, so we only load that.
-tables.putAll(fetchViewMetadata(keyspace, session, partitioner));
+tables.putAll(fetchViews(keyspace, session, partitioner, types));
 }
 }
 
@@ -96,6 +100,21 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
 tables.put(cfm.cfName, cfm);
 }
 
+private static Types fetchTypes(String keyspace, Session session)
+{
+String query = String.format("SELECT * FROM %s.%s WHERE keyspace_name 
= ?", SchemaKeyspace.NAME, SchemaKeyspace.TYPES);
+
+Types.RawBuilder types = Types.rawBuilder(keyspace);
+for (Row row : session.execute(query, keyspace))
+{
+String name = row.getString("type_name");
+List fieldNames = row.getList("field_names", String.class);
+List fieldTypes = row.getList("field_types", String.class);
+types.add(name, fieldNames, fieldTypes);
+}
+return types.build();
+}
+
 /*
  * The following is a slightly simplified but otherwise duplicated version 
of
  * SchemaKeyspace.createTableFromTableRowAndColumnRows().
@@ -105,7 +124,7 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
  * Note: It is not safe for this class to use static methods from 
SchemaKeyspace (static final fields are ok)
  * as that triggers initialization of the class, which fails in client 
mode.
  */
-private static Map fetchTablesMetadata(String 
keyspace, Session session, IPartitioner partitioner)
+private static Map fetchTables(String keyspace, 
Session session, IPartitioner partitioner, Types types)
 {
 Map tables = new HashMap<>();
 String query = 

[jira] [Comment Edited] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991648#comment-14991648
 ] 

Robert Stupp edited comment on CASSANDRA-10365 at 11/5/15 1:21 PM:
---

Some {{DROP (IF EXISTS) non_existing_keyspace.foo}} statements fail to execute 
(or raise an error).
Fix for that is in [this 
branch|https://github.com/snazy/cassandra/tree/10365-drop-if-exists] and 
cassci's working on [the 
utests|http://cassci.datastax.com/job/snazy-10365-drop-if-exists-testall/1/].

EDIT: damn auto-correction (cassci -> cassia)


was (Author: snazy):
Some {{DROP (IF EXISTS) non_existing_keyspace.foo}} statements fail to execute 
(or raise an error).
Fix for that is in [this 
branch|https://github.com/snazy/cassandra/tree/10365-drop-if-exists] and 
cassia's working on [the 
utests|http://cassci.datastax.com/job/snazy-10365-drop-if-exists-testall/1/].

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Don't account for unconsumed data when checking if past sstable index

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk be2cf1afa -> 65000b3c2


Don't account for unconsumed data when checking if past sstable index

patch by slebresne; reviewed by blambov for CASSANDRA-10590


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a00438a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a00438a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a00438a

Branch: refs/heads/trunk
Commit: 4a00438a27cfa5bcd29cdcd9b30eb6d69e836a1f
Parents: 242b973
Author: Sylvain Lebresne 
Authored: Wed Oct 28 12:21:36 2015 +0100
Committer: Sylvain Lebresne 
Committed: Thu Nov 5 11:07:09 2015 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/UnfilteredDeserializer.java| 45 +++-
 .../columniterator/AbstractSSTableIterator.java |  4 +-
 .../columniterator/SSTableReversedIterator.java | 17 ++--
 4 files changed, 51 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a00438a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5fdeae5..1ff2fdb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Fix reading of legacy sstables (CASSANDRA-10590)
  * Use CQL type names in schema metadata tables (CASSANDRA-10365)
  * Guard batchlog replay against integer division by zero (CASSANDRA-9223)
  * Fix bug when adding a column to thrift with the same name than a primary 
key (CASSANDRA-10608)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a00438a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
--
diff --git a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java 
b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
index 52de159..66f6b71 100644
--- a/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
+++ b/src/java/org/apache/cassandra/db/UnfilteredDeserializer.java
@@ -29,6 +29,7 @@ import org.slf4j.LoggerFactory;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.io.util.DataInputPlus;
+import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.net.MessagingService;
 
 /**
@@ -107,6 +108,20 @@ public abstract class UnfilteredDeserializer
  */
 public abstract void skipNext() throws IOException;
 
+
+/**
+ * For the legacy layout deserializer, we have to deal with the fact that 
a row can span multiple index blocks and that
+ * the call to hasNext() reads the next element upfront. We must take that 
into account when we check in AbstractSSTableIterator if
+ * we're past the end of an index block boundary as that check expect to 
account for only consumed data (that is, if hasNext has
+ * been called and made us cross an index boundary but neither readNext() 
or skipNext() as yet been called, we shouldn't consider
+ * the index block boundary crossed yet).
+ *
+ * TODO: we don't care about this for the current file format because a 
row can never span multiple index blocks (further, hasNext()
+ * only just basically read 2 bytes from disk in that case). So once we 
drop backward compatibility with pre-3.0 sstable, we should
+ * remove this.
+ */
+public abstract long bytesReadForUnconsumedData();
+
 private static class CurrentDeserializer extends UnfilteredDeserializer
 {
 private final ClusteringPrefix.Deserializer clusteringDeserializer;
@@ -216,6 +231,13 @@ public abstract class UnfilteredDeserializer
 isReady = false;
 isDone = false;
 }
+
+public long bytesReadForUnconsumedData()
+{
+// In theory, hasNext() does consume 2-3 bytes, but we don't care 
about this for the current file format so returning
+// 0 to mean "do nothing".
+return 0;
+}
 }
 
 public static class OldFormatDeserializer extends UnfilteredDeserializer
@@ -233,8 +255,8 @@ public abstract class UnfilteredDeserializer
 // The Unfiltered as read from the old format input
 private final UnfilteredIterator iterator;
 
-// Tracks which tombstone are opened at any given point of the 
deserialization. Note that this
-// is directly populated by UnfilteredIterator.
+// The position in the input after the last data consumption 
(readNext/skipNext).
+private long lastConsumedPosition;
 
 private OldFormatDeserializer(CFMetaData metadata,
   DataInputPlus in,
@@ -245,6 +267,7 @@ public abstract class UnfilteredDeserializer
  

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991481#comment-14991481
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

Well, that's annoying. {{prepareInternal}} it is then.

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991530#comment-14991530
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

[~slebresne] simple fix 
[here|https://github.com/iamaleksey/cassandra/commits/10365-counters-fix], does 
the minimal job to handle just supercolumns with counters, and doesn't go any 
further (and perhaps shouldn't).

Also, entirely unrelated, but it seems like we do allow counters as map keys, 
or at least don't validate it {{RawCollection}} (will follow up after 3.0).

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991544#comment-14991544
 ] 

Sylvain Lebresne commented on CASSANDRA-10365:
--

bq. simple fix here, does the minimal job to handle just supercolumns with 
counters

That's exactly what I had in mind so lgtm.

bq. it seems like we do allow counters as map keys, or at least don't validate 
it {{RawCollection}}

Saw that too and not sure why that is. Seems like an oversight though maybe we 
check some place else. We'll have to check.

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10652) Tracing prevents startup after upgrading

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991649#comment-14991649
 ] 

Sylvain Lebresne commented on CASSANDRA-10652:
--

There seems to be a hole in how we deal with upgrade of that trace keyspace. 
Basically, on upgrades, even if the schema of a trace table changed in 
{{TraceKeyspace}}, this won't be used because the table was pre-existing and 
the old schema is used. That's not a 3.0 bug per-se btw, I've manually checked 
that if you create a trace in 2.1 and upgrade to 2.2, then you won't get the 
{{client}} column (added by CASSANDRA-8162) back. In practice, this means 
CASSANDRA-8162 is only visible on new 2.2 clusters, but useless otherwise, 
which is kind of a problem.

Now, it doesn't have much more consequence in 2.1 -> 2.2 upgrade than making 
that new client column inaccessible, but 3.0 is more picky: while the node will 
use the old metadata, the rest of the tracing code *assumes* the client column 
exists and will write it, and 3.0 don't like writes to columns that don't exist.

Not sure what is the best fix yet. I'm not sure why we don't force the schema 
of those tracing tables to the most recent version. Probably going to try and 
see if something breaks if we do that, but if someone more familiar with the 
dealing of this keyspace wants to chime in with some other idea, feel free to.

> Tracing prevents startup after upgrading
> 
>
> Key: CASSANDRA-10652
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10652
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Priority: Blocker
> Fix For: 3.0.0
>
>
> After upgrading from 2.1 to 3.0, the {{system_traces.sessions}} table is not 
> properly upgraded to include the {{client}} column added in CASSANDRA-8162. 
> Just upgrading from a clean 2.2 install to 3.0 won't show this error because 
> the column was included in 2.2, it just doesn't break the queries in that 
> release.
> The errors I get when querying {{system_traces.sessions}}:
> {noformat}
> java.lang.RuntimeException: java.lang.IllegalStateException: 
> [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2350)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.lang.IllegalStateException: [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
> ~[main/:na]
>   at 
> 

[jira] [Commented] (CASSANDRA-10513) Update cqlsh for new driver execution API

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991475#comment-14991475
 ] 

Aleksey Yeschenko commented on CASSANDRA-10513:
---

Yes, sorry for my phrasing. Did not mean to make it sound like an oversight. 
Just suggested Paulo could do it as he's done the similar steps for cqlsh.

That said, given the time zones and the urgency, it will probably be handled by 
someone else (I'm on it too atm).

> Update cqlsh for new driver execution API
> -
>
> Key: CASSANDRA-10513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10513
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Minor
>  Labels: cqlsh
> Fix For: 2.2.4, 3.0.0
>
> Attachments: 10513-2.1.txt, 10513-2.2.txt, 10513-3.0-fix.patch, 
> 10513.txt
>
>
> The 3.0 Python driver will have a few tweaks to the execution API. We'll need 
> to update cqlsh in a couple of minor ways.
> [Results are always returned as an iterable 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-368]
> [Trace data is always attached to the 
> ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-318] (instead of 
> being attached to a statement)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10571) ClusteringIndexNamesFilter::shouldInclude is not implemented, SinglePartitionNamesCommand not discarding the sstables it could

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991880#comment-14991880
 ] 

Aleksey Yeschenko commented on CASSANDRA-10571:
---

+1

> ClusteringIndexNamesFilter::shouldInclude is not implemented, 
> SinglePartitionNamesCommand not discarding the sstables it could
> --
>
> Key: CASSANDRA-10571
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10571
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0
>
>
> Now that we use {{SinglePartitionNamesCommand}} in more places - where we'd 
> previously use what is now {{SinglePartitionSliceCommand}} - not being able 
> to skip sstables with non-overlapping clusterings is actually a performance 
> regression.
> {{SinglePartitionNamesCommand::queryMemtableAndDiskInternal}} should prune 
> sstables based on {{ClusteringIndexNamesFilter::shouldInclude}} output, and 
> the latter must be replaced with an actual implementation instead of a 
> {{TODO}}.
> This is also a potentially a big regression in performance for counter writes 
> (say, with DTCS), since before 3.0, the read-before-write code would use 
> {{collectAllData}}, and that *was* pruning sstable with non-overlapping 
> clusterings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10584) reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a ClassCastException

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991900#comment-14991900
 ] 

Carl Yeksigian commented on CASSANDRA-10584:


Yeah, I didn't notice that there was feedback on the dtest; my fault. I'll make 
sure to address.

> reads with EACH_QUORUM  on keyspace with SimpleTopologyStrategy throw a 
> ClassCastException
> --
>
> Key: CASSANDRA-10584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10584
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0
>
>
> I think this may be a regression introduced w/ [CASSANDRA-9602].  Starting 
> with C* 3.0.0-rc2 an error is returned when querying a keyspace with 
> {{SimpleTopologyStrategy}} using EACH_QUORUM CL:
> {noformat}
> cqlsh> create keyspace test with replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test (k int PRIMARY KEY, i int);
> cqlsh> consistency EACH_QUORUM;
> Consistency level set to EACH_QUORUM.
> cqlsh> select * from test.test;
> ServerError:  message="java.lang.ClassCastException: 
> org.apache.cassandra.locator.SimpleStrategy cannot be cast to 
> org.apache.cassandra.locator.NetworkTopologyStrategy">
> {noformat}
> The exception yielded in the system logs:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-10-23 13:02:15,405 ErrorMessage.java:336 - 
> Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy 
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForEachQuorum(ConsistencyLevel.java:227)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:188)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:180)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1795)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1762)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149) 
> ~[guava-18.0.jar:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1814)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1799)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1925)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1892)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager$PagerIterator.hasNext(AbstractQueryPager.java:99)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:327)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:205)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:236) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:221) 
> ~[main/:na]
> 

[jira] [Commented] (CASSANDRA-9930) Add test coverage for 2.1 -> 3.0 upgrades

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991998#comment-14991998
 ] 

Aleksey Yeschenko commented on CASSANDRA-9930:
--

[~rhatch] Can the issue be closed now?

> Add test coverage for 2.1 -> 3.0 upgrades
> -
>
> Key: CASSANDRA-9930
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9930
> Project: Cassandra
>  Issue Type: Test
>Reporter: Ryan McGuire
>Assignee: Andrew Hust
> Fix For: 3.0.0
>
>
> Once we move to a [tick-tock 
> model|http://www.mail-archive.com/dev%40cassandra.apache.org/msg07771.html] 
> we will need the ability to upgrade from stable release to stable release, 
> skipping a single version in the process.
> Example: 3.0 -> 3.2 (skipping the pass-through upgrade to 3.1 we normally 
> have to do)
> I would recommend we code this so that we can upgrade 2.1 directly to 3.0 as 
> well, it's probably too much work to do this for versions prior to that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10407) Benchmark and evaluate CASSANDRA-8894 improvements

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10407:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Benchmark and evaluate CASSANDRA-8894 improvements
> --
>
> Key: CASSANDRA-10407
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10407
> Project: Cassandra
>  Issue Type: Test
>Reporter: Aleksey Yeschenko
>Assignee: Alan Boudreault
> Fix For: 3.1
>
> Attachments: 3_0_head_mmap_wo_ra.nps, 3_0_head_std_wo_ra.nps, 
> 8894_tiny.yaml, allocateDirect.png, flight-recordings.tar.gz, 
> reBufferStandardTime.png, size.png, test-with-8894-tiny.json, 
> test-without-8894-tiny.json
>
>
> The original ticket (CASSANDRA-8894) was committed to 3.0 alpha1 two months 
> ago. We need to get proper performance tests before GA.
> See [~benedict]'s 
> [comment|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14631203=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14631203]
>  for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10563) Integrate new upgrade test into dtest upgrade suite

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10563:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Integrate new upgrade test into dtest upgrade suite
> ---
>
> Key: CASSANDRA-10563
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10563
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 3.1
>
>
> This is a follow-up ticket for CASSANDRA-10360, specifically [~slebresne]'s 
> comment here:
> https://issues.apache.org/jira/browse/CASSANDRA-10360?focusedCommentId=14966539=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14966539
> These tests should be incorporated into the [{{upgrade_tests}} in 
> dtest|https://github.com/riptano/cassandra-dtest/tree/master/upgrade_tests]. 
> I'll take this on; [~nutbunnies] is also a good person for it, but I'll 
> likely get to it first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10476) Fix upgrade paging dtest failures on 2.2->3.0 path

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10476:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Fix upgrade paging dtest failures on 2.2->3.0 path
> --
>
> Key: CASSANDRA-10476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10476
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Benjamin Lerer
> Fix For: 3.1
>
>
> EDIT: this list of failures is no longer current; see comments for current 
> failures.
> The following upgrade tests for paging features fail or flap on the upgrade 
> path from 2.2 to 3.0:
> - {{upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions}}
> - 
> {{upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging/}}
> I've grouped them all together because I don't know how to tell if they're 
> related; once someone triages them, it may be appropriate to break this out 
> into multiple tickets.
> The failures can be found here:
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingData/static_columns_paging_test/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingSize/test_undefined_page_size_default/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/42/testReport/upgrade_tests.paging_test/TestPagingSize/test_with_more_results_than_page_size/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_failure_threshold_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_multiple_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_cell_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingWithDeletions/test_single_row_deletions/history/
> http://cassci.datastax.com/view/Upgrades/job/storage_engine_upgrade_dtest-22_tarball-30_HEAD/44/testReport/upgrade_tests.paging_test/TestPagingDatasetChanges/test_cell_TTL_expiry_during_paging/
> Once [this dtest PR|https://github.com/riptano/cassandra-dtest/pull/586] is 
> merged, these tests should also run with this upgrade path on normal 3.0 
> jobs. Until then, you can run them with the following command:
> {code}
> SKIP=false CASSANDRA_VERSION=binary:2.2.0 UPGRADE_TO=git:cassandra-3.0 
> nosetests 
> upgrade_tests/paging_test.py:TestPagingData.static_columns_paging_test 
> upgrade_tests/paging_test.py:TestPagingSize.test_undefined_page_size_default 
> upgrade_tests/paging_test.py:TestPagingSize.test_with_more_results_than_page_size
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_failure_threshold_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_multiple_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_cell_deletions
>  
> upgrade_tests/paging_test.py:TestPagingWithDeletions.test_single_row_deletions
> upgrade_tests/paging_test.py:TestPagingDatasetChanges.test_cell_TTL_expiry_during_paging
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10656) Fix failing DeleteTest

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-10656:
-
Reviewer: Aleksey Yeschenko

> Fix failing DeleteTest
> --
>
> Key: CASSANDRA-10656
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10656
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.0
>
>
> DeleteTest fails in two ways:
> * it may not be able to setup the Java Driver (locally)
> * it fails with {{Not enough replicas available for query at consistency 
> LOCAL_QUORUM}} on cassci



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10584) reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a ClassCastException

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991872#comment-14991872
 ] 

Sylvain Lebresne commented on CASSANDRA-10584:
--

Looks like test eng was waiting on feedback from Carl. [~carlyeks] can you have 
a look?

> reads with EACH_QUORUM  on keyspace with SimpleTopologyStrategy throw a 
> ClassCastException
> --
>
> Key: CASSANDRA-10584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10584
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0
>
>
> I think this may be a regression introduced w/ [CASSANDRA-9602].  Starting 
> with C* 3.0.0-rc2 an error is returned when querying a keyspace with 
> {{SimpleTopologyStrategy}} using EACH_QUORUM CL:
> {noformat}
> cqlsh> create keyspace test with replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test (k int PRIMARY KEY, i int);
> cqlsh> consistency EACH_QUORUM;
> Consistency level set to EACH_QUORUM.
> cqlsh> select * from test.test;
> ServerError:  message="java.lang.ClassCastException: 
> org.apache.cassandra.locator.SimpleStrategy cannot be cast to 
> org.apache.cassandra.locator.NetworkTopologyStrategy">
> {noformat}
> The exception yielded in the system logs:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-10-23 13:02:15,405 ErrorMessage.java:336 - 
> Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy 
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForEachQuorum(ConsistencyLevel.java:227)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:188)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:180)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1795)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1762)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149) 
> ~[guava-18.0.jar:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1814)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1799)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1925)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1892)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager$PagerIterator.hasNext(AbstractQueryPager.java:99)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:327)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:205)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:236) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:221) 
> ~[main/:na]
> 

[jira] [Updated] (CASSANDRA-10612) Protocol v3 upgrade tests on 2.1->3.0 path fail when compaction is interrupted

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10612:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Protocol v3 upgrade tests on 2.1->3.0 path fail when compaction is interrupted
> --
>
> Key: CASSANDRA-10612
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10612
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Russ Hatch
> Fix For: 3.1
>
>
> The following tests in the upgrade_through_versions dtest suite fail:
> * 
> upgrade_through_versions_test.py:TestRandomPartitionerUpgrade.rolling_upgrade_test
> * 
> upgrade_through_versions_test.py:TestRandomPartitionerUpgrade.rolling_upgrade_with_internode_ssl_test
> * 
> upgrade_through_versions_test.py:TestUpgradeThroughVersions.rolling_upgrade_with_internode_ssl_test
> * 
> upgrade_through_versions_test.py:TestUpgradeThroughVersions.rolling_upgrade_test
> * 
> upgrade_through_versions_test.py:TestUpgrade_from_2_1_latest_tag_to_cassandra_3_0_HEAD.rolling_upgrade_test
> * 
> upgrade_through_versions_test.py:TestUpgrade_from_2_1_latest_tag_to_cassandra_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
> * 
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.rolling_upgrade_test
> * 
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.rolling_upgrade_with_internode_ssl_test
> * 
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.rolling_upgrade_test
> See this report:
> http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/
> They fail with the following error:
> {code}
> A subprocess has terminated early. Subprocess statuses: Process-41 (is_alive: 
> True), Process-42 (is_alive: False), Process-43 (is_alive: True), Process-44 
> (is_alive: False), attempting to terminate remaining subprocesses now.
> {code}
> and with logs that look like this:
> {code}
> Unexpected error in node1 node log: ['ERROR [SecondaryIndexManagement:1] 
> 2015-10-27 00:06:52,335 CassandraDaemon.java:195 - Exception in thread 
> Thread[SecondaryIndexManagement:1,5,main] java.lang.RuntimeException: 
> java.util.concurrent.ExecutionException: 
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
> interrupted: Secondary index 
> build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 578160/1663620)bytes
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) 
> ~[main/:na]
> at 
> org.apache.cassandra.index.internal.CassandraIndex.buildBlocking(CassandraIndex.java:688)
>  ~[main/:na]
> at 
> org.apache.cassandra.index.internal.CassandraIndex.lambda$getBuildIndexTask$206(CassandraIndex.java:658)
>  ~[main/:na]
> at 
> org.apache.cassandra.index.internal.CassandraIndex$$Lambda$151/1841229245.call(Unknown
>  Source) ~[na:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] Caused by: 
> java.util.concurrent.ExecutionException: 
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
> interrupted: Secondary index 
> build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 
> 578160/{code}1663620)bytes
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_51]
> at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_51]
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:364) 
> ~[main/:na]
> ... 7 common frames omitted Caused by: 
> org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction 
> interrupted: Secondary index 
> build@41202370-7c3e-11e5-9331-6bb6e58f8b1b(upgrade, cf, 578160/1663620)bytes
> at 
> org.apache.cassandra.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:67)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1269)
>  ~[main/:na]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_51]
> ... 4 common frames omitted', 'ERROR [HintsDispatcher:2] 2015-10-27 
> 00:08:48,520 CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]', 'ERROR [HintsDispatcher:2] 2015-10-27 
> 00:11:58,336 CassandraDaemon.java:195 - Exception in thread 
> Thread[HintsDispatcher:2,1,main]']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10610) Protocol upgrade tests for bootstrapping failing on 2.1->3.0 path

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10610:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Protocol upgrade tests for bootstrapping failing on 2.1->3.0 path
> -
>
> Key: CASSANDRA-10610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10610
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Russ Hatch
> Fix For: 3.1
>
>
> This ticket is for a couple failures on protocol v3 upgrade test job on 
> CassCI:
> http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/
> The failures are:
> * 
> [upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.bootstrap_multidc_test|http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD/bootstrap_multidc_test/]
> * 
> [upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag.bootstrap_multidc_test|http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_latest_tag/bootstrap_multidc_test/]
> They fail with the following error:
> {code}
> code=1000 [Unavailable exception] message="Cannot achieve consistency level 
> ALL" info={'required_replicas': 3, 'alive_replicas': 0, 'consistency': 'ALL'}
> {code}
> Assigning [~rhatch], since you're the most likely to understand what's going 
> on here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[snazy]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a38f362a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a38f362a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a38f362a

Branch: refs/heads/trunk
Commit: a38f362a04f5378266e8d466026634962f8d664e
Parents: c42fcf5 5289a56
Author: Robert Stupp 
Authored: Thu Nov 5 18:42:13 2015 +0100
Committer: Robert Stupp 
Committed: Thu Nov 5 18:42:13 2015 +0100

--
 .../org/apache/cassandra/cql3/DeleteTest.java   | 38 ++--
 1 file changed, 11 insertions(+), 27 deletions(-)
--

cassandra git commit: Fix failing DeleteTest

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9940892c3 -> 5289a564c


Fix failing DeleteTest

patch by Robert Stupp; reviewed by Aleksey Yeschenko for CASSANDRA-10656


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5289a564
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5289a564
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5289a564

Branch: refs/heads/cassandra-3.0
Commit: 5289a564cf929fe2ca9e8a37e1b8df53f2be0c6e
Parents: 9940892
Author: Robert Stupp 
Authored: Thu Nov 5 18:42:02 2015 +0100
Committer: Robert Stupp 
Committed: Thu Nov 5 18:42:02 2015 +0100

--
 .../org/apache/cassandra/cql3/DeleteTest.java   | 38 ++--
 1 file changed, 11 insertions(+), 27 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5289a564/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 0b88586..3c95a6c 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -17,27 +17,17 @@
  */
 package org.apache.cassandra.cql3;
 
-
-import org.junit.AfterClass;
 import org.junit.Assert;
-import org.junit.BeforeClass;
+import org.junit.Before;
 import org.junit.Test;
 
-import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.ConsistencyLevel;
 import com.datastax.driver.core.PreparedStatement;
 import com.datastax.driver.core.ResultSetFuture;
 import com.datastax.driver.core.Session;
-import org.apache.cassandra.SchemaLoader;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.Schema;
-import org.apache.cassandra.service.EmbeddedCassandraService;
 
-public class DeleteTest extends SchemaLoader
+public class DeleteTest extends CQLTester
 {
-private static EmbeddedCassandraService cassandra;
-
-private static Cluster cluster;
-private static Session session;
 private static PreparedStatement pstmtI;
 private static PreparedStatement pstmtU;
 private static PreparedStatement pstmtD;
@@ -47,16 +37,13 @@ public class DeleteTest extends SchemaLoader
 private static PreparedStatement pstmt4;
 private static PreparedStatement pstmt5;
 
-@BeforeClass
-public static void setup() throws Exception
+@Before
+public void prepare() throws Exception
 {
-Schema.instance.clear();
+// Schema.instance.clear();
 
-cassandra = new EmbeddedCassandraService();
-cassandra.start();
-
-cluster = 
Cluster.builder().addContactPoint("127.0.0.1").withPort(DatabaseDescriptor.getNativeTransportPort()).build();
-session = cluster.connect();
+Session session = sessionNet();
+
session.getCluster().getConfiguration().getQueryOptions().setConsistencyLevel(ConsistencyLevel.ONE);
 
 session.execute("drop keyspace if exists junit;");
 session.execute("create keyspace junit WITH REPLICATION = { 'class' : 
'SimpleStrategy', 'replication_factor' : 2 };");
@@ -106,15 +93,10 @@ public class DeleteTest extends SchemaLoader
 pstmt5 = session.prepare("select id, cid, inh_c, val from 
junit.tpc_inherit_c where id=? and cid=?");
 }
 
-@AfterClass
-public static void tearDown() throws Exception
-{
-cluster.close();
-}
-
 @Test
 public void lostDeletesTest()
 {
+Session session = sessionNet();
 
 for (int i = 0; i < 500; i++)
 {
@@ -151,6 +133,8 @@ public class DeleteTest extends SchemaLoader
 }
 
 private ResultSetFuture[] load() {
+Session session = sessionNet();
+
 return new ResultSetFuture[]{
 session.executeAsync(pstmt1.bind(1, 1)),
 session.executeAsync(pstmt2.bind(1, 1)),

[jira] [Commented] (CASSANDRA-10648) Native protocol is dead after running some Hive queries

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992116#comment-14992116
 ] 

Jim Witschey commented on CASSANDRA-10648:
--

[~alexliu68] How confident are you that this is a Cassandra issue, rather than 
a DSE Hive issue?

If possible, I'd be interested to see if using a different version of C* on the 
backend results in the same problem.

And, just to clarify, you think C* becomes unavailable because of performance 
issues, not some other bug making connecting via the default port not work 
anymore?

> Native protocol is dead after running some Hive queries
> ---
>
> Key: CASSANDRA-10648
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10648
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Liu
> Fix For: 2.2.x
>
> Attachments: 10_day_loss.q, deadlock.txt, deadlock1.txt
>
>
> When test on DSE portfolio demo, which basically creates a few C* tables and 
> inserts some data into the tables, then run some Hive queries on the tables.
> I attach the Hive queries
> After some queries are done, C* node is dead on native port, cqlsh can't 
> login any more.
> Some thread dumps are attached. Too many threads are in waiting mode and 
> system is not responding.
> It's tested on C* 2.2.3 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10650) Store UDA initcond as CQL literal in the schema table, instead of a blob

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10650:
--
Summary: Store UDA initcond as CQL literal in the schema table, instead of 
a blob  (was: Fix INITCOND after 10365)

> Store UDA initcond as CQL literal in the schema table, instead of a blob
> 
>
> Key: CASSANDRA-10650
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10650
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0
>
>
> CASSANDRA-10365 changed UDA's _initcond_ type to text. That required 
> CASSANDRA-10617 to be able to transform the serialized data to a valid CQL3 
> literal.
> This ticket is about to fix the _initcond_ thing broken in CASSANDRA-10365 
> after CASSANDRA-10617.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10645) sstableverify_test dtest fails on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10645:
--
Fix Version/s: (was: 3.0.0)
   3.1

> sstableverify_test dtest fails on Windows
> -
>
> Key: CASSANDRA-10645
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10645
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Joel Knighton
> Fix For: 3.1
>
>
> {{offline_tools_test.py:TestOfflineTools.sstableverify_test}} fails on CassCI 
> on C* 3.0 on Windows
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/offline_tools_test/TestOfflineTools/sstableverify_test/history/
> It fails consistently, but not always after checking the same number of 
> sstables. For instance:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/offline_tools_test/TestOfflineTools/sstableverify_test/
> vs.
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/93/testReport/junit/offline_tools_test/TestOfflineTools/sstableverify_test/
> I'm not sure if that's significant, but just in case, I thought it was worth 
> noting.
> Doesn't look like anyone's worked on this test particularly recently; 
> [~JoshuaMcKenzie], can you please find an assignee for this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10647) manual 2i rebuilding dtest failure on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10647:
--
Fix Version/s: (was: 3.0.0)
   3.1

> manual 2i rebuilding dtest failure on Windows
> -
>
> Key: CASSANDRA-10647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10647
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.1
>
>
> {{secondary_indexes_test.py:TestSecondaryIndexes.test_manual_rebuild_index}} 
> failed once on CassCI running C* 3.0 under Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/secondary_indexes_test/TestSecondaryIndexes/test_manual_rebuild_index/history/
> It fails here:
> https://github.com/riptano/cassandra-dtest/blob/master/secondary_indexes_test.py#L294
> with the error {{1 != 0}}. [~beobal] IIRC you understand these tests. Any 
> idea why it'd flap like this? or could this be a genuine regression?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10632) sstableutil tests failing on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10632:
--
Fix Version/s: (was: 3.0.0)
   3.1

> sstableutil tests failing on Windows
> 
>
> Key: CASSANDRA-10632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10632
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
> Fix For: 3.1
>
>
> {{sstableutil_test.py:SSTableUtilTest.abortedcompaction_test}} and 
> {{sstableutil_test.py:SSTableUtilTest.compaction_test}} fail on Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/abortedcompaction_test/
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/sstableutil_test/SSTableUtilTest/compaction_test/
> This is a pretty simple failure -- looks like the underlying behavior is ok, 
> but string comparison fails when the leading {{d}} in the filename is 
> lowercase as returned by {{sstableutil}} (see the [{{_invoke_sstableutil}} 
> test 
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L128]),
>  but uppercase as returned by {{glob.glob}} (see the [{{_get_sstable_files}} 
> test 
> function|https://github.com/riptano/cassandra-dtest/blob/master/sstableutil_test.py#L160]).
> Do I understand correctly that Windows filenames are case-insensitive, 
> including the drive portion? If that's the case, then we can just lowercase 
> the file names in the test helper functions above when the tests are run on 
> Windows. [~JoshuaMcKenzie] can you confirm? I'll fix this in the tests if so. 
> If I'm wrong, and something in {{sstableutil}} needs to be fixed, could you 
> find an assignee?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9930) Add test coverage for 2.1 -> 3.0 upgrades

[Russ Hatch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Russ Hatch resolved CASSANDRA-9930.
---
Resolution: Fixed

Yup, I think so. [~nutbunnies] feel free to reopen if there's something needed 
here that's not captured on another ticket.

> Add test coverage for 2.1 -> 3.0 upgrades
> -
>
> Key: CASSANDRA-9930
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9930
> Project: Cassandra
>  Issue Type: Test
>Reporter: Ryan McGuire
>Assignee: Andrew Hust
> Fix For: 3.0.0
>
>
> Once we move to a [tick-tock 
> model|http://www.mail-archive.com/dev%40cassandra.apache.org/msg07771.html] 
> we will need the ability to upgrade from stable release to stable release, 
> skipping a single version in the process.
> Example: 3.0 -> 3.2 (skipping the pass-through upgrade to 3.1 we normally 
> have to do)
> I would recommend we code this so that we can upgrade 2.1 directly to 3.0 as 
> well, it's probably too much work to do this for versions prior to that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10573) select_distinct_test flapping on 2.2 -> 3.0 upgrade path

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10573:
--
Fix Version/s: (was: 3.0.0)
   3.1

> select_distinct_test flapping on 2.2 -> 3.0 upgrade path
> 
>
> Key: CASSANDRA-10573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10573
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Benjamin Lerer
> Fix For: 3.1
>
>
> The {{upgrade_tests/cql_tests.py:TestCQLNodes2RF1.select_distinct_test 
> flaps}} dtest flaps on CassCI. It flaps locally for me as well:
> {code}
> CASSANDRA_VERSION=git:cassandra-3.0 nosetests 
> upgrade_tests/cql_tests.py:TestCQLNodes2RF1.select_distinct_test
> {code}
> Once [this dtest pr|https://github.com/riptano/cassandra-dtest/pull/624] is 
> merged, you can run against a local directory with 
> {code}
> CASSANDRA_VERSION=git:cassandra-3.0 OLD_CASSANDRA_VERSION=binary:2.2.3 
> UPGRADE_TO_DIR=~/path/to/cassandra nosetests 
> upgrade_tests/cql_tests.py:TestCQLNodes2RF1.select_distinct_test --no-skip
> {code}
> Apologies for the ugly interface for that. To run on Windows, use {{set}} to 
> set environment variables like {{CASSANDRA_VERSION}} instead of including 
> them in the command.
> The history for the test is here:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/280/testReport/junit/upgrade_tests.cql_tests/TestCQL/select_distinct_test/
> Note that the name of the class containing the test has changed from 
> {{TestCQL}} to ; its behavior hasn't.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991877#comment-14991877
 ] 

Aleksey Yeschenko commented on CASSANDRA-10365:
---

I think Paulo and I fixed most dtest issues, and none should be remaining that 
are caused by this particular commit, so I'm closing the ticket.

Thanks.

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10649) Cassandra 2.1 start failed with exception:Exception encountered during startup

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991971#comment-14991971
 ] 

Michael Shuler commented on CASSANDRA-10649:


In the attached cassandra.yaml file (is that your exact config?) there is an 
incorrectly (commented in attached file) spaced sub-config line {{#enabled: 
true}} at the beginning of the line after {{client_encryption_options:}} - this 
should have several spaces at the beginning of the line.

I would suggest downloading a fresh copy of a stock default cassandra.yaml file 
and carefully reviewing a diff of your changes.

2.1.11:
https://raw.githubusercontent.com/apache/cassandra/cassandra-2.1.11/conf/cassandra.yaml

2.1 branch HEAD:
https://raw.githubusercontent.com/apache/cassandra/cassandra-2.1/conf/cassandra.yaml

> Cassandra 2.1 start failed with exception:Exception encountered during startup
> --
>
> Key: CASSANDRA-10649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10649
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
> Environment: Linux: Fedora-16 64 bit
>Reporter: sandeep thakur
> Fix For: 2.1.x
>
> Attachments: cassandra.yaml
>
>
> I am trying to setup cassandra single node cluster. i've downloaded below 
> build:
> apache-cassandra-2.1.11-bin.tar.gz
> I've upgraded Java to 1.8 as well, as earlier it was throwing errors related 
> to Java version.
> {code}
> [root@localhost cassandra]# java -version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> {code}
> I've also verified the cassandra.yaml file from "http://www.yamllint.com/; as 
> well. But while starting cassandra, I am getting vague exception as below:
> {code}
> INFO  15:52:11 Compacting 
> [SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-18-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-17-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-20-Data.db'),
>  
> SSTableReader(path='/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-19-Data.db')]
> INFO  15:52:11 Node localhost/127.0.0.1 state jump to normal
> INFO  15:52:11 Netty using native Epoll event loop
> ERROR 15:52:11 Exception encountered during startup
> java.lang.NullPointerException: null
> at org.apache.cassandra.transport.Server.run(Server.java:171) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at org.apache.cassandra.transport.Server.start(Server.java:117) 
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
>  [apache-cassandra-2.1.11.jar:2.1.11]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651) 
> [apache-cassandra-2.1.11.jar:2.1.11]
> java.lang.NullPointerException
> at org.apache.cassandra.transport.Server.run(Server.java:171)
> at org.apache.cassandra.transport.Server.start(Server.java:117)
> at 
> org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:492)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:575)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651)
> Exception encountered during startup: null
> INFO  15:52:11 Announcing shutdown
> INFO  15:52:11 Node localhost/127.0.0.1 state jump to normal
> INFO  15:52:11 Compacted 4 sstables to 
> [/home/sandeep/bck_up/data/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-21,].
>   11,427 bytes to 5,749 (~50% of original) in 199ms = 0.027551MB/s.  4 total 
> partitions merged to 1.  Partition merge counts were {4:1, }
> INFO  15:52:13 Waiting for messaging service to quiesce
> INFO  15:52:13 MessagingService has terminated the accept() thread
> [root@localhost server]#
> {code}
> I've also posted the issue on stack overflow as well:
> http://stackoverflow.com/questions/33514745/cassandra-startup-failed-with-exception-exception-encountered-during-startup
> Request some one to assist on this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10644) multiple repair dtest fails under Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10644:
--
Fix Version/s: (was: 3.0.0)
   3.1

> multiple repair dtest fails under Windows
> -
>
> Key: CASSANDRA-10644
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10644
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 2.2.x, 3.1
>
>
> {{incremental_repair_test.py:TestIncRepair.multiple_repair_test}} flaps on 
> CassCI Windows runs on C* 3.0:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/incremental_repair_test/TestIncRepair/multiple_repair_test/history/
> The error is {{An existing connection was forcibly closed by the remote 
> host}}, and happens consistently in the failing runs:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/incremental_repair_test/TestIncRepair/multiple_repair_test/
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/junit/incremental_repair_test/TestIncRepair/multiple_repair_test/
> [~yukim] Can you have a look? I feel like you're more likely than anyone else 
> to understand the streaming error. In particular: is this what happens when a 
> node goes down? This could be an environment error, rather than a C* bug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10166) Fix failing tests

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10166:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Fix failing tests
> -
>
> Key: CASSANDRA-10166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10166
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.1
>
>
> Until we find a better way to track those things, this is meant as a master 
> ticket to track tickets open regarding tests (unit test and dtests, though at 
> the time of this writing only dtest are still failing) that are still 
> failing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10638) logged_batch_compatibility tests fail on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10638:
--
Fix Version/s: (was: 3.0.0)
   3.1

> logged_batch_compatibility tests fail on Windows
> 
>
> Key: CASSANDRA-10638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10638
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Paulo Motta
> Fix For: 3.1
>
>
> A number of the batch tests fail on Windows with a couple different errors. 
> On one build, they all fail with a pair of errors, e.g.:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/batch_test/TestBatch/logged_batch_compatibility_2_test_2/
> and
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/batch_test/TestBatch/logged_batch_compatibility_2_test/
> I can't tell if this is a C* failure or an environment issue, where the node 
> fails to start after being upgraded.
> On other runs, the tests flap when the test fails to delete temporary files. 
> Consider the failures here:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/batch_test/
> It could be an environmental issue, but I think I'd expect to see this error 
> in in other tests, but I haven't yet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10614) AssertionError while flushing memtables

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992013#comment-14992013
 ] 

Tyler Hobbs commented on CASSANDRA-10614:
-

Looks like the dtest is still problematic.  I've merged in the latest 
{{cassandra-3.0}} to trigger another run, hopefully that goes better.

> AssertionError while flushing memtables
> ---
>
> Key: CASSANDRA-10614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: debug.log, system.log
>
>
> While running mvbench against a single local node, I noticed this stacktrace 
> showing up multiple times in the logs:
> {noformat}
> ERROR 16:40:01 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.AssertionError: null
>   at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:49) 
> ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:45)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.SSTableTxnWriter.append(SSTableTxnWriter.java:52)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:389)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:352) 
> ~[main/:na]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[main/:na]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[main/:na]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45]
> {noformat}
> To reproduce, run mvbench with the regular schema and the following arguments:
> {noformat}
> mvn exec:java -Dexec.args="--num-users 10 --num-songs 100 
> --num-artists 1 -n 50 --endpoint 127.0.0.1"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10365) Store types by their CQL names in schema tables instead of fully-qualified internal class names

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991869#comment-14991869
 ] 

Robert Stupp commented on CASSANDRA-10365:
--

Well, you're right and I'm right ;)
10365 introduced NPEs on {{DROP TABLE (IF EXISTS)}} and throws 
{{ConfigurationException}} on {{DROP AGGREGATE/FUNCTION IF EXISTS}}.
2.1 and 2.2 are also affected by {{InvalidRequestException}} for {{DROP TYPE IF 
EXISTS}}
(opened CASSANDRA-10658)

> Store types by their CQL names in schema tables instead of fully-qualified 
> internal class names
> ---
>
> Key: CASSANDRA-10365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10365
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: client-impacting
> Fix For: 3.0.0
>
>
> Consider saving CQL types names for column, UDF/UDA arguments and return 
> types, and UDT components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10658) Some DROP ... IF EXISTS incorrectly result in exceptions on non-existing KS

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-10658:


 Summary: Some DROP ... IF EXISTS incorrectly result in exceptions 
on non-existing KS
 Key: CASSANDRA-10658
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10658
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
 Fix For: 2.1.x, 3.0.0


2.1, 2.2 and 3.0 incorrectly throws {{InvalidRequestException}} on non-existing 
keyspace for {{DROP TYPE IF EXISTS}}

3.0 incorrectly throws {{ConfigurationException}} for {{DROP AGGREGATE IF 
EXISTS}} with type arguments.
3.0 incorrectly throws {{ConfigurationException}} for {{DROP FUNCTION IF 
EXISTS}} with type arguments.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10570) HSHA test_closing_connections test flapping on 3.0

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991985#comment-14991985
 ] 

Joshua McKenzie commented on CASSANDRA-10570:
-

PR is merged. Is this good to close?

> HSHA test_closing_connections test flapping on 3.0
> --
>
> Key: CASSANDRA-10570
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10570
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Carl Yeksigian
> Fix For: 3.0.0
>
>
> See history here:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/280/testReport/thrift_hsha_test/ThriftHSHATest/test_closing_connections/history/
> The 3 most recent failures were:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/272/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/272/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/280/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/
> I haven't seen this failure on CassCI on the 2.2 branch, so it may be a 
> regression; I'm not sure.
> This is related to CASSANDRA-9369; it presents itself the same way. Looks 
> like that ticket was closed after a fix was merged to the dtests, but it 
> didn't fix this particular error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10646) crash_during_decommission_test dtest fails on windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10646:
--
Fix Version/s: (was: 3.0.0)
   3.1

> crash_during_decommission_test dtest fails on windows
> -
>
> Key: CASSANDRA-10646
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10646
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.1
>
>
> {{topology_test.py:TestTopology.crash_during_decommission_test}} flaps on on 
> C* 3.0 on Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/topology_test/TestTopology/crash_during_decommission_test/history/
> Since this test raises 2 errors on failure, there are 2 histories on CassCI 
> for it:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/topology_test/TestTopology/crash_during_decommission_test_2/history/
> It looks like it fails because of contention over the temporary file where 
> {{cassandra.env}} is stored:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/junit/topology_test/TestTopology/crash_during_decommission_test/
> Looks like this happens when {{nodetool status}} is called, since 
> {{nodetool}} sources {{cassandra-env.sh}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10656) Fix failing DeleteTest

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992061#comment-14992061
 ] 

Aleksey Yeschenko commented on CASSANDRA-10656:
---

+1

> Fix failing DeleteTest
> --
>
> Key: CASSANDRA-10656
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10656
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.0
>
>
> DeleteTest fails in two ways:
> * it may not be able to setup the Java Driver (locally)
> * it fails with {{Not enough replicas available for query at consistency 
> LOCAL_QUORUM}} on cassci



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10659) Windows CassCI: Fail on timed-out tests

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992110#comment-14992110
 ] 

Philip Thompson commented on CASSANDRA-10659:
-

I support option 3. (1) is too blunt of a solution, and fixing (2) ourselves 
may end up being more work than we need to solve our own problem.

> Windows CassCI: Fail on timed-out tests
> ---
>
> Key: CASSANDRA-10659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10659
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> On our Windows CassCI environments, it looks like some dtests are prone to 
> hanging, e.g.:
> https://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-10641_windows-dtest_win32/1/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/131/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/129/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/128/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/126/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/125/
> Ideally these tests wouldn't hang, but regardless, we should figure out a way 
> to make them fail, rather than timing out Jenkins and botching the rest of 
> the test run.
> The built-in [{{nosetests}} {{multiprocess}} 
> plugin|http://nose.readthedocs.org/en/latest/plugins/multiprocess.html] would 
> solve this problem for us -- we could run the tests with {{nosetests 
> --processes=1 --process-timeout=X}} and it would stop the test and fail if 
> the test took too long. However, it's broken on Windows. I've filed [a quick 
> issue on the {{nose}} GitHub|https://github.com/nose-devs/nose/issues/966], 
> but in the meantime, we should figure out how to avoid this.
> Possible solutions:
> # [~philipthompson] had a script that would shell out to {{nosetests}} for 
> each test and kill that process if it took too long. If I understand 
> correctly, that script is broken, or assumes things that are no longer true. 
> We can revamp it if we want.
> # We could make a patch for {{nose}} to fix the {{multiprocess}} plugin.
> # We could hack in some of {{multiprocessing}}'s functionality into the 
> {{dtest}} suite itself.
> 3. may be the best workaround for this problem -- our timeouts aren't caused 
> just when a tests runs long, but when Jenkins doesn't get any output on 
> stdout from a hanging test. We may be able to monitor stdout from a second 
> process and fail the test before Jenkins would time out.
> Pinging [~JoshuaMcKenzie] as this is a Windows issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10658) Some DROP ... IF EXISTS incorrectly result in exceptions on non-existing KS

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10658:
--
Fix Version/s: (was: 3.0.0)
   3.0.x
   2.2.x

> Some DROP ... IF EXISTS incorrectly result in exceptions on non-existing KS
> ---
>
> Key: CASSANDRA-10658
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10658
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> 2.1, 2.2 and 3.0 incorrectly throws {{InvalidRequestException}} on 
> non-existing keyspace for {{DROP TYPE IF EXISTS}}
> 3.0 incorrectly throws {{ConfigurationException}} for {{DROP AGGREGATE IF 
> EXISTS}} with type arguments.
> 3.0 incorrectly throws {{ConfigurationException}} for {{DROP FUNCTION IF 
> EXISTS}} with type arguments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10656) Fix failing DeleteTest

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991955#comment-14991955
 ] 

Robert Stupp commented on CASSANDRA-10656:
--

CI looks fine (0 failed tests)

> Fix failing DeleteTest
> --
>
> Key: CASSANDRA-10656
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10656
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.0
>
>
> DeleteTest fails in two ways:
> * it may not be able to setup the Java Driver (locally)
> * it fails with {{Not enough replicas available for query at consistency 
> LOCAL_QUORUM}} on cassci



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Fix failing DeleteTest

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk c42fcf570 -> a38f362a0


Fix failing DeleteTest

patch by Robert Stupp; reviewed by Aleksey Yeschenko for CASSANDRA-10656


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5289a564
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5289a564
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5289a564

Branch: refs/heads/trunk
Commit: 5289a564cf929fe2ca9e8a37e1b8df53f2be0c6e
Parents: 9940892
Author: Robert Stupp 
Authored: Thu Nov 5 18:42:02 2015 +0100
Committer: Robert Stupp 
Committed: Thu Nov 5 18:42:02 2015 +0100

--
 .../org/apache/cassandra/cql3/DeleteTest.java   | 38 ++--
 1 file changed, 11 insertions(+), 27 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5289a564/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 0b88586..3c95a6c 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -17,27 +17,17 @@
  */
 package org.apache.cassandra.cql3;
 
-
-import org.junit.AfterClass;
 import org.junit.Assert;
-import org.junit.BeforeClass;
+import org.junit.Before;
 import org.junit.Test;
 
-import com.datastax.driver.core.Cluster;
+import com.datastax.driver.core.ConsistencyLevel;
 import com.datastax.driver.core.PreparedStatement;
 import com.datastax.driver.core.ResultSetFuture;
 import com.datastax.driver.core.Session;
-import org.apache.cassandra.SchemaLoader;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.Schema;
-import org.apache.cassandra.service.EmbeddedCassandraService;
 
-public class DeleteTest extends SchemaLoader
+public class DeleteTest extends CQLTester
 {
-private static EmbeddedCassandraService cassandra;
-
-private static Cluster cluster;
-private static Session session;
 private static PreparedStatement pstmtI;
 private static PreparedStatement pstmtU;
 private static PreparedStatement pstmtD;
@@ -47,16 +37,13 @@ public class DeleteTest extends SchemaLoader
 private static PreparedStatement pstmt4;
 private static PreparedStatement pstmt5;
 
-@BeforeClass
-public static void setup() throws Exception
+@Before
+public void prepare() throws Exception
 {
-Schema.instance.clear();
+// Schema.instance.clear();
 
-cassandra = new EmbeddedCassandraService();
-cassandra.start();
-
-cluster = 
Cluster.builder().addContactPoint("127.0.0.1").withPort(DatabaseDescriptor.getNativeTransportPort()).build();
-session = cluster.connect();
+Session session = sessionNet();
+
session.getCluster().getConfiguration().getQueryOptions().setConsistencyLevel(ConsistencyLevel.ONE);
 
 session.execute("drop keyspace if exists junit;");
 session.execute("create keyspace junit WITH REPLICATION = { 'class' : 
'SimpleStrategy', 'replication_factor' : 2 };");
@@ -106,15 +93,10 @@ public class DeleteTest extends SchemaLoader
 pstmt5 = session.prepare("select id, cid, inh_c, val from 
junit.tpc_inherit_c where id=? and cid=?");
 }
 
-@AfterClass
-public static void tearDown() throws Exception
-{
-cluster.close();
-}
-
 @Test
 public void lostDeletesTest()
 {
+Session session = sessionNet();
 
 for (int i = 0; i < 500; i++)
 {
@@ -151,6 +133,8 @@ public class DeleteTest extends SchemaLoader
 }
 
 private ResultSetFuture[] load() {
+Session session = sessionNet();
+
 return new ResultSetFuture[]{
 session.executeAsync(pstmt1.bind(1, 1)),
 session.executeAsync(pstmt2.bind(1, 1)),

[jira] [Updated] (CASSANDRA-10639) Commitlog compression test fails on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10639:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Commitlog compression test fails on Windows
> ---
>
> Key: CASSANDRA-10639
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10639
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Joshua McKenzie
> Fix For: 3.1
>
>
> {{commitlog_test.py:TestCommitLog.test_compression_error}} fails on Windows 
> under CassCI. It fails in a number of different ways. Here, it looks like 
> reading the CRC fails:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/commitlog_test/TestCommitLog/test_compression_error/
> Here, I believe it fails when trying to validate the CRC header:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/99/testReport/commitlog_test/TestCommitLog/test_compression_error/
> https://github.com/riptano/cassandra-dtest/blob/master/commitlog_test.py#L497
> Here's another failure where the header has a {{Q}} written in it instead of 
> a closing brace:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/91/testReport/junit/commitlog_test/TestCommitLog/test_compression_error/
> https://github.com/riptano/cassandra-dtest/blob/master/commitlog_test.py#L513
> [~bdeggleston] Do I remember correctly that you wrote this test? Can you take 
> this on?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10642) cqlsh COPY bulk round trip dtest flaps on Windows

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10642:
--
Fix Version/s: (was: 3.0.0)
   3.1

> cqlsh COPY bulk round trip dtest flaps on Windows
> -
>
> Key: CASSANDRA-10642
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10642
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.1
>
>
> {{cqlsh_tests/cqlsh_copy_tests.py:CqlshCopyTest.test_bulk_round_trip}} flaps 
> on Windows under both C* 3.0 and 2.2:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_bulk_round_trip/history/
> http://cassci.datastax.com/view/win32/job/cassandra-2.2_dtest_win32/127/testReport/junit/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_bulk_round_trip/
> It fails because, after round-tripping with cqlsh COPY, it fails to find as 
> many values as expected with {{SELECT COUNT (\*)}}. The stderr from cqlsh 
> includes the following error:
> {code}
> (EE)  (EE)  :2:(EE)   out waiting for replica nodes' responses] message="Operation timed out - 
> received only 0 responses.">(EE)  :2:Aborting import at record #190. 
> Previously inserted records are still present, and some records after that 
> may be present as well.(EE)  :2:(EE)   [Unavailable exception] message="Cannot achieve consistency level ONE">(EE)  
> :2:Aborting import at record #637. Previously inserted records are 
> still present, and some records after that may be present as well.(EE)  
> {code}
> So, it looks like the load step may be timing out. See this run for the error:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/100/testReport/junit/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_bulk_round_trip/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10613) Upgrade test on 2.1->3.0 path fails with NPE in getExistingFiles (likely known bug)

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10613:
--
Fix Version/s: (was: 3.0.0)
   3.1

> Upgrade test on 2.1->3.0 path fails with NPE in getExistingFiles (likely 
> known bug)
> ---
>
> Key: CASSANDRA-10613
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10613
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Russ Hatch
> Fix For: 3.1
>
>
> In this job:
> http://cassci.datastax.com/view/Upgrades/job/cassandra_upgrade_2.1_to_3.0_proto_v3/10/
> The following tests fail due to an NPE in 
> {{org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles}}:
> upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.bootstrap_test
> upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.rolling_upgrade_test
> upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.parallel_upgrade_with_internode_ssl_test
> upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_1_HEAD_to_cassandra_3_0_HEAD.rolling_upgrade_with_internode_ssl_test
> upgrade_through_versions_test.py:TestUpgrade_from_3_0_latest_tag_to_3_0_HEAD.parallel_upgrade_test
> I believe this is likely happening because of CASSANDRA-10602, so let's hold 
> off on messing with this until that's merged.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10659) Windows CassCI: Fail on timed-out tests

[Jim Witschey (JIRA)]

Jim Witschey created CASSANDRA-10659:


 Summary: Windows CassCI: Fail on timed-out tests
 Key: CASSANDRA-10659
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10659
 Project: Cassandra
  Issue Type: Bug
Reporter: Jim Witschey
Assignee: Jim Witschey


On our Windows CassCI environments, it looks like some dtests are prone to 
hanging, e.g.:

https://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-10641_windows-dtest_win32/1/
http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/131/
http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/129/
http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/128/
http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/126/
http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/125/

Ideally these tests wouldn't hang, but regardless, we should figure out a way 
to make them fail, rather than timing out Jenkins and botching the rest of the 
test run.

The built-in [{{nosetests}} {{multiprocess}} 
plugin|http://nose.readthedocs.org/en/latest/plugins/multiprocess.html] would 
solve this problem for us -- we could run the tests with {{nosetests 
--processes=1 --process-timeout=X}} and it would stop the test and fail if the 
test took too long. However, it's broken on Windows. I've filed [a quick issue 
on the {{nose}} GitHub|https://github.com/nose-devs/nose/issues/966], but in 
the meantime, we should figure out how to avoid this.

Possible solutions:

# [~philipthompson] had a script that would shell out to {{nosetests}} for each 
test and kill that process if it took too long. If I understand correctly, that 
script is broken, or assumes things that are no longer true. We can revamp it 
if we want.
# We could make a patch for {{nose}} to fix the {{multiprocess}} plugin.
# We could hack in some of {{multiprocessing}}'s functionality into the 
{{dtest}} suite itself.

3. may be the best workaround for this problem -- our timeouts aren't caused 
just when a tests runs long, but when Jenkins doesn't get any output on stdout 
from a hanging test. We may be able to monitor stdout from a second process and 
fail the test before Jenkins would time out.

Pinging [~JoshuaMcKenzie] as this is a Windows issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10660) Support user-defined compactions through nodetool

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-10660:
---

 Summary: Support user-defined compactions through nodetool
 Key: CASSANDRA-10660
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10660
 Project: Cassandra
  Issue Type: New Feature
  Components: Tools
Reporter: Tyler Hobbs
Priority: Minor


For a long time, we've supported running user-defined compactions through JMX.  
This comes in handy fairly often, mostly when dealing with low disk space or 
tombstone purging, so it would be good to add something to nodetool for this.  
An extra option for {{nodetool compact}} would probably suffice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-10600:

Attachment: 10600-v2.txt

LGTM, uploaded v2 version removing unnecessary trailing whitespaces.

(lots of unrelated dtests failures, probably caused by python driver update of 
CASSANDRA-10365, but in pair with latest dtest runs: 
[3.0|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/327/testReport/]
 and 
[trunk|http://cassci.datastax.com/view/trunk/job/trunk_dtest/731/testReport/].

> CqlInputFormat throws IOE if the size estimates are zero
> 
>
> Key: CASSANDRA-10600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.x, 3.0.x
>
> Attachments: 10600-v2.txt, 10600.txt
>
>
> {{CqlInputFormat.describeSplits}} handles the case of no entry in the 
> 'system.size_estimates' table but does not handle the situation when there is 
> a zero size estimate in the table. This can happen if an input job is started 
> immediately after data is added but before the {{SizeEstimatesRecorder}} has 
> run.
> {{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same 
> manner as no size estimate and not sub-split but return the full split. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-10634:
--
Fix Version/s: 3.0.0

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992394#comment-14992394
 ] 

Carl Yeksigian commented on CASSANDRA-10634:


+1

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10629) IllegalArgumentException throw from CFMetaData when read meta information for dropped column

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992184#comment-14992184
 ] 

Jim Witschey commented on CASSANDRA-10629:
--

[~philipthompson] This ties in to some of the Hadoop code -- could you have a 
look please?

> IllegalArgumentException throw from CFMetaData when read meta information for 
> dropped column
> 
>
> Key: CASSANDRA-10629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10629
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Fang
>
> Used Spotify HDFS2CASS to bulk load data, everything is fine. After dropped 
> some columns, then can't read meta data, and seems result.getMap() call has 
> issue in src/java/org/apache/cassandra/config/CFMetaData.java, can't compose 
> type data from binary data.
> {code}
> java.lang.RuntimeException: Could not retrieve endpoint ranges: 
> at 
> com.spotify.hdfs2cass.cassandra.cql.CrunchExternalClient.init(CrunchExternalClient.java:141)
> at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:156)
> at 
> com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:158)
> at 
> com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:144)
> at org.apache.crunch.io.CrunchOutputs.close(CrunchOutputs.java:138)
> at 
> org.apache.crunch.impl.mr.run.CrunchTaskContext.cleanup(CrunchTaskContext.java:72)
> at org.apache.crunch.impl.mr.run.CrunchReducer.cleanup(CrunchReducer.java:64)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:267)
> at 
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543)
> at 
> org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124)
> at 
> org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101)
> at 
> org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30)
> at 
> org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50)
> at org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68)
> at 
> org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:282)
> at 
> org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1815)
> at 
> org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1116)
> at 
> com.spotify.hdfs2cass.cassandra.cql.CrunchExternalClient.init(CrunchExternalClient.java:133)
> ... 14 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10627) java.lang.NoSuchMethodException: forceKeyspaceCompaction on nodetool compact when running locally

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10627:
-
Description: 
I pulled latest bits from HEAD on 10/30. Started Cassandra in Eclipse and tried 
to run "nodetool compact" which resulted in the exception in title. Other 
nodetool commands and cqlsh work fine, so there are no issues with my setup.

Note this works fine on older versions (I tried on 2.1.9 Tag). 

If there are tips on how to debug, happy to submit a patch.

Full stack below:

{code}
 9:08:35.35;E:\DI\cassandra\bin>nodetool compact
error: forceKeyspaceCompaction(java.lang.String, [Ljava.lang.String;)
-- StackTrace --
java.lang.NoSuchMethodException: forceKeyspaceCompaction(java.lang.String, 
[Ljava.lang.String;)
at com.sun.jmx.mbeanserver.PerInterface.noSuchMethod(Unknown Source)
at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown 
Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown 
Source)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown 
Source)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown 
Source)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown 
Source)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
{code}

  was:
I pulled latest bits from HEAD on 10/30. Started Cassandra in Eclipse and tried 
to run "nodetool compact" which resulted in the exception in title. Other 
nodetool commands and cqlsh work fine, so there are no issues with my setup.

Note this works fine on older versions (I tried on 2.1.9 Tag). 

If there are tips on how to debug, happy to submit a patch.

Full stack below:

 9:08:35.35;E:\DI\cassandra\bin>nodetool compact
error: forceKeyspaceCompaction(java.lang.String, [Ljava.lang.String;)
-- StackTrace --
java.lang.NoSuchMethodException: forceKeyspaceCompaction(java.lang.String, 
[Ljava.lang.String;)
at com.sun.jmx.mbeanserver.PerInterface.noSuchMethod(Unknown Source)
at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown 
Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown 
Source)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown 
Source)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown 
Source)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown 
Source)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)


> java.lang.NoSuchMethodException: forceKeyspaceCompaction on nodetool compact 
> when running locally
> -
>
> Key: CASSANDRA-10627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10627
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Development environment.
>Reporter: Anubhav Kale
>Priority: Minor
> Fix For: 3.x
>
>
> I pulled latest bits from HEAD on 10/30. Started Cassandra in Eclipse and 
> tried to run "nodetool compact" which resulted in the exception in title. 
> Other nodetool commands and cqlsh work fine, so there are no issues with my 
> setup.
> Note this works fine on older versions (I tried on 2.1.9 Tag). 
> If there are tips on how to debug, happy to submit a patch.
> Full stack below:
> {code}
>  9:08:35.35;E:\DI\cassandra\bin>nodetool compact
> error: forceKeyspaceCompaction(java.lang.String, [Ljava.lang.String;)
> -- StackTrace --
> java.lang.NoSuchMethodException: forceKeyspaceCompaction(java.lang.String, 
> [Ljava.lang.String;)
> at com.sun.jmx.mbeanserver.PerInterface.noSuchMethod(Unknown Source)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
> at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown Source)
> at 

[jira] [Commented] (CASSANDRA-10659) Windows CassCI: Fail on timed-out tests

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992284#comment-14992284
 ] 

Joshua McKenzie commented on CASSANDRA-10659:
-

I'd recommend a brief inspection of the amount of work required for #2 before 
taking on the burden of porting in and maintaining multiprocessing in the dtest 
suite. I agree w/Philip that #1 is best left alone.

> Windows CassCI: Fail on timed-out tests
> ---
>
> Key: CASSANDRA-10659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10659
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> On our Windows CassCI environments, it looks like some dtests are prone to 
> hanging, e.g.:
> https://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-10641_windows-dtest_win32/1/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/131/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/129/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/128/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/126/
> http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/125/
> Ideally these tests wouldn't hang, but regardless, we should figure out a way 
> to make them fail, rather than timing out Jenkins and botching the rest of 
> the test run.
> The built-in [{{nosetests}} {{multiprocess}} 
> plugin|http://nose.readthedocs.org/en/latest/plugins/multiprocess.html] would 
> solve this problem for us -- we could run the tests with {{nosetests 
> --processes=1 --process-timeout=X}} and it would stop the test and fail if 
> the test took too long. However, it's broken on Windows. I've filed [a quick 
> issue on the {{nose}} GitHub|https://github.com/nose-devs/nose/issues/966], 
> but in the meantime, we should figure out how to avoid this.
> Possible solutions:
> # [~philipthompson] had a script that would shell out to {{nosetests}} for 
> each test and kill that process if it took too long. If I understand 
> correctly, that script is broken, or assumes things that are no longer true. 
> We can revamp it if we want.
> # We could make a patch for {{nose}} to fix the {{multiprocess}} plugin.
> # We could hack in some of {{multiprocessing}}'s functionality into the 
> {{dtest}} suite itself.
> 3. may be the best workaround for this problem -- our timeouts aren't caused 
> just when a tests runs long, but when Jenkins doesn't get any output on 
> stdout from a hanging test. We may be able to monitor stdout from a second 
> process and fail the test before Jenkins would time out.
> Pinging [~JoshuaMcKenzie] as this is a Windows issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9328) WriteTimeoutException thrown when LWT concurrency > 1, despite the query duration taking MUCH less than cas_contention_timeout_in_ms

[Aaron Whiteside (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Whiteside updated CASSANDRA-9328:
---
Comment: was deleted

(was: Completely agree here, if you need to add some sort of 
versioning/transaction id to detect changes then using CAS/LWT is pointless and 
you can achieve the same result with Cassandra's default eventual consistency 
behavior + versioning/transaction id. 

Which means CAS/LWT are completely broken and meaningless.)

> WriteTimeoutException thrown when LWT concurrency > 1, despite the query 
> duration taking MUCH less than cas_contention_timeout_in_ms
> 
>
> Key: CASSANDRA-9328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9328
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aaron Whiteside
> Fix For: 2.1.x
>
> Attachments: CassandraLWTTest.java, CassandraLWTTest2.java
>
>
> WriteTimeoutException thrown when LWT concurrency > 1, despite the query 
> duration taking MUCH less than cas_contention_timeout_in_ms.
> Unit test attached, run against a 3 node cluster running 2.1.5.
> If you reduce the threadCount to 1, you never see a WriteTimeoutException. If 
> the WTE is due to not being able to communicate with other nodes, why does 
> the concurrency >1 cause inter-node communication to fail?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10652) Tracing prevents startup after upgrading

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-10652:
---
Reviewer: Carl Yeksigian

> Tracing prevents startup after upgrading
> 
>
> Key: CASSANDRA-10652
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10652
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Carl Yeksigian
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 2.2.x, 3.0.0
>
>
> After upgrading from 2.1 to 3.0, the {{system_traces.sessions}} table is not 
> properly upgraded to include the {{client}} column added in CASSANDRA-8162. 
> Just upgrading from a clean 2.2 install to 3.0 won't show this error because 
> the column was included in 2.2, it just doesn't break the queries in that 
> release.
> The errors I get when querying {{system_traces.sessions}}:
> {noformat}
> java.lang.RuntimeException: java.lang.IllegalStateException: 
> [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2350)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: java.lang.IllegalStateException: [ColumnDefinition{name=client, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=command, 
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=coordinator, 
> type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=duration, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, 
> type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, 
> position=-1}, ColumnDefinition{name=parameters, 
> type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),
>  kind=REGULAR, position=-1}] is not a subset of [coordinator duration request 
> started_at parameters]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:465) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:298)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
>  

[jira] [Created] (CASSANDRA-10662) Make UFTest.testAmokUDF non-flappy

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-10662:


 Summary: Make UFTest.testAmokUDF non-flappy
 Key: CASSANDRA-10662
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10662
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
 Fix For: 3.1


UFTest.testAmokUDF currently relies on some (strict) timeouts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10629) IllegalArgumentException throw from CFMetaData when read meta information for dropped column

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-10629:
-
Description: 
Used Spotify HDFS2CASS to bulk load data, everything is fine. After dropped 
some columns, then can't read meta data, and seems result.getMap() call has 
issue in src/java/org/apache/cassandra/config/CFMetaData.java, can't compose 
type data from binary data.

{code}
java.lang.RuntimeException: Could not retrieve endpoint ranges: 
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchExternalClient.init(CrunchExternalClient.java:141)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:156)
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:158)
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:144)
at org.apache.crunch.io.CrunchOutputs.close(CrunchOutputs.java:138)
at 
org.apache.crunch.impl.mr.run.CrunchTaskContext.cleanup(CrunchTaskContext.java:72)
at org.apache.crunch.impl.mr.run.CrunchReducer.cleanup(CrunchReducer.java:64)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543)
at 
org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124)
at 
org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101)
at 
org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30)
at 
org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50)
at org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68)
at 
org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:282)
at 
org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1815)
at org.apache.cassandra.config.CFMetaData.fromThriftCqlRow(CFMetaData.java:1116)
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchExternalClient.init(CrunchExternalClient.java:133)
... 14 more
{code}

  was:
Used Spotify HDFS2CASS to bulk load data, everything is fine. After dropped 
some columns, then can't read meta data, and seems result.getMap() call has 
issue in src/java/org/apache/cassandra/config/CFMetaData.java, can't compose 
type data from binary data.

java.lang.RuntimeException: Could not retrieve endpoint ranges: 
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchExternalClient.init(CrunchExternalClient.java:141)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:156)
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:158)
at 
com.spotify.hdfs2cass.cassandra.cql.CrunchCqlBulkRecordWriter.close(CrunchCqlBulkRecordWriter.java:144)
at org.apache.crunch.io.CrunchOutputs.close(CrunchOutputs.java:138)
at 
org.apache.crunch.impl.mr.run.CrunchTaskContext.cleanup(CrunchTaskContext.java:72)
at org.apache.crunch.impl.mr.run.CrunchReducer.cleanup(CrunchReducer.java:64)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:543)
at 
org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:124)
at 
org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:101)
at 
org.apache.cassandra.serializers.MapSerializer.deserializeForNativeProtocol(MapSerializer.java:30)
at 
org.apache.cassandra.serializers.CollectionSerializer.deserialize(CollectionSerializer.java:50)
at org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:68)
at 
org.apache.cassandra.cql3.UntypedResultSet$Row.getMap(UntypedResultSet.java:282)
at 

[jira] [Commented] (CASSANDRA-10648) Native protocol is dead after running some Hive queries

[Alex Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992236#comment-14992236
 ] 

Alex Liu commented on CASSANDRA-10648:
--

C* 2.1.x works fine. I shut down all other programs and C* node native is dead 
forever without any client running on C*.

> Native protocol is dead after running some Hive queries
> ---
>
> Key: CASSANDRA-10648
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10648
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Liu
> Fix For: 2.2.x
>
> Attachments: 10_day_loss.q, deadlock.txt, deadlock1.txt
>
>
> When test on DSE portfolio demo, which basically creates a few C* tables and 
> inserts some data into the tables, then run some Hive queries on the tables.
> I attach the Hive queries
> After some queries are done, C* node is dead on native port, cqlsh can't 
> login any more.
> Some thread dumps are attached. Too many threads are in waiting mode and 
> system is not responding.
> It's tested on C* 2.2.3 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10641) simultaneous_bootstrap_test fails on Windows

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992291#comment-14992291
 ] 

Joshua McKenzie commented on CASSANDRA-10641:
-

Hm. Test passes locally for me. Did you get more detail on the failure 
condition from that test run?

> simultaneous_bootstrap_test fails on Windows
> 
>
> Key: CASSANDRA-10641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10641
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Joshua McKenzie
> Fix For: 2.2.x, 3.0.0
>
> Attachments: 10641.txt
>
>
> {{bootstrap_test.py:TestBootstrap.simultaneous_bootstrap_test}} fails on 
> Windows on C* 3.0 and 2.2:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/99/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> http://cassci.datastax.com/view/win32/job/cassandra-2.2_dtest_win32/127/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> It expects there to be a warning in the stderr of the node attempting to 
> start while another node boostraps, but the stderr is empty. It fails on this 
> line:
> https://github.com/riptano/cassandra-dtest/blob/master/bootstrap_test.py#L469
> [~yukim] I believe you're the person to handle this ticket? I'll assign you, 
> but feel free to reassign.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10584) reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a ClassCastException

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992332#comment-14992332
 ] 

Carl Yeksigian commented on CASSANDRA-10584:


I've updated https://github.com/riptano/cassandra-dtest/pull/606 to also 
include tests for this patch, which pass.

> reads with EACH_QUORUM  on keyspace with SimpleTopologyStrategy throw a 
> ClassCastException
> --
>
> Key: CASSANDRA-10584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10584
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0
>
>
> I think this may be a regression introduced w/ [CASSANDRA-9602].  Starting 
> with C* 3.0.0-rc2 an error is returned when querying a keyspace with 
> {{SimpleTopologyStrategy}} using EACH_QUORUM CL:
> {noformat}
> cqlsh> create keyspace test with replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> create table test.test (k int PRIMARY KEY, i int);
> cqlsh> consistency EACH_QUORUM;
> Consistency level set to EACH_QUORUM.
> cqlsh> select * from test.test;
> ServerError:  message="java.lang.ClassCastException: 
> org.apache.cassandra.locator.SimpleStrategy cannot be cast to 
> org.apache.cassandra.locator.NetworkTopologyStrategy">
> {noformat}
> The exception yielded in the system logs:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-10-23 13:02:15,405 ErrorMessage.java:336 - 
> Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy 
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForEachQuorum(ConsistencyLevel.java:227)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:188)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:180)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1795)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1762)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149) 
> ~[guava-18.0.jar:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1814)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1799)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1925)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1892)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager$PagerIterator.hasNext(AbstractQueryPager.java:99)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:327)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:205)
>  ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:236) 
> ~[main/:na]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:221) 
> 

[jira] [Commented] (CASSANDRA-10621) Error while bootstraping a new node with Materialized Views

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992360#comment-14992360
 ] 

Carl Yeksigian commented on CASSANDRA-10621:


+1

> Error while bootstraping a new node with Materialized Views
> ---
>
> Key: CASSANDRA-10621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Alan Boudreault
>Assignee: Joel Knighton
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: system.log
>
>
> If I try to add a new node in a cluster that has materialized views, I get 
> the following error:
> {code}
>  ERROR 19:05:15 Unknown exception caught while attempting to update 
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a 
> non-data replica
> at 
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
>  ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) 
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) 
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) 
> [main/:na]
> at 
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
>  [main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:15 Error applying streamed sstable: 
> java.lang.RuntimeException: Trying to get the view natural endpoint on a 
> non-data replica
> at 
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
>  ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) 
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) 
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) 
> ~[main/:na]
> at 
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
>  ~[main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_45]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> WARN  19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142 
> bytes)
> WARN  19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746 
> bytes)
> WARN  19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337 
> bytes)
> ERROR 19:05:33 Unknown exception caught while attempting to update 
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a 
> non-data replica
> at 
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
>  ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) 
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) 
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) 
> [main/:na]
> at 
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
>  [main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
> at 

[jira] [Commented] (CASSANDRA-10592) IllegalArgumentException in DataOutputBuffer.reallocate

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992388#comment-14992388
 ] 

Ariel Weisberg commented on CASSANDRA-10592:


[~benedict] The utests look good. The dtests I don't know. I might have to 
rebase to perform a comparison with tip of trunk.

I'm going to do that on a new branch and we can make the decision what/when to 
merge in parallel.

|[2.2 
Code|https://github.com/apache/cassandra/compare/cassandra-2.2...aweisberg:CASSANDRA-10592-2.2-squashed-v2?expand=1]|[utests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10592-2.2-squashed-v2-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10592-2.2-squashed-v2-dtest/]|
|[3.0 
Code|https://github.com/apache/cassandra/compare/cassandra-3.0...aweisberg:CASSANDRA-10592-3.0-squashed-v2?expand=1]|[utests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10592-3.0-squashed-v2-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-10592-3.0-squashed-v2-dtest/]|


> IllegalArgumentException in DataOutputBuffer.reallocate
> ---
>
> Key: CASSANDRA-10592
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10592
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sebastian Estevez
>Assignee: Ariel Weisberg
> Fix For: 2.2.4, 3.0.0
>
>
> CORRECTION-
> It turns out the exception occurs when running a read using a thrift jdbc 
> driver. Once you have loaded the data with stress below, run 
> SELECT * FROM "autogeneratedtest"."transaction_by_retailer" using this tool - 
> http://www.aquafold.com/aquadatastudio_downloads.html
>  
> The exception:
> {code}
> WARN  [SharedPool-Worker-1] 2015-10-22 12:58:20,792 
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.RuntimeException: java.lang.IllegalArgumentException
>   at 
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2366)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[main/:na]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [main/:na]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> Caused by: java.lang.IllegalArgumentException: null
>   at java.nio.ByteBuffer.allocate(ByteBuffer.java:334) ~[na:1.8.0_60]
>   at 
> org.apache.cassandra.io.util.DataOutputBuffer.reallocate(DataOutputBuffer.java:63)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.util.DataOutputBuffer.doFlush(DataOutputBuffer.java:57)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:151)
>  ~[main/:na]
>   at 
> org.apache.cassandra.utils.ByteBufferUtil.writeWithVIntLength(ByteBufferUtil.java:296)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.marshal.AbstractType.writeValue(AbstractType.java:374)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.BufferCell$Serializer.serialize(BufferCell.java:263)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:183)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:96)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:381)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[main/:na]
>   at 
> 

[jira] [Commented] (CASSANDRA-10660) Support user-defined compactions through nodetool

[Kenneth Failbus (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992175#comment-14992175
 ] 

Kenneth Failbus commented on CASSANDRA-10660:
-

This would be a good enhancement to the node tool to have option to compact and 
purging of tombstones.

> Support user-defined compactions through nodetool
> -
>
> Key: CASSANDRA-10660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10660
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Tyler Hobbs
>Priority: Minor
>  Labels: lhf
>
> For a long time, we've supported running user-defined compactions through 
> JMX.  This comes in handy fairly often, mostly when dealing with low disk 
> space or tombstone purging, so it would be good to add something to nodetool 
> for this.  An extra option for {{nodetool compact}} would probably suffice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10641) simultaneous_bootstrap_test fails on Windows

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992242#comment-14992242
 ] 

Jim Witschey commented on CASSANDRA-10641:
--

If I read the console output from the dtest test run correctly, it looks like 
this still fails?

{code}
00:13:37 simultaneous_bootstrap_test (bootstrap_test.TestBootstrap) ... 
Started: node1 with pid: 4912
00:13:50 Created keyspaces. Sleeping 1s for propagation.
00:13:51 Sleeping 2s...
00:13:53 Warming up WRITE with 5 iterations...
00:13:54 INFO  00:13:50 Using data-center name 'datacenter1' for 
DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct 
datacenter name with DCAwareRoundRobinPolicy constructor)
00:13:54 Connected to cluster: test
00:13:54 Datatacenter: datacenter1; Host: /127.0.0.1; Rack: rack1
00:13:54 INFO  00:13:50 New Cassandra host /127.0.0.1:9042 added
00:14:04 Running WRITE with 10 threads for 50 iteration
00:14:04 type,  total ops,op/s,pk/s,   row/s,mean, med, 
.95, .99,.999, max,   time,   stderr, errors,  gc: #,  max ms,  sum 
ms,  sdv ms,  mb
00:14:05 total,  6458,6445,6445,6445, 1.5, 1.2, 
2.4, 5.0,17.9,18.4,1.0,  0.0,  0,  1,  17,  
17,   0,  37
00:14:06 total, 13582,7060,7060,7060, 1.4, 1.0, 
2.1, 2.9,18.7,19.2,2.0,  0.03252,  0,  2,  16,  
31,   1,  73
00:14:07 total, 20931,7319,7319,7319, 1.3, 1.0, 
2.1, 2.8,15.7,17.9,3.0,  0.03011,  0,  1,  15,  
15,   0,  35
00:14:08 total, 27634,6654,6654,6654, 1.5, 1.2, 
2.3, 3.0,16.1,17.0,4.0,  0.02471,  0,  1,  15,  
15,   0,  39
00:14:09 total, 33211,5460,5460,5460, 1.8, 1.8, 
3.2, 6.2,21.4,21.7,5.0,  0.04431,  0,  1,  19,  
19,   0,  35
00:14:10 total, 39158,5867,5867,5867, 1.7, 1.7, 
2.7, 6.0,15.9,16.1,6.1,  0.04128,  0,  4,  15,  
43,   3, 155
00:14:11 total, 46558,7362,7362,7362, 1.3, 1.0, 
2.1, 2.3,15.9,16.3,7.1,  0.03906,  0,  1,  14,  
14,   0,  36
00:14:12 total, 54112,7516,7516,7516, 1.3, 1.0, 
2.1, 2.3,18.1,19.1,8.1,  0.03733,  0,  1,  17,  
17,   0,  34
00:14:13 total, 61494,7299,7299,7299, 1.3, 1.1, 
2.1, 2.6,19.1,20.7,9.1,  0.03418,  0,  1,  18,  
18,   0,  33
00:14:14 total, 68902,7306,7306,7306, 1.3, 1.1, 
2.1, 2.7,18.1,19.0,   10.1,  0.03141,  0,  1,  16,  
16,   0,  37
00:14:15 total, 76493,7553,7553,7553, 1.3, 1.0, 
2.1, 2.4,20.3,21.1,   11.1,  0.02932,  0,  2,  19,  
37,   1,  72
00:14:16 total, 83918,7378,7378,7378, 1.3, 1.0, 
2.1, 2.3,21.1,22.5,   12.1,  0.02737,  0,  1,  19,  
19,   0,  33
00:14:17 total, 90079,6130,6130,6130, 1.6, 1.2, 
2.8, 5.9,22.3,23.7,   13.1,  0.02677,  0,  2,  18,  
35,   1,  75
00:14:18 total, 96600,6494,6494,6494, 1.5, 1.3, 
2.3, 3.1,11.0,11.8,   14.1,  0.02544,  0,  3,   9,  
24,   1, 120
00:14:19 total,102474,5844,5844,5844, 1.7, 1.7, 
3.0, 4.9,16.1,16.4,   15.1,  0.02579,  0,  1,  14,  
14,   0,  32
00:14:20 total,109440,6945,6945,6945, 1.4, 1.1, 
2.1, 2.9,19.4,21.0,   16.1,  0.02420,  0,  2,  17,  
31,   2,  70
00:14:21 total,116446,6970,6970,6970, 1.4, 1.1, 
2.1, 2.7,17.8,18.4,   17.1,  0.02279,  0,  1,  16,  
16,   0,  37
00:14:22 total,123709,7234,7234,7234, 1.4, 1.1, 
2.1, 2.9,18.1,18.3,   18.1,  0.02174,  0,  1,  16,  
16,   0,  36
00:14:23 total,131224,7477,7477,7477, 1.3, 1.0, 
2.1, 2.5,20.9,21.4,   19.1,  0.02113,  0,  1,  19,  
19,   0,  32
00:14:24 total,138590,7336,7336,7336, 1.3, 1.1, 
2.1, 2.5,17.3,19.2,   20.1,  0.02030,  0,  1,  16,  
16,   0,  37
00:14:25 total,145668,7018,7018,7018, 1.4, 1.1, 
2.1, 2.9,19.2,20.6,   21.1,  0.01933,  0,  

[jira] [Commented] (CASSANDRA-10401) Improve json2sstable error reporting on nonexistent column

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992141#comment-14992141
 ] 

Jim Witschey commented on CASSANDRA-10401:
--

The dtest results look right to me too. +1.

> Improve json2sstable error reporting on nonexistent column
> --
>
> Key: CASSANDRA-10401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10401
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra 2.1.8.621
>Reporter: Jose Martinez Poblete
>Assignee: Paulo Motta
>
> We have the following table...
> {noformat}
> CREATE TABLE keyspace_name.table_name (
> col1 text,
> col2 text,
> col3 text,
> col4 text,
> PRIMARY KEY ((col1, col2), col3)
> ) WITH CLUSTERING ORDER BY (col3 ASC)
> {noformat}
> And the following  json in a file created from sstable2json tool
> {noformat}
> [
> {"key": "This is col1:This is col2,
>  "cells": [["This is col3:","",1443217787319002],
>["This is col3:"col4","This is col4",1443217787319002]]}
> ]
> {noformat}
> Let's say we deleted that record form the DB and wanted to bring it back
> If we try to create an sstable from this data in a json file named 
> test_file.json, we get a NPE 
> {noformat}
> -bash-4.1$ json2sstable -K elp -c table_name-3264cbe063c211e5bc34e746786b7b29 
> test_file.json  
> /var/lib/cassandra/data/keyspace_name/table_name-3264cbe063c211e5bc34e746786b7b29/keyspace_name-table_name-ka-1-Data.db
> Importing 1 keys...
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.tools.SSTableImport.getKeyValidator(SSTableImport.java:442)
>   at 
> org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:316)
>   at 
> org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:287)
>   at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:514)
> ERROR: null
> -bash-4.1$
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10342) Read defragmentation can cause unnecessary repairs

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992158#comment-14992158
 ] 

Jim Witschey commented on CASSANDRA-10342:
--

[~krummas] Thanks for the ping, and sorry for letting this fall on the floor. 
I've put it on my list of things to look at post-3.0 release.

> Read defragmentation can cause unnecessary repairs
> --
>
> Key: CASSANDRA-10342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10342
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Olsson
>Assignee: Marcus Eriksson
>Priority: Minor
>
> After applying the fix from CASSANDRA-10299 to the cluster we started having 
> a problem of ~20k small sstables appearing for the table with static data 
> when running incremental repair.
> In the logs there were several messages about flushes for that table, one for 
> each repaired range. The flushed sstables were 0.000kb in size with < 100 ops 
> in each. When checking cfstats there were several writes to that table, even 
> though we were only reading from it and read repair did not repair anything.
> After digging around in the codebase I noticed that defragmentation of data 
> can occur while reading, depending on the query and some other conditions. 
> This causes the read data to be inserted again to have it in a more recent 
> sstable, which can be a problem if that data was repaired using incremental 
> repair. The defragmentation is done in 
> [CollationController.java|https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/db/CollationController.java#L151].
> I guess this wasn't a problem with full repairs since I assume that the 
> digest should be the same even if you have two copies of the same data. But 
> with incremental repair this will most probably cause a mismatch between 
> nodes if that data already was repaired, since the other nodes probably won't 
> have that data in their unrepaired set.
> --
> I can add that the problems on our cluster was probably due to the fact that 
> CASSANDRA-10299 caused the same data to be streamed multiple times and ending 
> up in several sstables. One of the conditions for the defragmentation is that 
> the number of sstables read during a read request have to be more than the 
> minimum number of sstables needed for a compaction(> 4 in our case). So 
> normally I don't think this would cause ~20k sstables to appear, we probably 
> hit an extreme.
> One workaround for this is to use another compaction strategy than STCS(it 
> seems to be the only affected strategy, atleast in 2.1), but the solution 
> might be to either make defragmentation configurable per table or avoid 
> reinserting the data if any of the sstables involved in the read are repaired.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10641) simultaneous_bootstrap_test fails on Windows

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992185#comment-14992185
 ] 

Philip Thompson commented on CASSANDRA-10641:
-

+1

> simultaneous_bootstrap_test fails on Windows
> 
>
> Key: CASSANDRA-10641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10641
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Joshua McKenzie
> Fix For: 2.2.x, 3.0.0
>
> Attachments: 10641.txt
>
>
> {{bootstrap_test.py:TestBootstrap.simultaneous_bootstrap_test}} fails on 
> Windows on C* 3.0 and 2.2:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/99/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> http://cassci.datastax.com/view/win32/job/cassandra-2.2_dtest_win32/127/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> It expects there to be a warning in the stderr of the node attempting to 
> start while another node boostraps, but the stderr is empty. It fails on this 
> line:
> https://github.com/riptano/cassandra-dtest/blob/master/bootstrap_test.py#L469
> [~yukim] I believe you're the person to handle this ticket? I'll assign you, 
> but feel free to reassign.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992221#comment-14992221
 ] 

Paulo Motta edited comment on CASSANDRA-10600 at 11/5/15 6:46 PM:
--

LGTM, uploaded v2 version removing unnecessary trailing whitespaces.

(lots of unrelated dtests failures, probably caused by python driver update of 
CASSANDRA-10365, but in pair with latest dtest runs: 
[3.0|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/327/testReport/]
 and 
[trunk|http://cassci.datastax.com/view/trunk/job/trunk_dtest/731/testReport/])


was (Author: pauloricardomg):
LGTM, uploaded v2 version removing unnecessary trailing whitespaces.

(lots of unrelated dtests failures, probably caused by python driver update of 
CASSANDRA-10365, but in pair with latest dtest runs: 
[3.0|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/327/testReport/]
 and 
[trunk|http://cassci.datastax.com/view/trunk/job/trunk_dtest/731/testReport/].

> CqlInputFormat throws IOE if the size estimates are zero
> 
>
> Key: CASSANDRA-10600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.x, 3.0.x
>
> Attachments: 10600-v2.txt, 10600.txt
>
>
> {{CqlInputFormat.describeSplits}} handles the case of no entry in the 
> 'system.size_estimates' table but does not handle the situation when there is 
> a zero size estimate in the table. This can happen if an input job is started 
> immediately after data is added but before the {{SizeEstimatesRecorder}} has 
> run.
> {{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same 
> manner as no size estimate and not sub-split but return the full split. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10661) Integrate SASI to Cassandra

[Pavel Yaskevich (JIRA)]

Pavel Yaskevich created CASSANDRA-10661:
---

 Summary: Integrate SASI to Cassandra
 Key: CASSANDRA-10661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10661
 Project: Cassandra
  Issue Type: Improvement
  Components: core, index
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 3.x


We have recently released new secondary index engine 
(https://github.com/xedin/sasi) build using SecondaryIndex API, there are still 
couple of things to work out regarding 3.x since it's currently targeted on 2.0 
released. I want to make this an umbrella issue to all of the things related to 
integration of SASI, which are also tracked in 
[sasi_issues|https://github.com/xedin/sasi/issues], into mainline Cassandra 3.x 
release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10641) simultaneous_bootstrap_test fails on Windows

[Jim Witschey (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992371#comment-14992371
 ] 

Jim Witschey commented on CASSANDRA-10641:
--

Ah, I've got it passing locally; it was failing with {{don't know how to parse 
type string u'map...}} (etc.), so I updated the driver and it 
passed. +1 on the patch, and I'll talk to the windows worker admins about how 
the driver updates.

> simultaneous_bootstrap_test fails on Windows
> 
>
> Key: CASSANDRA-10641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10641
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Joshua McKenzie
> Fix For: 2.2.x, 3.0.0
>
> Attachments: 10641.txt
>
>
> {{bootstrap_test.py:TestBootstrap.simultaneous_bootstrap_test}} fails on 
> Windows on C* 3.0 and 2.2:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/99/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> http://cassci.datastax.com/view/win32/job/cassandra-2.2_dtest_win32/127/testReport/junit/bootstrap_test/TestBootstrap/simultaneous_bootstrap_test/
> It expects there to be a warning in the stderr of the node attempting to 
> start while another node boostraps, but the stderr is empty. It fails on this 
> line:
> https://github.com/riptano/cassandra-dtest/blob/master/bootstrap_test.py#L469
> [~yukim] I believe you're the person to handle this ticket? I'll assign you, 
> but feel free to reassign.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992385#comment-14992385
 ] 

Joel Knighton commented on CASSANDRA-10634:
---

I've pushed a version with the requested changes 
[here|https://github.com/jkni/cassandra/commits/10634].

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992385#comment-14992385
 ] 

Joel Knighton edited comment on CASSANDRA-10634 at 11/5/15 8:16 PM:


I've pushed a branch with the requested changes 
[here|https://github.com/jkni/cassandra/commits/10634].


was (Author: jkni):
I've pushed a version with the requested changes 
[here|https://github.com/jkni/cassandra/commits/10634].

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992368#comment-14992368
 ] 

Carl Yeksigian commented on CASSANDRA-10634:


Proposed changes look good -- can we change the names of the variables to just 
"baseEndpoints" and "viewEndpoints". Also, can you add a sentence about 
DC-locality to the comment at the top of the method; it's unclear that it is 
doing anything special with DC's from the comment.

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[aleksey]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/718c4789
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/718c4789
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/718c4789

Branch: refs/heads/trunk
Commit: 718c4789c90c25b676f04437b3a31c30b4b460cf
Parents: a38f362 9c9f263
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 22:32:34 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 22:32:34 2015 +

--
 CHANGES.txt |  1 +
 .../cassandra/hadoop/cql3/CqlInputFormat.java   | 22 ++--
 2 files changed, 17 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/718c4789/CHANGES.txt
--
diff --cc CHANGES.txt
index 14b6007,b469594..72dc800
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,10 -1,5 +1,11 @@@
 +3.2
 + * Added graphing option to cassandra-stress (CASSANDRA-7918)
 + * Abort in-progress queries that time out (CASSANDRA-7392)
 + * Add transparent data encryption core classes (CASSANDRA-9945)
 +
 +
  3.0
+  * (Hadoop) fix CIF describeSplits() not handling 0 size estimates 
(CASSANDRA-10600)
   * Fix reading of legacy sstables (CASSANDRA-10590)
   * Use CQL type names in schema metadata tables (CASSANDRA-10365)
   * Guard batchlog replay against integer division by zero (CASSANDRA-9223)

[jira] [Commented] (CASSANDRA-10600) CqlInputFormat throws IOE if the size estimates are zero

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992620#comment-14992620
 ] 

Aleksey Yeschenko commented on CASSANDRA-10600:
---

Committed as 
[9c9f263822ab0c7669f162380af6645ef8309083|https://github.com/apache/cassandra/commit/9c9f263822ab0c7669f162380af6645ef8309083]
 to 3.0 and merged with trunk, thanks.

> CqlInputFormat throws IOE if the size estimates are zero
> 
>
> Key: CASSANDRA-10600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10600
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10600-v2.txt, 10600.txt
>
>
> {{CqlInputFormat.describeSplits}} handles the case of no entry in the 
> 'system.size_estimates' table but does not handle the situation when there is 
> a zero size estimate in the table. This can happen if an input job is started 
> immediately after data is added but before the {{SizeEstimatesRecorder}} has 
> run.
> {{CqlInputFormat.describeSplits}} should handle 0 size estimate in the same 
> manner as no size estimate and not sub-split but return the full split. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix NativeSSTableLoaderClient to use the right ClusteringOrder class

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 9c9f26382 -> 4db577abf


Fix NativeSSTableLoaderClient to use the right ClusteringOrder class


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4db577ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4db577ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4db577ab

Branch: refs/heads/cassandra-3.0
Commit: 4db577abf8e0cfda655d16abc5d99aeaafba02d0
Parents: 9c9f263
Author: Aleksey Yeschenko 
Authored: Thu Nov 5 22:52:23 2015 +
Committer: Aleksey Yeschenko 
Committed: Thu Nov 5 22:52:23 2015 +

--
 src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4db577ab/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index cebee4a..840ef26 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -23,6 +23,7 @@ import java.util.*;
 import com.datastax.driver.core.*;
 
 import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.config.ColumnDefinition.ClusteringOrder;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.cql3.ColumnIdentifier;
 import org.apache.cassandra.db.marshal.*;

[jira] [Commented] (CASSANDRA-10634) Materialized Views filter paired endpoints to local DC even when not using DC-aware replication

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992671#comment-14992671
 ] 

Aleksey Yeschenko commented on CASSANDRA-10634:
---

Committed as 
[1bd2c94252b52d99b0c49257d2365f7c85935dcd|https://github.com/apache/cassandra/commit/1bd2c94252b52d99b0c49257d2365f7c85935dcd]
 to 3.0 and merged with trunk, thanks.

> Materialized Views filter paired endpoints to local DC even when not using 
> DC-aware replication
> ---
>
> Key: CASSANDRA-10634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Joel Knighton
>Assignee: Joel Knighton
> Fix For: 3.0.0
>
>
> When calculating paired view replicas on MV writes, we filter out base and 
> view replicas that aren't in the local datacenter. 
> If we are using SimpleStrategy, this filtering is incorrect, since our 
> replica placement is not datacenter aware.
> We should behave the same as DC-aware consistency levels and only perform 
> this datacenter-aware filtering if the replication strategy is an instance of 
> NetworkTopologyStrategy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)