date:20180412


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves reassigned CASSANDRA-14332:


Assignee: Kurt Greaves

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14332) Fix unbounded validation compactions on repair


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14332:
-
Status: Patch Available  (was: Open)

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14332) Fix unbounded validation compactions on repair


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436783#comment-16436783
 ] 

Kurt Greaves edited comment on CASSANDRA-14332 at 4/13/18 4:26 AM:
---

|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...kgreav:13797-r-3.0]|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...kgreav:13797-r-3.11]|
 
|[utests|https://circleci.com/gh/kgreav/cassandra/152]|[utests|https://circleci.com/gh/kgreav/cassandra/154]|

I think reverting this is the correct solution for 3.0 and 3.x. Changing 
defaults in minors while in this case is probably fine, it's not good practice. 
Backporting CASSANDRA-13521 completely is also a bit questionable. 

The original problem from CASSANDRA-13797 of the interrupted exceptions is only 
very minor, and likely not possible outside of CCM clusters.


was (Author: kurtg):
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...kgreav:13797-r-3.0]|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...kgreav:13797-r-3.11]|
 
|[utests|https://circleci.com/gh/kgreav/cassandra/152]|[utests|https://circleci.com/gh/kgreav/cassandra/154]|

I think reverting this is the correct solution for 3.0 and 3.x. Changing 
defaults in minors while in this case is probably fine, it's not good practice. 
Backporting CASSANDRA-13521 completely is also a bit questionable. 

The original problem from CASSANDRA-13797 of the interrupted exceptions is only 
very minor, and likely not possible outside of CCM clusters.

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Priority: Major
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14332) Fix unbounded validation compactions on repair


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436783#comment-16436783
 ] 

Kurt Greaves commented on CASSANDRA-14332:
--

|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...kgreav:13797-r-3.0]|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...kgreav:13797-r-3.11]|
 
|[utests|https://circleci.com/gh/kgreav/cassandra/152]|[utests|https://circleci.com/gh/kgreav/cassandra/154]|

I think reverting this is the correct solution for 3.0 and 3.x. Changing 
defaults in minors while in this case is probably fine, it's not good practice. 
Backporting CASSANDRA-13521 completely is also a bit questionable. 

The original problem from CASSANDRA-13797 of the interrupted exceptions is only 
very minor, and likely not possible outside of CCM clusters.

> Fix unbounded validation compactions on repair
> --
>
> Key: CASSANDRA-14332
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14332
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
>Priority: Major
>
> After CASSANDRA-13797 it's possible to cause unbounded, simultaneous 
> validation compactions as we no longer wait for validations to finish. 
> Potential fix is to have a sane default for the # of concurrent validation 
> compactions by backporting CASSANDRA-13521 and setting a sane default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436769#comment-16436769
 ] 

Jay Zhuang commented on CASSANDRA-14381:


The snapshot is taken and the user is still able to restore data for system 
keyspaces, just the snapshot is not in the {{listsnapshots}} output. Sometimes 
we do need to restore data for system tables, for example, if the schema is 
corrupted: 
[{{SchemaKeyspace.java:951}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L951]
@[~aweisberg], @[~stefania_alborghetti], what do you think to list 
system/system_schema snapshots?

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to

2018-04-12 Thread Alex Lourie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436764#comment-16436764
 ] 

Alex Lourie commented on CASSANDRA-13010:
-

[~rustyrazorblade], I don't have a clue what happened during that previous 
"fix". I've rebased my local stuff again, so hopefully, that is now truly fixed.

Additionally, I did a bit of cleanup and commenting, so now more stuff is 
commented on and the changeset got smaller by 2 files :)

So I hope it's looking better now and is much easier to read and review.

Thanks!

> nodetool compactionstats should say which disk a compaction is writing to
> -
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Tools
>Reporter: Jon Haddad
>Assignee: Alex Lourie
>Priority: Major
>  Labels: lhf
> Attachments: 13010.patch, cleanup.png, multiple operations.png
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14298) cqlshlib tests broken on b.a.o


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang reassigned CASSANDRA-14298:
--

Assignee: Patrick Bannister

> cqlshlib tests broken on b.a.o
> --
>
> Key: CASSANDRA-14298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Patrick Bannister
>Priority: Major
>
> It appears that cqlsh-tests on builds.apache.org on all branches stopped 
> working since we removed nosetests from the system environment. See e.g. 
> [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
>  Looks like we either have to make nosetests available again or migrate to 
> pytest as we did with dtests. Giving pytest a quick try resulted in many 
> errors locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14380) Cassandra crashes after fsync exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436743#comment-16436743
 ] 

Ariel Weisberg commented on CASSANDRA-14380:


Created CASSANDRA-14383

> Cassandra crashes after fsync exception
> ---
>
> Key: CASSANDRA-14380
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14380
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Adam Geiger
>Priority: Critical
> Attachments: debug.log, debug.log.1.zip, 
> logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt
>
>
> Running Cassandra with a Rook Ceph filesystem within Kubernetes.  During the 
> startup, the following Warnings in the debug log pop up and then Cassandra 
> crashes shortly after and restarts.  It looks like before hitting this error, 
> it is doing a lot of writing and flushing
> WARN [MemtableFlushWriter:2] 2018-04-11 14:34:42,748 NativeLibrary.java:328 - 
> fsync(666) failed, errorno (22) {}
> com.sun.jna.LastErrorException: [22] Invalid argument
>  at org.apache.cassandra.utils.NativeLibraryLinux.fsync(Native Method) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.callFsync(NativeLibraryLinux.java:107)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.NativeLibrary.trySync(NativeLibrary.java:317) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySync(SyncUtil.java:179) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySyncDir(SyncUtil.java:190) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:107)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:141)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:402)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:53)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:368)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1153)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1086)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
>  [na:1.8.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>  [na:1.8.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$12.BCF32600.run(Unknown
>  Source) ~[na:na]
>  at java.lang.Thread.run(Thread.java:811) ~[na:2.9 (12-15-2017)]
>  
> Syslog shows the following 
> (logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt):
> INFO  [main] 2018-04-11 14:49:01,848 ColumnFamilyStore.java:406 - 
> Initializing apim.ur_to_op_by_op
> INFO  [MemoryMXBean notification dispatcher] 2018-04-11 14:49:25,889 
> GCInspector.java:284 - global GC in 206ms.  class storage: 28700680 -> 
> 28692744; miscellaneous non-heap storage: 49871216 -> 53570176; 
> nursery-allocate: 1296878920 -> 149116672; tenured-SOA: 140321968 -> 139143760
> #0: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x302a94) 
> [0x7f17e4f10a94]
> #1: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x306b2d) 
> [0x7f17e4f14b2d]
> #2: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xc82da) 
> [0x7f17e4cd62da]
> #3: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9prt29.so(+0x22056) 
> [0x7f17e6531056]
> #4: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f17ed0de390]
> #5: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2c4e1f) 
> [0x7f17e4ed2e1f]
> #6: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x158c04) 
> [0x7f17e4d66c04]
> #7: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x542d24) 
> [0x7f17e5150d24]
> #8:

[jira] [Resolved] (CASSANDRA-14384) If fsync fails it's always an issue and continuing execution is suspect


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg resolved CASSANDRA-14384.

Resolution: Duplicate

Duplicate of CASSANDRA-14383

> If fsync fails it's always an issue and continuing execution is suspect
> ---
>
> Key: CASSANDRA-14384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14384
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Major
> Fix For: 2.1.x, 3.0.x, 3.11.x, 4.x
>
>
> We can't catch fsync errors and continue so we shouldn't have code that does 
> that in C*. There was a Postgres bug where fsync returned an error and the FS 
> lost data, but subsequent fsyncs succeeded.
> The [LastErrorException code in 
> NativeLibrary.trySync|https://github.com/apache/cassandra/commit/be313935e54be450d9aaabda7965a2f266e922c9#diff-4258621cdf765f0fea6770db5d40038fR307]
>  looks a little janky. What's up with that? When would trySync be something 
> we would merely try? If try is good enough why do it at all considering try 
> is the default behavior of a series of unsynced filesystem operations.
> Also when we fsync in FD it's not just fsyncing that file the FS is 
> potentially fsyncing other data and the error code we get could be related to 
> that other data so we can't safely ignore it. The filesystem could be 
> internally inconsistent as well. This happens because the FS journaling may 
> force the FS to flush other data as well to preserve the ordering 
> requirements of journaled metadata.
> If we ignore fsync errors it needs to be for whitelisted reasons such as a 
> bad FD.
> I know we have FSErrorHandler and it makes sense for reads, but I'm not sold 
> on it being the right answer for writes. We don't retry flushing a memtable 
> or writing to the commit log to my knowledge. We could go read only and I 
> need to check if that is w



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14383) If fsync fails it's always an issue and continuing execution is suspect

Ariel Weisberg created CASSANDRA-14383:
--

Summary: If fsync fails it's always an issue and continuing
execution is suspect
Key: CASSANDRA-14383
URL: https://issues.apache.org/jira/browse/CASSANDRA-14383
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 4.0.x, 2.1.x, 3.0.x, 3.11.x

We can't catch fsync errors and continue so we shouldn't have code that does
that in C*. There was a Postgres bug where fsync returned an error and the FS
lost data, but subsequent fsyncs succeeded.

The [LastErrorException code in
NativeLibrary.trySync|https://github.com/apache/cassandra/commit/be313935e54be450d9aaabda7965a2f266e922c9#diff-4258621cdf765f0fea6770db5d40038fR307]
looks a little janky. What's up with that? When would trySync be something we
would merely try? If try is good enough why do it at all considering try is the
default behavior of a series of unsynced filesystem operations.

Also when we fsync in FD it's not just fsyncing that file the FS is potentially
fsyncing other data and the error code we get could be related to that other
data so we can't safely ignore it. The filesystem could be internally
inconsistent as well. This happens because the FS journaling may force the FS
to flush other data as well to preserve the ordering requirements of journaled
metadata.

If we ignore fsync errors it needs to be for whitelisted reasons such as a bad
FD.

I know we have FSErrorHandler and it makes sense for reads, but I'm not sold on
it being the right answer for writes. We don't retry flushing a memtable or
writing to the commit log to my knowledge. We could go read only and I need to
check if that is w

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14384) If fsync fails it's always an issue and continuing execution is suspect

Ariel Weisberg created CASSANDRA-14384:
--

Summary: If fsync fails it's always an issue and continuing
execution is suspect
Key: CASSANDRA-14384
URL: https://issues.apache.org/jira/browse/CASSANDRA-14384
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 2.1.x, 3.0.x, 3.11.x, 4.x

We can't catch fsync errors and continue so we shouldn't have code that does
that in C*. There was a Postgres bug where fsync returned an error and the FS
lost data, but subsequent fsyncs succeeded.

If we ignore fsync errors it needs to be for whitelisted reasons such as a bad
FD.

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14375) Digest mismatch Exception when sending raw hints in cluster


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14375:
---
Description: 
We have 14 nodes cluster where we seen hints file getting corrupted and 
resulting in the following error
{noformat}
ERROR [HintsDispatcher:1] 2018-04-06 16:26:44,423 CassandraDaemon.java:228 - 
Exception in thread Thread[HintsDispatcher:1,1,main]
 org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch 
exception
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:298)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:263)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:169) 
~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:128)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:113) 
~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:94) 
~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:278)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:260)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:238)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:217)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_141]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_141]
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
~[na:1.8.0_141]
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[na:1.8.0_141]
 at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
 [apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_141]
 Caused by: java.io.IOException: Digest mismatch exception
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:315)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:289)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 ... 16 common frames omitted
{noformat}
Notes on cluster and investigation done so far
1. Cassandra used here is built locally from 3.11.1 branch along with following 
patch from issue: CASSANDRA-14080
 
[https://github.com/apache/cassandra/commit/68079e4b2ed4e58dbede70af45414b3d4214e195]
2. The bootstrap of 14 nodes happens in the following way:
 - Out of 14 nodes only 3 nodes are picked as seed nodes.
 - Only 1 out 3 seed nodes is started and schema is created if it was not 
created previously.
 - Post this, rest of nodes are bootstrapped.
 - In failure scenario, only 5 out of 14 succesfully formed the cassandra 
cluster. The failed nodes include two seed nodes.
3. We confirmed the following patch from issue: CASSANDRA-13696 has been 
applied. From confirmed from Jay Zhuang that this is different issue from what 
was previously fixed.
"this should be a different issue, as HintsDispatcher.java:128 sends hints with 
\{{buffer}}s, this patch is only to fix the digest mismatch for 
HintsDispatcher.java:129, which sends hints one by one."
4. Application uses java driver with quoram setting for cassandra
5. We saw this issue on 7 node cluster too (different from 14 node cluster)
6. We are able to workaround by running nodetool truncatehints on failed nodes 
and restarting cassandra.

  was:
We have 14 nodes cluster where we seen hints file getting corrupted and 
resulting in the following error

ERROR [HintsDispatcher:1] 2018-04-06 16:26:44,423 CassandraDaemon.java:228 - 
Exception in thread Thread[HintsDispatcher:1,1,main]
 org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch 
exception
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:298)
 ~[apache-cassandra-3.11.1.jar:3.11.1-SNAPSHOT]
 at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:263)

[jira] [Commented] (CASSANDRA-14375) Digest mismatch Exception when sending raw hints in cluster


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436714#comment-16436714
 ] 

Jay Zhuang commented on CASSANDRA-14375:


We saw the same issue in {{3.0.14}} 2 times in the last one week:
{noformat}
ERROR [HintsDispatcher:1] 2018-04-10 23:43:47,930 
HintsDispatchExecutor.java:234 - Failed to dispatch hints file 
d921cf74-c064-465d-82b4-aa964cb3b8f6-1523401451406-1.hints: file is corrupted 
({})
org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch 
exception
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:296)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:261)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:157) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:138)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:123) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:95) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:268)
 [apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:251)
 [apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:229)
 [apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:208)
 [apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_121]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_121]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_121]
at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
 [apache-cassandra-3.0.14.x.jar:3.0.14.x]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
Caused by: java.io.IOException: Digest mismatch exception
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:313)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:287)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
... 16 common frames omitted
ERROR [HintsDispatcher:1] 2018-04-10 23:43:47,931 CassandraDaemon.java:207 - 
Exception in thread Thread[HintsDispatcher:1,1,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch 
exception
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:296)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:261)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:157) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:138)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:123) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:95) 
~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:268)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:251)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at 
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:229)
 ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
at

[jira] [Commented] (CASSANDRA-14298) cqlshlib tests broken on b.a.o

2018-04-12 Thread Patrick Bannister (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436702#comment-16436702
 ] 

Patrick Bannister commented on CASSANDRA-14298:
---

[~spo...@gmail.com], [~krummas], I'm interested in taking on the cqlsh tests if 
you aren't already working on them. It looks like there's a mix of different 
things going on. I've started looking into it, I'll post details as they become 
clear.

> cqlshlib tests broken on b.a.o
> --
>
> Key: CASSANDRA-14298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14298
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Testing
>Reporter: Stefan Podkowinski
>Priority: Major
>
> It appears that cqlsh-tests on builds.apache.org on all branches stopped 
> working since we removed nosetests from the system environment. See e.g. 
> [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console].
>  Looks like we either have to make nosetests available again or migrate to 
> pytest as we did with dtests. Giving pytest a quick try resulted in many 
> errors locally, but I haven't inspected them in detail yet. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13475) Pluggable storage engine design

2018-04-12 Thread Brian O'Neill (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436701#comment-16436701
 ] 

Brian O'Neill commented on CASSANDRA-13475:
---

 A "storage engine" is analogous to the storage engine concept of MySQL. InnoDB 
and MyISAM are MySQL storage engines, and a few others exist as well. A driver 
swap is analogous to swapping MySQL for PostgreSQL, which aren't fully 
compatible. Pluggable storage inside Cassandra allows applications to continue 
using Cassandra, and with a high degree of compatibility.

An application can define tables against the engine which make the most sense, 
and all of them can co-exist. Over time, the preferred storage engine might 
change, much like how InnoDB is preferred over MyISAM, which was the original.

> Pluggable storage engine design
> ---
>
> Key: CASSANDRA-13475
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13475
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Major
>
> In this jira, we discuss how to make Cassandra's storage engine to be 
> pluggable. We will discuss the scope, expectation, and guideline for this 
> project, as well as a detailed design so that we can create sub tasks for 
> each small project.
> Here is a design doc we are currently working on:  
> https://docs.google.com/document/d/1suZlvhzgB6NIyBNpM9nxoHxz_Ri7qAm-UEO8v8AIFsc



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to

2018-04-12 Thread Jon Haddad (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436656#comment-16436656
 ] 

Jon Haddad commented on CASSANDRA-13010:


Hey [~alourie] I just took a look at the patch.  There's still a pretty big 
(200 LOC) conflict between trunk and what's in your branch.  Would you mind 
rebasing your branch off trunk so it applies cleanly?  According to the git 
history you merged in some changes from trunk, which are now making a lot 
harder to do the review, as I have to look through the history to determine 
what's actually been deleted and what's a change you made.  For instance it 
looks like you deleted {{doValidationCompaction}}, which you didn't, Blake 
Eggleston did in {{ c5a7fcaa8e000}}.

A few other notes while I'm in here to avoid lots of iterations:

# there's almost no comments added despite it touching almost 20 files.  I 
realize it's not the best commented codebase, but I'd like to see comments on 
any new variables like {{targetDirectory}}.  Specifically, consider why 
something is there {{// needed for nodetool compactioninfo output}} is better 
than {{// holds directory name}}.  Please add comments conveying intent for 
each class method and variable added.
# {{import org.apache.cassandra.cql3.Operation}} was added as an import to 
{{src/java/org/apache/cassandra/index/internal/CollatedViewIndexBuilder.java}} 
but not used

Outside that, I think it's looking pretty good, I'll be pretty happy to get 
this merged in soon!

> nodetool compactionstats should say which disk a compaction is writing to
> -
>
> Key: CASSANDRA-13010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13010
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction, Tools
>Reporter: Jon Haddad
>Assignee: Alex Lourie
>Priority: Major
>  Labels: lhf
> Attachments: 13010.patch, cleanup.png, multiple operations.png
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14380) Cassandra crashes after fsync exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436642#comment-16436642
 ] 

Jeff Jirsa commented on CASSANDRA-14380:


[~aweisberg] - worth trying to invoke some special version of an FSError 
handler and/or force a hard/fast shutdown instead of letting it run until it 
crashes? 

> Cassandra crashes after fsync exception
> ---
>
> Key: CASSANDRA-14380
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14380
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Adam Geiger
>Priority: Critical
> Attachments: debug.log, debug.log.1.zip, 
> logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt
>
>
> Running Cassandra with a Rook Ceph filesystem within Kubernetes.  During the 
> startup, the following Warnings in the debug log pop up and then Cassandra 
> crashes shortly after and restarts.  It looks like before hitting this error, 
> it is doing a lot of writing and flushing
> WARN [MemtableFlushWriter:2] 2018-04-11 14:34:42,748 NativeLibrary.java:328 - 
> fsync(666) failed, errorno (22) {}
> com.sun.jna.LastErrorException: [22] Invalid argument
>  at org.apache.cassandra.utils.NativeLibraryLinux.fsync(Native Method) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.callFsync(NativeLibraryLinux.java:107)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.NativeLibrary.trySync(NativeLibrary.java:317) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySync(SyncUtil.java:179) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySyncDir(SyncUtil.java:190) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:107)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:141)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:402)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:53)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:368)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1153)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1086)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
>  [na:1.8.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>  [na:1.8.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$12.BCF32600.run(Unknown
>  Source) ~[na:na]
>  at java.lang.Thread.run(Thread.java:811) ~[na:2.9 (12-15-2017)]
>  
> Syslog shows the following 
> (logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt):
> INFO  [main] 2018-04-11 14:49:01,848 ColumnFamilyStore.java:406 - 
> Initializing apim.ur_to_op_by_op
> INFO  [MemoryMXBean notification dispatcher] 2018-04-11 14:49:25,889 
> GCInspector.java:284 - global GC in 206ms.  class storage: 28700680 -> 
> 28692744; miscellaneous non-heap storage: 49871216 -> 53570176; 
> nursery-allocate: 1296878920 -> 149116672; tenured-SOA: 140321968 -> 139143760
> #0: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x302a94) 
> [0x7f17e4f10a94]
> #1: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x306b2d) 
> [0x7f17e4f14b2d]
> #2: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xc82da) 
> [0x7f17e4cd62da]
> #3: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9prt29.so(+0x22056) 
> [0x7f17e6531056]
> #4: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f17ed0de390]
> #5: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2c4e1f) 
> [0x7f17e4ed2e1f]
> #6: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x158c04) 
>

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

2018-04-12 Thread Cyril Scetbon (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436640#comment-16436640
 ] 

Cyril Scetbon commented on CASSANDRA-14381:
---

hmm, and it's there in 2.1 too 
[https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2629-L2630]
 and it's been there for 4 years 
[https://github.com/apache/cassandra/commit/719103b649c1c5459683a8ffd1c013664f1ffbb6]

I really don't know why it's there. What if we need to restore the whole 
node/cluster for some reason ??

 

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14382) Use timed wait from Futures in Guava

2018-04-12 Thread Simon Zhou (JIRA)

Simon Zhou created CASSANDRA-14382:
--

 Summary: Use timed wait from Futures in Guava
 Key: CASSANDRA-14382
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14382
 Project: Cassandra
  Issue Type: Bug
Reporter: Simon Zhou


We upgraded Guava to 23.3 in trunk and there is timed wait feature 
(Futures.withTimeout) that we should use. Otherwise we have a whole bunch of 
stability issues. Generally if something fails or is unresponsive, lots of 
thread will hang. For example, validation in repair.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436463#comment-16436463
 ] 

Jay Zhuang commented on CASSANDRA-14381:


{{listsnapshots}} excludes local system keyspaces ({{system}} and 
{{system_schema}}): 
[{{StorageService.java:3290}}|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/service/StorageService.java#L3290],
 but not sure why.

> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

2018-04-12 Thread Cyril Scetbon (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cyril Scetbon updated CASSANDRA-14381:
--
Description: 
The output of *nodetool listsnapshots* is inconsistent with the snapshots 
created :
{code:java}
$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ nodetool snapshot -t tag1 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
options {skipFlush=false}
Snapshot directory: tag1

$ nodetool snapshot -t tag2 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
options {skipFlush=false}
Snapshot directory: tag2

$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ ls 
/usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
tag1 tag2{code}
 

 

  was:
the output of `nodetool listsnapshots` is inconsistent with the snapshots 
created :

```

$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ nodetool snapshot -t tag1 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
options \{skipFlush=false}
Snapshot directory: tag1

$ nodetool snapshot -t tag2 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
options \{skipFlush=false}
Snapshot directory: tag2

$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ ls 
/usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
tag1 tag2

```


> nodetool listsnapshots is missing snapshots
> ---
>
> Key: CASSANDRA-14381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: MacOs 10.12.5
> Java 1.8.0_144
> Cassandra 3.11.2 (brew install)
>Reporter: Cyril Scetbon
>Priority: Major
>
> The output of *nodetool listsnapshots* is inconsistent with the snapshots 
> created :
> {code:java}
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ nodetool snapshot -t tag1 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
> options {skipFlush=false}
> Snapshot directory: tag1
> $ nodetool snapshot -t tag2 --table local system
> Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
> options {skipFlush=false}
> Snapshot directory: tag2
> $ nodetool listsnapshots
> Snapshot Details:
> There are no snapshots
> $ ls 
> /usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
> tag1 tag2{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14381) nodetool listsnapshots is missing snapshots

2018-04-12 Thread Cyril Scetbon (JIRA)

Cyril Scetbon created CASSANDRA-14381:
-

 Summary: nodetool listsnapshots is missing snapshots
 Key: CASSANDRA-14381
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14381
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: MacOs 10.12.5

Java 1.8.0_144

Cassandra 3.11.2 (brew install)
Reporter: Cyril Scetbon


the output of `nodetool listsnapshots` is inconsistent with the snapshots 
created :

```

$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ nodetool snapshot -t tag1 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag1] and 
options \{skipFlush=false}
Snapshot directory: tag1

$ nodetool snapshot -t tag2 --table local system
Requested creating snapshot(s) for [system] with snapshot name [tag2] and 
options \{skipFlush=false}
Snapshot directory: tag2

$ nodetool listsnapshots
Snapshot Details:
There are no snapshots

$ ls 
/usr/local/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/snapshots/
tag1 tag2

```



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14379:
---
Status: Patch Available  (was: Open)

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436378#comment-16436378
 ] 

Jay Zhuang commented on CASSANDRA-14379:


| Branch | uTest | dTest |
| [14379-3.0|https://github.com/cooldoger/cassandra/tree/14379-3.0] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14379-3.0.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14379-3.0]
 | 
[#522|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/522]
| [14379-3.11|https://github.com/cooldoger/cassandra/tree/14379-3.11] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14379-3.11.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14379-3.11]
 | 
[#523|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/523]
| [14379-trunk|https://github.com/cooldoger/cassandra/tree/14379-trunk] | 
[!https://circleci.com/gh/cooldoger/cassandra/tree/14379-trunk.svg?style=svg!|https://circleci.com/gh/cooldoger/cassandra/tree/14379-trunk]
 | 
[#524|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/524]

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14346) Scheduled Repair in Cassandra

2018-04-12 Thread Joseph Lynch (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436371#comment-16436371
 ] 

Joseph Lynch commented on CASSANDRA-14346:
--

[~bdeggleston] I took a whack at speccing out the design for a repair sidecar 
tool might have per your suggestion in the [design 
doc|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit#heading=h.5f10ng8gzle8].
 Please let me know if this was the direction you were suggesting or if there 
are areas you'd like to see flushed out before we start working on the port. 
For what it's worth the design is basically exactly how we have it implemented 
in Priam right now (sans guice and tomcat and such), so porting should be 
pretty easy.

If the general sidecar ends up getting merged we can always merge the repair 
sidecar into it, until then it's "just another tool" like {{nodetool}} or 
{{stress}} (except it runs a HTTP server ...).

> Scheduled Repair in Cassandra
> -
>
> Key: CASSANDRA-14346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14346
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Repair
>Reporter: Joseph Lynch
>Priority: Major
>  Labels: CommunityFeedbackRequested
> Fix For: 4.0
>
> Attachments: ScheduledRepairV1_20180327.pdf
>
>
> There have been many attempts to automate repair in Cassandra, which makes 
> sense given that it is necessary to give our users eventual consistency. Most 
> recently CASSANDRA-10070, CASSANDRA-8911 and CASSANDRA-13924 have all looked 
> for ways to solve this problem.
> At Netflix we've built a scheduled repair service within Priam (our sidecar), 
> which we spoke about last year at NGCC. Given the positive feedback at NGCC 
> we focussed on getting it production ready and have now been using it in 
> production to repair hundreds of clusters, tens of thousands of nodes, and 
> petabytes of data for the past six months. Also based on feedback at NGCC we 
> have invested effort in figuring out how to integrate this natively into 
> Cassandra rather than open sourcing it as an external service (e.g. in Priam).
> As such, [~vinaykumarcse] and I would like to re-work and merge our 
> implementation into Cassandra, and have created a [design 
> document|https://docs.google.com/document/d/1RV4rOrG1gwlD5IljmrIq_t45rz7H3xs9GbFSEyGzEtM/edit?usp=sharing]
>  showing how we plan to make it happen, including the the user interface.
> As we work on the code migration from Priam to Cassandra, any feedback would 
> be greatly appreciated about the interface or v1 implementation features. I 
> have tried to call out in the document features which we explicitly consider 
> future work (as well as a path forward to implement them in the future) 
> because I would very much like to get this done before the 4.0 merge window 
> closes, and to do that I think aggressively pruning scope is going to be a 
> necessity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14380) Cassandra crashes after fsync exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436357#comment-16436357
 ] 

Ariel Weisberg commented on CASSANDRA-14380:


This looks like a bad FD, but I just wanted to drop a note here that we can't 
catch fsync errors and continue so we shouldn't add code that does that in C*. 
There was a Postgres bug where fsync returned an error and the FS lost data, 
but subsequent fsyncs succeeded.

The LastErrorException code in NativeLibrary looks a little janky. What's up 
with that? When would trySync be something we would merely try? If try is good 
enough why do it at all considering try is the default behavior of a series of 
unsynced filesystem operations.

> Cassandra crashes after fsync exception
> ---
>
> Key: CASSANDRA-14380
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14380
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Adam Geiger
>Priority: Critical
> Attachments: debug.log, debug.log.1.zip, 
> logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt
>
>
> Running Cassandra with a Rook Ceph filesystem within Kubernetes.  During the 
> startup, the following Warnings in the debug log pop up and then Cassandra 
> crashes shortly after and restarts.  It looks like before hitting this error, 
> it is doing a lot of writing and flushing
> WARN [MemtableFlushWriter:2] 2018-04-11 14:34:42,748 NativeLibrary.java:328 - 
> fsync(666) failed, errorno (22) {}
> com.sun.jna.LastErrorException: [22] Invalid argument
>  at org.apache.cassandra.utils.NativeLibraryLinux.fsync(Native Method) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.NativeLibraryLinux.callFsync(NativeLibraryLinux.java:107)
>  ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.NativeLibrary.trySync(NativeLibrary.java:317) 
> ~[apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySync(SyncUtil.java:179) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at org.apache.cassandra.utils.SyncUtil.trySyncDir(SyncUtil.java:190) 
> [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:107)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:141)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:402)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:53)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:368)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1153)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1086)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
>  [na:1.8.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
>  [na:1.8.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
>  [apache-cassandra-3.11.0.jar:3.11.0]
>  at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$12.BCF32600.run(Unknown
>  Source) ~[na:na]
>  at java.lang.Thread.run(Thread.java:811) ~[na:2.9 (12-15-2017)]
>  
> Syslog shows the following 
> (logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt):
> INFO  [main] 2018-04-11 14:49:01,848 ColumnFamilyStore.java:406 - 
> Initializing apim.ur_to_op_by_op
> INFO  [MemoryMXBean notification dispatcher] 2018-04-11 14:49:25,889 
> GCInspector.java:284 - global GC in 206ms.  class storage: 28700680 -> 
> 28692744; miscellaneous non-heap storage: 49871216 -> 53570176; 
> nursery-allocate: 1296878920 -> 149116672; tenured-SOA: 140321968 -> 139143760
> #0: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x302a94) 
> [0x7f17e4f10a94]
> #1: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x306b2d) 
> [0x7f17e4f14b2d]
> #2:

[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative

2018-04-12 Thread Preetika Tyagi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436299#comment-16436299
 ] 

Preetika Tyagi commented on CASSANDRA-13853:


Thank you! :)

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0
>
> Attachments: cassandra-13853-v6.patch, jira_13853_dtest_v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13853) nodetool describecluster should be more informative


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-13853:
---
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

Committed as cassandra 
[40aeaf0c12b24dbc0f554fbd00a42fe739767ac7|https://github.com/apache/cassandra/commit/40aeaf0c12b24dbc0f554fbd00a42fe739767ac7]
 and dtest 
[3f89310c05c96afd7e86eb6763386dbb41068878|https://github.com/apache/cassandra-dtest/commit/3f89310c05c96afd7e86eb6763386dbb41068878].
 Thanks! [~pree] congratulations on your first Cassandra commit.

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.0
>
> Attachments: cassandra-13853-v6.patch, jira_13853_dtest_v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: nodetool describecluster should be more informative

2018-04-12 Thread aweisberg

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 95735a4d0 -> 3f89310c0


nodetool describecluster should be more informative

Patch by Preetika Tyagi; Reviewed by Ariel Weisberg for CASSANDRA-13853


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/3f89310c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/3f89310c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/3f89310c

Branch: refs/heads/master
Commit: 3f89310c05c96afd7e86eb6763386dbb41068878
Parents: 95735a4
Author: Preetika Tyagi 
Authored: Wed Apr 11 15:16:22 2018 -0400
Committer: Ariel Weisberg 
Committed: Thu Apr 12 16:12:34 2018 -0400

--
 nodetool_test.py | 64 +++
 1 file changed, 64 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/3f89310c/nodetool_test.py
--
diff --git a/nodetool_test.py b/nodetool_test.py
index e913b30..096e763 100644
--- a/nodetool_test.py
+++ b/nodetool_test.py
@@ -324,3 +324,67 @@ class TestNodetool(Tester):
 assert 'Number of concurrent view builders should be greater than 
0.', e.message
 else:
 self.fail("Expected error when setting and invalid value")
+
+def test_describecluster_more_information_three_datacenters(self):
+"""
+nodetool describecluster should be more informative. It should include 
detailes
+for total node count, list of datacenters, RF, number of nodes per dc, 
how many
+are down and version(s).
+@jira_ticket CASSANDRA-13853
+@expected_result This test invokes nodetool describecluster and 
matches the output with the expected one
+"""
+cluster = self.cluster
+cluster.populate([2, 3, 1]).start(wait_for_binary_proto=True)
+
+node1_dc1, node2_dc1, node1_dc2, node2_dc2, node3_dc2, node1_dc3 = 
cluster.nodelist()
+
+session_dc1 = self.patient_cql_connection(node1_dc1)
+session_dc1.execute("create KEYSPACE ks1 WITH replication = {'class': 
'NetworkTopologyStrategy', 'dc1': 3, 'dc2':5, 'dc3':1}")
+
+session_dc3 = self.patient_cql_connection(node1_dc3)
+session_dc3.execute("create KEYSPACE ks2 WITH replication = {'class': 
'NetworkTopologyStrategy', 'dc1': 3, 'dc2':5, 'dc3':1}")
+
+out_node1_dc1, err, _ = node1_dc1.nodetool('describecluster')
+assert 0 == len(err), err
+
+out_node2_dc1, err, _ = node2_dc1.nodetool('describecluster')
+assert 0 == len(err), err
+assert out_node1_dc1 == out_node2_dc1
+
+out_node1_dc2, err, _ = node1_dc2.nodetool('describecluster')
+assert 0 == len(err), err
+assert out_node1_dc1 == out_node1_dc2
+
+out_node2_dc2, err, _ = node2_dc2.nodetool('describecluster')
+assert 0 == len(err), err
+assert out_node1_dc1 == out_node2_dc2
+
+out_node2_dc3, err, _ = node3_dc2.nodetool('describecluster')
+assert 0 == len(err), err
+assert out_node1_dc1 == out_node2_dc3
+
+out_node1_dc3, err, _ = node1_dc3.nodetool('describecluster')
+assert 0 == len(err), err
+assert out_node1_dc1 == out_node1_dc3
+
+logger.debug(out_node1_dc1)
+assert 'Live: 6' in out_node1_dc1
+assert 'Joining: 0' in out_node1_dc1
+assert 'Moving: 0' in out_node1_dc1
+assert 'Leaving: 0' in out_node1_dc1
+assert 'Unreachable: 0' in out_node1_dc1
+assert 'Data Centers:' in out_node1_dc1
+assert 'dc1 #Nodes: 2 #Down: 0' in out_node1_dc1
+assert 'dc2 #Nodes: 3 #Down: 0' in out_node1_dc1
+assert 'dc3 #Nodes: 1 #Down: 0' in out_node1_dc1
+assert 'Database versions:' in out_node1_dc1
+assert '4.0.0: [127.0.0.6:7000, 127.0.0.5:7000, 127.0.0.4:7000, 
127.0.0.3:7000, 127.0.0.2:7000, 127.0.0.1:7000]' in out_node1_dc1
+assert 'Keyspaces:' in out_node1_dc1
+assert 'system_schema -> Replication class: LocalStrategy {}' in 
out_node1_dc1
+assert 'system -> Replication class: LocalStrategy {}' in out_node1_dc1
+assert 'system_traces -> Replication class: SimpleStrategy 
{replication_factor=2}' in out_node1_dc1
+assert 'system_distributed -> Replication class: SimpleStrategy 
{replication_factor=3}' in out_node1_dc1
+assert 'system_auth -> Replication class: SimpleStrategy 
{replication_factor=1}' in out_node1_dc1
+assert 'ks1 -> Replication class: NetworkTopologyStrategy {dc2=5, 
dc1=3, dc3=1}' in out_node1_dc1
+assert 'ks2 -> Replication class: NetworkTopologyStrategy {dc2=5, 
dc1=3, dc3=1}' in out_node1_dc1
+assert 'Cluster

cassandra git commit: nodetool describecluster should be more informative [Forced Update!]

2018-04-12 Thread aweisberg

Repository: cassandra
Updated Branches:
  refs/heads/trunk 42efbddd3 -> 40aeaf0c1 (forced update)


nodetool describecluster should be more informative

Patch by Preetika Tyagi, Reviewed by Ariel Weisberg for CASSANDRA-13853


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/40aeaf0c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/40aeaf0c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/40aeaf0c

Branch: refs/heads/trunk
Commit: 40aeaf0c12b24dbc0f554fbd00a42fe739767ac7
Parents: 3747a6c
Author: Preetika Tyagi 
Authored: Wed Apr 11 15:18:17 2018 -0400
Committer: Ariel Weisberg 
Committed: Thu Apr 12 16:11:18 2018 -0400

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 22 +
 .../org/apache/cassandra/gms/GossiperMBean.java |  4 +
 .../cassandra/service/StorageService.java   | 11 +++
 .../cassandra/service/StorageServiceMBean.java  |  5 ++
 .../org/apache/cassandra/tools/NodeProbe.java   | 10 +++
 .../tools/nodetool/DescribeCluster.java | 94 +++-
 7 files changed, 144 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/40aeaf0c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 754b9f3..0935434 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * nodetool describecluster should be more informative (CASSANDRA-13853)
  * Compaction performance improvements (CASSANDRA-14261) 
  * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
  * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/40aeaf0c/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 24b659b..0f3a5cd 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -32,6 +32,7 @@ import javax.management.ObjectName;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
 import com.google.common.util.concurrent.Uninterruptibles;
 
 import org.apache.cassandra.locator.InetAddressAndPort;
@@ -1732,6 +1733,27 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 return state != null ? state.getReleaseVersion() : null;
 }
 
+public Map getReleaseVersionsWithPort()
+{
+Map results = new HashMap();
+Iterable allHosts = 
Iterables.concat(Gossiper.instance.getLiveMembers(), 
Gossiper.instance.getUnreachableMembers());
+
+for (InetAddressAndPort host : allHosts)
+{
+CassandraVersion version = getReleaseVersion(host);
+String stringVersion = version == null ? "" : version.toString();
+List hosts = results.get(stringVersion);
+if (hosts == null)
+{
+hosts = new ArrayList<>();
+results.put(stringVersion, hosts);
+}
+hosts.add(host.getHostAddress(true));
+}
+
+return results;
+}
+
 @Nullable
 public UUID getSchemaVersion(InetAddressAndPort ep)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/40aeaf0c/src/java/org/apache/cassandra/gms/GossiperMBean.java
--
diff --git a/src/java/org/apache/cassandra/gms/GossiperMBean.java 
b/src/java/org/apache/cassandra/gms/GossiperMBean.java
index 1b116e1..92df2cd 100644
--- a/src/java/org/apache/cassandra/gms/GossiperMBean.java
+++ b/src/java/org/apache/cassandra/gms/GossiperMBean.java
@@ -19,6 +19,7 @@ package org.apache.cassandra.gms;
 
 import java.net.UnknownHostException;
 import java.util.List;
+import java.util.Map;
 
 public interface GossiperMBean
 {
@@ -34,4 +35,7 @@ public interface GossiperMBean
 
 public List getSeeds();
 
+/** Returns each node's database release version */
+public Map getReleaseVersionsWithPort();
+
 }
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/cassandra/blob/40aeaf0c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index

cassandra git commit: nodetool describecluster should be more informative

2018-04-12 Thread aweisberg

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3747a6cd9 -> 42efbddd3


nodetool describecluster should be more informative

Patch by Preetika Tyagi, Reviewed by Ariel Weisberg for CASSANDRA-13853


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/42efbddd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/42efbddd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/42efbddd

Branch: refs/heads/trunk
Commit: 42efbddd3aa942e2f325b035ace05fc3755d021f
Parents: 3747a6c
Author: Ariel Weisberg 
Authored: Wed Apr 11 15:18:17 2018 -0400
Committer: Ariel Weisberg 
Committed: Thu Apr 12 16:08:27 2018 -0400

--
 CHANGES.txt |  1 +
 src/java/org/apache/cassandra/gms/Gossiper.java | 22 +
 .../org/apache/cassandra/gms/GossiperMBean.java |  4 +
 .../cassandra/service/StorageService.java   | 11 +++
 .../cassandra/service/StorageServiceMBean.java  |  5 ++
 .../org/apache/cassandra/tools/NodeProbe.java   | 10 +++
 .../tools/nodetool/DescribeCluster.java | 94 +++-
 7 files changed, 144 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/42efbddd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 754b9f3..0935434 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * nodetool describecluster should be more informative (CASSANDRA-13853)
  * Compaction performance improvements (CASSANDRA-14261) 
  * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
  * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/42efbddd/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 24b659b..0f3a5cd 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -32,6 +32,7 @@ import javax.management.ObjectName;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
 import com.google.common.util.concurrent.Uninterruptibles;
 
 import org.apache.cassandra.locator.InetAddressAndPort;
@@ -1732,6 +1733,27 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 return state != null ? state.getReleaseVersion() : null;
 }
 
+public Map getReleaseVersionsWithPort()
+{
+Map results = new HashMap();
+Iterable allHosts = 
Iterables.concat(Gossiper.instance.getLiveMembers(), 
Gossiper.instance.getUnreachableMembers());
+
+for (InetAddressAndPort host : allHosts)
+{
+CassandraVersion version = getReleaseVersion(host);
+String stringVersion = version == null ? "" : version.toString();
+List hosts = results.get(stringVersion);
+if (hosts == null)
+{
+hosts = new ArrayList<>();
+results.put(stringVersion, hosts);
+}
+hosts.add(host.getHostAddress(true));
+}
+
+return results;
+}
+
 @Nullable
 public UUID getSchemaVersion(InetAddressAndPort ep)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/42efbddd/src/java/org/apache/cassandra/gms/GossiperMBean.java
--
diff --git a/src/java/org/apache/cassandra/gms/GossiperMBean.java 
b/src/java/org/apache/cassandra/gms/GossiperMBean.java
index 1b116e1..92df2cd 100644
--- a/src/java/org/apache/cassandra/gms/GossiperMBean.java
+++ b/src/java/org/apache/cassandra/gms/GossiperMBean.java
@@ -19,6 +19,7 @@ package org.apache.cassandra.gms;
 
 import java.net.UnknownHostException;
 import java.util.List;
+import java.util.Map;
 
 public interface GossiperMBean
 {
@@ -34,4 +35,7 @@ public interface GossiperMBean
 
 public List getSeeds();
 
+/** Returns each node's database release version */
+public Map getReleaseVersionsWithPort();
+
 }
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/cassandra/blob/42efbddd/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 91206c1..e5911f3 100644
---

[jira] [Commented] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436197#comment-16436197
 ] 

Jeff Jirsa commented on CASSANDRA-12238:


Re-opened

(Also cc [~iamaleksey] because counters)


> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
>

[jira] [Updated] (CASSANDRA-14380) Cassandra crashes after fsync exception

2018-04-12 Thread Adam Geiger (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Geiger updated CASSANDRA-14380:

Description: 
Running Cassandra with a Rook Ceph filesystem within Kubernetes.  During the 
startup, the following Warnings in the debug log pop up and then Cassandra 
crashes shortly after and restarts.  It looks like before hitting this error, 
it is doing a lot of writing and flushing

WARN [MemtableFlushWriter:2] 2018-04-11 14:34:42,748 NativeLibrary.java:328 - 
fsync(666) failed, errorno (22) {}
com.sun.jna.LastErrorException: [22] Invalid argument
 at org.apache.cassandra.utils.NativeLibraryLinux.fsync(Native Method) 
~[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.utils.NativeLibraryLinux.callFsync(NativeLibraryLinux.java:107)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.NativeLibrary.trySync(NativeLibrary.java:317) 
~[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.SyncUtil.trySync(SyncUtil.java:179) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.SyncUtil.trySyncDir(SyncUtil.java:190) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:107)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:141) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:402)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:53)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:368)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1153)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1086)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) 
[na:1.8.0]
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 
[na:1.8.0]
 at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$12.BCF32600.run(Unknown
 Source) ~[na:na]
 at java.lang.Thread.run(Thread.java:811) ~[na:2.9 (12-15-2017)]

 
Syslog shows the following 
(logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt):

INFO  [main] 2018-04-11 14:49:01,848 ColumnFamilyStore.java:406 - Initializing 
apim.ur_to_op_by_op
INFO  [MemoryMXBean notification dispatcher] 2018-04-11 14:49:25,889 
GCInspector.java:284 - global GC in 206ms.  class storage: 28700680 -> 
28692744; miscellaneous non-heap storage: 49871216 -> 53570176; 
nursery-allocate: 1296878920 -> 149116672; tenured-SOA: 140321968 -> 139143760
#0: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x302a94) 
[0x7f17e4f10a94]
#1: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x306b2d) 
[0x7f17e4f14b2d]
#2: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xc82da) 
[0x7f17e4cd62da]
#3: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9prt29.so(+0x22056) 
[0x7f17e6531056]
#4: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f17ed0de390]
#5: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2c4e1f) 
[0x7f17e4ed2e1f]
#6: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x158c04) 
[0x7f17e4d66c04]
#7: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x542d24) 
[0x7f17e5150d24]
#8: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x542e0b) 
[0x7f17e5150e0b]
#9: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x54981a) 
[0x7f17e515781a]
#10: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x5494c8) 
[0x7f17e51574c8]
#11: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x548dd2) 
[0x7f17e5156dd2]
#12: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2d7019) 
[0x7f17e4ee5019]
#13: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xd31ee) 
[0x7f17e4ce11ee]
#14: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xd3e51) 
[0x7f17e4ce1e51]
#15:

[jira] [Created] (CASSANDRA-14380) Cassandra crashes after fsync exception

2018-04-12 Thread Adam Geiger (JIRA)

Adam Geiger created CASSANDRA-14380:
---

 Summary: Cassandra crashes after fsync exception
 Key: CASSANDRA-14380
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14380
 Project: Cassandra
  Issue Type: Bug
Reporter: Adam Geiger
 Attachments: debug.log, debug.log.1.zip, 
logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt

Running Cassandra with a Rook Ceph filesystem within Kubernetes.  During the 
startup, the following Warnings in the debug log pop up and then Cassandra 
crashes shortly after and restarts:

WARN [MemtableFlushWriter:2] 2018-04-11 14:34:42,748 NativeLibrary.java:328 - 
fsync(666) failed, errorno (22) {}
com.sun.jna.LastErrorException: [22] Invalid argument
 at org.apache.cassandra.utils.NativeLibraryLinux.fsync(Native Method) 
~[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.utils.NativeLibraryLinux.callFsync(NativeLibraryLinux.java:107)
 ~[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.NativeLibrary.trySync(NativeLibrary.java:317) 
~[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.SyncUtil.trySync(SyncUtil.java:179) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at org.apache.cassandra.utils.SyncUtil.trySyncDir(SyncUtil.java:190) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.util.SequentialWriter.openChannel(SequentialWriter.java:107)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.util.SequentialWriter.(SequentialWriter.java:141) 
[apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:402)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:53)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:368)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:281)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1153)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1086)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) 
[na:1.8.0]
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 
[na:1.8.0]
 at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
 [apache-cassandra-3.11.0.jar:3.11.0]
 at 
org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$12.BCF32600.run(Unknown
 Source) ~[na:na]
 at java.lang.Thread.run(Thread.java:811) ~[na:2.9 (12-15-2017)]

 
Syslog shows the following 
(logs-from-cassandra-in-r97bb66e967-apiconnect-cc-0.txt):

INFO  [main] 2018-04-11 14:49:01,848 ColumnFamilyStore.java:406 - Initializing 
apim.ur_to_op_by_op
INFO  [MemoryMXBean notification dispatcher] 2018-04-11 14:49:25,889 
GCInspector.java:284 - global GC in 206ms.  class storage: 28700680 -> 
28692744; miscellaneous non-heap storage: 49871216 -> 53570176; 
nursery-allocate: 1296878920 -> 149116672; tenured-SOA: 140321968 -> 139143760
#0: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x302a94) 
[0x7f17e4f10a94]
#1: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x306b2d) 
[0x7f17e4f14b2d]
#2: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xc82da) 
[0x7f17e4cd62da]
#3: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9prt29.so(+0x22056) 
[0x7f17e6531056]
#4: /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f17ed0de390]
#5: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2c4e1f) 
[0x7f17e4ed2e1f]
#6: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x158c04) 
[0x7f17e4d66c04]
#7: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x542d24) 
[0x7f17e5150d24]
#8: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x542e0b) 
[0x7f17e5150e0b]
#9: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x54981a) 
[0x7f17e515781a]
#10: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x5494c8) 
[0x7f17e51574c8]
#11: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x548dd2) 
[0x7f17e5156dd2]
#12: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0x2d7019) 
[0x7f17e4ee5019]
#13: /opt/ibm/java/jre/lib/amd64/compressedrefs/libj9jit29.so(+0xd31ee)

[jira] [Commented] (CASSANDRA-13426) Make all DDL statements idempotent and not dependent on global state

2018-04-12 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436169#comment-16436169
 ] 

Aleksey Yeschenko commented on CASSANDRA-13426:
---

Status update: all unit tests are 
[passing|https://circleci.com/gh/iamaleksey/cassandra/178] and so are all 
[dtests|https://circleci.com/gh/iamaleksey/cassandra/179]. 
{{sstablesplit_test.py::TestSSTableSplit::test_single_file_split}} is the only 
failure on the branch, and it's unrelated - CASSANDRA-14371 is the JIRA for it.

Occasionally, 
{{materialized_views_test.py::TestMaterializedViews::test_populate_mv_after_insert_wide_rows}}
 would fail too, because materialized views are amazing, and a subtle change in 
ordering of local/remote schema application exposes an existing race condition 
in MV building. I'm not going to fix it, as this patch is not the cause of the 
issue, but merely exposes it, and I have better things to do with my life.

Next step: address review feedback and slightly optimise the diffing code to 
take into account on-disk representation.

> Make all DDL statements idempotent and not dependent on global state
> 
>
> Key: CASSANDRA-13426
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13426
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 4.0
>
>
> A follow-up to CASSANDRA-9425 and a pre-requisite for CASSANDRA-10699.
> It's necessary for the latter to be able to apply any DDL statement several 
> times without side-effects. As part of the ticket I think we should also 
> clean up validation logic and our error texts. One example is varying 
> treatment of missing keyspace for DROP TABLE/INDEX/etc. statements with IF 
> EXISTS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12238:
---
Fix Version/s: 4.x
   3.11.x

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>

[jira] [Updated] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12238:
---
Component/s: Core

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>

[jira] [Updated] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12238:
---
Fix Version/s: 3.0.x

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>

[jira] [Updated] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12238:
---
Reproduced In: 3.11.2, 3.7  (was: 3.7)

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>

[jira] [Updated] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12238:
---
Labels: counters  (was: )

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>

[jira] [Reopened] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reopened CASSANDRA-12238:


> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>  Labels: counters
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.closeIfEmpty(Filter.java:49) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:22) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.Filter.applyToPartition(Filter.java:6) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
>

[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative

2018-04-12 Thread Preetika Tyagi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436126#comment-16436126
 ] 

Preetika Tyagi commented on CASSANDRA-13853:


Great. Thank you!

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.x
>
> Attachments: cassandra-13853-v6.patch, jira_13853_dtest_v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436122#comment-16436122
 ] 

Ariel Weisberg commented on CASSANDRA-13853:


You don't need to do anything else I just need to commit it. Should happen 
today or Friday.

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.x
>
> Attachments: cassandra-13853-v6.patch, jira_13853_dtest_v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative

2018-04-12 Thread Preetika Tyagi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436034#comment-16436034
 ] 

Preetika Tyagi commented on CASSANDRA-13853:


[~aweisberg] I just wanted to follow-up on the next steps that are required for 
the patch submission. Could you please let me know if there is anything I need 
to get done?

> nodetool describecluster should be more informative
> ---
>
> Key: CASSANDRA-13853
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13853
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Jon Haddad
>Assignee: Preetika Tyagi
>Priority: Minor
>  Labels: lhf
> Fix For: 4.x
>
> Attachments: cassandra-13853-v6.patch, jira_13853_dtest_v2.patch
>
>
> Additional information we should be displaying:
> * Total node count
> * List of datacenters, RF, with number of nodes per dc, how many are down, 
> * Version(s)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12882) Deadlock in MemtableAllocator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435947#comment-16435947
 ] 

Marcus Eriksson commented on CASSANDRA-12882:
-

[~nob13] could you attach a full stack trace for the node? jstack -l 

> Deadlock in MemtableAllocator
> -
>
> Key: CASSANDRA-12882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12882
> Project: Cassandra
>  Issue Type: Bug
> Environment: Ubuntu 14.40
> Cassandra 3.5
>Reporter: Nimi Wariboko Jr.
>Priority: Major
> Fix For: 3.11.x
>
> Attachments: cassandra.yaml, threaddump.txt
>
>
> I'm seeing an issue where a node will eventually lock up and their thread 
> pools - I looked into jstack, and a lot of threads are stuck in the Memtable 
> Allocator
> {code}
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
>   at 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:198)
>   at 
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
>   at 
> org.apache.cassandra.utils.memory.MemtableBufferAllocator.clone(MemtableBufferAllocator.java:41)
> {code}
> I looked into the code, and its not immediately apparent to me what thread 
> might hold the relevant lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12238) Unexpected exception during request java.lang.IndexOutOfBoundsException: null

2018-04-12 Thread Steve Severance (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435918#comment-16435918
 ] 

Steve Severance commented on CASSANDRA-12238:
-

I have I think a good understanding of what causes this. I have a counter table 
with the following schema:

CREATE TABLE WorkQueueIndex_v2(
 partitionid int,
 shardid uuid,
 count counter,
 complete counter,
 size counter,
 PRIMARY KEY ((partitionid),shardid)
);

If I run the following query at consistency level ONE it works fine:

SELECT shardid, complete, count FROM workqueueindex_v2 where partitionid = 14;

However if you run the same query with consistency level QUORUM it fails. The 
workaround is to select all the counter columns in the table instead of a 
subset of them.

[~ifesdjeen]: Can we reopen this issue?

> Unexpected exception during request java.lang.IndexOutOfBoundsException: null
> -
>
> Key: CASSANDRA-12238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12238
> Project: Cassandra
>  Issue Type: Bug
> Environment: datastax-ddc-3.7.0, CentOS 7, x86, JDK 1.8.0u91
>Reporter: Gábor Auth
>Priority: Major
>
> Our two DCs and eight nodes cluster throws sometimes:
> {code}
> ERROR [SharedPool-Worker-25] 2016-07-19 16:22:14,643 ErrorMessage.java:338 - 
> Unexpected exception during request
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[na:1.8.0_91]
> at java.nio.HeapByteBuffer.getShort(HeapByteBuffer.java:314) 
> ~[na:1.8.0_91]
> at 
> org.apache.cassandra.db.context.CounterContext.headerLength(CounterContext.java:141)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.access$100(CounterContext.java:76)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.(CounterContext.java:758)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext$ContextState.wrap(CounterContext.java:765)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.context.CounterContext.merge(CounterContext.java:271) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.Conflicts.mergeCounterValues(Conflicts.java:76) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Cells.reconcile(Cells.java:143) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:614)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:572)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:549) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at 
> org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:30) 
>

[jira] [Updated] (CASSANDRA-14261) Compaction Profiling Improvements


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-14261:
---
Status: Ready to Commit  (was: Patch Available)

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.0
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14261) Compaction Profiling Improvements


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-14261:
---
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

Nice catch. Backed out that change, added a comment on commit, and committed to 
trunk as 3747a6cd993c1a4d1a4a9345f8efe3ebc6cb482b


> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.0
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Compaction performance improvements

2018-04-12 Thread jjirsa

Repository: cassandra
Updated Branches:
  refs/heads/trunk 60563f4e8 -> 3747a6cd9


Compaction performance improvements

Patch by Jeff Jirsa; Reviewed by Marcus Eriksson for CASSANDRA-14261


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3747a6cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3747a6cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3747a6cd

Branch: refs/heads/trunk
Commit: 3747a6cd993c1a4d1a4a9345f8efe3ebc6cb482b
Parents: 60563f4
Author: Jeff Jirsa 
Authored: Thu Apr 12 08:45:27 2018 -0700
Committer: Jeff Jirsa 
Committed: Thu Apr 12 08:45:27 2018 -0700

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/concurrent/SEPWorker.java  |  3 ++-
 .../db/partitions/UnfilteredPartitionIterators.java  | 11 ++-
 .../apache/cassandra/db/rows/SerializationHelper.java|  5 +
 src/java/org/apache/cassandra/schema/ColumnMetadata.java |  4 ++--
 5 files changed, 20 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3747a6cd/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2dc2021..754b9f3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Compaction performance improvements (CASSANDRA-14261) 
  * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
  * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)
  * Rename internals to reflect CQL vocabulary (CASSANDRA-14354)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3747a6cd/src/java/org/apache/cassandra/concurrent/SEPWorker.java
--
diff --git a/src/java/org/apache/cassandra/concurrent/SEPWorker.java 
b/src/java/org/apache/cassandra/concurrent/SEPWorker.java
index d3c87c6..918349f 100644
--- a/src/java/org/apache/cassandra/concurrent/SEPWorker.java
+++ b/src/java/org/apache/cassandra/concurrent/SEPWorker.java
@@ -17,6 +17,7 @@
  */
 package org.apache.cassandra.concurrent;
 
+import java.util.concurrent.ThreadLocalRandom;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicReference;
 import java.util.concurrent.locks.LockSupport;
@@ -230,7 +231,7 @@ final class SEPWorker extends 
AtomicReference implements Runnabl
 // we should always have a thread about to wake up, but most threads 
are sleeping
 long sleep = 1L * pool.spinningCount.get();
 sleep = Math.min(100, sleep);
-sleep *= Math.random();
+sleep *= ThreadLocalRandom.current().nextDouble();
 sleep = Math.max(1, sleep);
 
 long start = System.nanoTime();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3747a6cd/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
--
diff --git 
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java 
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
index edb7833..f3c965a 100644
--- 
a/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
+++ 
b/src/java/org/apache/cassandra/db/partitions/UnfilteredPartitionIterators.java
@@ -139,10 +139,19 @@ public abstract class UnfilteredPartitionIterators
 {
 UnfilteredRowIterators.MergeListener rowListener = 
listener.getRowMergeListener(partitionKey, toMerge);
 
+// Make a single empty iterator object to merge, we don't need 
toMerge.size() copiess
+UnfilteredRowIterator empty = null;
+
 // Replace nulls by empty iterators
 for (int i = 0; i < toMerge.size(); i++)
+{
 if (toMerge.get(i) == null)
-toMerge.set(i, EmptyIterators.unfilteredRow(metadata, 
partitionKey, isReverseOrder));
+{
+if (null == empty)
+empty = EmptyIterators.unfilteredRow(metadata, 
partitionKey, isReverseOrder);
+toMerge.set(i, empty);
+}
+}
 
 return UnfilteredRowIterators.merge(toMerge, nowInSec, 
rowListener);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3747a6cd/src/java/org/apache/cassandra/db/rows/SerializationHelper.java
--
diff --git a/src/java/org/apache/cassandra/db/rows/SerializationHelper.java 
b/src/java/org/apache/cassandra/db/rows/SerializationHelper.java
index c7fb8e4..db23cb8 100644
---

[jira] [Commented] (CASSANDRA-12882) Deadlock in MemtableAllocator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435829#comment-16435829
 ] 

Jeff Jirsa commented on CASSANDRA-12882:


Have seen something similar when either a flush thread or read thread is 
holding the memtable oporder barrier and the slab allocator can't allocate 
because it's at it's max allocation size.

I don't see a blocked flush thread in your stack, and it's waiting on:

{code}
"CompactionExecutor:87214" #790268 daemon prio=1 os_prio=4 
tid=0x7f8f60346800 nid=0x2d3e runnable [0x7fd8acf9b000]
   java.lang.Thread.State: RUNNABLE
at org.apache.cassandra.dht.Bounds.contains(Bounds.java:52)
at org.apache.cassandra.dht.Bounds.intersects(Bounds.java:80)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:534)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:520)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:651)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:330)
- locked <0x0005580ed570> (a 
org.apache.cassandra.db.compaction.LeveledManifest)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
- locked <0x000558020740> (a 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
at 
org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:108)
- locked <0x000558056bd0> (a 
org.apache.cassandra.db.compaction.CompactionStrategyManager)
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:258)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

Is that thread making progress? Do you have some incredible number of sstables? 
How often are you trying to flush (what are your memtable settings)? 


> Deadlock in MemtableAllocator
> -
>
> Key: CASSANDRA-12882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12882
> Project: Cassandra
>  Issue Type: Bug
> Environment: Ubuntu 14.40
> Cassandra 3.5
>Reporter: Nimi Wariboko Jr.
>Priority: Major
> Fix For: 3.11.x
>
> Attachments: cassandra.yaml, threaddump.txt
>
>
> I'm seeing an issue where a node will eventually lock up and their thread 
> pools - I looked into jstack, and a lot of threads are stuck in the Memtable 
> Allocator
> {code}
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
>   at 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:198)
>   at 
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
>   at 
> org.apache.cassandra.utils.memory.MemtableBufferAllocator.clone(MemtableBufferAllocator.java:41)
> {code}
> I looked into the code, and its not immediately apparent to me what thread 
> might hold the relevant lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-12882) Deadlock in MemtableAllocator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435829#comment-16435829
 ] 

Jeff Jirsa edited comment on CASSANDRA-12882 at 4/12/18 3:57 PM:
-

Have seen something similar when either a flush thread or read thread is 
holding the memtable oporder barrier and the slab allocator can't allocate 
because it's at it's max allocation size.

I see a blocked flush thread in your stack, and it's waiting on:

{code}
"CompactionExecutor:87214" #790268 daemon prio=1 os_prio=4 
tid=0x7f8f60346800 nid=0x2d3e runnable [0x7fd8acf9b000]
   java.lang.Thread.State: RUNNABLE
at org.apache.cassandra.dht.Bounds.contains(Bounds.java:52)
at org.apache.cassandra.dht.Bounds.intersects(Bounds.java:80)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:534)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:520)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:651)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:330)
- locked <0x0005580ed570> (a 
org.apache.cassandra.db.compaction.LeveledManifest)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
- locked <0x000558020740> (a 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
at 
org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:108)
- locked <0x000558056bd0> (a 
org.apache.cassandra.db.compaction.CompactionStrategyManager)
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:258)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

Is that thread making progress? Do you have some incredible number of sstables? 
How often are you trying to flush (what are your memtable settings)? 



was (Author: jjirsa):
Have seen something similar when either a flush thread or read thread is 
holding the memtable oporder barrier and the slab allocator can't allocate 
because it's at it's max allocation size.

I don't see a blocked flush thread in your stack, and it's waiting on:

{code}
"CompactionExecutor:87214" #790268 daemon prio=1 os_prio=4 
tid=0x7f8f60346800 nid=0x2d3e runnable [0x7fd8acf9b000]
   java.lang.Thread.State: RUNNABLE
at org.apache.cassandra.dht.Bounds.contains(Bounds.java:52)
at org.apache.cassandra.dht.Bounds.intersects(Bounds.java:80)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:534)
at 
org.apache.cassandra.db.compaction.LeveledManifest.overlapping(LeveledManifest.java:520)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:651)
at 
org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:330)
- locked <0x0005580ed570> (a 
org.apache.cassandra.db.compaction.LeveledManifest)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
- locked <0x000558020740> (a 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
at 
org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:108)
- locked <0x000558056bd0> (a 
org.apache.cassandra.db.compaction.CompactionStrategyManager)
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:258)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

Is that thread making progress? Do you have some incredible number of sstables? 
How often are you trying to flush (what are your memtable settings)? 


> Deadlock in MemtableAllocator
> -
>
> Key: CASSANDRA-12882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12882
> Project: Cassandra
>  Issue

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

2018-04-12 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435587#comment-16435587
 ] 

Stefan Podkowinski commented on CASSANDRA-12151:


I must have missed that log statement when filtering by user. Its a bit strange 
that failed login attempts don't get logged, if you filter by a particular 
user. But that's probably more a minor issue compared to the the question 
regarding how we should go on with prepared statements. Cause I can't see how 
we can get away with not logging these, if we want to address any compliance or 
security use cases.

Any ideas?

> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12882) Deadlock in MemtableAllocator

2018-04-12 Thread Norbert Schultz (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435522#comment-16435522
 ] 

Norbert Schultz commented on CASSANDRA-12882:
-

We have the same Problem on fairly huge machines with Cassandra 3.0.14 & DSE 
5.0.11

A lot of Threads waiting for the MemTableAllocator, the whole instance is 
blocking and does not respond to anything.

 
{code:java}
"SharedPool-Worker-19" daemon prio=5 tid=1542 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
Local Variable: org.apache.cassandra.utils.concurrent.WaitQueue$AnySignal#10
at 
org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:162)
at 
org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:82)
at 
org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
at 
org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
...{code}

> Deadlock in MemtableAllocator
> -
>
> Key: CASSANDRA-12882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12882
> Project: Cassandra
>  Issue Type: Bug
> Environment: Ubuntu 14.40
> Cassandra 3.5
>Reporter: Nimi Wariboko Jr.
>Priority: Major
> Fix For: 3.11.x
>
> Attachments: cassandra.yaml, threaddump.txt
>
>
> I'm seeing an issue where a node will eventually lock up and their thread 
> pools - I looked into jstack, and a lot of threads are stuck in the Memtable 
> Allocator
> {code}
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
>   at 
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279)
>   at 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.allocate(MemtableAllocator.java:198)
>   at 
> org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:89)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
>   at 
> org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
>   at 
> org.apache.cassandra.utils.memory.MemtableBufferAllocator.clone(MemtableBufferAllocator.java:41)
> {code}
> I looked into the code, and its not immediately apparent to me what thread 
> might hold the relevant lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14261) Compaction Profiling Improvements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435468#comment-16435468
 ] 

Marcus Eriksson edited comment on CASSANDRA-14261 at 4/12/18 12:35 PM:
---

this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{code}
cqlsh> describe sc.counters;

   
CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
{code}

the "for thrift" comment should be removed though


was (Author: krummas):
this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{code}
cqlsh> describe sc.counters;

   
CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
{code}

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14261) Compaction Profiling Improvements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435468#comment-16435468
 ] 

Marcus Eriksson edited comment on CASSANDRA-14261 at 4/12/18 12:30 PM:
---

this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{code}
cqlsh> describe sc.counters;

   
CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
{code}


was (Author: krummas):
{{this lgtm, but I don't think the }}{{isCounterColumn}}{{ change is correct - 
it seems super columns with counters are represented as a map with counter 
values, like this (in 3.0, upgraded from 2.1):}}
{{cqlsh> describe sc.counters;}}{{CREATE TABLE sc.counters (}}
{{ key text,}}
{{ column1 text,}}
{{ column2 blob,}}
{{ "" map,}}
{{ value counter,}}
{{ PRIMARY KEY (key, column1, column2)}}
{{ ) WITH COMPACT STORAGE}}
{{ AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)}}
{{ AND bloom_filter_fp_chance = 0.01}}
{{ AND caching =}}{{{'keys': 'ALL', 'rows_per_partition': 'NONE'}}}{{AND 
comment = ''}}
{{ AND compaction =}}{{{'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}}}{{AND compression 
=}}{{{'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}}}{{AND crc_check_chance = 
1.0}}
{{ AND dclocal_read_repair_chance = 0.1}}
{{ AND default_time_to_live = 0}}
{{ AND gc_grace_seconds = 864000}}
{{ AND max_index_interval = 2048}}
{{ AND memtable_flush_period_in_ms = 0}}
{{ AND min_index_interval = 128}}
{{ AND read_repair_chance = 0.0}}
{{ AND speculative_retry = 'NONE';}}
{{ that "for thrift" comment should be removed though}}

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14261) Compaction Profiling Improvements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435468#comment-16435468
 ] 

Marcus Eriksson edited comment on CASSANDRA-14261 at 4/12/18 12:28 PM:
---

{{this lgtm, but I don't think the }}{{isCounterColumn}}{{ change is correct - 
it seems super columns with counters are represented as a map with counter 
values, like this (in 3.0, upgraded from 2.1):}}
{{cqlsh> describe sc.counters;}}{{CREATE TABLE sc.counters (}}
{{ key text,}}
{{ column1 text,}}
{{ column2 blob,}}
{{ "" map,}}
{{ value counter,}}
{{ PRIMARY KEY (key, column1, column2)}}
{{ ) WITH COMPACT STORAGE}}
{{ AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)}}
{{ AND bloom_filter_fp_chance = 0.01}}
{{ AND caching =}}{{{'keys': 'ALL', 'rows_per_partition': 'NONE'}}}{{AND 
comment = ''}}
{{ AND compaction =}}{{{'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}}}{{AND compression 
=}}{{{'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}}}{{AND crc_check_chance = 
1.0}}
{{ AND dclocal_read_repair_chance = 0.1}}
{{ AND default_time_to_live = 0}}
{{ AND gc_grace_seconds = 864000}}
{{ AND max_index_interval = 2048}}
{{ AND memtable_flush_period_in_ms = 0}}
{{ AND min_index_interval = 128}}
{{ AND read_repair_chance = 0.0}}
{{ AND speculative_retry = 'NONE';}}
{{ that "for thrift" comment should be removed though}}


was (Author: krummas):
this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{{cqlsh> describe sc.counters;  

 

CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
}}
that "for thrift" comment should be removed though

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14261) Compaction Profiling Improvements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435468#comment-16435468
 ] 

Marcus Eriksson edited comment on CASSANDRA-14261 at 4/12/18 12:27 PM:
---

this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{{cqlsh> describe sc.counters;  

 

CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
}}
that "for thrift" comment should be removed though


was (Author: krummas):
this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{{cqlsh> describe sc.counters;  

 

CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
}}
that "for thrift" comment should be removed though

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435468#comment-16435468
 ] 

Marcus Eriksson commented on CASSANDRA-14261:
-

this lgtm, but I don't think the {{isCounterColumn}} change is correct - it 
seems super columns with counters are represented as a map with counter values, 
like this (in 3.0, upgraded from 2.1):
{{cqlsh> describe sc.counters;  

 

CREATE TABLE sc.counters (
key text,
column1 text,
column2 blob,
"" map,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (column1 ASC, column2 ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = 'NONE';
}}
that "for thrift" comment should be removed though

> Compaction Profiling Improvements
> -
>
> Key: CASSANDRA-14261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14261
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
> Attachments: patched-hot-threads.png, patched-tlab.png, 
> unpatched-hot-threads-top.png, unpatched-hot-threads.png, unpatched-tlab.png
>
>
> There's some low hanging fruit in some laptop compaction runs, such as 
> creating a ton of the same object unnecessarily and hashing cell names 
> repeatedly to see if a column is dropped even when we should know that the 
> table has no dropped columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14055) Index redistribution breaks SASI index

2018-04-12 Thread Ludovic Boutros (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435231#comment-16435231
 ] 

Ludovic Boutros commented on CASSANDRA-14055:
-

Any news on this ? Thx.

> Index redistribution breaks SASI index
> --
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Ludovic Boutros
>Assignee: Jordan West
>Priority: Major
>  Labels: patch, sasi
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, 
> CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, 
> CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : 
> [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14304) DELETE after INSERT IF NOT EXISTS does not work

2018-04-12 Thread Julien (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435165#comment-16435165
 ] 

Julien commented on CASSANDRA-14304:


We suspect that an important condition to reproduce the issue is that the 
client and server machines clocks are not synchronized.

It is necessary to synchronize all the server clocks, but as far as I know, 
this requirement does not apply to the client machine, or does it ?

> DELETE after INSERT IF NOT EXISTS does not work
> ---
>
> Key: CASSANDRA-14304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14304
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Julien
>Assignee: Vinay Chella
>Priority: Major
> Attachments: debug.log, system.log
>
>
> DELETE a row immediately after INSERT IF NOT EXISTS does not work.
> Can be reproduced with this CQL script:
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 1 };
> CREATE TABLE ks.ta ( id text PRIMARY KEY, col text );
> INSERT INTO ks.ta (id, col) VALUES ('myId', 'myCol') IF NOT EXISTS;
> DELETE FROM ks.ta WHERE id = 'myId';
> SELECT * FROM ks.ta WHERE id='myId';
> {code}
> {code:java}
> [cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> WARNING: pyreadline dependency missing.  Install to enable tab completion.
> cqlsh> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 1 };
> cqlsh> CREATE TABLE ks.ta ( id text PRIMARY KEY, col text );
> cqlsh> INSERT INTO ks.ta (id, col) VALUES ('myId', 'myCol') IF NOT EXISTS;
>  [applied]
> ---
>   True
> cqlsh> DELETE FROM ks.ta WHERE id = 'myId';
> cqlsh> SELECT * FROM ks.ta WHERE id='myId';
>  id   | col
> --+---
>  myId | myCol
> {code}
>  * Only happens if the client is on a different host (works as expected on 
> the same host)
>  * Works as expected without IF NOT EXISTS
>  * A ~500 ms delay between INSERT and DELETE fixes the issue.
> Logs attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14239) OutOfMemoryError when bootstrapping with less than 100GB RAM

2018-04-12 Thread Sergey Kirillov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435158#comment-16435158
 ] 

Sergey Kirillov commented on CASSANDRA-14239:
-

[~pauloricardomg] with mutation_repair_rows_per_batch=1000 I got OOM (heap size 
is 31G), with mutation_repair_rows_per_batch=500 everything was very similar to 
default mutation_repair_rows_per_batch=100.

It seems that the only way to fix this for me is to remove MVs.

> OutOfMemoryError when bootstrapping with less than 100GB RAM
> 
>
> Key: CASSANDRA-14239
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14239
> Project: Cassandra
>  Issue Type: Bug
> Environment: Details of the bootstrapping Node
>  * ProLiant BL460c G7
>  * 56GB RAM
>  * 2x 146GB 10K HDD (One dedicated for Commitlog, one for Data, Hints and 
> saved_caches)
>  * CentOS 7.4 on SD-Card
>  * /tmp and /var/log on tmpfs
>  * Oracle JDK 1.8.0_151
>  * Cassandra 3.11.1
> Cluster
>  * 10 existing Nodes (Up and Normal)
>Reporter: Jürgen Albersdorfer
>Priority: Major
>  Labels: materializedviews
> Attachments: Objects-by-class.csv, 
> Objects-with-biggest-retained-size.csv, Selection_420.png, Selection_421.png, 
> cassandra-env.sh, cassandra.yaml, dstat.png, gc.log.0.201804111524.zip, 
> gc.log.0.current.zip, gc.log.20180441.zip, jvm.options, jvm_opts.txt, 
> stack-traces.txt
>
>
> Hi, I face an issue when bootstrapping a Node having less than 100GB RAM on 
> our 10 Node C* 3.11.1 Cluster.
> During bootstrap, when I watch the cassandra.log I observe a growth in JVM 
> Heap Old Gen which gets not significantly freed up any more.
> I know that JVM collects on Old Gen only when really needed. I can see 
> collections, but there is always a remainder which seems to grow forever 
> without ever getting freed.
> After the Node successfully Joined the Cluster, I can remove the extra RAM I 
> have given it for bootstrapping without any further effect.
> It feels like Cassandra will not forget about every single byte streamed over 
> the Network over time during bootstrapping, - which would be a memory leak 
> and a major problem, too.
> I was able to produce a HeapDumpOnOutOfMemoryError from a 56GB Node (40 GB 
> assigned JVM Heap). YourKit Profiler shows huge amount of Memory allocated 
> for org.apache.cassandra.db.Memtable (22 GB) 
> org.apache.cassandra.db.rows.BufferCell (19 GB) and java.nio.HeapByteBuffer 
> (11 GB)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14377) Returning invalid JSON for NaN and Infinity float values

2018-04-12 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-14377:
--

Assignee: Benjamin Lerer

> Returning invalid JSON for NaN and Infinity float values
> 
>
> Key: CASSANDRA-14377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14377
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Piotr Sarna
>Assignee: Benjamin Lerer
>Priority: Minor
>
> After inserting special float values like NaN and Infinity into a table:
> {{CREATE TABLE testme (t1 bigint, t2 float, t3 float, PRIMARY KEY (t1));}}
> {{INSERT INTO testme (t1, t2, t3) VALUES (7, NaN, Infinity);}}
> and returning them as JSON...
> {{cqlsh:demodb> select json * from testme;}}
> {{ [json]}}
> {{--}}
> {{ \{"t1": 7, "t2": NaN, "t3": Infinity}}}
>  
> ... the result will not be validated (e.g. with 
> [https://jsonlint.com/|https://jsonlint.com/)] ) because neither NaN nor 
> Infinity is a valid JSON value. The consensus seems to be returning JSON's 
> `null` in these cases, based on this article 
> [https://stackoverflow.com/questions/1423081/json-left-out-infinity-and-nan-json-status-in-ecmascript]
>  and other similar ones.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14261) Compaction Profiling Improvements