[jira] [Created] (CASSANDRA-10908) client_encryption seems to prevent startup in 3.1

2015-12-18 Thread Will Hayworth (JIRA)
Will Hayworth created CASSANDRA-10908:
-

 Summary: client_encryption seems to prevent startup in 3.1
 Key: CASSANDRA-10908
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
 Project: Cassandra
  Issue Type: Bug
 Environment: Amazon Linux 2015.09
Reporter: Will Hayworth
 Attachments: client_encryption.log

Hi there! I'm a total Cassandra novice, so please forgive me if this report is 
lacking or malformed. I've been setting up a new cluster and trying to use 3.1 
but I've noticed that so much as *enabling* client-to-node encryption causes 
the node to shut down after determining there's no gossip backlog. Notably, I 
set up node-to-node encryption beforehand and that was working just fine. I 
thought they might be interfering with each other, but that seems not to be the 
case. Logs attached. (Please let me know how I can be of further help!)

Thanks for looking at this,
Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-10908) client_encryption seems to prevent startup in 3.1

2015-12-18 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-10908:
--
Attachment: 2.2.4-client-log-trace.log

> client_encryption seems to prevent startup in 3.1
> -
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-10908) client_encryption seems to prevent startup in 3.1

2015-12-18 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065289#comment-15065289
 ] 

Will Hayworth commented on CASSANDRA-10908:
---

I'm noticing this happening with 2.2.4, too. Puzzling. :( Attaching trace-level 
logs--the server just dies silently.

> client_encryption seems to prevent startup in 3.1
> -
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-10908) client_encryption seems to prevent startup in 2.2.4 and 3.1

2015-12-18 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-10908:
--
Summary: client_encryption seems to prevent startup in 2.2.4 and 3.1  (was: 
client_encryption seems to prevent startup in 3.1)

> client_encryption seems to prevent startup in 2.2.4 and 3.1
> ---
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-10908) client_encryption seems to prevent startup in 2.2.4 and 3.1

2015-12-19 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065294#comment-15065294
 ] 

Will Hayworth commented on CASSANDRA-10908:
---

I'm so sorry--I've screwed up here.

{quote}
ERROR [main] 2015-12-19 08:15:55,528 CassandraDaemon.java:651 - Exception 
encountered during startup
java.lang.RuntimeException: Unable to create thrift socket to /0.0.0.0:9160
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$Factory.buildTServer(CustomTThreadPoolServer.java:267)
 ~[apache-cassandra-2.2.4.jar:2.2.4]
at 
org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:46)
 ~[apache-cassandra-2.2.4.jar:2.2.4]
at 
org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.(ThriftServer.java:131)
 ~[apache-cassandra-2.2.4.jar:2.2.4]
at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:58) 
~[apache-cassandra-2.2.4.jar:2.2.4]
at 
org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:450) 
[apache-cassandra-2.2.4.jar:2.2.4]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:544) 
[apache-cassandra-2.2.4.jar:2.2.4]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:638) 
[apache-cassandra-2.2.4.jar:2.2.4]
Caused by: org.apache.thrift.transport.TTransportException: Could not bind to 
port 9160
at 
org.apache.thrift.transport.TSSLTransportFactory.createServer(TSSLTransportFactory.java:120)
 ~[libthrift-0.9.2.jar:0.9.2]
at 
org.apache.thrift.transport.TSSLTransportFactory.getServerSocket(TSSLTransportFactory.java:105)
 ~[libthrift-0.9.2.jar:0.9.2]
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$Factory.buildTServer(CustomTThreadPoolServer.java:255)
 ~[apache-cassandra-2.2.4.jar:2.2.4]
... 6 common frames omitted
Caused by: java.lang.IllegalArgumentException: Cannot support 
TLS_RSA_WITH_AES_256_CBC_SHA with currently installed providers
at sun.security.ssl.CipherSuiteList.(CipherSuiteList.java:92) 
~[na:1.8.0_65]
at 
sun.security.ssl.SSLServerSocketImpl.setEnabledCipherSuites(SSLServerSocketImpl.java:200)
 ~[na:1.8.0_65]
at 
org.apache.thrift.transport.TSSLTransportFactory.createServer(TSSLTransportFactory.java:115)
 ~[libthrift-0.9.2.jar:0.9.2]
... 8 common frames omitted
{quote}

Found in {{system.log}}. Enough said. Sorry for the noise.

> client_encryption seems to prevent startup in 2.2.4 and 3.1
> ---
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-10908) INVALID: client_encryption seems to prevent startup in 2.2.4 and 3.1

2015-12-19 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-10908:
--
Summary: INVALID: client_encryption seems to prevent startup in 2.2.4 and 
3.1  (was: client_encryption seems to prevent startup in 2.2.4 and 3.1)

> INVALID: client_encryption seems to prevent startup in 2.2.4 and 3.1
> 
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-10908) bad report: client_encryption seems to prevent startup in 2.2.4 and 3.1

2015-12-19 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-10908:
--
Summary: bad report: client_encryption seems to prevent startup in 2.2.4 
and 3.1  (was: INVALID: client_encryption seems to prevent startup in 2.2.4 and 
3.1)

> bad report: client_encryption seems to prevent startup in 2.2.4 and 3.1
> ---
>
> Key: CASSANDRA-10908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10908
> Project: Cassandra
>  Issue Type: Bug
> Environment: Amazon Linux 2015.09
>Reporter: Will Hayworth
> Attachments: 2.2.4-client-log-trace.log, client_encryption.log
>
>
> Hi there! I'm a total Cassandra novice, so please forgive me if this report 
> is lacking or malformed. I've been setting up a new cluster and trying to use 
> 3.1 but I've noticed that so much as *enabling* client-to-node encryption 
> causes the node to shut down after determining there's no gossip backlog. 
> Notably, I set up node-to-node encryption beforehand and that was working 
> just fine. I thought they might be interfering with each other, but that 
> seems not to be the case. Logs attached. (Please let me know how I can be of 
> further help!)
> Thanks for looking at this,
> Will



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-10980) nodetool scrub NPEs when keyspace isn't specified

2016-01-06 Thread Will Hayworth (JIRA)
Will Hayworth created CASSANDRA-10980:
-

 Summary: nodetool scrub NPEs when keyspace isn't specified
 Key: CASSANDRA-10980
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10980
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Cassandra (and nodetool) version 3.1
Reporter: Will Hayworth
Priority: Trivial
 Attachments: nodetool_scrub_npe.txt

I've attached logs of what I saw. Running nodetool scrub without anything else 
specified resulted in the NPE. Running with the keyspace specified saw 
successful termination.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-10980) nodetool scrub NPEs when keyspace isn't specified

2016-01-06 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-10980:
--
Reproduced In:   (was: 3.1)

> nodetool scrub NPEs when keyspace isn't specified
> -
>
> Key: CASSANDRA-10980
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10980
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: Cassandra (and nodetool) version 3.1
>Reporter: Will Hayworth
>Priority: Trivial
> Attachments: nodetool_scrub_npe.txt
>
>
> I've attached logs of what I saw. Running nodetool scrub without anything 
> else specified resulted in the NPE. Running with the keyspace specified saw 
> successful termination.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-16 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149812#comment-15149812
 ] 

Will Hayworth commented on CASSANDRA-11172:
---

I'm seeing this exact problem with full repairs on C* 3.3.

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-16 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11172:
--
Attachment: beep.txt

Seeing this on C* 3.3 after running a full repair on another node. {{nodetool 
repair --full -pr -j 4 my_keyspace_name}}

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Attachments: beep.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Issue Comment Deleted] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-16 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11172:
--
Comment: was deleted

(was: I'm seeing this exact problem with full repairs on C* 3.3.)

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Attachments: beep.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-16 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149814#comment-15149814
 ] 

Will Hayworth edited comment on CASSANDRA-11172 at 2/17/16 4:16 AM:


Seeing this on C* 3.3 after running a full repair on another node. {{nodetool 
repair --full -pr -j 4 my_keyspace_name}} I can provide whatever further 
details you'd like, obviously.


was (Author: _wsh):
Seeing this on C* 3.3 after running a full repair on another node. {{nodetool 
repair --full -pr -j 4 my_keyspace_name}}

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Attachments: beep.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-16 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150049#comment-15150049
 ] 

Will Hayworth commented on CASSANDRA-11172:
---

debug.log? I can go hunt through my compressed files but this quickly pushes 
the other logs out of rotation. I've watched it happen. Thousands upon 
thousands of lines. And it's happened for a bunch of different nodes. The 
solution has been to kill them and restart but it's temporary.

I just checked--all 20 zipped system.logs are filled with 40,000+ lines like 
the ones in my sample file. I was going to upload some to you but they're just 
identical, including the earliest lines I have:
{noformat}
INFO  [CompactionExecutor:35] 2016-02-17 01:59:03,111 LeveledManifest.java:438 
- Adding high-level (L0) 
BigTableReader(path='/var/lib/cassandra/data/segmentation/domain_events_by_event_domain_time-e81d74f0cd3a11e5aad8e7b84e29e52f/ma-3663-big-Data.db')
 to candidates
INFO  [CompactionExecutor:37] 2016-02-17 01:59:03,112 LeveledManifest.java:438 
- Adding high-level (L3) 
BigTableReader(path='/var/lib/cassandra/data/segmentation/times_by_event_domain_user-e6112a30cd3a11e5ba896547d15a24f6/ma-4586-big-Data.db')
 to candidates
INFO  [CompactionExecutor:35] 2016-02-17 01:59:03,276 LeveledManifest.java:438 
- Adding high-level (L0) 
BigTableReader(path='/var/lib/cassandra/data/segmentation/domain_events_by_event_domain_time-e81d74f0cd3a11e5aad8e7b84e29e52f/ma-3663-big-Data.db')
 to candidates
INFO  [CompactionExecutor:37] 2016-02-17 01:59:03,276 LeveledManifest.java:438 
- Adding high-level (L3) 
BigTableReader(path='/var/lib/cassandra/data/segmentation/times_by_event_domain_user-e6112a30cd3a11e5ba896547d15a24f6/ma-4586-big-Data.db')
 to candidates
{noformat}

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Attachments: beep.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction

2016-02-17 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11172:
--
Attachment: trapped_in_compaction.txt
trapped_in_compaction_mixed.txt
tpstats_compaction.txt

I don't know if these are worth anything but, in case they are: {{nodetool 
tpstats}} before I shut down the node and both mixed (native) and only-Java 
thread dumps.

> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Attachments: beep.txt, tpstats_compaction.txt, 
> trapped_in_compaction.txt, trapped_in_compaction_mixed.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will 
> enter an infinite loop with vast amounts of logs lines recording, "Adding 
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam 
> quickly rotates out possibly useful earlier debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-01 Thread Will Hayworth (JIRA)
Will Hayworth created CASSANDRA-11293:
-

 Summary: NPE when using CQLSSTableWriter
 Key: CASSANDRA-11293
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
 Project: Cassandra
  Issue Type: Bug
 Environment: C* 3.3
Reporter: Will Hayworth


Hi all!

I'm trying to using CQLSSTableWriter to load a bunch of historical data into my 
cluster and I'm getting NullPointerExceptions consistently after having written 
a few million rows (anywhere from 0.5 to 1.5 GB of data).

{code}
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) 
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) at 
java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
 at 
com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)

Caused by: java.lang.NullPointerException at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
 at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
 at java.util.TimSort.binarySort(TimSort.java:296) at 
java.util.TimSort.sort(TimSort.java:239) at 
java.util.Arrays.sort(Arrays.java:1512) at 
org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) at 
org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
 at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
{code}

This may be a red herring, but I started encountering this when I parallelized 
writes. (I wasn't aware that doing so was safe until I saw CASSANDRA-7463; I 
Googled in vain for a while before that.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-01 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11293:
--
Description: 
Hi all!

I'm trying to using CQLSSTableWriter to load a bunch of historical data into my 
cluster and I'm getting NullPointerExceptions consistently after having written 
a few million rows (anywhere from 0.5 to 1.5 GB of data).

{code}
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) 
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) at 
java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
 at 
com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)

Caused by: java.lang.NullPointerException at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
 at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
 at java.util.TimSort.binarySort(TimSort.java:296) at 
java.util.TimSort.sort(TimSort.java:239) at 
java.util.Arrays.sort(Arrays.java:1512) at 
org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) at 
org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
 at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
{code}

This may be a red herring, but I started encountering this when I parallelized 
writes. (I wasn't aware that doing so was safe until I saw CASSANDRA-7463; I 
Googled in vain for a while before that.) I'm also definitely not passing any 
nulls in my {{addRow}} calls.

  was:
Hi all!

I'm trying to using CQLSSTableWriter to load a bunch of historical data into my 
cluster and I'm getting NullPointerExceptions consistently after having written 
a few million rows (anywhere from 0.5 to 1.5 GB of data).

{code}
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) 
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) at 
java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
 at 
com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)

Caused by: java.lang.NullPointerException at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
 at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
 at java.util.TimSort.binarySort(TimSort.java:296) at 
java.util.TimSort.sort(TimSort.java:239) at 
java.util.Arrays.sort(Arrays.java:1512) at 
org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) at 
org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
org.apache.cassandra.db.partitions.PartitionUpdate.bui

[jira] [Updated] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-01 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11293:
--
Description: 
Hi all!

I'm trying to using CQLSSTableWriter to load a bunch of historical data into my 
cluster and I'm getting NullPointerExceptions consistently after having written 
a few million rows (anywhere from 0.5 to 1.5 GB of data).

{code}
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) 
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) at 
java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
 at 
com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)

Caused by: java.lang.NullPointerException at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
 at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
 at
java.util.TimSort.binarySort(TimSort.java:296) at
java.util.TimSort.sort(TimSort.java:239) at
java.util.Arrays.sort(Arrays.java:1512) at
org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) at 
org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
 at 
org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
 at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
 at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
{code}

This may be a red herring, but I started encountering this when I parallelized 
writes. (I wasn't aware that doing so was safe until I saw CASSANDRA-7463; I 
Googled in vain for a while before that.) I'm also definitely not passing any 
nulls in my {{addRow}} calls.

  was:
Hi all!

I'm trying to using CQLSSTableWriter to load a bunch of historical data into my 
cluster and I'm getting NullPointerExceptions consistently after having written 
a few million rows (anywhere from 0.5 to 1.5 GB of data).

{code}
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) 
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) at 
java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
 at 
com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)

Caused by: java.lang.NullPointerException at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
 at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
 at java.util.TimSort.binarySort(TimSort.java:296) at 
java.util.TimSort.sort(TimSort.java:239) at 
java.util.Arrays.sort(Arrays.java:1512) at 
org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) at 
org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
org.apache.cassandra.db.partitions.PartitionUpdate.build(

[jira] [Commented] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-02 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175739#comment-15175739
 ] 

Will Hayworth commented on CASSANDRA-11293:
---

Also seeing this on trunk, too:

{code}
Caused by: java.lang.NullPointerException
at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:123)
at 
org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:41)
at java.util.TimSort.countRunAndMakeAscending(TimSort.java:356)
at java.util.TimSort.sort(TimSort.java:234)
at java.util.Arrays.sort(Arrays.java:1512)
at org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027)
at 
org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036)
at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075)
at 
org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:583)
at 
org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:573)
at 
org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:388)
at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
at 
org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
{code}

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-02 Thread Will Hayworth (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Hayworth updated CASSANDRA-11293:
--
Environment: C* 3.3, C* trunk  (was: C* 3.3)

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3, C* trunk
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-06 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182621#comment-15182621
 ] 

Will Hayworth commented on CASSANDRA-11293:
---

Just a note that going singlethreaded magically fixed everything. I'm guessing 
something isn't synchronized that needs to be? :S

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3, C* trunk
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-07 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182840#comment-15182840
 ] 

Will Hayworth edited comment on CASSANDRA-11293 at 3/7/16 10:18 AM:


I was, and this is very unclear. No assertions are made anywhere I can find in 
the docs about {{CQLSSTableWriter}}'s thread safety or lack thereof, and when I 
saw this on a resolved issue:

{quote}
Currently it is not possible to programatically write multiple SSTables for the 
same table in parallel using the CQLSSTableWriter. 
{quote}

my inference was that it was now possible to do so. That ticket also says the 
*same* table. Is it really possible to use multiple {{CQLSSTableWriter}}s 
writing to the same directory at the same time? I'd be astonished if that were 
so.


was (Author: _wsh):
I was, and this is very unclear. No assertions are made anywhere I can find in 
the docs about {{CQLSSTableWriter}}'s thread safety or lack thereof, and when I 
saw this on a resolved issue:

{quote}
Currently it is not possible to programatically write multiple SSTables for the 
same table in parallel using the CQLSSTableWriter. 
{quote}

my inference was that it was now possible to do so. That ticket also says the 
*same* table. Is it really possible to use multiple {{CQLSSTableWriters}}s 
writing to the same directory at the same time? I'd be astonished if that were 
so.

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3, C* trunk
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-07 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182840#comment-15182840
 ] 

Will Hayworth edited comment on CASSANDRA-11293 at 3/7/16 10:18 AM:


I was, and this is very unclear. No assertions are made anywhere I can find in 
the docs about {{CQLSSTableWriter}}'s thread safety or lack thereof, and when I 
saw this on a resolved issue:

{quote}
Currently it is not possible to programatically write multiple SSTables for the 
same table in parallel using the CQLSSTableWriter. 
{quote}

my inference was that it was now possible to do so. That ticket also says the 
*same* table. Is it really possible to use multiple CQLSSTableWriters writing 
to the same directory at the same time? I'd be astonished if that were so.


was (Author: _wsh):
I was, and this is very unclear. No assertions are made anywhere I can find in 
the docs about {{CQLSSTableWriter}}'s thread safety or lack thereof, and when I 
saw this on a resolved issue:

{quote}
Currently it is not possible to programatically write multiple SSTables for the 
same table in parallel using the CQLSSTableWriter. 
{quote}

my inference was that it was now possible to do so. That ticket also says the 
*same* table. Is it really possible to use multiple {{CQLSSTableWriter}}s 
writing to the same directory at the same time? I'd be astonished if that were 
so.

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3, C* trunk
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-11293) NPE when using CQLSSTableWriter

2016-03-07 Thread Will Hayworth (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182840#comment-15182840
 ] 

Will Hayworth commented on CASSANDRA-11293:
---

I was, and this is very unclear. No assertions are made anywhere I can find in 
the docs about {{CQLSSTableWriter}}'s thread safety or lack thereof, and when I 
saw this on a resolved issue:

{quote}
Currently it is not possible to programatically write multiple SSTables for the 
same table in parallel using the CQLSSTableWriter. 
{quote}

my inference was that it was now possible to do so. That ticket also says the 
*same* table. Is it really possible to use multiple {{CQLSSTableWriters}}s 
writing to the same directory at the same time? I'd be astonished if that were 
so.

> NPE when using CQLSSTableWriter
> ---
>
> Key: CASSANDRA-11293
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11293
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 3.3, C* trunk
>Reporter: Will Hayworth
>
> Hi all!
> I'm trying to using CQLSSTableWriter to load a bunch of historical data into 
> my cluster and I'm getting NullPointerExceptions consistently after having 
> written a few million rows (anywhere from 0.5 to 1.5 GB of data).
> {code}
> java.lang.NullPointerException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at 
> java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
>  at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) 
> at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) at 
> java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) 
> at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
>  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
>  at 
> com.atlassian.engagementengine.segmentation.helenus.Daemon.main(Daemon.java:24)
> Caused by: java.lang.NullPointerException at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:126)
>  at 
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:44)
>  at
> java.util.TimSort.binarySort(TimSort.java:296) at
> java.util.TimSort.sort(TimSort.java:239) at
> java.util.Arrays.sort(Arrays.java:1512) at
> org.apache.cassandra.utils.btree.BTree$Builder.sort(BTree.java:1027) at 
> org.apache.cassandra.utils.btree.BTree$Builder.autoEnforce(BTree.java:1036) 
> at org.apache.cassandra.utils.btree.BTree$Builder.build(BTree.java:1075) at 
> org.apache.cassandra.db.partitions.PartitionUpdate.build(PartitionUpdate.java:572)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.maybeBuild(PartitionUpdate.java:562)
>  at 
> org.apache.cassandra.db.partitions.PartitionUpdate.holder(PartitionUpdate.java:370)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:177)
>  at 
> org.apache.cassandra.db.partitions.AbstractBTreePartition.unfilteredIterator(AbstractBTreePartition.java:172)
>  at 
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:209)
> {code}
> This may be a red herring, but I started encountering this when I 
> parallelized writes. (I wasn't aware that doing so was safe until I saw 
> CASSANDRA-7463; I Googled in vain for a while before that.) I'm also 
> definitely not passing any nulls in my {{addRow}} calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)