[jira] [Issue Comment Edited] (CASSANDRA-2863) NPE when writing SSTable generated via repair

2011-09-23 Thread chris erway (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113501#comment-13113501
 ] 

chris erway edited comment on CASSANDRA-2863 at 9/23/11 3:18 PM:
-

Since my last two comments, I increased my RF and started a rolling repair on 
my nodes.  This has caused this NPE to pop up on all the boxes over the last 
couple of days as they process SSTables.  Again, all the nodes are fresh 0.8.6 
installs from a few days ago using the ComboAMI.


 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,086 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF1-g-535
 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,172 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF2-g-350
 INFO [CompactionExecutor:648] 2011-09-22 04:36:01,721 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF3-g-456
ERROR [Thread-3658] 2011-09-22 04:36:04,821 AbstractCassandraDaemon.java (line 
139) Fatal exception in thread Thread[Thread-3658,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154)
at 
org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
at 
org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
Caused by: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138)
... 3 more
Caused by: java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
ERROR [CompactionExecutor:648] 2011-09-22 04:36:04,823 
AbstractCassandraDaemon.java (line 139) Fatal exception in thread 
Thread[CompactionExecutor:648,1,main]
java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)


  was (Author: cce):
Since my last two comments, I increased my RF and started a rolling repair 
on my nodes.  This has caused this NPE to pop up on all the boxes as the 
process SSTables.  Again, all the nodes are fresh 0.8.6 installs from a few 
days ago using the ComboAMI.


 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,086 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF1-g-535
 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,172 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF2-g-350
 INFO [CompactionExecutor:648] 2011-09-22 04:36:01,721 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF3-g-456
ERROR [Thread-3658] 2011-09-22 04:36:04,821 AbstractCassandraDaemon.java (line 
139) Fatal exception in thread Thread[Thread-3658,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at

[jira] [Issue Comment Edited] (CASSANDRA-2863) NPE when writing SSTable generated via repair

2011-09-23 Thread chris erway (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113501#comment-13113501
 ] 

chris erway edited comment on CASSANDRA-2863 at 9/23/11 3:27 PM:
-

Since my last two comments, I increased my RF and started a rolling repair on 
my nodes.  This has caused this NPE to pop up on all the boxes over the last 
couple of days as they process SSTables.  Again, all the nodes are fresh 0.8.6 
installs from a few days ago using the ComboAMI.  I've seen backtraces like the 
one below appear at least a couple of times on each node in my cluster as I was 
repairing..

 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,086 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF1-g-535
 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,172 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF2-g-350
 INFO [CompactionExecutor:648] 2011-09-22 04:36:01,721 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF3-g-456
ERROR [Thread-3658] 2011-09-22 04:36:04,821 AbstractCassandraDaemon.java (line 
139) Fatal exception in thread Thread[Thread-3658,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154)
at 
org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
at 
org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
Caused by: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138)
... 3 more
Caused by: java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
ERROR [CompactionExecutor:648] 2011-09-22 04:36:04,823 
AbstractCassandraDaemon.java (line 139) Fatal exception in thread 
Thread[CompactionExecutor:648,1,main]
java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)


  was (Author: cce):
Since my last two comments, I increased my RF and started a rolling repair 
on my nodes.  This has caused this NPE to pop up on all the boxes over the last 
couple of days as they process SSTables.  Again, all the nodes are fresh 0.8.6 
installs from a few days ago using the ComboAMI.


 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,086 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF1-g-535
 INFO [CompactionExecutor:648] 2011-09-22 04:35:51,172 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF2-g-350
 INFO [CompactionExecutor:648] 2011-09-22 04:36:01,721 SSTableReader.java (line 
162) Opening /raid0/cassandra/data/Keyspace/CF3-g-456
ERROR [Thread-3658] 2011-09-22 04:36:04,821 AbstractCassandraDaemon.java (line 
139) Fatal exception in thread Thread[Thread-3658,5,main]
java.lang.RuntimeException:

[jira] [Issue Comment Edited] (CASSANDRA-2863) NPE when writing SSTable generated via repair

2011-09-20 Thread chris erway (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109079#comment-13109079
 ] 

chris erway edited comment on CASSANDRA-2863 at 9/20/11 11:20 PM:
--

I just got a similar-looking NPE stack trace.  The node that raised the 
exception was receiving streams from a node being decommissioned (with 
nodetool decommission).  Both nodes were running 0.8.6; both had been 
upgraded a few hours earlier from 0.8.4.

The first:  

ERROR [CompactionExecutor:72] 2011-09-20 22:34:20,892 
AbstractCassandraDaemon.java (line 139) Fatal exception in thread 
Thread[CompactionExecutor:72,1,main]
java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Then half a minute later:

 INFO [CompactionExecutor:72] 2011-09-20 22:34:46,923 SSTableReader.java (line 
162) Opening /mnt/cassandra/data/Keyspace/CF1-g-1536
ERROR [Thread-785] 2011-09-20 22:34:52,054 AbstractCassandraDaemon.java (line 
139) Fatal exception in thread Thread[Thread-785,5,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154)
at 
org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
at 
org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
Caused by: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138)
... 3 more
Caused by: java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)


  was (Author: cce):
I just got a similar-looking NPE stack trace.  The node that raised the 
exception was receiving streams from a node being decommissioned (with 
nodetool decommission).  Both nodes were running 0.8.6; both had been 
upgraded a few hours earlier from 0.8.4.

The first:  

ERROR [CompactionExecutor:72] 2011-09-20 22:34:20,892 
AbstractCassandraDaemon.java (line 139) Fatal exception in thread 
Thread[CompactionExecutor:72,1,main]
java.lang.NullPointerException
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
at 
org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
at 
org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1108)
at 
org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1099)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at

[jira] [Issue Comment Edited] (CASSANDRA-2863) NPE when writing SSTable generated via repair

2011-07-21 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13068956#comment-13068956
 ] 

Héctor Izquierdo edited comment on CASSANDRA-2863 at 7/21/11 12:36 PM:
---

I have a patch from #2818 (2818-v4) applied, if that's of any help. The patch 
only touches messaging classes though.

  was (Author: hector.izquierdo):
I have a patch from 2818 (2818-v4) applied, if that's of any help. The 
patch only touches messaging classes though.
  
 NPE when writing SSTable generated via repair
 -

 Key: CASSANDRA-2863
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2863
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 0.8.1
Reporter: Héctor Izquierdo
Assignee: Sylvain Lebresne
 Fix For: 0.8.2


 A NPE is generated during repair when closing an sstable generated via 
 SSTable build. It doesn't happen always. The node had been scrubbed and 
 compacted before calling repair.
  INFO [CompactionExecutor:2] 2011-07-06 11:11:32,640 SSTableReader.java (line 
 158) Opening /d2/cassandra/data/sbs/walf-g-730
 ERROR [CompactionExecutor:2] 2011-07-06 11:11:34,327 
 AbstractCassandraDaemon.java (line 113) Fatal exception in thread 
 Thread[CompactionExecutor:2,1,main] 
 java.lang.NullPointerException
   at 
 org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.close(SSTableWriter.java:382)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter$RowIndexer.index(SSTableWriter.java:370)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:315)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1103)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:1094)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira