[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2021-07-27 Thread Ekaterina Dimitrova (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388128#comment-17388128
 ] 

Ekaterina Dimitrova commented on CASSANDRA-13087:
-

Ping [~blerer] :)

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
>Priority: Normal
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2020-04-07 Thread Antonio (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077049#comment-17077049
 ] 

Antonio commented on CASSANDRA-13087:
-

i also have the same problem in 2.1.15 with nodetool ‘compact’， can anyone help？

java.lang.IllegalArgumentException: Not enough bytes. Offset: 2. Length: 26163. 
Buffer size: 25139
 at 
org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
 ~[apache-cassandra-2.1.15.jar:2.1.15]
 at 
org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
 ~[apache-cassandra-2.1.15.jar:2.1.15]
 at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
 ~[apache-cassandra-2.1.15.jar:2.1.15]
 at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
 ~[apache-cassandra-2.1.15.jar:2.1.15]
 at 
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
 ~[apache-cassandra-2.1.15.jar:2.1.15]
 at org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
~[apache-cassandra-2.1.15.jar:2.1.15]
 at org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
~[apache-cassandra-2.1.15.jar:2.1.15]

 

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
>Priority: Normal
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-08-30 Thread Carlos Rolo (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147432#comment-16147432
 ] 

Carlos Rolo commented on CASSANDRA-13087:
-

Hello,

Facing the same bug, patched Cassandra with this same patch, and still having 
the same problem. Might be due to a different issue (But we get the exactly 
same stack trace).

ERROR [CompactionExecutor:3] 2017-08-29 22:35:14,591 CassandraDaemon.java:231 - 
Exception in thread Thread[CompactionExecutor:3,1,main]
java.lang.IllegalArgumentException: Not enough bytes. Offset: 14390. Length: 
25398. Buffer size: 25400
at 
org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
 ~[main/:na]
at 
org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
 ~[main/:na]
at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
 ~[main/:na]
at 
org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:193)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:174)
 ~[main/:na]
at 
org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:129)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:278)
 ~[main/:na]
at 
org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:231)
 ~[main/:na]
at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 ~[guava-16.0.jar:na]
at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
~[guava-16.0.jar:na]
at 
org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:211) 
~[main/:na]
at 
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:146)
 ~[main/:na]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:125)
 ~[main/:na]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:99)
 ~[main/:na]
at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
 ~[guava-16.0.jar:na]
at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
~[guava-16.0.jar:na]
at 
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:191)
 ~[main/:na]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
 ~[main/:na]
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
 ~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_65]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_65]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_65]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]

Will Run scrub now to see if it does throw the same error.

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-06-21 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057354#comment-16057354
 ] 

Benjamin Lerer commented on CASSANDRA-13087:


[~fabrice.facorat] Thanks for the information. I will review the patch.

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>Assignee: Cyril Scetbon
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-06-21 Thread FACORAT (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057323#comment-16057323
 ] 

FACORAT commented on CASSANDRA-13087:
-

We fix the issues with attached patch.
scrubbing the sstables at the origin won't help as Cassandra and nodetool scrub 
will happily accept orphaned partition keys (i.e partition keys with not 
liveness information and no cells)

[
{"key": "pk_1", "cells": []}

]

However the serialization during streaming will incorrectly handle this use 
case and produce and badly formed sstable.

Only attached patch allow to fix serialization bug for streamed sstables

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-06-21 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057189#comment-16057189
 ] 

Benjamin Lerer commented on CASSANDRA-13087:


[~fabrice.facorat] do you still have that problem or can it be closed? 

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-03-30 Thread Paulo Motta (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949861#comment-15949861
 ] 

Paulo Motta commented on CASSANDRA-13087:
-

This is probably a leftover from the bug CASSANDRA-10791 and is be fixed by 
scrubbing the faulty sstables on the source and destination nodes before 
running repair again.

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-02-08 Thread FACORAT (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858393#comment-15858393
 ] 

FACORAT commented on CASSANDRA-13087:
-

Normal data structure that pose no problem when streamed:
[
{"key": "pk_1",
"cells": [["ck1_myType1:ck2_myProv1:ck3_myData:","",1485975340370722],
   
["ck1_myType1:ck2_myProv1:ck3_myData:dvalue","fooBar1",1485975340370722]]}
]
[
{"key": "pk_1",
"cells": 
[["ck1_myType1:ck2_myProv1:ck3_myData:dvalue",1485975924,1485975924865834,"d"]]}
]

Data structure that produce compaction exceptions on stream destination node:

[
{"key": "pk_1",
"cells": []}
]


> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-02-08 Thread FACORAT (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858387#comment-15858387
 ] 

FACORAT commented on CASSANDRA-13087:
-

Orphans partition keys can be identified with sstable2json as you will see 
empty cells structure in the output like this

{"key": "CLP0100441361214:PID",
 "cells": []},


Please note that the table have a coumpound partition key, but also clustering 
keys.

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
> Attachments: CASSANDRA-13087.patch
>
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-01-27 Thread Sorin Manolache (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842419#comment-15842419
 ] 

Sorin Manolache commented on CASSANDRA-13087:
-

Hello,

We think we have identified the cause of the error: An end-of-row marker 
(0x) without row header and with no columns in the data file of the sstable.

If we have a row without any columns then the execution does not enter the 
"while (iter.hasNext())" loop in db.SSTableWriter.appendFromStream. Therefore 
the db.ColumnIndex.add is not invoked before the execution arrives at

columnIndexer.maybeWriteEmptyRowHeader();
dataFile.stream.writeShort(END_OF_ROW);

in db.SSTableWriter.appendFromStream.

If the row without columns is live (and we have such rows) then the row header 
is not added.

I propose that
columnIndexer.maybeWriteEmptyRowHeader();
in db.SSTableWriter.appendFromStream is replaced by
columnIndexer.maybeWriteRowHeader();
(i.e. maybeWriteRowHeader instead of maybeWriteEmptyRowHeader).

If we apply this modification, we don't have the exception any more. We would 
like your confirmation that the proposed modification does not have other 
undesired side-effects.

Best regards,
Sorin

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-01-10 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816206#comment-15816206
 ] 

Cyril Scetbon commented on CASSANDRA-13087:
---

Hey [~pauloricardomg], did you see that one ? It looks pretty similar to the 
one you fixed.

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>

[jira] [Commented] (CASSANDRA-13087) Not enough bytes exception during compaction

2017-01-02 Thread FACORAT (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792724#comment-15792724
 ] 

FACORAT commented on CASSANDRA-13087:
-

This seems closely related to 
[CASSANDRA-10961|https://issues.apache.org/jira/browse/CASSANDRA-10961] issue

> Not enough bytes exception during compaction
> 
>
> Key: CASSANDRA-13087
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13087
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Ubuntu 14.04.3 LTS, Cassandra 2.1.14
>Reporter: FACORAT
>
> After a repair we have compaction exceptions on some nodes and its spreading
> {noformat}
> ERROR [CompactionExecutor:14065] 2016-12-30 14:45:07,245 
> CassandraDaemon.java:229 - Exception in thread 
> Thread[CompactionExecutor:14065,1,main]
> java.lang.IllegalArgumentException: Not enough bytes. Offset: 5. Length: 
> 20275. Buffer size: 12594
> at 
> org.apache.cassandra.db.composites.AbstractCType.checkRemaining(AbstractCType.java:378)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:100)
>  ~[apache-cassandra-2.1.14.ja
> r:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:171)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) 
> ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>  ~[guava-16.0.jar:na]
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) 
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:166)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:193) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
>  ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-2.1.14.jar:2
> .1.14]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>